January 23, 2015

| By Brendon Dorn, Systems Analyst

View the original blog post

Is Your Energy Benchmarking Data Up to the Task?

Posted by Brendon Dorn on Jan 23, 2015 12:44:23 PM


How do you measure the projected savings of an energy-efficiency project?

Today, there’s a lot of data that can help answer this question. The issue, however, is whether we’re looking at the right data in the right context – particularly given the complex array of factors (different building systems, functions, etc.) that will affect the outcome.

Many in our industry rely on historical comparisons for savings calculations. These comparisons look at an historical baseline of energy consumption and an equivalent period after the project is complete. These two measurement periods are normalized for variables such as weather and occupancy before being contrasted to determine savings. It’s a time-honored approach, but by itself it may not be robust enough for today’s challenges.

How Historical Comparisons Fall Short

The main problem is that historical comparison provides only one lens with which to view results. Those measuring the data often falsely attribute all of the success (or lack thereof) to a particular project. As a result, they’re either too confident or too skeptical of a prospective investment.  Either way, that’s not a situation that breeds success.

To make a more accurate prediction, we have to make sure the data are more relevant and complete. A more robust approach should include two distinct practices: historical comparison and a methodology called peer group analysis.

What Is Peer Group Analysis?

Peer group analysis is a similar practice to experiments where scientists observe a test group and a control group. In peer group analysis, the test group consists of the sites where the project is being implemented. The control group or peer group contains the sites that will be used for quantifying the effect of the project. The nuances of this practice must be dealt with carefully, but the three main tenets are:

  1. Selection – Peer group sites should be chosen based on building characteristics such as physical traits, geographic location and energy consumption behavior.
  2. Measurement – Best accomplished in two ways. The first is comparing between the two groups during the same time frame. The second is comparing each group to its own historical baseline.
  3. Control – There are always cases where exact matches are hard to find. In these instances, match the groups as closely as possible and control for all variables that are dissimilar.

A Case for Combining the Two Approaches

Here’s an example of a recent project where we’re using a blended approach at ESI. A customer piloted an energy-efficiency measure at a limited number of buildings in several states. We analyzed the results using the traditional historical methodology and calculated an approximate 16% savings in the first year. The results were more than satisfactory to justify the pilot investment. The savings were also substantial enough to start assembling estimates for implementing this energy-efficiency measure at a larger number of sites. However, using peer group analysis, the customer came up with an approximate savings of 5.1%. It was quite a difference, and it no longer justified the initial investment or any future investment.

But that wasn’t the end of it, because neither method by itself gave us the full picture. Together, we examined both analyses more closely. In the process, a new insight emerged: the pilot worked better in warmer climates, enough to justify further investment in those areas. We’d moved from a broad rejection to a targeted, strategic rollout of the investment (the project is in progress).

Elevating the Conversation with Greater Accuracy

In all energy project measurement and verification, there are unknowns that must be filled in as well as we can. It’s essential to know what tools to use when and how to use them properly. There certainly are limitations to both peer group analysis and historical comparison, which is why it’s important to use both methods together to fill in the blanks.

In the many conversations surrounding new development or retrofits, energy projects tend to take a back seat. Energy is cheap, waste is hard to perceive and many people are tired of overpromising in the industry. So how can we stay in the conversation? Accuracy and transparency, through more robust data analysis, is one crucial piece of the puzzle.