Cause and effect, means and ends are often confused and confusing. So what do you need to do to differentiate between assessing if you have reached your goals and evaluating the effectiveness of tools you use to achieve them? In a world where smart devices are seen as the solution, what other skills do we need to ensure we meet our aspirations?

For example, can we save the planet and save money by installing a smart device?

Let’s take a look.

"The UK’s energy network is changing” said the email I received from my energy provider. As the UK Government addressed climate change and energy use behaviour Smart Meters are seen as a tool for achieving the outcomes of reduced carbon emissions. And so there is a target to install 37 million smart meters by 2024 or 84% of all meters in the UK .

But do Smart meters do what the energy industry and government have in mind? Will they reduce bills and ultimately cut carbon emissions? That depends on a lot of things in between.

This is the claim that a government backed advertising campaign made in 2019. Install a smart meter to reduce energy use, lower your bills and help make a greener future.

But do smart meters alone do this for consumers?

“No, not on their own” says money saving expert Martin Lewis.

Is a digital smart meter really an improvement on the analogue. They both provide realtime information about how much energy is being used, one uses dials the other numbers on a screen. 

It is the convenient location of the meter that might be significant. Whilst older meters tend to be located in cupboards and out of sight, smart meters are often put in full view of the householder so that can see how much energy they are using in real time. 

But this alone doesn’t achieve the outcome. For energy use to be cut the householder must adjust their behaviour in some way. And here is where the challenge might lie.

Around 15 years ago good friends of mine installed an OWL device, a predecessor to the smart meter. It too gave realtime information about electricity use.

One half of the couple monitored the meter fastidiously and when the readings increased would run up to their partner in a panic.

Who just happened to be doing the ironing at the time, a device which eats electricity.

But it only eats electricity for a short time whilst doing the ironing.

Put simply, as one partner said to the other, have lower electricity bills or have crisply ironed shirts for work.

You see the smart meter is a tool, a useful device for providing quantitative data. But without an understanding of the wider context it is difficult to interpret the meaning of the data. And this is vital for the outcome of energy use to be effectively achieved 

My friend isn’t alone. The assumption that the smart meters alone will achieve better energy use doesn’t stack up. Nor does it factor in the various negative impacts that their use might bring about.

In the present climate of rapidly rising energy prices a Smart meter can become a compulsive addiction. An distraction rather than a help. An added layer of stress inducing complexity rather than clarity.

They are also a tool that can be used for other means which were hitherto ignored in the debate. There is no mention of energy company’s ability to remotely switch you to a pre-payment account because you have a smart meter installed.

But even if we focus on that initial goal of energy reduction, in the case of smart energy meters a consumer needs to have an understanding about what they are doing, what appliances and devices they are using and for how long when their energy use is flagged as high. 

If the peaks are short lived or they can be attributed to particular events - darker winter nights requiring more light, heating and comfort slow cooking for example - it is difficult to see how behaviours can be adjusted further without sacrifices. 

And this isn’t what the advert is suggesting to the consumer.

The context of data is always important.

The link between the smart meter and behaviour is yet to be proved sufficiently. Therefore setting a target of installations is not the same as meeting the aspiration of reducing carbon. It is a means to an end, not an end in itself. Only by asking additional questions and exploring behaviour changed through qualitative methods can these links be established.

The installation target is based on outputs (number of smart meters installed). To measure outcomes requires much more - the installations is a poor proxy.

It another good illustration of Goodharts Law where the measure itself is the focus rather than what the measure means as I highlighted in an earlier episode about evaluating parcel deliveries.

Smart meters are another example of how our understanding of data requires both the empirical, quantitative figures to be placed into their wider qualitative context. Without the latter, the data is open to far too much misinterpretation and panic over the ironing.  

In seeking to install a set number of smart meters it demonstrates how a means for achieving a target will not automatically achieve the desired outcome on its own.

This is not to say that smart meters are unhelpful. Smart meters are a useful tool in helping consumers be aware of and monitor their energy use but on their own and without changes in consumers behaviour as a result they do no more than the analogue dials they replace. 

Installing a smart meter might be a help but on its own it won’t save the planet. 

We all need to get a bit smarter about how data can help and hinder those aspirations.