“Measuring what’s measurable rather than what’s important” – Better indicators part 1: Donella Meadows

A banger report from the late, great Donella Meadows, talking about sustainable indicators.

WAY too much to cover, so extracts over 3-4 separate posts.

Today is background on indicators before the specific and meatier stuff.

Extracts: (** Part 2 in comments **)

· “We have many words for indicator— sign, symptom, omen, signal, tip, clue, grade, rank, data, pointer, dial, warning light, instrument, measurement”

· “Indicators arise from values (we measure what we care about), and they create values (we care about what we measure”

· Some indicators are quantitatively measurable “while others, which may be equally important, can only be felt qualitatively”

· “Not only do we measure what we value, we also come to value what we measure … No one cared about a blood cholesterol level until over 200 doctors started including it in our annual checkups”

· “Indicators can be tools of change, learning, and propaganda. Their presence, absence, or prominence affect behavior”

· “The world would be a very different place if nations prided themselves not on their high GDPs but on their low infant mortality rates”

· “We try to measure what we value. We come to value what we measure. This feedback process is common, inevitable, useful, and full of pitfalls”

· “When indicators are poorly chosen, they can cause serious malfunctions”

· “Indicators are both important and dangerous because they sit at the center of the decision-making process”

· “Action is taken depending on the discrepancy between the desired state or goal and the perceived state of the system”

· “The perceived state is an indicator. It may not be measured accurately. It may measure not the actual system state, but some proxy or associated state”

· “The indicator may be delayed. It may be “noisy,” so its central tendency is hard to deduce. It may be deliberately or accidentally biased”

· “Misleading indicators will cause over- or under-reactions, changes that are too weak or too strong to bring the system exactly to the desired state”

Several pitfalls in selecting indicators exist:

· Overaggregation: “If too many things are lumped together, their combined message may be indecipherable” – GDP is an example where it doesn’t highlight the specific ‘goods’ and ‘bads’ in shifts

· “Measuring what is measurable, rather than what is important. The area covered by forest rather than the size, diversity, or health of the trees”

· “Dependence on a false model. We may think that the birth rate reflects the availability of family planning programs, when it may actually reflect the freedom of women to use those programs”

· “Deliberate falsification. If an index carries bad news, someone may be tempted to alter it, delay it, change terms or definitions, unfund it, lose it, or otherwise suppress it.”

· “Diverting attention form direct experience .Indicators may mesmerize people with numbers and blind them to their own perceptions”

· “Overconfidence. Indicators may lead people to think they know what they’re doing, or to think what they’re doing is working, when in fact the indicators may be faulty”

· “Incompletenes. Indicators are not the real system. They may miss, many of the subtleties, beauties, wonders, warnings, diversities, possibilities, or perversities of the real system”