Data Authors: Skilled Designers of Data Presentations - Data Fluency: Empowering Your Organization with Effective Data Communication - page 93

Although the requirements are waived in many states, districts and
schools have dramatically reshaped their behavior to focus efforts on a
new “bottom line” of education—state standardized test scores.

Metrics can create focus and alignment in an organization by providing clarity
about what improvement looks like. Metrics can also create behaviors that are
counter-intuitive or contrary to organizational goals. Beware of the call center
that measures agent performance based on average call duration!

Despite the importance of choosing the right metrics, it is common to not
think deeply about what to measure. Here are a few of the most common
mistakes in choosing what metrics to focus on:

Historical conventions —History isn't always a good guide. Many
organizations blindly follow the conventional wisdom in their industry
or simply follow the path of history without thinking about the impli-
cations.

⬛

Simplistic metrics —The path of least resistance may be to take
what your data gives you. Time spent on a website is easily tracked
but doesn't tell you anything about whether the users were actively
engaged or running off to make a cup of coffee.

⬛

Complex metrics —The opposite of simplistic metrics are complex,
contrived metrics that combine data from many data sources. If people
don't understand how a value is created, they may react skeptically to
your analyses.

⬛

Too many metrics —Many organizations fall into a trap of trying to
have a comprehensive collection of metrics that track everything that
managers care about. If you work with dozens of key metrics, those
metrics may not deserve to be called “key.”

⬛

Vanity metrics —Some metrics can make you or your organization
look good but don't tie to outcomes that are important. Be careful that
you don't select metrics simply because you know they'll make you
look good.

⬛

To avoid these common pitfalls and choose the best possible metric, consider
the following framework, one that balances four factors, as shown in Figure 5-5.