The Data Fluent Culture - Data Fluency: Empowering Your Organization with Effective Data Communication - page 145

uNDERSTANDING AND APPRECIATING CREDIBLE,
RELIABLE DATA SOuRCES
“Consider the source,” is a warning with much relevance for modern organiza-
tions. When making decisions or forming opinions, thoughtful people consider
the source of information and the methods with which it was collected. Data
sources carry a reputation, like people.
Outside data sources can have a reputation for credibility. The Dartmouth Atlas
of Health Care is an academic source that has built a strong reputation for
gathering geographically based information about the use of health resources.
For measuring online audiences, comScore has emerged as the standard for
advertisers and agencies when negotiating ad spend. In other sectors, govern-
ment sources like the Bureau of Labor Statistics and Department of Education
have established a role in delivering credible data. In part, these reputations
are developed because these data sources have established protocols and
processes to vet and verify sources.
Sources of internal organizational data also develop a reputation over time.
Many times we've worked with organizations that have one database that is
considered the trusted source of truth, whereas another data set seems to
capture similar information but is known to be faulty. The difference between
these data sources is based on how the data is collected, stored, accessed and
perceived. Reputation of data sources is also influenced by employee, team,
department, organization and industry biases, whether warranted or not. A
data fluent organization needs to create broad awareness of these issues to
guide analyses toward the data sources that are most credible.
understand the Strengths and Weaknesses of Data Sources
Data, a piece of information in and of itself, has limited utility. First, data fluent
members of an organization must understand the importance of reliable data,
or what is referred to as internal validity . Next, to make inferences about the data
to a larger population, we move into the terrain of generalizability, or external
validity . A leader has to know whether the data collected will be generalizable
to other situations. Do the processes or protocols used in one situation apply
to another situation, or are the populations quite different? Leaders, over time,
often develop the skill to determine if the data collected will be generalizable
to other markets, regions, sectors, populations, or situations. Following are a set
of conditions that impact the reliability and generalizability of data. For each
condition, data fluent organizations ask themselves if there is any way that the
condition can explain the story that we would otherwise think the data tells.