How to Check Measures for Adequacy - Text Mining Techniques for Healthcare Provider Quality Determination - page 390

is less than predicted, then it must be because the provider has better quality than if the actual mortal-
ity is greater than predicted. Another assumption using this measure is that a logistic regression model
that has a high misclassification rate in the predicted mortality is useful in assessing the quality of a
provider. Such an assumption tends to penalize those providers with low actual mortality because there
will be little difference to the predicted mortality. However, logistic regression was never designed to
be used in this way.
Also, the general use of logistic regression assumes that the underlying population is normal; as we
showed in Chapter 3, this assumption is not valid. Because of the nature of healthcare outcomes, patient
results tend to be skewed and to follow a gamma distribution. The use of data visualization and kernel
density estimation can be used to investigate the assumption of the population distribution so that the
model can be modified to accommodate the nature of that distribution.
The use of text mining to define a patient severity index does not require a number of the basic as-
sumptions that are required for the more traditional definitions. It is more versatile in that it can be used
with a variety of outcomes, including the occurrence of nosocomial infection. Since outcomes are not
used to define the patient severity index with text mining, outcomes can be used to validate the results.
Text mining is a little used technique in the investigation of health outcomes, but it is extremely versatile,
and does not require the uniformity of data entry since it is the linkage between patient conditions that is
utilized to define a patient severity index rather than just relying upon the conditions themselves. It is the
only method defined in this topic that accepts the assumption that patient co-morbidities are related.
If a severity index is defined using one outcome variable such as mortality, other outcomes should
be used to validate the results; outcomes should be considered multi-dimensional. Instead of relying
on the difference between observed and expected values in a regression that primarily take advantage
of misclassification in the model (in contrast to the original definition of regression), reasons for the
difference in rate should be investigated. Otherwise, providers with low actual rates will be penalized
compared to providers with high rates where the deviation between actual and predicted can be much
greater. Because of the concern with nosocomial infection, the likelihood and risk of acquiring such
infection should be a prime component in any measure of patient quality. Another measure that is im-
portant is the occurrence of adverse events and errors, especially errors in medication dispensing and
prescribing, and patient falls.
We suggest that scorecards should have diminished importance compared to a careful examination
of mortality, morbidity, nosocomial problems, and errors. Patient treatment is multi-faceted, and mor-
tality is not the only outcome that is of importance to patients. The advantage of using text clusters is
that they can accommodate the multi-faceted outcomes, and are flexible enough to enable researchers
to drill down into details. In combination with kernel density estimation to examine possibilities, we
can examine the occurrence of nosocomial problems, which should be given considerable weight when
investigating the quality of provider care. Certainly patients will be concerned if there is a considerable
risk of acquiring infection because of a hospital stay. One of the biggest problems with investigating
nosocomial infections is that it is under-reported, and it is difficult to extract information from the
ICD9 secondary diagnoses. As we demonstrated in Chapter 10, text clusters can be used to separate
community-acquired from nosocomoal infections in order to investigate hospitals that have problems
with nosocomial infections. Such an examination again demonstrates the versatility of using text mining
to investigate patient outcomes.
If we do not examine severity indices more carefully, their use can penalize providers that provide
good quality care while rewarding those who focus their efforts on “gaming” the index. As the move