KEY CONCEPTS:
Upon completion of this topic, it is expected that the reader will understand these following concepts:
• Paramedics continually evaluate practices, protocols, and procedures
• Connection between Paramedic practice and evidence-based practice
• Different types of research appropriate for differing research questions
• The research format and ability to identify errors
• Ethical concerns associated with research
• Types of research that can improve "the bottom line"
Case study:
At a QA/QI meeting a Paramedic presents a case to her coworkers and medical director that introduces the need for medication-facilitated intubation (MFI). This developed after the Paramedic was presented with a patient who—due to the patient’s physical condition and medical emergency—could have been greatly aided by MFI. The Paramedic is charged with researching the topic and presenting her findings at the next meeting. The Paramedic has been asked to examine the success or effectiveness of having MFI as an advanced airway skill at other agencies with MFI programs and to consult published research in an effort to support an evidence-based practice change.
OVERVIEW
Hippocrates stated in his exposition "as to diseases, make a habit of two things—help or at least to do no harm" (Epidemics, Bk I, Section XI). Paramedics driven by the maxim "do no harm" often ask themselves if what they do truly helps, or if, at a minimum, no harm has been caused to the patient.
Practice, Protocols, and Procedures
The question "does it help?" naturally leads to the question of where our practice, protocols, and procedures originated. Most current EMS practice parameters, protocols and procedures, originated from anecdotal experience. These are the result of apparently successful previous practices and of intu-itiveness (i.e., this ought to work).
In situations where science did not exist to support a practice, Paramedics and medical directors often used an analogy to think through a situation (i.e., this worked for this other problem in the past and the present situation is similar). Experienced Paramedics call this "common sense" while Paramedic educators refer to it as "pattern recognition" (i.e., comparison of similarities).
Unfortunately, some EMS practices have been driven by correction of earlier misadventures. Practice improvement arrived at by conscious decision (i.e., by committee) is no more reliable because it is simply the group’s combined anecdotal experience.
As a result, Paramedic practice is often more likely to be a function of what does not harm the patient. This form of practice can be ineffective and leads to the practice of defensive medicine as well as the misdirected application of resources.
The best way to determine effective Paramedic practices is to look at those practices from a scientific vantage. Use of the scientific method, the acquisition of knowledge through objective observation and considered reasoning, corrects previous misconceptions and integrates new conceptual frameworks for Paramedic practice.2 The application of the scientific method will lead to the improvement of patient care.
For example, scientific research created the paradigm shift in trauma care that concludes that limited resuscitation (i.e., permissive hypotension) may be more advantageous to the patient than previously thought. Also, in the past it seemed logical that survival chances for trauma patients were increased by replacing blood loss with intravenous solutions in a 3:1 ratio. Well-designed studies have demonstrated the fallacy of that thinking and Paramedics have adjusted their trauma care accordingly—saving lives in the process.
While every EMS call can be unique, the purpose of scientific EMS research is to establish a Paramedic practice that is defensible. A Paramedic practice is defendable if it establishes evidence-based medical care is provided in specific circumstances, such care can be independently evaluated by other Paramedics, it can be applied to a number of same or similar circumstances, and it is the most effective means of delivering desired patient outcomes. Such evidence-based practice is more likely to weather the scrutiny of cautious public officials and alert community leaders.
Evidence-Based Practice
Emergency medical service in general—and Paramedic practice in particular—are being attacked by some critics who say that EMS is costly to the public and an ineffective means to delivering patient care.
To survive in the current cost-cutting environment Paramedics must prove that their practice is valuable. Paramedicine must prove that Paramedics can decrease morbidity (e.g., through decreased hospital stays or length of stays) and/or decrease mortality and should therefore be seen as valuable.5-7
To transform a practice to evidence-based practice, either through updated protocols or continuing education, the first step is to look at the research that already exists.
While EMS research is still limited at this time, Paramedics can look to the research of other allied health professions, such as medicine, nursing, respiratory care, and so on, for support of paramedic practice.
A literature search for research pertaining to shared practice issues (e.g., safety in medication administration) may reveal previous clinical research on the subject that could potentially be applied to the practice of paramedicine.
Similarly, Paramedics can look to the research of other professions, such as business and education, for evidence-based practices. For example, operational issues, such as effective human resource allocation, have already been researched by hospital administrators and major businesses.
Unfortunately, these studies can suggest solutions that are impractical in the out-of-hospital setting or are cost prohibitive for EMS. Furthermore, the practice of out-of-hospital care is unique in many cases and there may be no analogous studies from other allied health professions to draw upon. The best support for Paramedic practice is research done in the prehospital setting, by Paramedics, physicians, and others interested in advancing prehospital patient care. Details on how to start EMS research follows shortly.
Performing a Literature Search
The first step in utilizing research to create evidence-based practice is to ask key questions. These key questions should focus on topics that are important to the Paramedics’ practice. An example of a question is: "Does pediatric intubation by Paramedics improve patient outcomes?"
After deciding on the question, the Paramedic should perform a search of the current literature that is available on the topic of interest. By reviewing the published reports of research, called the literature, the Paramedic may find studies on the topic or studies that ask a question that is similar to the question at hand.
High-quality EMS research, when completed, is published in various academic journals or works. Unlike the popular press, these journals are peer reviewed.811
A peer-reviewed journal accepts submissions for publication and circulates them to other experts in the field for inspection and critical analysis. This process is called refer-eeing. This intradisciplinary review provides readers with a degree of confidence that what they are reading meets the profession’s standards and is a scholarly work.
However, even when a work has been properly vetted there may still be some errors. In medicine the saying goes, "One study does not make a practice." It is important that a Paramedic carefully read the entire study to see if the same conditions exist in his or her system such that the study results can reasonably be applied to that practice.
The most effective means of performing a literature search is a computerized search. The most inclusive search engine for medicine is the electronic search engine called MEDLINE, formerly the paper "Index Medicus®." MEDLINE provides a list of most published medical research that is searchable by key words or medical subject headings (MeSH).12-15
Other research search engines that can also be used include PubMed, a search engine of the National Library of Medicine (http://www.ncbi.nlm.nih.gov/pubmed/); the Cumulative Index to Nursing and Allied Health (CINAHL – http://www .ebscohost.com/cinahl/); as well as the Educational Resources Information Center (ERIC – http://www.eric.ed.gov/). Even a search of popular search engines like Google® scholar can be helpful.
Hypothetically, a Paramedic could find dozens of citations on a subject, especially if the key words have broad application, like the subject of pediatric intubation.
To separate the "wheat from the chaff," the Paramedic should review the research study’s abstract. The abstract is an abbreviated "executive" summary that hits a study’s highlights.
After reading the abstracts, and eliminating non-related articles, the Paramedic should take the reduced list of studies and review the studies directly. With the reference information (i.e., author’s name, journal name, journal volume number, and research title) in hand, the Paramedic may elect to either go directly on-line to read the article or proceed to a medical library.
Most medical libraries, and many university libraries, have a reference librarian. The reference librarian is trained in research techniques and can help the Paramedic develop a search strategy to identify which articles will be most helpful.
Reviewing the Literature
After obtaining relevant research articles, the Paramedic needs to identify the kind and type of research that was performed in the study.
Currently, the most common kind of EMS research in the literature is retrospective research. A question is raised and Paramedics look at past practice patterns, typically from documentation on the patient care reports, to determine effective versus ineffective practice.
Retrospective data analysis is often used in performance improvement. The danger of retrospective studies are the numerous variables involved in the particular patient scenarios that could account for the patient changes and which are not controlled. For example, if the rate of ventilation in a cardiac arrested patient treated by Paramedics is being measured, how does the researcher know that every Paramedic counts respirations the same way or that every Paramedic even counts respirations, perhaps leaving out the respiratory rate in the documentation by stating that manual ventilation was performed with a bag-valve-mask assembly? As a result, randomness could be an explanation for the results.
Data dredging (data mining) is conducting research without a scientific question in mind (i.e., without a predefined hypothesis). The application of mathematical tests of "statistical significance" to data and trying to observe patterns in that data, and then attempting to form a cause and effect conclusion, is not scientific research.16
The most scientifically valid research is prospective research. In prospective research, an attempt is made to account for all predictable or known confounding variables, to control those variables, and then add a treatment. If change occurs then it may be reasonable to conclude that the treatment may have caused that change.
The gold standard for research is the double-blinded randomized clinical trial (RCT).17,18 This technique is a prospective scientific study that controls known and unknown variables (which could result in spurious results), leaving only one variable to be manipulated. Subjects are then chosen at random to be included in either the experimental treatment group or in the control group (the control group receives standard treatment). The key is that the treatment group, those who receive the experimental treatment, are subjects chosen at random.19
The results of the treatment of the experimental group subjects is then compared to the results of the control group. Ultimately a conclusion is drawn.
The use of statistically equivalent groups (i.e., patient populations having all the same characteristics [variables] except the one being tested) lends credence to the claim that the procedure/medication/and so on worked as predicted and did not occur as a result of random chance or some other variable.
Clinical Trials
In a clinical trial (i.e., experimental medical research), subjects are assigned at random to either the treatment group or to the non-treatment group (i.e., those receiving standard care [control group]).
To limit bias, the participants may not be aware of which treatment group they are in. In a single-blind study, the subject does not know which group he is in. In a double-blind study, both the researcher and the participants are unaware of which group the subject is in.17,20,21
Research often uses inactive drugs, called placebos, or ineffective devices, called shams, that appear similar to the actual drug or device in order to create blinding for the participants.22-24
Statistical Evidence
The key to utilizing experimental research is to understand the statistical methods used to either confirm the hypothesis or reject it.
Classical hypothesis testing compares the results of two treatment groups, statistically, in order to obtain a degree of confidence that the treatment actually caused the effect.
Always skeptical, the researcher’s first assumption is that the effect was not caused by the treatment but rather by random chance. With this assumption that the null hypothesis is true (i.e., the treatment did not cause the desired effect but rather random chance could account for the change), the probability is calculated. The probability of random chance causing the changes, rather than the treatment, is called the p value. An acceptable p value is arbitrarily assigned by the researcher prior to the start of the study and is symbolized as a.
The calculated p value is then compared to the selected a. If the p value is less than the a value, the alternative hypothesis is accepted, and is considered "statistically significant."25 Traditionally, in the medical community a values of 0.05 are considered the standard for probabilities.26 In other words, if the p value in a study is less than 0.05 then this result may be considered to mean that the treatment caused the intended effect and that the researcher is willing to accept the notion that there is a 5% chance that the improvement in outcome occurred by random chance.
Types of Research
Generally, research can be broken down into three types: descriptive studies, observational studies, and experimental studies.
Descriptive Studies
The descriptive study simply states the prevalence of a condition and is often illustrative of a problem, without trying to offer an explanation.
A case report or case series is an example of a descriptive study. By reporting interesting or unique cases, Paramedics can help other Paramedics gain insight into a problem. The utility in a case study may be the development of theories of causation, leading to further research.
There is precedent for case reports, such as case law. Case law has been used for hundreds of years to support decisions which are based on decisions made by an earlier court. Like case law, case reports may be used in a court of law, or by a Paramedic in front of a medical director, to defend a decision in a highly unusual circumstance.27
In dealing with instances of rare diseases, such as Ebola virus, or an exceptional event, such as a plane crash, the case report may be the only means of educating other Paramedics as to the nature and scope of the atypical problem.
Another example of descriptive research is a cross-sectional survey. The cross-sectional survey is essentially a snapshot of a certain aspect of a population at a given moment in time that the researcher is interested in. It is obtained by means of observation, usually utilizing a written tool such as a survey. A cross-sectional survey can look at a specific population and a specific disease, for example.
The National Health and Nutrition Examination Survey (NHANES) conducted by the Centers for Disease Control and Prevention has established the prevalence of obesity in the United States and might be used to help support the decision to purchase a bariatric ambulance. By using an analysis of a cross-sectional survey, Paramedics can use the prevalence of certain diseases, conditions, and so on, to determine operational, medical, and educational priorities.
The results of a cross-sectional survey of one population may not be applicable to another patient population. Furthermore, any descriptive study, like the cross-sectional survey, does not prove a cause and effect relationship between various variables.
The final descriptive study is the ecological study, also called a correlational study. This type of research design serves to provide information about trends and rates of disease within a population. Often cited as X number of cases of Y disease per 1,000 or per 100,000 of Z population, the ecological study results are often quoted to emphasize the prevalence of a disease and therefore the need for research grants or funding for special projects.
Observational Studies
The observational study, in contrast to the descriptive study, asks a question and poses a simple explanation or hypothesis.
To have a scientifically valid result from an observational study, it is necessary to control extraneous confounding variables that could account for the desired change.
One such method of observational study is the case-control study. In the case-control study, the Paramedic would compare the cases—those patients with the disease—to the controls—those patients without the disease—and then examine the procedures performed on both to see if there was an association between outcomes.
For example, a case-control study might look at both patients who died and those who survived a cardiac arrest to see if there was a difference in the medications administered to the surviving patients that could have made a statistical difference (i.e., not attributed to chance).
Similar to a case-control study, a cohort study examines patients who have been exposed to a treatment and compares them to a group that was not exposed to the same treatment. The patients are followed to determine outcomes.
For example, a group of patients (such as pediatric patients) would be divided into two groups—intubated versus not intubated—and then a review of outcome data (i.e., the patient care report) would be analyzed to determine if one group had a statistically better outcome.
Experimental Studies
Classic research starts with a suggested explanation why something occurs or could occur. For example, a hypothesis might be that administration of oxygen to cardiac patients improves long-term survival for cardiac patients.
With the hypothesis in mind, the researcher uses the experiment to test, under controlled conditions, if a treatment created the predicted change.
When considering the results of the research, the researcher understands that there can be two plausible explanations why the change occurred. The first explanation is that the treatment did not create changes (i.e., any changes are purely random and coincidental). This hypothesis is called the null hypothesis. An alternative hypothesis, that the treatment is a plausible explanation for a change, is also considered.
Then a statistical test is applied to the outcome data to determine which hypothesis is most likely correct. The results of the statistical analysis either support the null hypothesis or the alternative hypothesis.
Errors in Research
A common error made in an experiment is to reject the null hypothesis and accept the alternative hypothesis when in fact it is not supported. This is called a type I error.
A type I error, also called a false positive, assumes a treatment effect where none exists. An example of a false positive would be the assumption that the administration of oxygen to a patient with carbon monoxide poisoning increased the patient’s oxygen saturation.
Alternatively, incorrectly failing to reject the null hypothesis is called a type II error, or a false negative. A type II error is a failure to observe the change created by the treatment when one did occur.28
An example of a type II error might be ascertaining a patient’s blood sugar was low and concluding that treatment is required and subsequently administering glucose, when in fact the glucometer is out of calibration and producing erroneous low readings.
In terms of patient care and test results, a false negative would give patients false reassurance that treatment was effective while a false positive may either lead to a wrong conclusion or may ignore that the patient’s condition could have an alternative explanation.
For example, an ST segment elevation on an electrocardiogram (ECG) tracing suggests myocardial infarction. However, some African Americans have naturally occurring ST segment elevations on their ECG tracing. To state to an African American male that this elevated finding was suggestive of a myocardial infarction would be a false positive.
One cause of a false negative is the limitations that a study group’s size places on the experiment. The number of participants in a treatment group, abbreviated N, may not be large enough for the statistical difference to become evident and support the alternative hypothesis. This should not be interpreted to mean that the opposite is true (i.e., that the null hypothesis is true). Instead, the evidence may have been inconclusive.
The results of small studies should be viewed as suspect and lacking power. The power of study (i.e., the ability to attribute the changes to the treatment rather than chance) is increased whenever there is an increase in the number of subjects (i.e., sample size) in the study.
