ACCREDITATION OF FORENSIC SCIENCE LABORATORIES

Introduction

Historically, testimony in court was based on a combination of school learning and on-the-job training. Because both criteria can be quite variable, the confidence in the interpretation of results was also quite variable. Few generally agreed upon procedures were available. The identification of a drug in a body fluid may rely upon spot tests, thin layer chromatography, high performance liquid chromatography, capillary electrophoresis (CE), fluorimetry, infrared, gas chromatography, gas chromatography/mass spec-trometry, immunoassay, or a combination of the above. Consensus on the superiority of one method over another is difficult to obtain. For example, Professor Manfred R. Moller opined that it would be easier to get a group of forensic scientists to share the same toothbrush than to share the same method of analysis.
In other analytical laboratories, such as clinical laboratories upon whose results medical diagnoses are based, a similar pattern of public outcry resulted in the imposition of stiffer laboratory performance requirements. US Congressional testimony citing laboratory errors in medical diagnoses and lack of clear standards for detection of infectious diseases resulted in the passage of the Clinical Laboratories Improvement Act (CLIA) of 1988. This act established quality standards for all medical laboratory testing to ensure the accuracy, reliability and timeliness of patient test results regardless of where the test was performed. The final CLIA regulations were published on 28, February 1992 and have resulted in steadily improving quality, as measured by the precision and accuracy of results. The regulations specifically exempt research, National Laboratory Certification Progam (NIDA) drug testing (exempt because NLCP maintains its own accreditation program specific to the tasks at hand), and forensic science testing.
Laboratory accreditation implements guidelines to provide quality and standardized results. However, accreditation is not without detractions. Certainly, costs are incurred in any proficiency testing program, which must be weighed against the results and simplicity of the tests. In addition, standardization and proficiency testing mandated by the CLIA has resulted in the reduction and elimination of certain tests performed in physicians’ offices; although the quality of results may have increased, the timeliness of results has certainly decreased. Recognition of the impact on family physicians has resulted in the US Congress reexamining the application of the CLIA to point-of-care testing.
Accreditation programs should examine both the specific procedure being followed and the general procedure for unknowns. Accreditation is more easily accomplished when several laboratories are performing the same basic tasks such as fingerprint analysis, routine drug and alcohol analysis, questioned document examination, hair and fiber comparisons, etc. Often in forensic cases, especially unique poisonings, the toxin or question to be answered may be so unusual that only general scientific principles can be evaluated for its detection rather than a set procedure. Certainly, for unusual toxins no accreditation program could reasonably include blind proficiency testing. Thus, the accreditation of forensic laboratories can be more lengthy and the procedure more difficult to define than that of general medical diagnosis.
Although older technology has its value, analytical technology is ever advancing to more sensitive, more reliable, and less error-prone techniques. Often, only after high visibility events, such as the Lindbergh kidnapping (1930s) (resulting in funding of FBI laboratory and local law enforcement laboratories), civil unrest (1960s) (with US Department of Justice funding of lab improvements, including proficiency testing), or more recently the OJ Simpson trial and FBI whistle blower scandal (with increased calls for better evidence collection procedures and more money for FBI lab personnel – larger numbers and American Society of Crime Laboratory Directors (ASCLD) certification), have major laboratory improvements been made. From nation to nation and jurisdiction to jurisdiction within autonomous states, new laws and court-imposed standards proscribe a new set of expectations on the forensic laboratory.
For example, in the US the acceptance of testimony about novel scientific methods has experienced rapid revision. Expanded from the 1923 ‘Frye standard’ that the scientific methodology be generally accepted, US Federal Rules of Evidence the 1993 US Supreme Court ruling in Daubert v. Merrill Dow Pharmaceuticals, Inc. (and subsequent refinements) state that courts should consider at least four aspects before admitting expert testimony. The four are: (1) whether the technique or method has been tested (i.e. reputable scientific journal articles, adequate controls used accurately, research done expressedly for litigation, blind trials, etc.); (2) whether it has a known error rate (i.e, determined through blind proficiency testing); (3) whether it has passed peer review scrutiny (without which courts should be skeptical); and (4) whether it is generally accepted by the scientific community. Laboratory accreditation facilitates many of these requirements, helping courts to make decisions about evidentiary reliability.
Internationally, the World Trade Organization (WTO) recognizes the lack of acceptance of laboratory test results as a technical barrier to international trade. Forensic science, worldwide, is benefiting from this enhanced attention to laboratory accreditation. Attention is being focused continually on the
International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). ISO/IEC Guide 2 defines laboratory accreditation as ‘formal recognition that a testing laboratory is competent to carry out specific tests or specific types of tests’ and may include ‘recognition of both the technical competence and impartiality of a testing laboratory.’ The US Drug Enforcement Administration has provided the venue for representatives from the United Nations, Canada, United Kingdom, Japan, Germany, the Netherlands and Australia to promote quality assurance (including certification and accreditation) in forensic drug analysis, for example, through the scientific working group (SWG), acronym SWGDRUG.
In India the National Accreditation Board for Testing and Calibration Laboratories (NABL) has instituted compliance requirements with the ISO/IEC Guide 58/EN-45003 with specific reference to accreditation of forensic laboratories in ISO/IEC Guide 25/EN-45001. This includes annual inspection for quality system management requirements, maintenance of all technical parameters, and satisfactory participation in interlaboratory proficiency testing. For this purpose, the National Physical Laboratory in New Delhi has been developing Certified Reference Materials for several years. Most recently (1998) the India Ministry of Home Affairs has directed forensic science laboratories under the Bureau of Police Research and Development to prepare for NABL full accreditation, including all analyses performed. Work is continuing in the area of uniformity in the approach of forensic science laboratory inspectors, including their qualifications and training.
In Europe the International Laboratory Accreditation Cooperation (ILAC) was established in the late 1970s. More recently, recognizing the importance of laboratory accreditation, the EU has established an independent European Accreditation Advisory Board (EAAB) to work with ILAC with the aim of promoting confidence in one-stop accreditation of industry and public authorities, including forensic laboratories. Separately, the European Network of Forensic Science Institutes (ENFSI) have initiated interlabora-tory projects, with member laboratories in more than 20 European countries. Specific areas of forensic expertise, such as handwriting examinations and shoeprint comparisons, have experienced difficulty standardizing quality assurance approaches specified under EN 45000 for example. In addition, standardization of proficiency testing among European countries is under development through the joint A-EUROLAB-EUROCHEM Working Group Proficiency Testing in Accreditation Procedures. In 1997 the European cooperation for Accreditation (A) merged the activities of the European Accreditation of Certification (EAC) with the European cooperation for the Accreditation of Laboratories (EAL). The United Kingdom’s network of forensic science service laboratories in England and Wales have maintained accreditation for several years under the National Accreditation of Measurement and Sampling (NAMAS) M10, a service of UKAS (United Kingdom Accreditation Service), and ISO 9000 series quality standards.

Laboratory accreditation defined

Laboratory accreditation encompasses external oversight of laboratory operations, including: whether the laboratory facilities are adequate, whether the laboratory personnel have the appropriate background (expertise and experience) and opportunities for continuing education to perform assigned tasks satisfactorily, whether the laboratory has a quality control program and the degree to which this program strives to achieve excellence, how the laboratory performs on proficiency tests, how the laboratory complies with established standards as determined by laboratory inspections, and other factors that affect the reliability and accuracy of testing and reporting done by the laboratory. An example of specific requirements for urine drug testing may be used as a guide. The following paragraphs describe aspects that comprise each of these areas.

Adequate laboratory facilities

Forensic laboratories require safe, secure, and un-contaminated work areas containing the proper equipment. An unsafe forensic laboratory not only jeopardizes the health and safety of workers, but risks the compromise of evidence. Security extends from normal working hours to whenever the laboratory is closed. Visitors (including all service personnel) must be documented and escorted at all times to protect the integrity of evidence testing and chain of custody. Scientific personnel should have access restricted to only those specific areas that their work requires. After-hours security should deter and detect any unauthorized entry to the laboratory. A convenient method of restricting access and recording entry is by the use of key cards connected to a central computer system for logging. Unnecessary clutter can be unsafe, and contamination must be minimized and assessed periodically to know its potential to affect results.
Finally, a laboratory cannot meet modern analytical expectations without the proper equipment, maintained and in good working order.

Laboratory personnel

The quality of work in any laboratory rests on good personnel. The expertise and experience of laboratory personnel require opportunities for continuing education to remain current in the performance of assigned tasks. Certain accrediting organizations require academic degrees for positions such as laboratory director (doctorate) or certifying scientist (master’s or bachelor’s degree in chemistry, biology, or forensic science). Frequently the accreditation organizations have rigid guidelines of educational attainment for specific positions, which may underestimate the value of on-the-job experience. Not all requirements of accreditation need be imposed simultaneously. Accreditation of a laboratory must proceed slowly to allow personnel time for educational improvements and to obtain qualified personnel – to avoid disruption of laboratory services. When new accreditation agencies are forming, often current personnel who do not meet the new, more stringent requirements may be ‘grand fathered’, granting them a special exception for a certain time. The work skills of laboratory personnel may be brought to acceptable standards and/or improved first by probationary training periods of up to three years; in-service continuing educational programs conducted within the laboratory itself; relevant scientific seminars, conferences, symposia, and meetings; and part-time completion of degree programs.

Quality control program

A laboratory’s quality control program indicates the extent to which excellence is a priority. One gauge may be the percentage of quality control samples per specimen analysis (frequently 10% in urine testing programs). Where applicable, standards and controls (from different sources) analyzed within established tolerances add confidence to identification and quan-titation, as do equipment calibration records (how often, how thorough, and what steps are taken when outside of established tolerances), and maintenance records. Written standard operating procedures are necessary in order to reduce subjectivity and to provide objective analysis and interpretation of results. Because reliability decreases near the limit of detection (LOD), it is important to define what method(s) are used to measure LOD and limit of quantitation (LOQ). For example, in forensic analytical toxicology, experimentally determined LOD and LOQ use signal-to-noise ratios of 3:1 and 10:1, respectively, measured with serial dilutions in the matrix of concern. Statistically determined LOD and LOQ rely on ‘quantitation’ of a series of blank samples (usually at least 10), calculation of the mean (*) and standard deviation (a), and applying the formulas: LOD = *+ 3a and LOQ = * + 10a. Although more difficult when analyzing unique substances, written procedures that detail the approach and specific criteria for the analysis of novel analytes provide useful guidance to analysts and crucial insight to those tasked with evaluating a laboratory’s performance. An approach to developing procedures for detecting novel analytes should consist, at minimum, of: searching the literature for relevant literature references; obtaining standard compounds; placement of reference compounds in a matrix similar to the specimen matrix; and analyzing with standard procedures, all before analysis of the questioned specimen. Sometimes a better approach for novel analytes would be to contact and transfer the specimen to another laboratory better equipped in its analysis. An example in forensic DNA typing is extensive quality control to assess contamination by polymerase chain reaction amplification products that could affect reliability of results and physical separation of extraction from analysis operations.

Proficiency test performance

Proficiency tests (PTs) measure laboratory performance by submitting specimens containing materials known only to the submitting agency. Open PTs are known to the laboratory to be PT samples, although the specific materials and/or their concentration are unknown. Blind PTs are submitted as any other sample from a client, so that the forensic laboratory does not recognize them as PTs. The quality of the PT program depends on the rigor of the PT challenge. For example, where cut-off concentrations are mandated by statute, such as in workplace drug testing, PT samples containing 75% and 125% of the cutoff would be more appropriate than ones containing 25% and 250% because the expectation is to distinguish concentrations at +20% around the cutoff. Similarly, PTs containing known interferences or metabolites, normally present in real specimens, represent more rigorous challenges. The frequency of PTs may vary from daily to yearly, depending on accreditation requirements and sample volume of the laboratory. Certain failures in the PT program can cause a laboratory to lose accreditation. For example, an accrediting organization may partially withdraw a laboratory’s accreditation for incorrectly quantitating a given percentage of specimens; however, laboratories can lose accreditation for reporting a single false positive result (such as reporting a drug present that is not present).

Laboratory inspections

Laboratories in an accreditation program should be inspected at least annually by an outside panel of experts knowledgeable in the work preformed by the laboratory. The inspection process should have several objective criteria such as: (1) examining recent analysis and checking for compliance with their own standard operating procedures manual (SOP); (2) does the SOP reflect established standards and current regulations? (3) where failure of the laboratory in previous proficiency testing programs has occurred, have specific steps been implemented to correct the deficiencies and are these steps adequate? The inspectors should provide a written report of the results of their inspection. Laboratory personnel should be at least familiar with how the work that they are performing compares to that done by other groups in the world. This can be accomplished by the laboratory director or members attending national and local conferences and reviewing the literature. Also, it is helpful to hold periodic staff meetings where discussions of problems with specimens are held, to review each person’s tasks, and for the staff to review specific literature and discuss that literature. Such a scheme allows the staff to remain current with technology and become aware of how their role in a multistep process may impact the subsequent steps.

Compliance with established standards

Complicance is determined by the following.
1. Any other factors than affect the reliability and accuracy of testing and reporting done by the laboratory. Paperwork trail: identification of samples, custody, documentation of exceptions (had to run it again because: spilled, seal broken, quality control not right, sample volume inadequate) are the samples appropriately stored (urine frozen, DNA dried out/frozen, mold growing on things, fire debris with bacterial decomposition, time window on certain things that are known to be lost due to evaporation and/or degradation).
2. Presence of retest criteria. One example of retest criteria for urine drug testing is the presence of the drug on retesting the sample (above LOD) rather than presence above the cut-off. Criteria for if the sample fails the retest, can you explain? Example:
benzoylecgonine degrades to ecgonine if the urine is basic and ecgonine is not normally detected. 3. Sample integrity checking. Are checks for adulteration routinely performed? If not, does the capability exist to conduct such tests if requested? Was the chain of custody documentation intact? Were discrepancies (broken seals; did submission form say it was dirt and it was actually paper; are signatures on custody documents? or are all samples present?) noted? Positive samples and all evidence should be saved and stored appropriately for a set period of time, in accordance with the laboratory SOP. Evidence is usually kept for one year or more and notification is normally made to the submitting organization before routine discarding of samples. Negative samples are normally discarded soon after testing.
A number of problems arise in establishing a good accreditation system for forensic laboratories. Unlike urine testing for drugs of abuse or medical samples, the work in forensic laboratories is varied and samples have a history and are frequently part of a larger body of evidence. This makes blind PT testing difficult as it would be likely that the examiner would know that a sample being submitted was a test sample rather than a case specimen. Frequently, a district attorney would not have the funds nor knowledge to submit samples to evaluate a laboratory.
Accreditation is an expensive undertaking. Smaller laboratories, associated with police units, or private laboratories may not have the resources to become accredited. Accreditation may force consolidation of the forensic testing system into state or regional laboratories. This has the advantage of concentrating resources and allowing modernization with the distinct disadvantage of loss of local control and possibly greater turn-around times. Loss of private laboratories could increase costs for private litigants and defendants. One possible solution would be for public laboratories to do private testing for a fee. This builds confidence in the public that the Government is doing a good job but opens them to criticism by unscrupulous private individuals. Participation in national round-robin tests, possibly sponsored by accreditation agencies or national standards agencies, could be an interim solution to full accreditation. Such participation would build confidence that a laboratory performed satisfactorily and make the transition to full accreditation easier or provide tiers of accreditation for forensic laboratories. A list of several accrediting organizations and their addresses is given in Table 1.

Table 1 Accreditation and information organizations