Introduction
This article addresses quality in the forensic science laboratory by considering the following questions:
• What are the necessary elements to ensure quality?
• How does the laboratory put these in place?
• How does it provide objective proof that it has done so?
The necessary elements to ensure quality make up the ‘quality system’.The laboratory puts the quality system in place through its ‘quality manual’ and an ‘internal audit program.’ The proof that it is implementing its quality system is shown by third-party assessment for ‘accreditation.’ The quality system is the total of policies, practices and procedures which form the basis for good laboratory practice.The system is captured as a quality manual, and accreditation examines the appropriateness of the quality system and compliance with it.
It is also necessary to know what we mean by ‘quality’.There are many definitions, but the one used in this article is:
• The quality of a forensic science service is measured by its fitness for the intended purpose.
This definition has the advantage of letting the provider and the user each have a role in achieving quality. The user is defining the purpose and the provider is responsible for ensuring the fitness for that purpose.
The Quality System
An effective quality system is based on three components: people, technical and documentation.We will look at each of these in turn.
People
To produce quality work needs people who know what they are doing, how to do it, why they are doing it, and how well they are doing it.That is, people quality depends on:
• job descriptions;
• recruitment;
• education, training and competency;
• supervision;
• evaluations.
Job description The purpose of the job description is to capture a definition of the job and to communicate that definition.The content of a typical job description will include:
• an understandable, meaningful title;
• responsibilities and accountabilities;
• job outcomes;
• key tasks to achieve outcomes;
• competencies required to perform the tasks successfully;
• education, training and experience standards to confirm the competencies;
• performance goals.
Recruitment The job description provides the framework within which recruitment can take place. Recruitment can be a complex and frustrating activity. There are two main concerns in regard to quality: what to do if there is no ideal candidate; and whether to recruit to promoted posts from within or by hiring ‘new blood’.
These questions were at the root of considerable debate in the United States recently when the Quality Assurance Standards for DNA analysis were being developed by the Federal DNA Advisory Board.The standards require that there be a technical leader who has a graduate degree with credits in certain subjects, including molecular biology and population genetics. The objective is to produce good ‘DNA problem solvers’ who can ensure the standard of operations and development in forensic laboratories, and match the sometimes very high ability of experts engaged by the defense.
Most technical leaders in post are also the managers of the section, and many do not have a graduate degree.There were also concerns about the ability of people with a PhD in Molecular Biology, but with no or limited forensic experience, to translate their technical knowledge into the particular application.The DNA Advisory Board standards address this through an experience requirement.The problem in regards to existing staff was solved by establishing a waiver process.However, the real answer is to make sure that the purpose of the technical leader is properly and clearly defined in the job description, and that future recruitment is targeted.All candidates can be measured against the elements of the job description and the best one selected.
Education, training and competency These deal with the ability of people to do the tasks assigned to them.Education provides a platform of theoretical and practical knowledge.Training provides continuous development of the theoretical and practical knowledge at a personal level after appointment, so that the individual can perform well as times and job requirements change.Competency is a measure of the overall capability of the person, arising from education, training and on-the-job experience.Competence can be linked to competencies, the knowledge, skills and abilities required to discharge the responsibilities and accountabilities required by the job.
Supervision Good quality requires appropriate supervision.Supervision should have two parts to it: firstly, coaching; and, secondly, responsibility and accountability.The coaching role is the difference between success and failure in delegation and empowerment.Coaching calls to mind theater or sporting analogies.The role of the coach is to make sure that the cast (or team) members know their roles and are able to carry them out.So it is with supervisors in the laboratory.The coaching role works best when coupled with the responsibility/accountability role. The supervisor has to make clear what are the responsibilities of the staff and how they will be held accountable for their performance.
Evaluations Science is about knowledge.Science uses a process of measuring and testing and using the result to further knowledge.lt is just the same with people and quality.The performance of people has to be measured and used to further performance.
There are many ways to carry out performance evaluations.Unfortunately, many laboratories have performance review forced upon them through application of an agency- or government-wide system that is divorced from the quality needs of the forensic science operations.Building performance evaluations into the laboratory quality system provides a way to get around the problem.Usually the issues addressed are similar in the laboratory and in the wider agency. The results of the laboratory evaluation can be used to provide the answers to the agency needs.
Technical
The technical overall aspects of a laboratory, and which contribute to quality, are its methods and its instrumentation.
Methods Forensic science operations make substantial and sometimes conflicting demands on methods used.The requirements for method selection are mostly conservative.Methods should be either ones generally accepted in the field, or ones that are supported by information collected in a scientific manner.That is, the methods should either be well established or should be thoroughly validated.The conflict arises when a request is made for testing which is outside the capability of the laboratory at the time.Maintaining quality provides for three responses to such situations: a method can be developed and validated; the request can be passed to another laboratory which is known to have such a method in use; or the request can be turned down.
General acceptability can be shown in several ways.The method may be in widespread use, have been published and extensively cited in the scientific literature, and possibly have been published as a standard method (for example as an American Society for Testing and Materials (ASTM) Stan-dard).One of the problems of ‘general acceptability’ is that the method may be outdated and there may be better methods available.For example, there is debate about the use of microcrystal tests for identification of controlled substances.There is a view that molecular confirmation techniques such as gas chromatography-mass spectrometry or gas chromatography-infrared spectroscopy (GC-IR) are much more objective.However, many well-established criminalists are able to point to data they have collected that show that reliable and rapid identification can be obtained from careful and knowledgeable application of microcrystal testing.
Validation is a more difficult process in developing a forensic method than in most other analytical applications.The main reason is the nature of the material tested.It is very variable in composition and can be impure.
The most detailed guidelines for method validation
in forensic science are to be found in the quality assurance standards for DNA analysis prepared by the DNA Advisory Board.
They define validations as: a process by which a procedure is evaluated to determine its efficacy and reliability for forensic casework analysis, including:
• Developmental validation, which is the acquisition of test data and determination of conditions and limitations of a new or novel DNA methodology for use on forensic samples.
• Internal validation, which is an accumulation of test data within the laboratory to demonstrate that established methods and procedures perform as expected in the laboratory.
The standards on internal validation have elements that could be applied to any forensic testing area.For example:
The procedure shall be tested using known and non-probative evidence samples.The laboratory shall monitor and document the reproducibility and precision of the procedure using human DNA control(s).
The laboratory shall establish and document match criteria based on empirical data.
Before the introduction of a procedure into forensic casework, the analyst or examination team shall successfully complete a qualifying test.
Material modifications made to analytical procedures shall be documented and subject to validation testing.
Where methods are not specified, the laboratory shall, wherever possible, select methods that have been published by reputable technical organizations or in relevant scientific texts or journals, or have been appropriately evaluated for a specific or unique application.(Quality Assurance Standards for Forensic DNA Testing Laboratories, as approved by the Director of the FBI, October 1998.)
The normal approaches of analytical chemistry can be used.Accuracy, precision and recovery studies, evaluated by standard statistical techniques, are important characteristics.A range of substrates/matrices should be tested, and aging and environmental insults must be studied.
Instruments The nature of the use to which results are put, together with the nature of the samples tested and the need to be able to select an approach best suited to the overall case conditions, means that instrument calibration and maintenance must be of the highest order.The starting point is to have an inventory of all equipment, the performance of which can affect analytical accuracy.The inventory can then be linked to a calibration and maintenance program.
The effect of this is that the laboratory has a program to ensure that the calibration is done and records of the results.The program itself can be staged.
Taking balances as an example.Each should have an entry on the inventory.The entry could show the following:
• Each time used check zero and standard mass.
• Every 6 months check zero, accuracy and linearity of range of standard masses, and off-center load.
• Annual routine preventative maintenance by contractor, with complete recalibration and certification.
The equipment itself must be suitable for the purpose. The laboratory should be able to show its calibration status.Note that ‘calibration’ here means:
• The set of operations which establish, under specified conditions, the relationship between values indicated by a measuring instrument or measuring system or values represented by a material and the corresponding known values.
The technique commonly referred to as ‘calibration’, in which a standard curve is prepared for quantitative analysis, is better referred to as ‘standardization’.
Documentation
A quality system cannot exist without proper documentation.The three rules are:
• If you do it, write it.
• If you write it, do it.
• If it isn’t written, it didn’t happen.
These three simple rules give guidance for writing, complying with and recording compliance with the quality system, as contained in a quality manual.
If you do it, write it Many laboratories preparing a quality system for the first time have problems writing their quality manual.The root of the problem is a failure to appreciate that the laboratory must be doing something right or it would not still be in business and seeking to improve its quality.Another common difficulty is that the laboratory tries to create a perfect manual at the first attempt, thus making what should be a simple task much more complex and daunting than it should be.The third common problem arises when the laboratory obtains copies of other laboratories’ manuals and seeks to mimic them.
The best way to write the quality manual is to begin by recording what the laboratory actually does.The language should be kept simple and at a high level.
For example, if the laboratory has developed a routine of only analyzing fiber evidence when the supervisor of the trace evidence section has evaluated the case circumstances and makes a judgment call based on experience, then the manual should state that. Trying to capture the exact basis of the judgment is not necessary, especially in the first attempt at compiling a manual.It is especially important to avoid wholesale adoption of another laboratory’s manuals, as inevitably there will be practices, policies and procedures in it that do not transfer well.
Starting by capturing what the laboratory actually does also has the advantage that it will result in a more familiar and understandable set of instructions. If the capture is achieved by a process involving staff, then it will have the further benefit of engaging commitment.
If you write it, do it, and if it isn’t written, it didn’t happen Having captured the policies and procedures current in the laboratory, the next step is to make sure that they are followed.The best way to make sure that the instructions are indeed being followed is through the use of proformas which record all vital steps.They also provide a database for evaluation and improvement.
The advice ‘if you write it, do it’ does not mean an unthinking and unchanging subservience to the ‘rule topic’.If experience following something in the manual shows that it can be improved then it should, and the method or procedure or policy should be updated.
The first entirely new policy to be written should, therefore, be the policy on policies.The policy on policies must include the procedures for making and authorizing change.
The Quality Manual and Internal Audit Program
Quality manual
The quality manual documents what is done, by whom, how, with what resources, and where it is done.It covers policy, procedure and practices.In many laboratories, much of the nontechnical content will be set by the parent agency.The quality manual will contain the laboratory’s policies and procedures for quality assurance and quality control. Quality assurance is:
• All activities and functions concerned with the attainment of quality in the laboratory, including: method development and selection, calibration, staff, procurement, reviews and audits, and subcontracting.
Quality control is:
• The operational techniques and activities that sustain the product or service quality to the specified requirements.In the laboratory, it includes: method evaluation, control of standards, materials and reagents, calibration, proficiency tests, and QC samples.
The manual must address the following:
• A quality policy statement including objectives and commitments by management.
• The organization and management structure of the laboratory, its place in any parent organization, and relevant organizational charts.
• The relationships and responsibilities of management, technical operations and support services in implementing the quality system.
• Job descriptions, education, and up-to-date training records of laboratory staff.
• Control and maintenance of documentation of case records and procedure manuals.
• The laboratory’s procedures for ensuring that measurements are traceable to appropriate standards, where available.
• The type and extent of examinations conducted by the laboratory.
• Validation and verification of test procedures used.
• Handling evidence items.
• An inventory of equipment, the performance of which affects the quality of the laboratory product.
• Reference measurement standards used.
• Calibration and maintenance of equipment.
• Physical environment control (such as access, temperature, lighting).
• Verification practices for ensuring continuing competence of examiners, including interlabora-tory comparisons, proficiency testing programs and internal quality control schemes (e.g. technical peer review).
• Gaining feedback and taking corrective action whenever analytical discrepancies are detected.
• Monitoring court testimony to ensure the reporting of scientific findings in an unbiased and effective manner.
• Laboratory protocol permitting departures from documented policies and procedures.
• Dealing with complaints.
• Disclosure of information and confidentiality.
• Audits, reviews and updates to the quality system.
• Health and safety.
• Accreditations and policy on subcontracting.
The quality manual is completed by the laboratory methods manual.The methods must show the requirements for sampling, equipment, reagents, safety, controls and standards, possible sources of error, and a step-by-step description of the procedures.
Each policy or procedure must declare its authority.That is, the title, date of issue, authorization, version number or other identification of currency of the policy or procedure.The quality manual is therefore a living document which gives realistic, authoritative, up-to-date and appropriate guidance. Document control is a vital part of achieving these outcomes.Without good document control, staff will not know whether an instruction is currently valid nor what authority is behind it.
Internal audit program
The internal audit program has two goals:
• to demonstrate compliance with the quality system;
• to provide a basis for improvement of the quality system.
The usual form is the vertical audit, in which all activities in a section are audited at the same time. For example, there may be a program in which the toxicology section is audited, then trace evidence, then DNA.A less common alternative is the horizontal audit, in which all instances of a specific activity are audited throughout the laboratory at the same time.For example, evidence control is audited, then training, then case records.
The audit An audit is defined as:
• A planned and documented activity performed to determine by investigation, examination or evaluation of objective evidence, the adequacy of and compliance with established procedures, and the effectiveness of implementation.
The conduct of an audit is the same whether it is a compliance or quality improvement audit, and whether it is a vertical or horizontal audit. Planning the audit requires attention to:
• Dates when the audit will take place, advising team members and those being audited;
• Team members should be independent of the area being audited and have had training in auditing;
• Scope what is being looked at;
• Documents what documentation is needed (policies, procedures, records).
Conducting the audit requires that the auditors obtain objective information about compliance with the quality system.This is achieved by:
• Questioning staff.
• Observing methods and procedures.
• Examining facilities, documents, records.
• Reviewing the information.
Above all else, it must be understood by the auditors and those audited that:
• An audit is a fact-finding mission and not a faultfinding safari.
Corrective actions If the audit produces objective evidence of nonconformancy, then the laboratory must deal with these through a corrective action program.A nonconformance should always be described in the words of the quality system standard or procedure.If this is not possible, then although the observation may be an opportunity for improvement, it is not a nonconformance.
Nonconformances should be recorded as corrective action reports (CARs).The CAR should describe the nonconformance and initiate a sequence designed to produce an effective rectification, thus:
• Request for rectification by audit team leader.
• Responsibility for rectification assigned by management.
• Root cause investigated by problem-solving team involving front-line staff.
• Corrective action required identified by team and reviewed by management.
• Verification of effectiveness of corrective action assessed by follow-up audit.
Accreditation
Accreditation in the crime laboratory
The objective proof that the laboratory has an adequate quality system and is following it is achieved through accreditation.Accreditation is defined as:
• The formal assessment and recognition by an impartial competent authority that a laboratory is capable of meeting and maintaining defined standards of performance, competence and professionalism.
The strength of accreditation lies in the involvement of the third-party competent authority to assess the performance of the laboratory.
Accreditation programs
There are two main accreditation programs in place in forensic science.The first is that of the American Society of Crime Laboratory Directors Laboratory
Accreditation Board, or ASCLD/LAB.ASCLD/LAB was established by the American Society of Crime Laboratory Directors as an independent incorporated body, with the objective of creating an accreditation scheme for crime laboratories, with the following objectives:
To improve the quality of laboratory services provided to the criminal justice system.
To develop and maintain criteria which can be used by a laboratory to assess its level of performance and so strengthen its operation.
To provide an independent, impartial, and objective system by which laboratories can benefit from a total operational review.
To offer to the general public and users of laboratory services a means of identifying those laboratories which have demonstrated that they meet established standards. (ASCLD/LAB, 1999 Accreditation program.)
The Illinois State Police system became the first accredited facility in 1982.In 1990, the Forensic Science Centre in South Australia became the first non-US facility to be accredited.Today there are about 200 accredited laboratories, including those in Singapore, Hong Kong and parts of Australia.
The second approach is to seek accreditation to the standards for good laboratory practice, as set out by the International Standards Organization in ISO Guide 25.Countries adopting this approach amplify the basic ISO requirements with some degree of field-specific requirements.This has been done in the UK, Australia, The Netherlands and Canada.The program in Australia is worthy of note, as the field-specific requirements were arrived at through a cooperative agreement with ASCLD/LAB.Finally, ASCLD/LAB is in the process of revising its program to become fully compliant with ISO Guide 25.
The ASCLD/LAB accreditation program
Some of the principal features of the ASCLD/LAB program will be used to illustrate how accreditation programs affect the quality of operations in a forensic science laboratory.
The program deals with laboratory management and operations, personnel qualifications and the physical plant.Each area is dealt with by describing a set of requirements, presented in the form of a principle, standards, criteria and a discussion.These are defined as:
Principle For each major section within the three divisions of these standards, a basic statement of principle is presented.Principle is defined as: a basic rule, assumption or quality; a fixed or predetermined policy or mode of action.
Standards The standards are statements which describe acceptable levels of performance, excellence, or attainment in that particular activity.
Evaluation criteria The criteria are objective tests to assess whether the laboratory activity meets the standard.This is often a restatement of the standard in the form of a question which can be answered ‘yes’, ‘no’, or ‘not applicable’.Criteria are each assigned a number in this manual.
Discussion At the end of each of the three divisions, there is discussion which sets forth the rationale used in the adoption of the standards and provides more detailed explanations of some criteria.(ASCLD/LAB, 1999 Accreditation program.)
Essential criteria in the ASCLD/LAB program The criteria are categorized as essential, important or desirable.Essential criteria are those which directly affect and have fundamental impact on the work product of the laboratory or the integrity of the evidence, and only these will be considered further in this article.The original program can be obtained from ASCLD/LAB (ASCLD/LAB, 1999 Accreditation program).
Policies and procedures The first essential criteria encountered are requirements for clearly written and well-understood policies on handling and preserving the integrity of evidence, and on laboratory security. These set the theme for the whole program, in that it requires policies for good laboratory practice as applied to a forensic setting, and then goes on to cover observation of these.Thus criteria rated as essential are next encountered in a set applying to evidence integrity.The policy on evidence integrity should provide the basis for compliance with these operational requirements.
Evidence integrity The first of the group dealing with evidence integrity is that the laboratory has to show that it has a written or secure electronic chain-of-custody record with all necessary data to provide complete tracking of all evidence.Next in the set is a requirement that all evidence is marked for identification.Neither of these is foreign to a forensic laboratory and neither causes much difficulty in interpretation or compliance.
They are followed by criteria requiring that evidence is stored under proper seal and is protected from loss, crosstransfer, contamination and/or deleterious change.Interpretation of these criteria requires careful attention to the wording in the Discussion.There we find a definition of ‘proper seal’.A container is ‘properly sealed’ only if its contents cannot readily escape and only if entering the container results in obvious damage/alteration to the container or its seal.Tape used to seal containers must be initialed (or otherwise identified) to document the person sealing the evidence.The proper seal provision only applies to evidence in storage.
Evidence in the process of being examined is covered by the criterion requiring it to be protected from loss, crosstransfer, etc.This means that it can be kept in packaging that is not sealed but which must be closed.It also means that it must be kept safely, including consideration of environmental temperature, and security of access.The access issue is addressed by the last of this set of essentials, requiring that there is secure storage space available for overnight and/or long-term storage of evidence.
Quality system The program requires that the laboratory has a quality manual, a quality manager, completes audits of the entire laboratory operations annually, and conducts an annual review of its quality system.
The review of the quality system should address the results of the audit and any other significant factors, such as legislative change, technology developments and issues identified in corrective action activities.
Procedures Procedures used in the laboratory should be generally accepted in the field or should be supported by validation data collected in a scientific manner.For new methods, the validations must be completed before the technique is used on case-work.The degree of validation depends on whether the method is being developed anew or is the introduction of an established method not previously used by the laboratory.In the latter case, the object is to show that the method works in the target laboratory, not that it is intrinsically valid – this having been previously established.Therefore, successful completion of samples in an interlaboratory study is acceptable validation.
All the procedures must be documented, and available to staff for consultation.The methods must specify the controls and standards to be used, and the laboratory must be able to show that the quality of the standards and reagents it uses is adequate for the procedure to be used.Finally, in this section, the laboratory must implement a program which checks the reliability of its reagents.This can be as simple as ensuring that test methods require the running of positive and negative controls before case samples.
Instrumentation Instruments must be properly calibrated.Typically this would be a cascading program, with an annual maintenance and calibration against a nationally traceable standard, supported by less stringent calibration checks at regular intervals.
Case records The program has very demanding standards in regard to case records.The reason is to ensure that their integrity can withstand any attack. It is required that notes, worksheets, photographs,spectra, printouts, charts and other data or records used by examiners to support their conclusions are maintained in a case record.The information must be kept as pages, each of which bears the case number or other unique identifier and the identity of the examiner or person responsible for the information.These labeling requirements encompass all documentation in the record, including administrative information. The requirement ensures, among other things, that it is possible to trace the authenticity of any extracts which may have been made from the file.
The records must also be proof against any allegation of alteration.It is thus not permitted to make any alterations in the form of obliterations (such as the use of correcting fluids).Rather, alterations must be made by a single strikeout which leaves the original record readable, and should be initialed by the person making the change.
The laboratory is also required to have a system of review of case records.There are two types of review. Administrative review covers typographical errors and completion of administrative documentation.It is required for all cases and may be performed by the analyst.Technical review covers the correctness of the technical information, including calculations and data transfer, and has to be done by a second person sufficiently knowledgeable of the area to know that the technical steps have been performed correctly and that the data support the conclusions drawn.Techni-cal review does not need to be carried out on all reports but it is suggested that the minimum is 20%, or six per analyst per month, whichever is the lesser.
Testimony monitoring and proficiency testing The laboratory is required to monitor the testimony of examiners at least annually and to provide feedback. The monitoring can be done in several ways, including observation, feedback from attorneys and transcript review.It should cover appearance, poise, performance under cross-examination, ability to present scientific information in an understandable way, and a determination that the testimony is consistent with the examinations and report.
Proficiency testing is dealt with as a set of somewhat complex standards and criteria.Each area of activity must successfully complete at least one external proficiency test per year.The test must be obtained from an approved external supplier and the results passed to the accreditation program for review.Each analyst (other than in DNA) must successfully complete a proficiency test each year, but this can be internally sourced – for example, by re-examination of case samples.
The requirements for DNA analysts are set by external regulation over and above the base requirements of the accreditation program.They require each DNA analyst to successfully complete an external test at intervals not exceeding 180 days.If the analyst performs restriction fragment length polymorphism (RFLP) and polymerase chain reaction (PCR) testing then he or she has to complete at least one proficiency test in each.
Corrective action The laboratory is required to have and implement a corrective action program to deal with significant technical problems.
Personnel qualifications Examiners in controlled substances, toxicology, serology, DNA and trace evidence are required to have a baccalaureate degree in an appropriate subject.Examiners in firearms and toolmarks, latent prints and questioned documents do not need a degree.All examiners must demonstrate that they understand the instruments, methods and procedures used.
Competency testing All new examiners (newly recruited or established examiners moving to a new discipline) must successfully complete a competency exam before being allowed to conduct independent casework.The competency test should cover theoretical knowledge (including relevant literature) and demonstrate practical skills, and include a moot court.
Technical support personnel must be able to demonstrate that they meet the requirements of their job descriptions, that their job descriptions and duties agree, and that they have successfully completed a competency test if they have casework responsibilities.
Access control The program requires that all exterior entrance and exit points have adequate security control, and that all internal areas requiring limited or controlled access have a lock system.All keys, magnetic cards and other devices used for access must be accounted for and have controlled distribution. The laboratory must be monitored out-of-hours by alarms or security patrols.
Summary
Forensic science laboratories have a particular responsibility in regard to quality.The results of their work have far-reaching consequences – life or death in some jurisdictions.The nature of the materials that they examine makes quality control much more difficult than in most instances of application of scientific analyses.
This article has illustrated some of the more significant factors that must be considered in quality assurance of a forensic science service.
