The Bottom-Up Approach to Quality Assurance

The importance of quality assurance in laboratory medicine seems so obvious that there should be no need to spell it out, but despite the criticality of measures to ensure reliability of laboratory testing, they are still plagued by problems. The key issues faced today fall into two groups – systematic problems with the way quality assessments are conducted, and poor communication between the concerned parties.

A major issue of internal quality control and external quality assessment (IQC/EQA) is the use of “processed” materials – that is, samples that have been pooled, stripped, dialyzed, and so on.

An Empowered Approach to Quality Control/Assessment

What?
The Empower Project is a bottom-up approach to quality control/assessment that facilitates collaboration between laboratories and IVD manufacturers. The aim is to offer involved parties evidence of performance quality on patient samples, better data access, and communication.

Why?
Laboratories need QC systems that offer better access to data from commutable samples. This should enable them to obtain a global picture of comparability of results across assays, detect analytical problems in their own assays and performance, trace the origins of the observed issues, and remediate them. The systems should also use proper peer grouping, so that laboratories can join forces in making claims for improvement to their IVD manufacturers. Companies themselves should also benefit from the opportunity for evidence-based dialogue with their customers.

When?
The Empower Project started in 2012. The pilot phase is still underway and will last until September 2015. Until then, laboratories are invited to enroll free of charge.

Who?
All kinds of laboratories are welcome. In the master comparison surveys, 125 laboratories can enroll; after this number is reached, a rotation system is implemented to allow other laboratories to participate from survey to survey. Manufacturers’ in-house laboratories also join the appropriate peer groups. In patient percentile monitoring, more than 120 laboratories are enrolled thus far, which translates to over 240 test systems and appropriate peer groups in the program.

Key achievements?
The project has seen a good level of participation, with a global distribution of laboratories from more than 20 countries. The project founders have also successfully established collaboration with IVD manufacturers and LIS providers.

Next challenges?
The main objective is for the performance standards adopted in the Empower Project to become accepted in the community, and for more LIS providers to adapt their software for automated participation in percentile monitoring. New software is currently being developed; one example is the “Flagger,” which will allow investigation of the effect of performance instability on the frequency of flagged results. The master comparison surveys are ongoing on an annual basis.

For further information, or to get involved in the Empower project, email [email protected] or [email protected]

These kinds of materials are cheap and available in high volumes, which makes them desirable for use as controls; they are also easy to alter if needed, for instance by supplementing to obtain pathological levels, or by lyophilizing to keep them stable. The problem is that these materials don’t necessarily reflect the reality of patient testing. In metrological terms, this is called “noncommutability.” If noncommutable materials are used for IQC/EQA, they may point to biases that don’t exist in patient samples, and vice versa. It also means that IQC/EQA across assays isn’t possible; quality assessment can only be done at the peer group level.

And building proper peer groups isn’t an easy task. Peers should ideally be homogeneous – that is, grouped by testing system (combination of reagent, calibrator and assay from the same in vitro diagnostic (IVD) manufacturer). However, small EQA schemes may never have enough laboratories using a single system to form homogeneous peer groups. They do peer grouping by method principle, which is inadequate because IVD manufacturers design and optimize their own testing systems differently – so, despite being based on the same principle, one company’s test may not be equivalent to another’s. We estimate a need for about 15–20 laboratories before homogeneous peer grouping can provide any meaningful conclusions on performance. That’s why we recommend that small EQA schemes join forces or participate in a program like the Empower Project (see sidebar “An Empowered Approach to Quality Control/Assessment”) to benefit from proper peer grouping.

Communication issues mainly center on access to data. For instance, in many commercial schemes offering combined IQC/EQA solutions, the external assessment is often only available on a monthly basis. This means that laboratories become aware of bias problems only after the event, rather than in a timely manner. As a result, they’re not able to remediate at the earliest possible stage. Laboratories are not the only parties affected by limited data access, though; IVD manufacturers can’t obtain extensive data records from individual customers, which limits their ability to detect analytical problems in their assays when applied to patient samples. This is why we advocate that both laboratories and IVD manufacturers would benefit from access – within confidentiality constraints – to externally maintained QC databases.

The bottom-up approach

Every health care provider’s goal is to provide the best possible patient care. For those working in laboratory medicine, focusing on test comparability and stability is a key component of reaching this goal. To achieve it, though, current quality systems need to improve. We suggest that shifting from a “top-down” to a “bottom-up” approach is an important first step. What this means is that, rather than a system imposed by authorities and associated with penalties for failure, we recommend one that involves voluntary participation. This could move the goalposts for some labs – instead of aiming for performance at a “reasonable” level, participants aim for a “desirable” level. The involved parties must, of course, be in agreement over what constitutes desirable quality and be willing to closely collaborate and communicate. Only this can ensure that problems identified by QC processes are easily traced to their sources and resolved. The use of commutable samples that resemble patient samples as closely as possible is another change that would improve existing QC systems – which means that, if any problems are discovered, their effect on patient results is clear.

Of course, the changes don’t stop there. Along with restructuring to a bottom-up QC system, laboratories need to establish appropriate performance specifications. Modern quality assurance systems can help define realistic, but meaningful, quality goals.

This might seem to the laboratory community to be a lot of changes to make, but we suggest that the changes first be implemented for common, high-volume tests, so that any bugs can be ironed out. Only once the community feels confident enough that the major problems have been resolved does it make sense to tackle the more complicated tests.

Practical implementation

Discussing systemic changes like the ones we’ve mentioned is easy, but when the discussion is over, how many of the ideas are actually translated into practice? No proposal is truly useful without a “product,” a practical solution that can be used. The Empower Project is structured around several products that can help turn intention into implementation.

One such product is the master comparison survey. These studies provide the participating laboratories, divided into homogeneous peer groups, with a panel of 20 samples to examine for eight different analytes. The samples, which consist of unprocessed clot serum from single blood donations, are commutable. This allows the surveys to provide comparability across assays and laboratories, and to set benchmarks for the intrinsic quality of commercial assays and for laboratory performance. All of these attributes ensure that master comparison surveys add value to conventional EQA – they provide evidence for quality of performance on real samples under “field” conditions.

Laboratories that want to add value to IQC as well as to EQA can make use of a second product, patient percentile monitoring. With this system, laboratories can monitor their performance for 20 common analytes by calculating their daily medians and sending them to the Empower Project’s database. A number of laboratory information system (LIS) vendors offer free solutions for automatically calculating and transferring daily medians, which makes the percentile monitoring program easy to join. For online monitoring, participants have password-protected access to a user interface (“The Percentiler”), which enables them to plot the course of the moving median over time for each analyte and even for individual instruments (Figure 1).

With the Percentiler, laboratories can identify aberrations in their own performance and trace their origins, and compare performance with their peers. The patient results stored and shared within this tool enable discussion between laboratories, but they also assist communication with manufacturers; laboratories can use the results to suggest improvements.

From thinking to doing

The biggest challenge in updating QC processes lies in moving from “thinking” to “doing.” The shift to a bottom-up approach with large networks of laboratories is the key to future progress, because when the motivation for good QC exists, all of the stakeholders work together to provide the best possible patient care. This leads to the adoption of better performance standards and to improved channels of communication. The addition of easy-to-implement products like master comparison surveys or patient percentile monitoring turn the wealth of ideas for improvement into real possibilities. The result is better comparability and stability of laboratory results. And, ultimately, better outcomes for patients.