It is well known that certain assays on certain instruments exhibit variation in quality control and patient values when reagents and / or calibrators are changed. This variation, which we will refer to as ‘lot-to-lot’ variation (L2L), must be dealt with whenever necessary. In order to do this, we need to know the cause(s) of it and try to eliminate or at least reduce the amount or frequency of such variation. Let us start with an example of within lot variation. Table 1 illustrates three levels of control, each analyzed 4 times and 4 patient samples also analyzed 4 times. For our discussion we will use a combined within-, plus across-run, variation of 5% (%CV).
It is important to be cognizant of the variation within and across runs before looking for L2L variation. In Table I, it is of course obvious that there is some variation in the control and patient results. It is possible that the L2L variation would be masked by the inherent variation in the method. Table II illustrates this point. We use the same data as in Table I but add a decrease in the overall mean of 5% with a new lot of reagent (or calibrator):
Looking at just the first value for each of the 7 samples, we note that 3 of the new lot values are actually higher than the old lot. The student t-test does not detect the 5% decrease (p = 0.81), even when each subset of 4 replicates with the old lot are compared to the corresponding 4 replicates of the new lot. IF a 5% change would be significant, this method would not be acceptable – both from a within lot and between lot variation. Let’s look at an example of a 10% increase change between lots (keeping a 5% variation within lot). Table III illustrates this:
An examination of all the raw data shows the increase, as does looking at just the first value of the 7 samples. It is noteworthy that the Student t-test does NOT detect the change of 10% (with a 5% within the lot variation). In this case the subsets, when examined with the t-test, generally detect the 10% increase. This approach suggests a possible way to detect a change between lots of reagent and or calibrator. But it would seem that in most cases with the current instrumentation and consistency of both reagents and calibrators between lots, that it is not necessary to spend the time and resources to look for a change that is probably not there. If so, is there a way to know which analyte(s) do have measurable and clinically important L2L variation? We think there is and that it is easy to uncover them using interlab (QAP) data. Table IV illustrates first an analyte (MNP) that does not show a L2L variation based on last year’s data from 7 lots of reagent (Table IVa).
While the means for the current month and the cumulative period (12 months with 7 changes of reagents and calibrators) are similar, they do not conclusively tell us that there is no L2L variation during that period (e.g. means of 46, 56 and 66 will yield a mean of 56 as will means of 54, 56 and 58). The SDs for both levels of control show little variation over the periods. The F-ratio indicates that the SDs are not statistically different. These then indicate that there is no significant L2L variation in MNP with this instrument’s reagent and calibrators. Now turn to Table IVb which illustrates another analyte (RST).
The story is different for RST. Note that the means are nearly the same, certainly not significantly different, yet the SDs are different as the F-ratio reveals [as well as our own eyes]. Table V illustrates the sort of data that RST could generate for Level I and II, over the changes of 4 lots. You will see that there are obvious differences between lots 1 and 2 as well as 3 and 4. Probably lots 1 and 4 are different. If these same differences exist in the patient data, most likely the physicians would call your attention to it. You cannot simply change the reference range in such a case. If the vendor tells you that the L2Lvariation is ‘about 10%,’ you need to ask whether that is the maximum value or an average. It matters. Our data in Table V could be a worst case scenario or an average one. Another possible means to cope with L2L variation is to use the cumulative SD to set the range. Assuming that a new lot of control would have nearly the same mean as the old lot and the SD would be about the same, we would have a Levey-Jennings chart as illustrated in the Figure below. {We used only Level II to keep the figure less busy.] It seems best to be aware that L2L changes can occur – mainly by studying the QAP data. The laboratory should try to select reagents and calibrators to match the current lot and order as much of it as possible (given the physical space constraints and the lot expiry dates). To use the new mean established with 8 replicates for each level of control (and use a patient sample to determine whether the shift is affecting only controls or both controls and patients). Use the new mean and 3 * the cumulative SD to establish a QC chart. David Plaut is a chemist and statistician in Plano, Texas. Nathalie Lepage is a clinical biochemist and a biochemical geneticist at the Children’s Hospital of Eastern Ontario and an associate professor in the Department of Pathology and Laboratory Medicine at the University of Ottawa, Ontario, Canada.
