Laboratory test menus change to meet the needs of physicians, introduce new technology, increase productivity or save money. When evaluating a new test, random patient samples are often run with the new and current or reference methods.
Laboratory staff need to use this data to know not only how well the new test performs, but also how many patients will be identified or excluded as having a disease, a critical manufacturer claim. By constructing a “truth table” and using a few simple calculations, you can do just that.
Sensitivity and Specificity
As we’re comparing positive and negative values between two test methods, let’s first consider diagnostic sensitivity and specificity.
Note: analytical sensitivity and specificity are different than diagnostic sensitivity and specificity. Analytical sensitivity is the smallest amount of substance in a sample that can be accurately measured; analytical specificity is how well the test measures one substance compared to others.
Diagnostic sensitivity and specificity describe how the test performs in a patient population.1 The latter is our concern.
Sensitivity is the “true positive” rate or ratio of patients correctly identified with disease to true positives and false negatives (patients with disease not identified by the assay). A test correctly identifying all patients with disease is 100 percent sensitive.
Higher sensitivity means fewer false negatives and a lower likelihood of missing a patient with disease, desirable for screening large populations (e.g., HIV testing).
Specificity, equally important, is the “true negative” rate or ratio of patients correctly identified as normal to true negatives and false positives (patients who test positive that don’t have disease).
A test that correctly identifies all normal patients is 100 percent specific. Higher specificity means fewer false positives and a lower likelihood of missing a normal patient, desirable for confirmatory testing (e.g., HIV PCR or Western Blot testing).
Although impractical, an HIV screening test is 100 percent sensitive if every patient tests positive and correctly identifies all patients with disease. Similarly, if all screens are negative, all normal patients without disease are correctly identified and the test is 100 percent specific. The two must be considered together.
Sensitivity and specificity are summarized in Table 1.2
TABLE 1: SENSITIVITY AND SPECIFICITY
| Calculation | Answers This Question: |
|
# true positives |
If the patient has disease, how likely is it that the test is positive? |
| # true negatives Specificity = ———————– x 100 (# true negatives + # false positives) |
If the patient is normal, how likely is it that the test is negative? |
Tables courtesy/Scott Warner
Why not just design tests that are 100 percent sensitive and specific, with no false results? This might be the Holy Grail of laboratory testing, but diagnostic test performance depends on the population testing and prevalence (how many patients per 100,000 that have disease at any one time).
Likelihood ratios are influenced by severity of illness, generally favoring advanced stages, and clinical features similar to the disease.3 No test is perfect.
Other useful calculations to consider are the positive predictive value (PPV) and negative predictive value (NPV) in Table 2.4
Let’s consider how a truth table can apply these concepts.
TABLE 2: POSITIVE AND NEGATIVE PREDICTIVE VALUE
| Calculation | Answers This Question: |
| # true positives PPV = —————————– x 100 (# true positives + # false positives) |
If the test is positive, how likely is it that the patient has disease? |
|
# true negatives |
If the test is negative, how likely is it that the patient is normal? |
Continued on page 2 …
Comparing Results
A truth table is a grid that compares one set of test results to another. It’s commonly used in electrical engineering and computer science to describe inputs and outputs, and in propositional calculus.5
The simplest laboratory application is qualitative testing, in which a test result is positive or negative. Table 3 presents a truth table illustrating possible combinations and their interpretations.
TABLE 3: TRUTH TABLE
| Reference Method – POS | Reference Method – NEG | |
| Comparison – POS |
Positive (true positive) | Positive (false positive) |
| Comparison – NEG | Negative (false negative) | Negative (true negative) |
When compared to a reference test or gold standard, for example, a serological test that is positive can either be a true or false positive. A test that is negative can either be a true or false negative.
A sample worksheet to tally results is shown in Table 4. For illustrative purposes, a sensitivity and specificity has been calculated from the “Totals” rows at the bottom of the Table.
TABLE 4: CORRELATION WORKSHEET EXAMPLE
| Reference Method-POS |
Reference Method-NEG |
|
| Sample 1 | positive | |
| Sample 2 | positive | |
| Sample 3 | negative | |
| Sample 4 | positive | |
| Sample 5 | negative | |
| Total-POS | 2 | 1 |
| Total-NEG | 1 | 1 |
Sensitivity = TP / (TP + FN) x 100 = 2 / (2 + 1) x 100 = 67 percent
Specificity = TN / (TN + FP) x 100 = 1 / (1 + 1) x 100 = 50 percent
The advantage of the truth table is to illustrate the relationship of the variables used to calculate diagnostic sensitivity and specificity, as well as positive and negative predictive value. Adding totals to the table also shows the sample size, a critical variable. Different tables can be compared depending on prevalence, age, gender and other variables.
A truth table is useful to summarize test performance, compare methods or highlight false-positive or false-negative results. It has many uses. For example, urinalysis testing that reflexes to culture can be summarized in a truth table; different tables can be constructed based on different criteria to calculate and compare sensitivity and specificity. Thus, it can be a useful quality assessment tool to compare input and output.
An alternative to creating separate truth tables is to construct a single table as in Table 5. This can be very useful when comparing two or more new tests, or when evaluating different criteria. In the Table, both Method A and B have the same sensitivity but different specificities.
TABLE 5: MULTIPLE METHODS TRUTH TABLE
| TP | TN | FP | FN | |
| Method A | 20 | 10 | 5 | 5 |
| Method B | 24 | 9 | 1 | 6 |
Method A Sensitivity = TP / (TP + FN) x 100 = 20 / (20 + 5) x 100 = 80 percent
Method A Specificity = TN / (TN + FP) x 100 = 10 / (10 + 5) x 100 = 67 percent
Method B Sensitivity = TP / (TP + FN) x 100 = 24 / (24 + 6) x 100 = 80 percent
Method B Specificity = TN / (TN + FP) x 100 = 9 / (9 + 6) x 100 = 60 percent
Making a Template
Spreadsheet software is a natural choice for creating a template, such as Microsoft Excel or another product. Formulae can be entered into the spreadsheet to calculate the sensitivity and specificity as suggested in Table 6.
TABLE 6: SAMPLE SPREADSHEET TEMPLATE AND FORMULAE
| A | B | C | |
| 1 | POS | NEG | |
| 2 | POS | ||
| 3 | NEG |
Sensitivity = B2 / (B2 + C2) * 100
Specificity = C3 / (C3 + B3) * 100
A spreadsheet is a two-dimensional grid, just like a truth table, defined by letters and numbers. Each “cell” is referenced in xy (column-row) notation. Thus, the top left cell is A1, the next cell to the right is B1, the cell directly below that is B2, and so on.
Formulae are entered into the cell (in Excel use the F2 key) using cell references. (Tip: preceding a column or row designation with $ makes the reference absolute; it doesn’t change when copied and pasted to another cell.) Most spreadsheets have built-in calculations to perform arithmetic, statistical, and business operations on numbers, but that’s not necessary here.
Once your template is created, it can be used to create a method validation summary for review by your staff and medical director, or cut-and-pasted into a memorandum for your medical staff.
Sensitivity and specificity are important laboratory concepts, useful in test evaluation and validation. A truth table is a practical tool to illustrate and highlight test performance and, along with a spreadsheet template, can be used to calculate and compare diagnostic test performance. This can help standardize your approaches to validation and quality assessment, leading to better patient care.
Scott Warner is lab manager at Penobscot Valley Hospital, Lincoln, ME.
References
1. Saah A et al. “Sensitivity” and “specificity” reconsidered: the meaning of these terms in analytical and diagnostic settings. Available at: www.annals.org/content/126/1/91.full. Last accessed May 16, 2010.
2. Wikipedia. Sensitivity and specificity. Available at: http://www.google.com/url?q=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FSensitivity_and_specificity%23Specificity&sa=D&sntz=1&usg=AFQjCNHKnIxjOapbJDAqsbGXXeDVm1fzUA. Last accessed May 16, 2010.
3. Lofgren RP: The dynamic nature of sensitivity and specificity. Available at: www.springerlink.com/content/172x5m41055412x4. Last accessed May 16, 2010.
4. Bean P. Sensitivity, specificity, and predictive values. The Specialty Labs web site. Available at: http://www.google.com/url?q=http%3A%2F%2Fwww.specialtylabs.com%2Fbooks%2Fdisplay.asp%3Fid%3D126&sa=D&sntz=1&usg=AFQjCNFd3u68voiPHGkAVO8sDd_M1swkvw. Last accessed May 16, 2010.
5. Wikipedia. Truth tables. Available at: http://www.google.com/url?q=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTruth_table&sa=D&sntz=1&usg=AFQjCNHRnF_dmuvxgZk6cE2K6w_rN-6fyQ. Last accessed May 16, 2010.
