The Title From the Supervisor
Dr. Sumaira Talib came straight from the emergency theatre, still in her scrubs, holding a single printed sheet.
“Sir, my supervisor has finally given me my synopsis title.” She read it out. “Diagnostic accuracy of ultrasound in the diagnosis of acute appendicitis, taking histopathology as the gold standard.”
“A classic CPSP title,” I said. “What will you measure?”
“That is the problem, Sir. I know how to do an appendicectomy. I do not know what ‘diagnostic accuracy’ actually means in numbers. Is it one percentage? Do I compare two groups? Do I need a hypothesis?”
Dr. Hammad Ali looked relieved that, for once, the confused trainee was not him.
“Sumaira,” I said, “every one of us uses diagnostic tests a hundred times a day: a urine dipstick, a troponin, an X-ray, a Pap smear. Very few of us have ever asked how often each test is wrong. Tonight we learn to ask that question properly.”
Two Tests on the Same Patient
I wrote two words on the board: index test and reference standard.
“The index test is the test you are evaluating. For Sumaira, it is ultrasound. The reference standard, which CPSP titles still call the gold standard, is the best available way of knowing whether the disease is truly present. For appendicitis, it is histopathology of the removed appendix.”
“So every patient gets both tests,” said Dr. Junaid Rashid.
“Every patient gets both, and the two results are compared. That is the whole design. In structure, it is a cross-sectional study: each patient is examined once, at one point in their illness. There is no intervention, and no comparison of two groups of patients. You are comparing two tests on the same people.”
“Then I do not need a hypothesis?” Sumaira asked.
“For a single test against a reference standard, no. You are estimating how accurate the test is, not testing whether one group differs from another. You need a hypothesis only when you compare two tests against each other, for example ultrasound against the Alvarado score in the same patients.”
The Table With Four Boxes
“Suppose you finish your study with 200 patients,” I said, and drew the most important table in diagnostic research.
| Histopathology: appendicitis | Histopathology: normal appendix | Total | |
|---|---|---|---|
| Ultrasound positive | 136 (true positive, a) | 8 (false positive, b) | 144 |
| Ultrasound negative | 24 (false negative, c) | 32 (true negative, d) | 56 |
| Total | 160 | 40 | 200 |
“Always put the reference standard across the top and the index test down the side. Then every number you need comes from these four boxes.”
“Sensitivity asks: of the patients who truly had appendicitis, how many did ultrasound pick up? That is a divided by a plus c: 136 out of 160, which is 85%.”
“Specificity asks: of the patients with a normal appendix, how many did ultrasound correctly call normal? That is d divided by b plus d: 32 out of 40, which is 80% [1].”
Dr. Hassan Raza frowned. “Sir, these two always confuse me. Which one do I want high?”
“It depends on what the test is for. Students remember it as SnNout and SpPin. A highly sensitive test, when negative, rules the disease out, because it rarely misses anyone. A highly specific test, when positive, rules the disease in, because it rarely raises a false alarm. A screening test needs high sensitivity. A test that sends a patient to theatre needs high specificity.”
The Question the Patient Actually Asks
Dr. Bushra Fatima had been quiet. “Sir, no patient asks me about sensitivity. When the scan report says positive, the mother asks: ‘Does my son have appendicitis or not?’”
“And that is a different question, read across the rows instead of down the columns. The positive predictive value is: of all patients with a positive scan, how many truly had appendicitis? That is 136 out of 144, about 94%. The negative predictive value is: of all patients with a negative scan, how many were truly normal? That is 32 out of 56, only 57% [2].”
“So a negative scan in your study means almost nothing,” said Bushra.
“In this study, yes. And now the part that surprises most consultants. Predictive values are not fixed properties of the test. They change with how common the disease is in the patients you test.”
I explained. “Sumaira’s patients are those already admitted by a surgeon and taken to theatre, so 160 out of 200, that is 80%, truly had appendicitis. Now take the same ultrasound, with the same 85% sensitivity and 80% specificity, to a general OPD where only 20% of patients with abdominal pain have appendicitis.”
| Setting (same test: sensitivity 85%, specificity 80%) | Prevalence | Positive predictive value | Negative predictive value |
|---|---|---|---|
| Surgical ward, patients taken to theatre | 80% | 94% | 57% |
| General OPD, out of 1,000 patients with abdominal pain | 20% | 170 / 330 = 52% | 640 / 670 = 96% |
“The same machine, the same sonologist. In the ward a positive scan is almost always right. In the OPD, half the positive scans are false alarms, but a negative scan is very reassuring. That is why you must always report the prevalence in your sample, and why sensitivity and specificity are the numbers that travel from one hospital to another.”
Junaid was writing quickly. “Sir, is there a number that does not change with prevalence but still answers the patient’s question?”
“The likelihood ratio. The positive likelihood ratio is sensitivity divided by one minus specificity: 0.85 divided by 0.20, about 4.3. It tells you how much a positive result raises the odds of disease, whatever the starting point. The negative likelihood ratio is one minus sensitivity divided by specificity: 0.15 divided by 0.80, about 0.19. A positive ratio above 10 or a negative ratio below 0.1 usually changes management decisively [3]. You will meet these again in the statistics part, along with ROC curves for tests that give a number rather than a yes or no.”
“And diagnostic accuracy itself?” Sumaira asked. “My title uses that phrase.”
“Overall accuracy is all correct results divided by all patients: 136 plus 32, out of 200, which is 84%. CPSP synopses usually report it, but never report it alone. A test can look accurate simply because one kind of patient dominates the sample. Report all five: sensitivity, specificity, PPV, NPV and accuracy.”
Where the Biases Hide
Dr. Zunaira Malik had been studying the table. “Sir, only patients who are operated on get histopathology. What about the patients whose ultrasound was negative and who were sent home? Nobody ever knows if they had appendicitis.”
“Zunaira has found the commonest bias in diagnostic research,” I said. “It is called verification bias, or partial verification. If the result of the index test decides who gets the reference standard, the study quietly loses most of its false negatives, and sensitivity looks better than it really is. In oral pathology, you face the same problem with any screening test for oral lesions: if only suspicious lesions are biopsied, you never learn what the test missed.”
“So what does Sumaira do?”
“She defines her population as patients with clinically suspected appendicitis who are scheduled for appendicectomy, and every one of them has the scan before surgery, whatever its result. The surgeon’s decision to operate must not depend on the scan she is studying. And she writes that limitation honestly in her synopsis.”
I listed the others on the board.
| Bias | What goes wrong | How to prevent it |
|---|---|---|
| Verification bias | Only some patients, chosen by the index test result, get the reference standard | Apply the reference standard to every enrolled patient |
| Spectrum bias | Only obvious cases and clearly healthy people are enrolled, so the test looks better than it is in real practice | Enrol consecutive patients in whom the diagnosis is genuinely in doubt [4] |
| Review bias | The person reading one test knows the result of the other | The sonologist does not see the histopathology; the histopathologist does not see the scan report |
| Incorporation bias | The index test is part of the reference standard | Keep the two tests completely separate |
| Time-lag bias | Disease changes between the two tests | Perform the scan shortly before surgery, and state the maximum interval |
Hassan laughed. “Sir, in my hospital the sonologist calls the surgeon before writing the report, and the surgeon tells him what he thinks.”
“Then in your hospital, the ultrasound is partly measuring the surgeon’s opinion. The sonologist may know the clinical findings, because that is how scans are done in real life, but he must not know anyone else’s conclusion about this patient. Write down who performs each test, and who is blinded to what.”
The Sample Size Sumaira Did Not Expect
“Now your sample size,” I said. “CPSP expects a diagnostic accuracy study to be calculated from the expected sensitivity and specificity, not from a prevalence alone. The usual method is the one described by Buderer [5].”
“Suppose your literature search suggests a sensitivity of about 85%, a specificity of about 80%, and about 80% of operated patients having appendicitis on histopathology. You want each estimate within 10% either side, at 95% confidence.”
I worked it on the board.
- For sensitivity: 1.96² × 0.85 × 0.15 ÷ 0.10² = 49 patients who truly have appendicitis. Since only 80% of patients do, 49 ÷ 0.80 = 61.3, rounded up to 62 patients.
- For specificity: 1.96² × 0.80 × 0.20 ÷ 0.10² = 61.5 patients with a normal appendix. Since only 20% of patients have a normal appendix, 61.5 ÷ 0.20 = 307.4, rounded up to 308 patients.
- Take the larger figure: 308 patients.
Sumaira stared at the board. “Three hundred and eight? I thought sixty would be enough.”
“Sixty would be enough for sensitivity. But to estimate specificity, you need enough patients without the disease, and in a surgical ward they are rare. The rarer the normal appendix, the more patients you must enrol to find enough of them. This is why both figures must come from the same published study, and why the CPSP reviewer checks both.”
Putting It in the Synopsis
By the end of the session, Sumaira had rewritten the core of her synopsis, and I put it on the board for everyone.
| Objective | To determine the diagnostic accuracy of ultrasound in the diagnosis of acute appendicitis, taking histopathology as the gold standard |
| Study design | Cross-sectional (validation) study |
| Sampling | Non-probability consecutive sampling of patients with clinically suspected acute appendicitis scheduled for appendicectomy |
| Index test | Ultrasound positive: a non-compressible appendix with an outer diameter of more than 6 mm, performed by a consultant radiologist before surgery, blinded to histopathology |
| Reference standard | Histopathology positive: neutrophilic infiltration of the muscularis propria, reported by a consultant histopathologist blinded to the ultrasound result |
| Hypothesis | Not required (a single test against a reference standard) |
| Analysis | 2×2 table; sensitivity, specificity, PPV, NPV and diagnostic accuracy; stratified for age, gender, BMI and duration of pain |
“Notice that each operational definition says what counts as positive, who decides, and when,” I said. “And when you write the paper, follow the STARD 2015 checklist, including its flow diagram showing how many patients were eligible, how many had both tests, and why any were excluded [6]. When a systematic review later judges your study, its authors will use a tool called QUADAS-2 [7]. Design your study so that it passes every question on that tool.”
Hammad leaned back. “Sir, this is easier than my trial. No randomisation, no blinding of the patient.”
“It is simpler to run,” I said, “but no less easy to get wrong. A diagnostic study fails quietly, in who gets the reference standard and who knows which result. Research is not rocket science. It is a skill, and skills can be taught. Tonight’s skill is learning to ask of every test you order: how often is it wrong, and in which patients?”
Key Takeaways
- A diagnostic accuracy study compares an index test with a reference standard (gold standard) in the same patients. In structure, it is cross-sectional.
- Sensitivity and specificity are read down the columns of the 2×2 table; predictive values are read across the rows.
- Predictive values change with the prevalence of disease in the sample; always report the prevalence alongside them.
- Likelihood ratios show how much a result changes the odds of disease and do not depend on prevalence.
- Prevent verification, spectrum, review, incorporation and time-lag bias: every patient gets both tests, enrolled consecutively, with each reader blinded to the other result.
- For CPSP, calculate the sample size from both sensitivity and specificity, adjusted for prevalence, and take the larger figure. Report the study with STARD 2015.
References:
- Altman DG, Bland JM. Diagnostic tests 1: sensitivity and specificity. BMJ. 1994;308(6943):1552.
- Altman DG, Bland JM. Diagnostic tests 2: predictive values. BMJ. 1994;309(6947):102.
- Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ. 2004;329(7458):168-9.
- Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299(17):926-30.
- Buderer NM. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med. 1996;3(9):895-900.
- Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527.
- Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-36.