Mark AinsworthpersName, Monique AnderssonpersName, Kathryn AucklandpersName, J. Kenneth BailliepersName, Eleanor BarnespersName, Sally BeerpersName, Amy BeveridgepersName, Sagida BibipersName, Luke BlackwellpersName, Martyna BorakpersName, Abbie BownpersName, Tim BrookspersName, N. Burgess-BrownpersName, Susana CamarapersName, Matthew CattonpersName, Kevin ChaupersName, Thomas ChristottpersName, Elizabeth ClutterbuckpersName, Jesse A. CokerpersName, Richard J. CornallpersName, Stuart CoxpersName, David Crawford-JonespersName, Derrick W. CrookpersName, Silvia D’ArcangelopersName, Wanwisa DejnirattsaipersName, Julie DequairepersName, Stavros DimitriadispersName, Kate E. DinglepersName, George DohertypersName, Christina DoldpersName, Tao DongpersName, Susanna DunachiepersName, Daniel EbnerpersName, Marc EmmeneggerpersName, Alexis EspinosapersName, David W. EyrepersName, Rory FairheadpersName, Shayan FassihpersName, Conor FeehilypersName, Sally FellepersName, Alejandra Fernandez‐CidpersName, Maria Fernandez MendozapersName, Thomas FoordpersName, Thomas FordwohpersName, Deborah McKeepersName, John FraterpersName, Verónica SánchezpersName, Nick GentpersName, Dominique GeorgioupersName, Christopher J. GrovespersName, Bassam HallispersName, Peter HammondpersName, Stephanie B. HatchpersName, Heli HarvalapersName, Jennifer HillpersName, Sarah HoosdallypersName, Bryn HorsingtonpersName, Alison HowarthpersName, Tim JamespersName, Katie JefferypersName, Elizabeth JonespersName, Anita JusticepersName, Fredrik KarpepersName, James KavanaghpersName, David KimpersName, Richard KirtonpersName, Paul KlenermanpersName, Julian C. KnightpersName, Leonidas KoukouflispersName, Andrew KwokpersName, U LeuschnerpersName, Robert H. LevinpersName, Aline LinderpersName, Teresa LockettpersName, Sheila LumleypersName, Spyridoula MarinoupersName, Brian D. MarsdenpersName, José Luis MartínezpersName, Lucas Martins FerreirapersName, L. H. MasonpersName, Philippa C. MatthewspersName, Alexander J. MentzerpersName, Alexander MobbspersName, Juthathip MongkolsapayapersName, Jordan MorrowpersName, Shubhashish MukhopadhyaypersName, Matthew NevillepersName, Sarah OakleypersName, Marta OliveirapersName, Ashley OtterpersName, Kevin PaddonpersName, Jordan PascoepersName, Yanchun PengpersName, Elena PerezpersName, Prem PerumalpersName, Tim PetopersName, Hayleah PickfordpersName, Rutger J. PloegpersName, Andrew J. PollardpersName, Anastasia RichardsonpersName, Thomas RitterpersName, David J. RobertspersName, Gillian RodgerpersName, Christine S. RollierpersName, Cathy RowepersName, Justine RudkinpersName, Gavin ScreatonpersName, Malcolm G. SemplepersName, Alex SienkiewiczpersName, Laura Silva-ReyespersName, Donal SkellypersName, Alberto Sobrino DiazpersName, Lizzie StaffordpersName, Lisa StockdalepersName, Nicole StoesserpersName, Teresa StreetpersName, David I. StuartpersName, Angela SweedpersName, A TaylorpersName, Hannah ThravespersName, Hoi Pat TsangpersName, Marije K. VerheulpersName, Richard VipondpersName, A Sarah WalkerpersName, Susan WareingpersName, Yolanda WarrenpersName, Charlie WellspersName, Clare WilsonpersName, Kate WithycombepersName, Rebecca YoungpersName (2020date). “Performance characteristics of five immunoassays for SARS-CoV-2: a head-to-head benchmark comparison.” The Lancet Infectious Diseasestitle·venue.
Object of study — FIVE IMMUNOASSAYS MEASURING ONE CONSTRUCT ON ONE SAMPLE SET - 976 pre-pandemic and 536 PCR-confirmed bloods run through four commercial platforms and a novel ELISA, with the thresholds treated as adjustable
Research question — how five widely deployed SARS-CoV-2term antibody assays compare when run on identical samples, and what thresholds each needs to reach 98% sensitivityterm and specificityterm
Contribution — the first head-to-head assessmentterm of four commercial platforms plus a scalable 384-well ELISA on one sample set, with ROC-derived thresholds and an error-rate projection per million tests
Cross-disciplinary analysis
content-tags & facets:
phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | an instrument's measured accuracy as a property of the cut point somebody chose, in the absence of anything to check the instrument against — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | |
| claim-type | method — “At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%).” | |
| method — evidence | — “All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset.” | |
| object of study — evidence | — “At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%).” | |
| research question — evidence | — “However, immunoassay performance has not been compared on large, identical sample sets.” | |
| contribution — evidence | — “No published studies assessed all four commercial platforms using the same sample sets.” | |
| mechanism | THE PAPER SAYS OUTRIGHT THAT THERE IS NOTHING TO MEASURE THE INSTRUMENTS AGAINST: `Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response ... effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample ... or through a false-positive RT-PCR test`. FIVE WAYS TO GET A DISCORDANT RESULT AND ONLY ONE OF THEM IS THE ASSAY BEING WRONG — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | mechanisms[0] |
| mechanism | AND THE THRESHOLD IS WHAT MOVES, NOT THE ASSAY: at the manufacturers' cut points the five assays span 92.7% to 99.1% sensitivity, and then `All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset`. Same five machines, same 1,512 samples, and every one of them passes once someone else chooses the cut point — “All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset.” | mechanisms[1] |
| mechanism | and the authors add the caveat that makes it honest rather than circular: `opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity` — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity.” | mechanisms[2] |
| mechanism | AND THE COST OF THE CHOICE IS COUNTED IN PEOPLE: `at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests`. A five-fold difference in wrong answers, from a choice of platform — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | mechanisms[3] |
| mechanism | AND THE PRIOR LITERATURE HAS THE RIFLE PROBLEM: `For the Abbott assay, sensitivity RANGED FROM 93.4% TO 100% ... (70 to 680 samples) ... for the DiaSorin assay, we found A single sensitivity estimate of 94.4% ... (18 SAMPLES)`, and `No published studies assessed all four commercial platforms using the same sample sets`. The spread was never a property of the assays — “For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples).” | mechanisms[4] |
| extends | the eight prior studies of the Abbott and DiaSorin assays, whose sensitivity estimates ranged from 93.4% to 100% on sample sets of eighteen to 680 — “For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples).” | claim_relations[0] |
| contrasts-with | the manufacturers' own threshold settings, under which only two of the five assays reach 98% sensitivity and specificity — “At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%).” | claim_relations[1] |
| sense | `gold standard` (absent) - `no clear gold standard exists against which to assess these antibody tests`, and the substitute is explicitly a proxy: `RT-PCR positivity is a proxy for the expected presence of antibody` — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | senses[0] |
| sense | `sensitivity` (manufacturer sense) vs `sensitivity` (ROC-optimised sense) - the same word for two numbers on the same machine and the same samples. Abbott is 92.7% under one and at least 98% under the other — “All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset.” | senses[1] |
| sense | `false negative` - here NOT a property of the assay: an antibody-negative PCR-positive result can be immunocompromise, T-cell clearance, a bad sample, or a FALSE-POSITIVE PCR. Four alternatives to the instrument being wrong — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | senses[2] |
| sense | Head-to-head benchmark - all platforms are scored on one large, identical sample set rather than compared across separate studies — “In this study on a large, single sample set of more than 1500 samples, we compared four commercial SARS-CoV-2 immunoassays (Abbott, DiaSorin, Roche, and Siemens assays) with a global installed base, and a novel 384-well ELISA (the Oxford immunoassay) that could scale to be used for population-level surveillance. The Siemens assay and Oxford immunoassay achieved a sensitivity and specificity of at least 98% without optimisation (lower 95% CI ≥96%). The other assays could achieve sensitivities and specificities of at least 98% with assay threshold adjustment or deployment on samples taken at least 30 days after symptom onset. However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests. Implications of all the available evidence Understanding the comparative performance of immunoassays is crucial to the development of appropriate individual and population-level SARS-CoV-2 testing strategies. We showed that although precise performance metrics varied between immunoassay platforms, all assays that we assessed could be usefully deployed with careful consideration of use case, assay thresholds, and by considering symptom-to-sample timings, thus optimising the use of available serological testing resource and enabling the most widespread rollout” | senses[3] |
| sense | Sensitivity reference - antibody detection among RT-PCR-confirmed post-symptom samples, with RT-PCR acknowledged as only a proxy — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection. The three samples that were negative by all assays in our study could plausibly have had a biological absence of antibodies” | senses[4] |
| sense | Specificity reference - absence of positive calls among samples collected years before SARS-CoV-2 appeared — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | senses[5] |
| sense | Threshold-dependent performance - manufacturer cut points and ROC-optimised cut points produce different sensitivity-specificity pairs — “Using the pre-defined assay thresholds for calling test results positive or negative, only the Siemens assay and the Oxford immunoassay met the specificity and sensitivity target of at least 98%, achieving this threshold at all three timepoints assessed (≥14, ≥20, and ≥30 days post symptom onset; appendix pp 28, 29 ; figure 1 ; table ). The Roche assay also met the specificity and sensitivity target of at least 98% on samples taken at least 30 days post symptom onset ( appendix p 29 ; table ). ROC curve analysis showed that the Roche assay could also potentially meet the sensitivity and specificity targets on samples taken at least 14 and at least 20 days post symptom onset through threshold adjustment—eg, adjusting the threshold to at least 0·128 as opposed to the current threshold of more than 1·0 would result in a sensitivity and specificity of at least 98% (with a lower bound of the 95% CI of ≥96%) at 20 days or more post symptom onset ( figure 3 ; appendix p 30 for samples taken ≥14 days post symptom onset; table ). On samples taken at least 30 days post symptom onset (but not ≥14 or ≥20 days post symptom onset), the Abbott and DiaSorin assay thresholds could be optimised to achieve sensitivity and specificity of at least 98% (with a lower bound of ≥96%; appendix p 31 ; table ). Overall, on samples taken at least 30 days post symptom onset, optimisation of assay thresholds to ensure a specificity of at least 98% (with a lower bound of ≥96%) would result in sensitivities of at least 98% (with a lower bound of ≥96%) for all assays assessed” | senses[6] |
| sense | Predictive-value framing - expected false positives, false negatives, and total errors depend on population prevalence — “Although sensitivity and specificity are useful, positive and negative predictive values are more directly relevant to public health and clinical application. At 10% prevalence, the high specificity platforms (Abbott, Roche, and Siemens assays) had lower numbers of false positive tests per million tests compared with other assays (DiaSorin and Oxford immunoassay; figure 4 ; appendix p 21 ). However, the number of false negative tests per million was higher with the Abbott and DiaSorin assays and lowest with the Oxford immunoassay ( figure 4 ; appendix p 21 ). Overall, the Siemens assay had the fewest total errors per million tests for prevalences of 20% or lower” | senses[7] |
| sense | Use-case-dependent assay choice - nucleocapsid assays become useful for surveillance when vaccination is spike-protein based — “On a large panel of blood samples, we showed that the Siemens assay and the Oxford immunoassay both achieved sensitivity and specificity of at least 98% (with a lower bound of the 95% CI ≥96%) on samples taken at least 20 days post symptom onset without optimisation. However, all assays could potentially achieve these specifications through threshold adjustment, or by assessing samples collected at least 30 days post symptom onset, consistent with the time-dependent nature of antibody responses. Additionally, we highlighted the potentially scalable capacity of the Oxford immunoassay, which could enable centralised testing for large-scale projects (eg, national seroprevalence studies). This finding suggests that global serology testing needs can be met using different assays, mitigating against the risk of shortages, and allowing immediate deployment in laboratories with different analysers already installed for other testing purposes. The optimal deployment of assays should reflect use case—eg, an optimised assay measuring antibodies to nucleocapsid protein might be most relevant for disease surveillance in the context of a widespread rollout of vaccines, which are mostly spike-protein based. Although sensitivity and specificity are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted” | senses[8] |
| sense | Antibody-versus-immunity boundary - assays detect antibodies, while durability and protective meaning remain uncertain — “In our study, antibody titres within individuals were largely sustained beyond 2 months of symptom onset for all assays investigated, suggesting that serosurveillance with these assays is likely to be accurate within months of exposure at least. We did not observe any significant difference in assay sensitivity by disease severity; notably, however, the numbers in our non-mild disease category, including asymptomatic cases, were small, and we were limited by the bounds determined by the dynamic range of the assays and saturation. Inconsistencies have been observed in the association between antibody titres and disease severity in other studies, which might relate to assay target. 20 , 24 , 25 Further studies definitively assessing the longer-term trajectory of antibodies in larger longitudinal cohorts of individuals with different disease severities and taking the specific assay type into account would be of benefit. Although all these assays can effectively detect SARS-CoV-2 antibodies, the durability and nature of immunity conferred by these antibodies remains unclear. Few data are available on immunity to seasonal human coronaviruses (alphacoronaviruses NL63 and 229E, and betacoronaviruses HKU1 and OC43). In two small, controlled, human infection models, re-challenge at 8–12 months with 229E (nine individuals) 26 and 229E-related strains (six) 27 showed protection against symptoms, but virus shedding was detected in six (67%) of nine individuals. 26 In an observational study of NL63 in Kenya, reinfections over a 6-month period occurred in 46 (28%) of 163 individuals, with reinfections within 80 days largely associated (>80% cases) with lower viral titres and fewer symptoms compared with those associated with reinfections after more than 80 days. 28 Data on Middle East respiratory syndrome (MERS) and severe acute respiratory syndrome coronavirus suggest that antibody titres wane over periods of approximately 2 years, with more severe infection resulting in more measurable and durable antibody responses in MERS infection. 29 , 30 Animal data (re-challenge in four rhesus macaques) suggest protection from re-infection with SARS-CoV-2 extends to at least 28 days post infection. 31 The relationship between SARS-CoV-2 antibodies and neutralising activity remains an important research question. Because the spike protein is responsible for enabling cell entry, the spike protein appears to be the main antigen responsible for eliciting neutralising antibodies. In SARS-CoV-2, a correlation between anti-spike antibody titres when trimeric versions of spike are used as the antigen and neutralising capacity appears to be emerging. 24 , 32 If SARS-CoV-2 antibody titres are shown to clearly correlate directly or indirectly with immunity to infection or attenuation of disease severity, then more accurately evaluating titres by optimising the dynamic range of assays might become relevant” | senses[9] |
| sense | Post-hoc threshold optimisation - any changed cut point requires renewed verification of both sensitivity and specificity — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity. Assay selection should carefully reflect use case. This study represents a benchmark for future assessments of serological platforms. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level” | senses[10] |
| control | 976 pre-pandemic blood samples collected between September 2014 and October 2016 as the negative panel — held — “All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset.” — rules out specificity estimated on samples that might contain SARS-CoV-2 antibodies - the samples predate the virus — “At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%).” | controls[0] |
| control | running all five assays on one identical sample set — held — “For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples).” — rules out the between-study variation that made the prior literature unusable: 'No published studies assessed all four commercial platforms using the same sample sets' — “No published studies assessed all four commercial platforms using the same sample sets.” | controls[1] |
| control | a reference standard to score the assays against — absent-not-in-staged-text — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” — rules out nothing - it does not exist: 'no clear gold standard exists against which to assess these antibody tests', and RT-PCR positivity is only a proxy with four named failure modes — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | controls[2] |
| control | testing for antibodies to seasonal coronaviruses in the negative panel — absent-declared-unreported — “Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection.” — rules out nothing here - declared not done: the authors say they 'did not investigate our large negative sample set for the presence of antibodies against seasonal coronaviruses, the presence of which might have accounted for some of the false-positive results' — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity.” | controls[3] |
| weight | the eight prior studies of the Abbott and DiaSorin assays, whose sensitivity estimates ranged from 93.4% to 100% on sample sets of eighteen to 680 — 0.85 — “For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples).” | claim_relations[0] |
| weight | the manufacturers' own threshold settings, under which only two of the five assays reach 98% sensitivity and specificity — 0.9 — “At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%).” | claim_relations[1] |
| audit | Siemens assay —distinct-from→ DiaSorin assay — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span compares the two assays' error rates; distinct-from is loose. — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | concept_edges[0] |
| note | Siemens assay —distinct-from→ DiaSorin assay — 'at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests'. Two instruments for one construct, run on the same 1,512 samples, differing by a factor of five in the number of people they get wrong — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | concept_edges[0] |
| weight | Siemens assay —distinct-from→ DiaSorin assay — 0.95 — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | concept_edges[0] |
| audit | gold standard —operationalised-as→ RT-PCR positivity — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but 'no clear gold standard exists' and RT-PCR positivity is 'a proxy'; operationalised-as is loose. — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | concept_edges[1] |
| note | gold standard —operationalised-as→ RT-PCR positivity — THE CUE AND THE TYPE DISAGREE ON PURPOSE. The cue is KINDHOOD 'is a', in 'RT-PCR positivity is a proxy for the expected presence of antibody' - but the relation is not kindhood, it is substitution: the reference standard is absent ('no clear gold standard exists') and RT-PCR positivity stands in for it. Weight 0.9 because the paper states four ways the proxy and the target can disagree without either being wrong — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | concept_edges[1] |
| weight | gold standard —operationalised-as→ RT-PCR positivity — 0.9 — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | concept_edges[1] |
| note | SARS-CoV-2 —same-as→ Severe acute respiratory syndrome coronavirus 2 — Parenthetical abbreviation of the expanded pathogen name. — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections” | concept_edges[2] |
| note | novel 384-well ELISA —same-as→ Oxford immunoassay — The paper introduces the ELISA with Oxford immunoassay as its parenthetical name. — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concept_edges[3] |
| relation | novel 384-well ELISA —same-as→ Oxford immunoassay — Summary Background Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concept_edges[3] |
| note | Oxford immunoassay —is-a→ indirect ELISA — Direct class statement. — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein” | concept_edges[4] |
| note | pre-pandemic samples from healthy individuals aged at least 18 years —part-of→ blood (plasma and serum) sample collection — The collection explicitly includes the pre-pandemic panel. — “We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers” | concept_edges[5] |
| note | samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection —part-of→ blood (plasma and serum) sample collection — The collection explicitly includes the infection-confirmed panel. — “We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers” | concept_edges[6] |
| note | Readouts —operationalised-as→ fluorescent signal — The readouts are measured as fluorescent signal before calibration. — “Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons” | concept_edges[7] |
| note | symptom severity —operationalised-as→ asymptomatic, mild, severe, critical or death — The paper explicitly defines severity by these WHO-aligned categories. — “symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay” | concept_edges[8] |
| note | specificity —operationalised-as→ known negative pre-pandemic samples — Specificity is assessed with the pre-pandemic negative panel. — “Specificity was assessed using the known negative pre-pandemic samples, and sensitivity using the known positive samples from patients who had previously had laboratory-confirmed SARS–CoV–2 infection” | concept_edges[9] |
| note | sensitivity —operationalised-as→ known positive samples from patients who had previously had laboratory-confirmed SARS–CoV–2 infection — Sensitivity is assessed with the laboratory-confirmed positive panel. — “Specificity was assessed using the known negative pre-pandemic samples, and sensitivity using the known positive samples from patients who had previously had laboratory-confirmed SARS–CoV–2 infection” | concept_edges[10] |
| audit | first 3–4 weeks after symptom onset —causes→ Antibody responses — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but antibody responses rose OVER the first weeks; a time window as the cause is loose. — “Antibody responses increased over the first 3–4 weeks after symptom onset, with the maximum percentage of positive tests reached by day 27 across all assays” | concept_edges[11] |
| effect direction | first 3–4 weeks after symptom onset —causes→ Antibody responses — increases — “Antibody responses increased over the first 3–4 weeks after symptom onset, with the maximum percentage of positive tests reached by day 27 across all assays” | concept_edges[11] |
| evidential status (stated by the tagger, no quotation) | first 3–4 weeks after symptom onset —causes→ Antibody responses — observational temporal trajectory across assays, not an intervention | concept_edges[11] |
| note | first 3–4 weeks after symptom onset —causes→ Antibody responses — Antibody responses increased over this post-symptom period. — “Antibody responses increased over the first 3–4 weeks after symptom onset, with the maximum percentage of positive tests reached by day 27 across all assays” | concept_edges[11] |
| outcome | control — held — “At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%).” — “All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset.” | controls[0] |
| outcome | control — held — “No published studies assessed all four commercial platforms using the same sample sets.” — “For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples).” | controls[1] |
| outcome | control — absent-not-in-staged-text — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | controls[2] |
| outcome | control — absent-declared-unreported — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity.” — “Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection.” | controls[3] |
| relation | direct detection of the virus → real-time RT-PCR — Introduction Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme) — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | unhoused_relations[0] |
| relation | serology → presence of antibodies — Introduction Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme) — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | unhoused_relations[1] |
| relation | nucleocapsid protein → IgG assays — Procedures The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | unhoused_relations[2] |
| relation | EUROIMMUN assay → SARS-CoV-2 Total assay — Research in context Evidence before this study Substantial global interest exists in the use of serology to enable population-level surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2 serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency use in the USA and the EU include the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxiv up to May 31, 2020, using the following search terms: (SARS-CoV-2) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviews were done for 124 articles. Eight studies provided data on sensitivity and specificity for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivity estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificity estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample sets — “Substantial global interest exists in the use of serology to enable population-level surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2 serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency use in the USA and the EU include the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxiv up to May 31, 2020, using the following search terms: (SARS-CoV-2) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviews were done for 124 articles. Eight studies provided data on sensitivity and specificity for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivity estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificity estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample sets” | unhoused_relations[4] |
| no type for this | direct detection of the virus → real-time RT-PCR — Direct viral detection is performed with real-time RT-PCR; performed-with is absent from the eight governed types. — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | unhoused_relations[0] |
| no type for this | serology → presence of antibodies — Serology is used to investigate antibody presence; investigates is absent from the eight types. — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | unhoused_relations[1] |
| no type for this | nucleocapsid protein → IgG assays — The assay taxonomy crosses antigen target and antibody-isotype class; targets is absent from the eight governed types. — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | unhoused_relations[2] |
| no type for this | trimeric spike protein → Oxford immunoassay — Trimeric spike is the antigen measured by the Oxford immunoassay; target-of is absent from the eight types. — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein” | unhoused_relations[3] |
| no type for this | EUROIMMUN assay → SARS-CoV-2 Total assay — EUROIMMUN was replaced with Siemens in the head-to-head comparison; replaces or selected-instead-of is absent from the eight types. — “Substantial global interest exists in the use of serology to enable population-level surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2 serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency use in the USA and the EU include the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxiv up to May 31, 2020, using the following search terms: (SARS-CoV-2) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviews were done for 124 articles. Eight studies provided data on sensitivity and specificity for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivity estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificity estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample sets” | unhoused_relations[4] |
| no type for this | post-hoc changes to thresholds → repeat verification of sensitivity and specificity — Any post-hoc threshold change requires repeat verification; prerequisite-for is absent from the eight governed types. — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity. Assay selection should carefully reflect use case. This study represents a benchmark for future assessments of serological platforms. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level” | unhoused_relations[5] |
| no type for this | use case → Assay selection — Assay selection should reflect the use case; should-be-governed-by is absent from the eight types. — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity. Assay selection should carefully reflect use case. This study represents a benchmark for future assessments of serological platforms. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level” | unhoused_relations[6] |
| limitation | ONE INSTRUMENT WAS DROPPED FROM THE COMPARISON BEFORE IT BEGAN, ON THE STRENGTH OF A PRIOR REPORT: `the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens ...)`. A selection decision that shapes what the benchmark can find, made in the open and worth reading next to the finding that prior estimates ranged from 93.4% to 100% on samples as small as eighteen — “However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison.” | limitations[0] |
| limitation | AND THE ASSESSMENT CRITERIA THEMSELVES HAD BEEN RELAXED: `governments, regulators, and clinical laboratories have had to balance the urgent need ... with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)` — “To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme).” | limitations[1] |
| limitation | AND THE ANSWER DEPENDS ON WHAT THE TEST IS FOR: `The optimal deployment of assays should reflect use case - eg, an optimised assay measuring antibodies to NUCLEOCAPSID protein might be most relevant for disease surveillance in the context of A widespread rollout of vaccines, which are mostly spike-protein based`. The best instrument changes when the population changes — “The optimal deployment of assays should reflect use case—eg, an optimised assay measuring antibodies to nucleocapsid protein might be most relevant for disease surveillance in the context of a widespread rollout of vaccines, which are mostly spike-protein based.” | limitations[2] |
| limitation | and the headline metrics are not the useful ones: `Although sensitivity and specificity are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted` - because those depend on prevalence, which is what the assays are being used to estimate — “Although sensitivity and specificity are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted.” | limitations[3] |
| limitation | WHO IS NOT IN THE SAMPLE, STATED: `Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection` — “Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection.” | limitations[4] |
| limitation | and three discordant samples are left explicitly undecided rather than counted as failures: `The three samples that were negative by all assays in our study could plausibly have had a biological absence of antibodies` — “The three samples that were negative by all assays in our study could plausibly have had a biological absence of antibodies.” | limitations[5] |
| limitation | THE PUBLISHED RECORD OF THIS PAPER CARRIES ITS OWN CORRECTION: `This online publication has been corrected. The corrected version first appeared at thelancet.com/infection on November 25, 2020`, and the PMC record we staged is VERSION 2. The project's distinction applies - a correction re-grounds a record, it does not retire it — “This online publication has been corrected. The corrected version first appeared at thelancet.com/infection on November 25, 2020” | limitations[6] |
Content tags
content-observations:
finding · quantity · variable · definition · key-term — grounded in the full text| finding | — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | content_tags.findings[0] |
| finding | — “All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset.” | content_tags.findings[1] |
| finding | — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity.” | content_tags.findings[2] |
| finding | — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | content_tags.findings[3] |
| finding | — “For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples).” | content_tags.findings[4] |
| finding | — “No published studies assessed all four commercial platforms using the same sample sets.” | content_tags.findings[5] |
| finding | — “However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison.” | content_tags.findings[6] |
| finding | — “To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme).” | content_tags.findings[7] |
| finding | — “The optimal deployment of assays should reflect use case—eg, an optimised assay measuring antibodies to nucleocapsid protein might be most relevant for disease surveillance in the context of a widespread rollout of vaccines, which are mostly spike-protein based.” | content_tags.findings[8] |
| finding | — “Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection.” | content_tags.findings[9] |
| finding | — “This online publication has been corrected. The corrected version first appeared at thelancet.com/infection on November 25, 2020” | content_tags.findings[10] |
| quantity | 2800 errors per million tests — the Siemens assay's projected total errors at 10% seroprevalence - the best of the five — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | content_tags.quantities[0] |
| quantity | 13 700 errors per million tests — the DiaSorin assay's projected total errors at the same seroprevalence - a five-fold gap from a choice of platform — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | content_tags.quantities[1] |
| quantity | 18 samples — the size of the only prior sensitivity estimate for the DiaSorin assay in the literature the authors could find — “For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples).” | content_tags.quantities[2] |
| quantity | 98 % — the sensitivity and specificity every one of the five assays reaches once the threshold is chosen for it rather than by the manufacturer — “All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset.” | content_tags.quantities[3] |
| variable | RT-PCR positivity — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | content_tags.variables[0] |
| definition | gold standard — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | content_tags.definitions[0] |
Concepts
terms this paper carries, each with the sentence it was cut from — the layer that joins one paper to another. Terms with no quote yet are on CONCEPTS.html.
| concept | gold standard — not defined in this paper's own words; the reference the assays would be scored against, which the authors say does not exist for this measurement — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | concepts[0] |
| concept | RT-PCR positivity — not defined in this paper's own words; the proxy standing in for the absent gold standard, with four stated ways it can disagree with an antibody test without either being wrong — “Notably, no clear gold standard exists against which to assess these antibody tests. RT-PCR positivity is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody response (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2 infection.” | concepts[1] |
| concept | Siemens assay — not defined in this paper's own words; the platform with the lowest projected error rate at 10% seroprevalence - 2800 per million tests — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | concepts[2] |
| concept | DiaSorin assay — not defined in this paper's own words; the platform with the highest - 13 700 total errors per million tests at the same seroprevalence — “However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests.” | concepts[3] |
| concept | EUROIMMUN assay — not defined in this paper's own words; the assay excluded from the comparison before it began, on a prior report of approximately 90% sensitivity — “However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison.” | concepts[4] |
| concept | global pandemic — not defined in this paper's own words; worldwide SARS-CoV-2 outbreak context — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[5] |
| concept | Serology — not defined in this paper's own words; antibody testing used for surveillance and risk assessment — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[6] |
| concept | population-level surveillance — not defined in this paper's own words; population use case for serology — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[7] |
| concept | individual-level risk assessment — not defined in this paper's own words; potential individual use case for serology — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[8] |
| concept | head-to-head assessment — not defined in this paper's own words; same-sample comparison of five assays — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[9] |
| concept | four high-throughput commercial SARS-CoV-2 antibody immunoassays — not defined in this paper's own words; commercial platform group in the benchmark — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[10] |
| concept | novel 384-well ELISA — not defined in this paper's own words; academic scalable platform compared with the commercial assays — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[11] |
| concept | Oxford immunoassay — not defined in this paper's own words; name for the novel 384-well ELISA — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[12] |
| concept | pre-pandemic blood samples — not defined in this paper's own words; known-negative panel collected before SARS-CoV-2 — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[13] |
| concept | laboratory-confirmed SARS-CoV-2 infection — not defined in this paper's own words; RT-PCR-confirmed source condition for known-positive samples — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[14] |
| concept | Receiver operating characteristic (ROC) curves — not defined in this paper's own words; curves used to assess decision thresholds — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[15] |
| concept | assay thresholds — not defined in this paper's own words; decision cut points assessed and optimised — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[16] |
| concept | sensitivity — not defined in this paper's own words; true-positive performance metric — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[17] |
| concept | specificity — not defined in this paper's own words; true-negative performance metric — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a global pandemic in 2020. Testing is crucial for mitigating public health and economic effects. Serology is considered key to population-level surveillance and potentially individual-level risk assessment. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay). We derived sensitivity and specificity from 976 pre-pandemic blood samples (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curves were used to assess assay thresholds. Findings At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%). All assays achieved a sensitivity of at least 98% with thresholds optimised to achieve a specificity of at least 98% on samples taken 30 days or more post symptom onset. Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity of at least 98%. The Siemens assay and Oxford immunoassay achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health. Funding Public Health England and UK National Institute for Health Research” | concepts[18] |
| concept | Severe acute respiratory syndrome coronavirus 2 — not defined in this paper's own words; expanded pathogen name — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[19] |
| concept | SARS-CoV-2 — not defined in this paper's own words; abbreviation for the pathogen — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[20] |
| concept | direct detection of the virus — not defined in this paper's own words; first broad form of testing — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[21] |
| concept | real-time RT-PCR — not defined in this paper's own words; respiratory-sample method for direct virus detection — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[22] |
| concept | presence of antibodies — not defined in this paper's own words; target investigated by serology — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[23] |
| concept | specific types of antibody — not defined in this paper's own words; IgM or IgG targets of some immunoassays — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[24] |
| concept | total antibody — not defined in this paper's own words; combined antibody target of other immunoassays — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[25] |
| concept | early acute infection — not defined in this paper's own words; setting in which delayed antibodies are unreliable — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[26] |
| concept | population exposure — not defined in this paper's own words; quantity informed by population-level serology — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[27] |
| concept | global laboratory infrastructures — not defined in this paper's own words; installed systems compatible with commercial immunoassays — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[28] |
| concept | population-wide testing — not defined in this paper's own words; regular large-scale testing use case — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[29] |
| concept | typical assessment criteria — not defined in this paper's own words; standards relaxed under urgent demand and limited evidence — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[30] |
| concept | Emergency Use Authorization programme — not defined in this paper's own words; example of relaxed emergency assessment — “Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as a novel human pathogen, causing a global pandemic in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen. SARS-CoV-2 testing broadly takes two forms: first, direct detection of the virus in respiratory samples with real-time RT-PCR; and second, by using serology to investigate the presence of antibodies. 2 Immunoassays detect either specific types of antibody (eg, IgM or IgG) or total antibody. SARS-CoV-2 antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infection. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposure. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responses in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testing (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteria (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programme)” | concepts[31] |
| concept | regulatory approvals for emergency use — not defined in this paper's own words; approval status used to select widely available assays — “Substantial global interest exists in the use of serology to enable population-level surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2 serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency use in the USA and the EU include the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxiv up to May 31, 2020, using the following search terms: (SARS-CoV-2) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviews were done for 124 articles. Eight studies provided data on sensitivity and specificity for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivity estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificity estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample sets” | concepts[32] |
| concept | PubMed, BioRxiv, and MedRxiv — not defined in this paper's own words; sources searched for prior assay evaluations — “Substantial global interest exists in the use of serology to enable population-level surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2 serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency use in the USA and the EU include the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxiv up to May 31, 2020, using the following search terms: (SARS-CoV-2) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviews were done for 124 articles. Eight studies provided data on sensitivity and specificity for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivity estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificity estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample sets” | concepts[33] |
| concept | full-text reviews — not defined in this paper's own words; detailed evidence review step applied to 124 articles — “Substantial global interest exists in the use of serology to enable population-level surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2 serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency use in the USA and the EU include the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxiv up to May 31, 2020, using the following search terms: (SARS-CoV-2) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviews were done for 124 articles. Eight studies provided data on sensitivity and specificity for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivity estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificity estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample sets” | concepts[34] |
| concept | same sample sets — not defined in this paper's own words; comparability condition absent from prior four-platform studies — “Substantial global interest exists in the use of serology to enable population-level surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2 serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency use in the USA and the EU include the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxiv up to May 31, 2020, using the following search terms: (SARS-CoV-2) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviews were done for 124 articles. Eight studies provided data on sensitivity and specificity for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivity estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificity estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample sets” | concepts[35] |
| concept | SARS-CoV-2 Total assay — not defined in this paper's own words; Siemens platform selected for the head-to-head comparison — “Substantial global interest exists in the use of serology to enable population-level surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2 serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency use in the USA and the EU include the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivity of the EUROIMMUN assay has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assay (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxiv up to May 31, 2020, using the following search terms: (SARS-CoV-2) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviews were done for 124 articles. Eight studies provided data on sensitivity and specificity for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivity ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificity from 95·1% to 100% (nine to 1020); for the DiaSorin assay, we found a single sensitivity estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificity ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivity estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificity estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample sets” | concepts[36] |
| concept | large, single sample set of more than 1500 samples — not defined in this paper's own words; shared sample set used for direct comparison — “In this study on a large, single sample set of more than 1500 samples, we compared four commercial SARS-CoV-2 immunoassays (Abbott, DiaSorin, Roche, and Siemens assays) with a global installed base, and a novel 384-well ELISA (the Oxford immunoassay) that could scale to be used for population-level surveillance. The Siemens assay and Oxford immunoassay achieved a sensitivity and specificity of at least 98% without optimisation (lower 95% CI ≥96%). The other assays could achieve sensitivities and specificities of at least 98% with assay threshold adjustment or deployment on samples taken at least 30 days after symptom onset. However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests. Implications of all the available evidence Understanding the comparative performance of immunoassays is crucial to the development of appropriate individual and population-level SARS-CoV-2 testing strategies. We showed that although precise performance metrics varied between immunoassay platforms, all assays that we assessed could be usefully deployed with careful consideration of use case, assay thresholds, and by considering symptom-to-sample timings, thus optimising the use of available serological testing resource and enabling the most widespread rollout” | concepts[37] |
| concept | global installed base — not defined in this paper's own words; existing commercial-analyser footprint — “In this study on a large, single sample set of more than 1500 samples, we compared four commercial SARS-CoV-2 immunoassays (Abbott, DiaSorin, Roche, and Siemens assays) with a global installed base, and a novel 384-well ELISA (the Oxford immunoassay) that could scale to be used for population-level surveillance. The Siemens assay and Oxford immunoassay achieved a sensitivity and specificity of at least 98% without optimisation (lower 95% CI ≥96%). The other assays could achieve sensitivities and specificities of at least 98% with assay threshold adjustment or deployment on samples taken at least 30 days after symptom onset. However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests. Implications of all the available evidence Understanding the comparative performance of immunoassays is crucial to the development of appropriate individual and population-level SARS-CoV-2 testing strategies. We showed that although precise performance metrics varied between immunoassay platforms, all assays that we assessed could be usefully deployed with careful consideration of use case, assay thresholds, and by considering symptom-to-sample timings, thus optimising the use of available serological testing resource and enabling the most widespread rollout” | concepts[38] |
| concept | assay threshold adjustment — not defined in this paper's own words; post-measurement optimisation route — “In this study on a large, single sample set of more than 1500 samples, we compared four commercial SARS-CoV-2 immunoassays (Abbott, DiaSorin, Roche, and Siemens assays) with a global installed base, and a novel 384-well ELISA (the Oxford immunoassay) that could scale to be used for population-level surveillance. The Siemens assay and Oxford immunoassay achieved a sensitivity and specificity of at least 98% without optimisation (lower 95% CI ≥96%). The other assays could achieve sensitivities and specificities of at least 98% with assay threshold adjustment or deployment on samples taken at least 30 days after symptom onset. However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests. Implications of all the available evidence Understanding the comparative performance of immunoassays is crucial to the development of appropriate individual and population-level SARS-CoV-2 testing strategies. We showed that although precise performance metrics varied between immunoassay platforms, all assays that we assessed could be usefully deployed with careful consideration of use case, assay thresholds, and by considering symptom-to-sample timings, thus optimising the use of available serological testing resource and enabling the most widespread rollout” | concepts[39] |
| concept | symptom-to-sample timings — not defined in this paper's own words; time condition affecting assay performance — “In this study on a large, single sample set of more than 1500 samples, we compared four commercial SARS-CoV-2 immunoassays (Abbott, DiaSorin, Roche, and Siemens assays) with a global installed base, and a novel 384-well ELISA (the Oxford immunoassay) that could scale to be used for population-level surveillance. The Siemens assay and Oxford immunoassay achieved a sensitivity and specificity of at least 98% without optimisation (lower 95% CI ≥96%). The other assays could achieve sensitivities and specificities of at least 98% with assay threshold adjustment or deployment on samples taken at least 30 days after symptom onset. However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests. Implications of all the available evidence Understanding the comparative performance of immunoassays is crucial to the development of appropriate individual and population-level SARS-CoV-2 testing strategies. We showed that although precise performance metrics varied between immunoassay platforms, all assays that we assessed could be usefully deployed with careful consideration of use case, assay thresholds, and by considering symptom-to-sample timings, thus optimising the use of available serological testing resource and enabling the most widespread rollout” | concepts[40] |
| concept | serological testing resource — not defined in this paper's own words; testing capacity allocated by platform and use case — “In this study on a large, single sample set of more than 1500 samples, we compared four commercial SARS-CoV-2 immunoassays (Abbott, DiaSorin, Roche, and Siemens assays) with a global installed base, and a novel 384-well ELISA (the Oxford immunoassay) that could scale to be used for population-level surveillance. The Siemens assay and Oxford immunoassay achieved a sensitivity and specificity of at least 98% without optimisation (lower 95% CI ≥96%). The other assays could achieve sensitivities and specificities of at least 98% with assay threshold adjustment or deployment on samples taken at least 30 days after symptom onset. However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assay would generate an estimated 2800 total errors per million tests, versus the DiaSorin assay at a rate of 13 700 total errors per million tests. Implications of all the available evidence Understanding the comparative performance of immunoassays is crucial to the development of appropriate individual and population-level SARS-CoV-2 testing strategies. We showed that although precise performance metrics varied between immunoassay platforms, all assays that we assessed could be usefully deployed with careful consideration of use case, assay thresholds, and by considering symptom-to-sample timings, thus optimising the use of available serological testing resource and enabling the most widespread rollout” | concepts[41] |
| concept | blood (plasma and serum) sample collection — not defined in this paper's own words; benchmark collection containing negative and positive panels — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[42] |
| concept | pre-pandemic samples from healthy individuals aged at least 18 years — not defined in this paper's own words; negative-panel component of the collection — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[43] |
| concept | samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection — not defined in this paper's own words; positive-panel component of the collection — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[44] |
| concept | known negative samples — not defined in this paper's own words; pre-appearance samples used for specificity — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[45] |
| concept | known positive samples — not defined in this paper's own words; RT-PCR-positive post-symptom samples used for sensitivity — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[46] |
| concept | de-duplicated by individual — not defined in this paper's own words; one-person inclusion rule — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[47] |
| concept | latest sample from each individual — not defined in this paper's own words; within-person sample selected for analysis — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[48] |
| concept | eight-point 1:2 dilution series — not defined in this paper's own words; serial-dilution assessment of high, medium, and low titres — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[49] |
| concept | SARS-CoV-2 IgG antibodies — not defined in this paper's own words; antibodies represented at high, medium, and low titres in the dilution series — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[50] |
| concept | optical density — not defined in this paper's own words; sample signal divided by calibrator signal for the EUROIMMUN ratio — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[51] |
| concept | protocol before implementing the study — not defined in this paper's own words; prospectively drafted protocol shared with manufacturers and government — “To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2 immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collection including pre-pandemic samples from healthy individuals aged at least 18 years, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infection from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samples to be taken more than 6 months before the known appearance of SARS-CoV-2 (ie, earlier than July 2019), and known positive samples to be collected from individuals with a previous positive SARS-CoV-2 RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analyses. In each case samples were de-duplicated by individual, and the latest sample from each individual was analysed. Two assays (SARS-CoV-2 IgG assay [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2 S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assay [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down. Additionally, we analysed an eight-point 1:2 dilution series of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodies as assessed by the SARS-CoV-2 IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical density of the sample divided by the optical density of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassay. The NHSBT samples were correlated with national SARS-CoV-2 reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titre], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassay ( appendix p 10 ). We drafted a protocol before implementing the study, which was shared with the manufacturers and the UK Department of Health and Social Care” | concepts[52] |
| concept | nucleocapsid protein — not defined in this paper's own words; target antigen for Abbott and Roche — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | concepts[53] |
| concept | spike protein-based targets — not defined in this paper's own words; target class for DiaSorin and Siemens — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | concepts[54] |
| concept | IgG assays — not defined in this paper's own words; Abbott and DiaSorin antibody-isotype class — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | concepts[55] |
| concept | total antibody assays — not defined in this paper's own words; Roche and Siemens antibody class — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | concepts[56] |
| concept | trained laboratory staff — not defined in this paper's own words; operators following manufacturer instructions — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | concepts[57] |
| concept | specified controls and calibrants — not defined in this paper's own words; manufacturer-specified assay controls — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | concepts[58] |
| concept | unique study barcodes — not defined in this paper's own words; sample identifiers supporting blinded operation — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | concepts[59] |
| concept | sample metadata — not defined in this paper's own words; information withheld from laboratory staff before assay runs — “The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid protein (Abbott and Roche) and two had spike protein-based targets (DiaSorin and Siemens); two were IgG assays (Abbott and DiaSorin), and two were total antibody assays (Roche and Siemens). Manufacturers' sensitivity estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificity estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staff in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrants, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodes for this study, and laboratory staff doing the assays did not have access to sample metadata before running the assays” | concepts[60] |
| concept | indirect ELISA — not defined in this paper's own words; assay class of the Oxford immunoassay — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[61] |
| concept | serum IgG — not defined in this paper's own words; antibody isotype measured by the Oxford assay — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[62] |
| concept | trimeric spike protein — not defined in this paper's own words; Oxford assay antigen — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[63] |
| concept | Readouts — not defined in this paper's own words; Oxford assay outputs transformed into standard units — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[64] |
| concept | fluorescent signal — not defined in this paper's own words; raw readout form — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[65] |
| concept | natural cubic spline-based regression model — not defined in this paper's own words; calibration model enabling plate and batch comparison — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[66] |
| concept | across-plate and across-batch comparisons — not defined in this paper's own words; comparisons enabled by signal normalisation — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[67] |
| concept | 384-well plates — not defined in this paper's own words; high-throughput assay format — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[68] |
| concept | Janus automated liquid handler — not defined in this paper's own words; automated sample and plate handler — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[69] |
| concept | EnVision 2104 multilabel plate reader — not defined in this paper's own words; instrument used to read assay plates — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[70] |
| concept | low input sample volumes — not defined in this paper's own words; approximately two-microlitre serum requirement — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[71] |
| concept | dead volume requirements — not defined in this paper's own words; additional volume required for liquid handling — “The Oxford immunoassay is an indirect ELISA, measuring serum IgG against trimeric spike protein, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readouts were measured as a fluorescent signal, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression model to enable across-plate and across-batch comparisons ( appendix p 10 ). We derived thresholds for the Oxford immunoassay using an independent set of serum samples from 44 individuals with acute SARS-CoV-2 infection; 99 convalescent individuals post SARS-CoV-2 infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivity was 100% (120 of 120 [95% CI 97·0–100·0]) and specificity (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9). The Oxford immunoassay runs batches of up to 12 384-well plates, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handler (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate reader (Perkin Elmer). The assay requires low input sample volumes of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirements of approximately 10 μL are required for adequate liquid handling function. Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study” | concepts[72] |
| concept | sample size — not defined in this paper's own words; prospectively powered positive and negative panel sizes — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[73] |
| concept | 80% power — not defined in this paper's own words; prospective detection power — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[74] |
| concept | exact binomial 95% CIs — not defined in this paper's own words; uncertainty intervals for sensitivity and specificity — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[75] |
| concept | UK MHRA guidance — not defined in this paper's own words; performance benchmark requiring point and interval thresholds — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[76] |
| concept | primary analysis — not defined in this paper's own words; complete-case cross-platform analysis — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[77] |
| concept | secondary analyses — not defined in this paper's own words; all-sample, timing, severity, and threshold analyses — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[78] |
| concept | receiver operator characteristic (ROC) curves — not defined in this paper's own words; method defining sensitivity-specificity trade-offs — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[79] |
| concept | symptom severity — not defined in this paper's own words; clinical subgroup variable — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[80] |
| concept | asymptomatic, mild, severe, critical or death — not defined in this paper's own words; WHO-aligned severity categories — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[81] |
| concept | Positive and negative predictive values — not defined in this paper's own words; prevalence-dependent applied metrics — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[82] |
| concept | population prevalences of 5%, 10%, 20%, and 50% — not defined in this paper's own words; prevalence scenarios used to model predictive values — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[83] |
| concept | antibody titres — not defined in this paper's own words; quantitative response trajectory — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[84] |
| concept | spline curves — not defined in this paper's own words; models for antibody-titre trajectories — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[85] |
| concept | Akaike Information Criterion — not defined in this paper's own words; criterion selecting spline complexity — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[86] |
| concept | external review group — not defined in this paper's own words; independent group overseeing analyses and results — “Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample size was prospectively calculated to provide 80% power to detect a lower 95% CI of at least 96·0% if the actual assay sensitivity was 98·3% (which required ≥460 positive samples) and 80% power to detect a lower 95% CI of at least 96·0% if the actual assay specificity was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics. The data were collated, cleaned, and locked before analysis. Sensitivity and specificity with exact binomial 95% CIs for each assay were compared with current UK MHRA guidance 6 stipulating a requirement for sensitivity and specificity of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysis, and for all samples as part of secondary analyses. When more than one sample was available from the same individual, samples were de-duplicated by individual and only the last sample obtained was analysed, unless otherwise indicated. Other secondary analyses were done; first, using receiver operator characteristic (ROC) curves to define trade-offs in assay sensitivity and specificity. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivity with respect to symptom severity (defined as asymptomatic, mild, severe, critical or death, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive values at population prevalences of 5%, 10%, 20%, and 50% previous SARS-CoV-2 infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serology platform was assessed; for this analysis all samples were included. Trajectories of antibody titres with respect to days post symptom onset were modelled by fitting spline curves (three to five knots; model selection based on the Akaike Information Criterion); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review group” | concepts[87] |
| concept | known negative pre-pandemic samples — not defined in this paper's own words; specificity reference set — “From 1000 pre-pandemic samples (collected between Sept 4, 2014, and Oct 4, 2016), and 769 samples from 673 individuals with laboratory-confirmed SARS-CoV-2 infection (collected between Feb 1, and May 31, 2020), assay sensitivity and specificity were assessed on 976 known negative pre-pandemic samples and 536 samples from RT-PCR-confirmed COVID-19 patients, de-duplicated by individual for whom complete data across all platforms were available ( appendix p 24 ). No samples failed testing on any of the four commercial platforms. We found 18 liquid handling failures on the Oxford immunoassay resulting in empty wells. Sensitivity and specificity of each of the assays are shown in the table and figure 1 . Similar results were obtained analysing all available samples ( appendix p 27 ; table ). The clearest separation of known positive and known negative samples was shown for the Siemens assay” | concepts[88] |
| concept | known positive samples from patients who had previously had laboratory-confirmed SARS–CoV–2 infection — not defined in this paper's own words; sensitivity reference set — “Specificity was assessed using the known negative pre-pandemic samples, and sensitivity using the known positive samples from patients who had previously had laboratory-confirmed SARS–CoV–2 infection” | concepts[89] |
| concept | liquid handling failures — not defined in this paper's own words; Oxford-assay failures producing empty wells — “From 1000 pre-pandemic samples (collected between Sept 4, 2014, and Oct 4, 2016), and 769 samples from 673 individuals with laboratory-confirmed SARS-CoV-2 infection (collected between Feb 1, and May 31, 2020), assay sensitivity and specificity were assessed on 976 known negative pre-pandemic samples and 536 samples from RT-PCR-confirmed COVID-19 patients, de-duplicated by individual for whom complete data across all platforms were available ( appendix p 24 ). No samples failed testing on any of the four commercial platforms. We found 18 liquid handling failures on the Oxford immunoassay resulting in empty wells. Sensitivity and specificity of each of the assays are shown in the table and figure 1 . Similar results were obtained analysing all available samples ( appendix p 27 ; table ). The clearest separation of known positive and known negative samples was shown for the Siemens assay” | concepts[90] |
| concept | pre-defined assay thresholds — not defined in this paper's own words; manufacturer or prespecified decision rules — “Using the pre-defined assay thresholds for calling test results positive or negative, only the Siemens assay and the Oxford immunoassay met the specificity and sensitivity target of at least 98%, achieving this threshold at all three timepoints assessed (≥14, ≥20, and ≥30 days post symptom onset; appendix pp 28, 29 ; figure 1 ; table ). The Roche assay also met the specificity and sensitivity target of at least 98% on samples taken at least 30 days post symptom onset ( appendix p 29 ; table ). ROC curve analysis showed that the Roche assay could also potentially meet the sensitivity and specificity targets on samples taken at least 14 and at least 20 days post symptom onset through threshold adjustment—eg, adjusting the threshold to at least 0·128 as opposed to the current threshold of more than 1·0 would result in a sensitivity and specificity of at least 98% (with a lower bound of the 95% CI of ≥96%) at 20 days or more post symptom onset ( figure 3 ; appendix p 30 for samples taken ≥14 days post symptom onset; table ). On samples taken at least 30 days post symptom onset (but not ≥14 or ≥20 days post symptom onset), the Abbott and DiaSorin assay thresholds could be optimised to achieve sensitivity and specificity of at least 98% (with a lower bound of ≥96%; appendix p 31 ; table ). Overall, on samples taken at least 30 days post symptom onset, optimisation of assay thresholds to ensure a specificity of at least 98% (with a lower bound of ≥96%) would result in sensitivities of at least 98% (with a lower bound of ≥96%) for all assays assessed” | concepts[91] |
| concept | threshold adjustment — not defined in this paper's own words; route for Roche, Abbott, and DiaSorin to reach performance targets — “Using the pre-defined assay thresholds for calling test results positive or negative, only the Siemens assay and the Oxford immunoassay met the specificity and sensitivity target of at least 98%, achieving this threshold at all three timepoints assessed (≥14, ≥20, and ≥30 days post symptom onset; appendix pp 28, 29 ; figure 1 ; table ). The Roche assay also met the specificity and sensitivity target of at least 98% on samples taken at least 30 days post symptom onset ( appendix p 29 ; table ). ROC curve analysis showed that the Roche assay could also potentially meet the sensitivity and specificity targets on samples taken at least 14 and at least 20 days post symptom onset through threshold adjustment—eg, adjusting the threshold to at least 0·128 as opposed to the current threshold of more than 1·0 would result in a sensitivity and specificity of at least 98% (with a lower bound of the 95% CI of ≥96%) at 20 days or more post symptom onset ( figure 3 ; appendix p 30 for samples taken ≥14 days post symptom onset; table ). On samples taken at least 30 days post symptom onset (but not ≥14 or ≥20 days post symptom onset), the Abbott and DiaSorin assay thresholds could be optimised to achieve sensitivity and specificity of at least 98% (with a lower bound of ≥96%; appendix p 31 ; table ). Overall, on samples taken at least 30 days post symptom onset, optimisation of assay thresholds to ensure a specificity of at least 98% (with a lower bound of ≥96%) would result in sensitivities of at least 98% (with a lower bound of ≥96%) for all assays assessed” | concepts[92] |
| concept | samples taken at least 30 days post symptom onset — not defined in this paper's own words; timing condition under which all assays could meet targets — “Using the pre-defined assay thresholds for calling test results positive or negative, only the Siemens assay and the Oxford immunoassay met the specificity and sensitivity target of at least 98%, achieving this threshold at all three timepoints assessed (≥14, ≥20, and ≥30 days post symptom onset; appendix pp 28, 29 ; figure 1 ; table ). The Roche assay also met the specificity and sensitivity target of at least 98% on samples taken at least 30 days post symptom onset ( appendix p 29 ; table ). ROC curve analysis showed that the Roche assay could also potentially meet the sensitivity and specificity targets on samples taken at least 14 and at least 20 days post symptom onset through threshold adjustment—eg, adjusting the threshold to at least 0·128 as opposed to the current threshold of more than 1·0 would result in a sensitivity and specificity of at least 98% (with a lower bound of the 95% CI of ≥96%) at 20 days or more post symptom onset ( figure 3 ; appendix p 30 for samples taken ≥14 days post symptom onset; table ). On samples taken at least 30 days post symptom onset (but not ≥14 or ≥20 days post symptom onset), the Abbott and DiaSorin assay thresholds could be optimised to achieve sensitivity and specificity of at least 98% (with a lower bound of ≥96%; appendix p 31 ; table ). Overall, on samples taken at least 30 days post symptom onset, optimisation of assay thresholds to ensure a specificity of at least 98% (with a lower bound of ≥96%) would result in sensitivities of at least 98% (with a lower bound of ≥96%) for all assays assessed” | concepts[93] |
| concept | false positive tests per million tests — not defined in this paper's own words; prevalence-specific error quantity — “Although sensitivity and specificity are useful, positive and negative predictive values are more directly relevant to public health and clinical application. At 10% prevalence, the high specificity platforms (Abbott, Roche, and Siemens assays) had lower numbers of false positive tests per million tests compared with other assays (DiaSorin and Oxford immunoassay; figure 4 ; appendix p 21 ). However, the number of false negative tests per million was higher with the Abbott and DiaSorin assays and lowest with the Oxford immunoassay ( figure 4 ; appendix p 21 ). Overall, the Siemens assay had the fewest total errors per million tests for prevalences of 20% or lower” | concepts[94] |
| concept | false negative tests per million — not defined in this paper's own words; prevalence-specific error quantity — “Although sensitivity and specificity are useful, positive and negative predictive values are more directly relevant to public health and clinical application. At 10% prevalence, the high specificity platforms (Abbott, Roche, and Siemens assays) had lower numbers of false positive tests per million tests compared with other assays (DiaSorin and Oxford immunoassay; figure 4 ; appendix p 21 ). However, the number of false negative tests per million was higher with the Abbott and DiaSorin assays and lowest with the Oxford immunoassay ( figure 4 ; appendix p 21 ). Overall, the Siemens assay had the fewest total errors per million tests for prevalences of 20% or lower” | concepts[95] |
| concept | total errors per million tests — not defined in this paper's own words; combined error burden — “Although sensitivity and specificity are useful, positive and negative predictive values are more directly relevant to public health and clinical application. At 10% prevalence, the high specificity platforms (Abbott, Roche, and Siemens assays) had lower numbers of false positive tests per million tests compared with other assays (DiaSorin and Oxford immunoassay; figure 4 ; appendix p 21 ). However, the number of false negative tests per million was higher with the Abbott and DiaSorin assays and lowest with the Oxford immunoassay ( figure 4 ; appendix p 21 ). Overall, the Siemens assay had the fewest total errors per million tests for prevalences of 20% or lower” | concepts[96] |
| concept | prevalences of 20% or lower — not defined in this paper's own words; range in which Siemens had the fewest total errors — “Although sensitivity and specificity are useful, positive and negative predictive values are more directly relevant to public health and clinical application. At 10% prevalence, the high specificity platforms (Abbott, Roche, and Siemens assays) had lower numbers of false positive tests per million tests compared with other assays (DiaSorin and Oxford immunoassay; figure 4 ; appendix p 21 ). However, the number of false negative tests per million was higher with the Abbott and DiaSorin assays and lowest with the Oxford immunoassay ( figure 4 ; appendix p 21 ). Overall, the Siemens assay had the fewest total errors per million tests for prevalences of 20% or lower” | concepts[97] |
| concept | Antibody responses — not defined in this paper's own words; post-symptom response measured across assays — “Antibody responses increased over the first 3–4 weeks after symptom onset, with the maximum percentage of positive tests reached by day 27 across all assays ( appendix p 34 ). Antibody responses were sustained up to 73 days post symptom onset and up to 82 days post positive RT-PCR result ( appendix p 35 ); however, any declines in titres could not be specifically ruled out given that, by this stage, most quantitative results were at near-maximum values for each assay. For 158 (29%) of 536 individuals in the positive cohort for whom disease severity data were available, we found no evidence of a difference in immunoassay sensitivity by disease severity (asymptomatic n=13 [8%], mild n=122 [77%], severe n=16 [10%], critical or death n=7 [4%]) for any of the assays assessed ( appendix p 36 ); however numbers were small in the non-mild groups. We also undertook an analysis of discordance between assays ( appendix pp 22, 23 ). Most of the discordances were observed in samples taken before 20 days post symptom onset for SARS-CoV-2-positive patients confirmed by RT-PCR (37 [30%] of 124 samples taken <20 days post symptom onset vs 46 [9%] 538 samples taken ≥20 days post symptom onset; 27 [3%] of 976 pre-pandemic samples; p<0·0001). Three (<1%) of the samples classified as known positive (from two mild and one severe case) tested antibody-negative across all assays” | concepts[98] |
| concept | first 3–4 weeks after symptom onset — not defined in this paper's own words; period of increasing antibody responses — “Antibody responses increased over the first 3–4 weeks after symptom onset, with the maximum percentage of positive tests reached by day 27 across all assays ( appendix p 34 ). Antibody responses were sustained up to 73 days post symptom onset and up to 82 days post positive RT-PCR result ( appendix p 35 ); however, any declines in titres could not be specifically ruled out given that, by this stage, most quantitative results were at near-maximum values for each assay. For 158 (29%) of 536 individuals in the positive cohort for whom disease severity data were available, we found no evidence of a difference in immunoassay sensitivity by disease severity (asymptomatic n=13 [8%], mild n=122 [77%], severe n=16 [10%], critical or death n=7 [4%]) for any of the assays assessed ( appendix p 36 ); however numbers were small in the non-mild groups. We also undertook an analysis of discordance between assays ( appendix pp 22, 23 ). Most of the discordances were observed in samples taken before 20 days post symptom onset for SARS-CoV-2-positive patients confirmed by RT-PCR (37 [30%] of 124 samples taken <20 days post symptom onset vs 46 [9%] 538 samples taken ≥20 days post symptom onset; 27 [3%] of 976 pre-pandemic samples; p<0·0001). Three (<1%) of the samples classified as known positive (from two mild and one severe case) tested antibody-negative across all assays” | concepts[99] |
| concept | day 27 — not defined in this paper's own words; day by which positive-test percentage peaked — “Antibody responses increased over the first 3–4 weeks after symptom onset, with the maximum percentage of positive tests reached by day 27 across all assays ( appendix p 34 ). Antibody responses were sustained up to 73 days post symptom onset and up to 82 days post positive RT-PCR result ( appendix p 35 ); however, any declines in titres could not be specifically ruled out given that, by this stage, most quantitative results were at near-maximum values for each assay. For 158 (29%) of 536 individuals in the positive cohort for whom disease severity data were available, we found no evidence of a difference in immunoassay sensitivity by disease severity (asymptomatic n=13 [8%], mild n=122 [77%], severe n=16 [10%], critical or death n=7 [4%]) for any of the assays assessed ( appendix p 36 ); however numbers were small in the non-mild groups. We also undertook an analysis of discordance between assays ( appendix pp 22, 23 ). Most of the discordances were observed in samples taken before 20 days post symptom onset for SARS-CoV-2-positive patients confirmed by RT-PCR (37 [30%] of 124 samples taken <20 days post symptom onset vs 46 [9%] 538 samples taken ≥20 days post symptom onset; 27 [3%] of 976 pre-pandemic samples; p<0·0001). Three (<1%) of the samples classified as known positive (from two mild and one severe case) tested antibody-negative across all assays” | concepts[100] |
| concept | discordance between assays — not defined in this paper's own words; cross-platform disagreement outcome — “Antibody responses increased over the first 3–4 weeks after symptom onset, with the maximum percentage of positive tests reached by day 27 across all assays ( appendix p 34 ). Antibody responses were sustained up to 73 days post symptom onset and up to 82 days post positive RT-PCR result ( appendix p 35 ); however, any declines in titres could not be specifically ruled out given that, by this stage, most quantitative results were at near-maximum values for each assay. For 158 (29%) of 536 individuals in the positive cohort for whom disease severity data were available, we found no evidence of a difference in immunoassay sensitivity by disease severity (asymptomatic n=13 [8%], mild n=122 [77%], severe n=16 [10%], critical or death n=7 [4%]) for any of the assays assessed ( appendix p 36 ); however numbers were small in the non-mild groups. We also undertook an analysis of discordance between assays ( appendix pp 22, 23 ). Most of the discordances were observed in samples taken before 20 days post symptom onset for SARS-CoV-2-positive patients confirmed by RT-PCR (37 [30%] of 124 samples taken <20 days post symptom onset vs 46 [9%] 538 samples taken ≥20 days post symptom onset; 27 [3%] of 976 pre-pandemic samples; p<0·0001). Three (<1%) of the samples classified as known positive (from two mild and one severe case) tested antibody-negative across all assays” | concepts[101] |
| concept | centralised testing — not defined in this paper's own words; large-project use enabled by the Oxford platform — “On a large panel of blood samples, we showed that the Siemens assay and the Oxford immunoassay both achieved sensitivity and specificity of at least 98% (with a lower bound of the 95% CI ≥96%) on samples taken at least 20 days post symptom onset without optimisation. However, all assays could potentially achieve these specifications through threshold adjustment, or by assessing samples collected at least 30 days post symptom onset, consistent with the time-dependent nature of antibody responses. Additionally, we highlighted the potentially scalable capacity of the Oxford immunoassay, which could enable centralised testing for large-scale projects (eg, national seroprevalence studies). This finding suggests that global serology testing needs can be met using different assays, mitigating against the risk of shortages, and allowing immediate deployment in laboratories with different analysers already installed for other testing purposes. The optimal deployment of assays should reflect use case—eg, an optimised assay measuring antibodies to nucleocapsid protein might be most relevant for disease surveillance in the context of a widespread rollout of vaccines, which are mostly spike-protein based. Although sensitivity and specificity are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted” | concepts[102] |
| concept | use case — not defined in this paper's own words; deployment context that should govern assay selection — “On a large panel of blood samples, we showed that the Siemens assay and the Oxford immunoassay both achieved sensitivity and specificity of at least 98% (with a lower bound of the 95% CI ≥96%) on samples taken at least 20 days post symptom onset without optimisation. However, all assays could potentially achieve these specifications through threshold adjustment, or by assessing samples collected at least 30 days post symptom onset, consistent with the time-dependent nature of antibody responses. Additionally, we highlighted the potentially scalable capacity of the Oxford immunoassay, which could enable centralised testing for large-scale projects (eg, national seroprevalence studies). This finding suggests that global serology testing needs can be met using different assays, mitigating against the risk of shortages, and allowing immediate deployment in laboratories with different analysers already installed for other testing purposes. The optimal deployment of assays should reflect use case—eg, an optimised assay measuring antibodies to nucleocapsid protein might be most relevant for disease surveillance in the context of a widespread rollout of vaccines, which are mostly spike-protein based. Although sensitivity and specificity are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted” | concepts[103] |
| concept | disease surveillance — not defined in this paper's own words; nucleocapsid-assay use during spike-vaccine rollout — “On a large panel of blood samples, we showed that the Siemens assay and the Oxford immunoassay both achieved sensitivity and specificity of at least 98% (with a lower bound of the 95% CI ≥96%) on samples taken at least 20 days post symptom onset without optimisation. However, all assays could potentially achieve these specifications through threshold adjustment, or by assessing samples collected at least 30 days post symptom onset, consistent with the time-dependent nature of antibody responses. Additionally, we highlighted the potentially scalable capacity of the Oxford immunoassay, which could enable centralised testing for large-scale projects (eg, national seroprevalence studies). This finding suggests that global serology testing needs can be met using different assays, mitigating against the risk of shortages, and allowing immediate deployment in laboratories with different analysers already installed for other testing purposes. The optimal deployment of assays should reflect use case—eg, an optimised assay measuring antibodies to nucleocapsid protein might be most relevant for disease surveillance in the context of a widespread rollout of vaccines, which are mostly spike-protein based. Although sensitivity and specificity are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted” | concepts[104] |
| concept | widespread rollout of vaccines — not defined in this paper's own words; deployment context changing useful assay targets — “On a large panel of blood samples, we showed that the Siemens assay and the Oxford immunoassay both achieved sensitivity and specificity of at least 98% (with a lower bound of the 95% CI ≥96%) on samples taken at least 20 days post symptom onset without optimisation. However, all assays could potentially achieve these specifications through threshold adjustment, or by assessing samples collected at least 30 days post symptom onset, consistent with the time-dependent nature of antibody responses. Additionally, we highlighted the potentially scalable capacity of the Oxford immunoassay, which could enable centralised testing for large-scale projects (eg, national seroprevalence studies). This finding suggests that global serology testing needs can be met using different assays, mitigating against the risk of shortages, and allowing immediate deployment in laboratories with different analysers already installed for other testing purposes. The optimal deployment of assays should reflect use case—eg, an optimised assay measuring antibodies to nucleocapsid protein might be most relevant for disease surveillance in the context of a widespread rollout of vaccines, which are mostly spike-protein based. Although sensitivity and specificity are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted” | concepts[105] |
| concept | spike-protein based — not defined in this paper's own words; dominant vaccine target characteristic — “On a large panel of blood samples, we showed that the Siemens assay and the Oxford immunoassay both achieved sensitivity and specificity of at least 98% (with a lower bound of the 95% CI ≥96%) on samples taken at least 20 days post symptom onset without optimisation. However, all assays could potentially achieve these specifications through threshold adjustment, or by assessing samples collected at least 30 days post symptom onset, consistent with the time-dependent nature of antibody responses. Additionally, we highlighted the potentially scalable capacity of the Oxford immunoassay, which could enable centralised testing for large-scale projects (eg, national seroprevalence studies). This finding suggests that global serology testing needs can be met using different assays, mitigating against the risk of shortages, and allowing immediate deployment in laboratories with different analysers already installed for other testing purposes. The optimal deployment of assays should reflect use case—eg, an optimised assay measuring antibodies to nucleocapsid protein might be most relevant for disease surveillance in the context of a widespread rollout of vaccines, which are mostly spike-protein based. Although sensitivity and specificity are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted” | concepts[106] |
| concept | dynamic range of the assays — not defined in this paper's own words; upper/lower measurement limits constraining titre inference — “In our study, antibody titres within individuals were largely sustained beyond 2 months of symptom onset for all assays investigated, suggesting that serosurveillance with these assays is likely to be accurate within months of exposure at least. We did not observe any significant difference in assay sensitivity by disease severity; notably, however, the numbers in our non-mild disease category, including asymptomatic cases, were small, and we were limited by the bounds determined by the dynamic range of the assays and saturation. Inconsistencies have been observed in the association between antibody titres and disease severity in other studies, which might relate to assay target. 20 , 24 , 25 Further studies definitively assessing the longer-term trajectory of antibodies in larger longitudinal cohorts of individuals with different disease severities and taking the specific assay type into account would be of benefit. Although all these assays can effectively detect SARS-CoV-2 antibodies, the durability and nature of immunity conferred by these antibodies remains unclear. Few data are available on immunity to seasonal human coronaviruses (alphacoronaviruses NL63 and 229E, and betacoronaviruses HKU1 and OC43). In two small, controlled, human infection models, re-challenge at 8–12 months with 229E (nine individuals) 26 and 229E-related strains (six) 27 showed protection against symptoms, but virus shedding was detected in six (67%) of nine individuals. 26 In an observational study of NL63 in Kenya, reinfections over a 6-month period occurred in 46 (28%) of 163 individuals, with reinfections within 80 days largely associated (>80% cases) with lower viral titres and fewer symptoms compared with those associated with reinfections after more than 80 days. 28 Data on Middle East respiratory syndrome (MERS) and severe acute respiratory syndrome coronavirus suggest that antibody titres wane over periods of approximately 2 years, with more severe infection resulting in more measurable and durable antibody responses in MERS infection. 29 , 30 Animal data (re-challenge in four rhesus macaques) suggest protection from re-infection with SARS-CoV-2 extends to at least 28 days post infection. 31 The relationship between SARS-CoV-2 antibodies and neutralising activity remains an important research question. Because the spike protein is responsible for enabling cell entry, the spike protein appears to be the main antigen responsible for eliciting neutralising antibodies. In SARS-CoV-2, a correlation between anti-spike antibody titres when trimeric versions of spike are used as the antigen and neutralising capacity appears to be emerging. 24 , 32 If SARS-CoV-2 antibody titres are shown to clearly correlate directly or indirectly with immunity to infection or attenuation of disease severity, then more accurately evaluating titres by optimising the dynamic range of assays might become relevant” | concepts[107] |
| concept | neutralising activity — not defined in this paper's own words; functional antibody property related to spike titres — “In our study, antibody titres within individuals were largely sustained beyond 2 months of symptom onset for all assays investigated, suggesting that serosurveillance with these assays is likely to be accurate within months of exposure at least. We did not observe any significant difference in assay sensitivity by disease severity; notably, however, the numbers in our non-mild disease category, including asymptomatic cases, were small, and we were limited by the bounds determined by the dynamic range of the assays and saturation. Inconsistencies have been observed in the association between antibody titres and disease severity in other studies, which might relate to assay target. 20 , 24 , 25 Further studies definitively assessing the longer-term trajectory of antibodies in larger longitudinal cohorts of individuals with different disease severities and taking the specific assay type into account would be of benefit. Although all these assays can effectively detect SARS-CoV-2 antibodies, the durability and nature of immunity conferred by these antibodies remains unclear. Few data are available on immunity to seasonal human coronaviruses (alphacoronaviruses NL63 and 229E, and betacoronaviruses HKU1 and OC43). In two small, controlled, human infection models, re-challenge at 8–12 months with 229E (nine individuals) 26 and 229E-related strains (six) 27 showed protection against symptoms, but virus shedding was detected in six (67%) of nine individuals. 26 In an observational study of NL63 in Kenya, reinfections over a 6-month period occurred in 46 (28%) of 163 individuals, with reinfections within 80 days largely associated (>80% cases) with lower viral titres and fewer symptoms compared with those associated with reinfections after more than 80 days. 28 Data on Middle East respiratory syndrome (MERS) and severe acute respiratory syndrome coronavirus suggest that antibody titres wane over periods of approximately 2 years, with more severe infection resulting in more measurable and durable antibody responses in MERS infection. 29 , 30 Animal data (re-challenge in four rhesus macaques) suggest protection from re-infection with SARS-CoV-2 extends to at least 28 days post infection. 31 The relationship between SARS-CoV-2 antibodies and neutralising activity remains an important research question. Because the spike protein is responsible for enabling cell entry, the spike protein appears to be the main antigen responsible for eliciting neutralising antibodies. In SARS-CoV-2, a correlation between anti-spike antibody titres when trimeric versions of spike are used as the antigen and neutralising capacity appears to be emerging. 24 , 32 If SARS-CoV-2 antibody titres are shown to clearly correlate directly or indirectly with immunity to infection or attenuation of disease severity, then more accurately evaluating titres by optimising the dynamic range of assays might become relevant” | concepts[108] |
| concept | neutralising antibodies — not defined in this paper's own words; antibodies elicited against entry-enabling spike protein — “In our study, antibody titres within individuals were largely sustained beyond 2 months of symptom onset for all assays investigated, suggesting that serosurveillance with these assays is likely to be accurate within months of exposure at least. We did not observe any significant difference in assay sensitivity by disease severity; notably, however, the numbers in our non-mild disease category, including asymptomatic cases, were small, and we were limited by the bounds determined by the dynamic range of the assays and saturation. Inconsistencies have been observed in the association between antibody titres and disease severity in other studies, which might relate to assay target. 20 , 24 , 25 Further studies definitively assessing the longer-term trajectory of antibodies in larger longitudinal cohorts of individuals with different disease severities and taking the specific assay type into account would be of benefit. Although all these assays can effectively detect SARS-CoV-2 antibodies, the durability and nature of immunity conferred by these antibodies remains unclear. Few data are available on immunity to seasonal human coronaviruses (alphacoronaviruses NL63 and 229E, and betacoronaviruses HKU1 and OC43). In two small, controlled, human infection models, re-challenge at 8–12 months with 229E (nine individuals) 26 and 229E-related strains (six) 27 showed protection against symptoms, but virus shedding was detected in six (67%) of nine individuals. 26 In an observational study of NL63 in Kenya, reinfections over a 6-month period occurred in 46 (28%) of 163 individuals, with reinfections within 80 days largely associated (>80% cases) with lower viral titres and fewer symptoms compared with those associated with reinfections after more than 80 days. 28 Data on Middle East respiratory syndrome (MERS) and severe acute respiratory syndrome coronavirus suggest that antibody titres wane over periods of approximately 2 years, with more severe infection resulting in more measurable and durable antibody responses in MERS infection. 29 , 30 Animal data (re-challenge in four rhesus macaques) suggest protection from re-infection with SARS-CoV-2 extends to at least 28 days post infection. 31 The relationship between SARS-CoV-2 antibodies and neutralising activity remains an important research question. Because the spike protein is responsible for enabling cell entry, the spike protein appears to be the main antigen responsible for eliciting neutralising antibodies. In SARS-CoV-2, a correlation between anti-spike antibody titres when trimeric versions of spike are used as the antigen and neutralising capacity appears to be emerging. 24 , 32 If SARS-CoV-2 antibody titres are shown to clearly correlate directly or indirectly with immunity to infection or attenuation of disease severity, then more accurately evaluating titres by optimising the dynamic range of assays might become relevant” | concepts[109] |
| concept | immunity to infection — not defined in this paper's own words; clinical protection not established by antibody detection alone — “In our study, antibody titres within individuals were largely sustained beyond 2 months of symptom onset for all assays investigated, suggesting that serosurveillance with these assays is likely to be accurate within months of exposure at least. We did not observe any significant difference in assay sensitivity by disease severity; notably, however, the numbers in our non-mild disease category, including asymptomatic cases, were small, and we were limited by the bounds determined by the dynamic range of the assays and saturation. Inconsistencies have been observed in the association between antibody titres and disease severity in other studies, which might relate to assay target. 20 , 24 , 25 Further studies definitively assessing the longer-term trajectory of antibodies in larger longitudinal cohorts of individuals with different disease severities and taking the specific assay type into account would be of benefit. Although all these assays can effectively detect SARS-CoV-2 antibodies, the durability and nature of immunity conferred by these antibodies remains unclear. Few data are available on immunity to seasonal human coronaviruses (alphacoronaviruses NL63 and 229E, and betacoronaviruses HKU1 and OC43). In two small, controlled, human infection models, re-challenge at 8–12 months with 229E (nine individuals) 26 and 229E-related strains (six) 27 showed protection against symptoms, but virus shedding was detected in six (67%) of nine individuals. 26 In an observational study of NL63 in Kenya, reinfections over a 6-month period occurred in 46 (28%) of 163 individuals, with reinfections within 80 days largely associated (>80% cases) with lower viral titres and fewer symptoms compared with those associated with reinfections after more than 80 days. 28 Data on Middle East respiratory syndrome (MERS) and severe acute respiratory syndrome coronavirus suggest that antibody titres wane over periods of approximately 2 years, with more severe infection resulting in more measurable and durable antibody responses in MERS infection. 29 , 30 Animal data (re-challenge in four rhesus macaques) suggest protection from re-infection with SARS-CoV-2 extends to at least 28 days post infection. 31 The relationship between SARS-CoV-2 antibodies and neutralising activity remains an important research question. Because the spike protein is responsible for enabling cell entry, the spike protein appears to be the main antigen responsible for eliciting neutralising antibodies. In SARS-CoV-2, a correlation between anti-spike antibody titres when trimeric versions of spike are used as the antigen and neutralising capacity appears to be emerging. 24 , 32 If SARS-CoV-2 antibody titres are shown to clearly correlate directly or indirectly with immunity to infection or attenuation of disease severity, then more accurately evaluating titres by optimising the dynamic range of assays might become relevant” | concepts[110] |
| concept | sample volumes — not defined in this paper's own words; constraint preventing repeat analyses — “Our study was limited by sample volumes, especially given the constraints imposed by dead volume requirements for liquid handling, and we were unable to do repeat analyses. Subgroup analysis by timing of collection or disease severity was constrained by small numbers and larger studies would be of value. Neutralisation and pseudo-neutralisation assays provide an in-vitro approach to investigating antibodies to help address the question as to whether antibody titres correlate with functionality; this work is ongoing. Longitudinal prospective clinical studies are also needed to investigate whether repeat infections can arise in antibody-positive individuals. Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection. Additional performance assessments in these groups would be warranted. Although assessments of precision and reproducibility have to some extent been carried out by the immunoassay manufacturers, a complete validation of the individual assays was beyond the scope of our study. We did not investigate our large negative sample set for the presence of antibodies against seasonal coronaviruses, the presence of which might have accounted for some of the false-positive results. However, previous manufacturers' assessments for the Abbott and Roche immunoassays, and our assessment as part of this study for the Oxford immunoassay on small panels of sera taken from patients with respiratory virus infections including seasonal coronavirus infections, suggest good analytical specificity” | concepts[111] |
| concept | dead volume requirements for liquid handling — not defined in this paper's own words; specific source of sample-volume constraint — “Our study was limited by sample volumes, especially given the constraints imposed by dead volume requirements for liquid handling, and we were unable to do repeat analyses. Subgroup analysis by timing of collection or disease severity was constrained by small numbers and larger studies would be of value. Neutralisation and pseudo-neutralisation assays provide an in-vitro approach to investigating antibodies to help address the question as to whether antibody titres correlate with functionality; this work is ongoing. Longitudinal prospective clinical studies are also needed to investigate whether repeat infections can arise in antibody-positive individuals. Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection. Additional performance assessments in these groups would be warranted. Although assessments of precision and reproducibility have to some extent been carried out by the immunoassay manufacturers, a complete validation of the individual assays was beyond the scope of our study. We did not investigate our large negative sample set for the presence of antibodies against seasonal coronaviruses, the presence of which might have accounted for some of the false-positive results. However, previous manufacturers' assessments for the Abbott and Roche immunoassays, and our assessment as part of this study for the Oxford immunoassay on small panels of sera taken from patients with respiratory virus infections including seasonal coronavirus infections, suggest good analytical specificity” | concepts[112] |
| concept | Neutralisation and pseudo-neutralisation assays — not defined in this paper's own words; in-vitro approach to antibody functionality — “Our study was limited by sample volumes, especially given the constraints imposed by dead volume requirements for liquid handling, and we were unable to do repeat analyses. Subgroup analysis by timing of collection or disease severity was constrained by small numbers and larger studies would be of value. Neutralisation and pseudo-neutralisation assays provide an in-vitro approach to investigating antibodies to help address the question as to whether antibody titres correlate with functionality; this work is ongoing. Longitudinal prospective clinical studies are also needed to investigate whether repeat infections can arise in antibody-positive individuals. Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection. Additional performance assessments in these groups would be warranted. Although assessments of precision and reproducibility have to some extent been carried out by the immunoassay manufacturers, a complete validation of the individual assays was beyond the scope of our study. We did not investigate our large negative sample set for the presence of antibodies against seasonal coronaviruses, the presence of which might have accounted for some of the false-positive results. However, previous manufacturers' assessments for the Abbott and Roche immunoassays, and our assessment as part of this study for the Oxford immunoassay on small panels of sera taken from patients with respiratory virus infections including seasonal coronavirus infections, suggest good analytical specificity” | concepts[113] |
| concept | Longitudinal prospective clinical studies — not defined in this paper's own words; design needed to test repeat infection among antibody-positive people — “Our study was limited by sample volumes, especially given the constraints imposed by dead volume requirements for liquid handling, and we were unable to do repeat analyses. Subgroup analysis by timing of collection or disease severity was constrained by small numbers and larger studies would be of value. Neutralisation and pseudo-neutralisation assays provide an in-vitro approach to investigating antibodies to help address the question as to whether antibody titres correlate with functionality; this work is ongoing. Longitudinal prospective clinical studies are also needed to investigate whether repeat infections can arise in antibody-positive individuals. Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection. Additional performance assessments in these groups would be warranted. Although assessments of precision and reproducibility have to some extent been carried out by the immunoassay manufacturers, a complete validation of the individual assays was beyond the scope of our study. We did not investigate our large negative sample set for the presence of antibodies against seasonal coronaviruses, the presence of which might have accounted for some of the false-positive results. However, previous manufacturers' assessments for the Abbott and Roche immunoassays, and our assessment as part of this study for the Oxford immunoassay on small panels of sera taken from patients with respiratory virus infections including seasonal coronavirus infections, suggest good analytical specificity” | concepts[114] |
| concept | insufficient clinical meta-data — not defined in this paper's own words; sample-description limitation — “Our study was limited by sample volumes, especially given the constraints imposed by dead volume requirements for liquid handling, and we were unable to do repeat analyses. Subgroup analysis by timing of collection or disease severity was constrained by small numbers and larger studies would be of value. Neutralisation and pseudo-neutralisation assays provide an in-vitro approach to investigating antibodies to help address the question as to whether antibody titres correlate with functionality; this work is ongoing. Longitudinal prospective clinical studies are also needed to investigate whether repeat infections can arise in antibody-positive individuals. Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection. Additional performance assessments in these groups would be warranted. Although assessments of precision and reproducibility have to some extent been carried out by the immunoassay manufacturers, a complete validation of the individual assays was beyond the scope of our study. We did not investigate our large negative sample set for the presence of antibodies against seasonal coronaviruses, the presence of which might have accounted for some of the false-positive results. However, previous manufacturers' assessments for the Abbott and Roche immunoassays, and our assessment as part of this study for the Oxford immunoassay on small panels of sera taken from patients with respiratory virus infections including seasonal coronavirus infections, suggest good analytical specificity” | concepts[115] |
| concept | analytical specificity — not defined in this paper's own words; lack of cross-reactivity with seasonal respiratory-virus sera — “Our study was limited by sample volumes, especially given the constraints imposed by dead volume requirements for liquid handling, and we were unable to do repeat analyses. Subgroup analysis by timing of collection or disease severity was constrained by small numbers and larger studies would be of value. Neutralisation and pseudo-neutralisation assays provide an in-vitro approach to investigating antibodies to help address the question as to whether antibody titres correlate with functionality; this work is ongoing. Longitudinal prospective clinical studies are also needed to investigate whether repeat infections can arise in antibody-positive individuals. Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-data to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection. Additional performance assessments in these groups would be warranted. Although assessments of precision and reproducibility have to some extent been carried out by the immunoassay manufacturers, a complete validation of the individual assays was beyond the scope of our study. We did not investigate our large negative sample set for the presence of antibodies against seasonal coronaviruses, the presence of which might have accounted for some of the false-positive results. However, previous manufacturers' assessments for the Abbott and Roche immunoassays, and our assessment as part of this study for the Oxford immunoassay on small panels of sera taken from patients with respiratory virus infections including seasonal coronavirus infections, suggest good analytical specificity” | concepts[116] |
| concept | post-hoc changes to thresholds — not defined in this paper's own words; optimisation requiring fresh verification — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity. Assay selection should carefully reflect use case. This study represents a benchmark for future assessments of serological platforms. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level” | concepts[117] |
| concept | repeat verification of sensitivity and specificity — not defined in this paper's own words; required validation after threshold changes — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity. Assay selection should carefully reflect use case. This study represents a benchmark for future assessments of serological platforms. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level” | concepts[118] |
| concept | benchmark for future assessments of serological platforms — not defined in this paper's own words; stated role of the study — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity. Assay selection should carefully reflect use case. This study represents a benchmark for future assessments of serological platforms. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level” | concepts[119] |
| concept | Assay selection — not defined in this paper's own words; choice of serological platform conditioned on use case — “In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificity, and with opportunities to optimise sensitivity for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholds would require repeat verification of sensitivity and specificity. Assay selection should carefully reflect use case. This study represents a benchmark for future assessments of serological platforms. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level” | concepts[120] |
Statistics — cross-field join keys
field-invariant statistical terms, used in their statistical sense (gated on a number / symbol / test). A p-value or a standard deviation means the SAME thing in physics, biology, psychology and sociology — so these are the strongest cross-field join keys. Grounded verbatim.
| 92·7% (95% CI 90·2–94·8) | — “At the manufacturers' thresholds, for the Abbott assay sensitivity was 92·7% (95% CI 90·2–94·8) and specificity was 99·9% (99·4–100%); for the DiaSorin assay sensitivity was 96·2% (94·2–97·7) and specificity was 98·9% (98·0–99·4); for the Oxford immunoassay sensitivity was 99·1% (97·8–99·7) and specificity was 99·0% (98·1–99·5); for the Roche assay sensitivity was 97·2% (95·4–98·4) and specificity was 99·8% (99·3–100); and for the Siemens assay sensitivity was 98·1% (96·6–99·1) and specificity was 99·9% (99·4–100%).” | content_tags.statistics[0] |
Second readings
a facet read a second time, independently, from its own sentence — and whether the two readings agree
| claim type | method — a second reading reaching the same value; recorded: method — “We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassays and a novel 384-well ELISA. Methods We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay).” | alternatives.claim_type |
| method | diagnostic-accuracy-benchmark — a second reading DIFFERING from the recorded value; recorded: comparative — “We did a head-to-head assessment of SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assay (Siemens, Munich, Germany), and a novel 384-well ELISA (the Oxford immunoassay).” | alternatives.method |
| phenomenon | the sensitivity and specificity of five SARS-CoV-2 antibody immunoassays measured on identical sample sets — a second reading DIFFERING from the recorded value; recorded: an instrument's measured accuracy as a property of the cut point somebody chose, in the absence of anything to check the — “However, immunoassay performance has not been compared on large, identical sample sets.” | alternatives.phenomenon |
Full text — tagged
The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.
Lancet Infect Dis Lancet Infect Dis 445 elsevierwt 101130150 The Lancet. Infectious Diseases 1473-3099 1474-4457 pmc-is-collection-domain yes pmc-collection-title Elsevier Sponsored Documents PMC7511171 PMC7511171.2 7511171 7689287 32979318 10.1016/S1473-3099(20)30634-4 S1473-3099(20)30634-4 2 Articles Performance characteristics of five immunoassays for SARS-CoV-2term: a head-to-head benchmark comparison The National SARS-CoV-2term Serologyterm Assay Evaluation Group AinsworthpersName Mark AnderssonpersName Monique AucklandpersName Kathryn BailliepersName J Kenneth BarnespersName Eleanor BeerpersName Sally BeveridgepersName Amy BibipersName Sagida BlackwellpersName Luke BorakpersName Martyna BownpersName Abbie BrookspersName Tim Burgess-BrownpersName Nicola A CamarapersName Susana CattonpersName MatthewpersName ChaupersName Kevin K. ChristottpersName Thomas ClutterbuckpersName Elizabeth CokerpersName Jesse CornallpersName Richard J CoxpersName StuartpersName Crawford-JonespersName David CrookpersName Derrick W D'ArcangelopersName Silvia DejnirattsaipersName Wanwisa DequairepersName Julie M M DimitriadispersName Stavros DinglepersName Kate E DohertypersName George DoldpersName Christina DongpersName Tao DunachiepersName Susanna J EbnerpersName Daniel EmmeneggerpersName Marc EspinosapersName Alexis EyrepersName David W FairheadpersName Rory FassihpersName Shayan FeehilypersName Conor FellepersName Sally Fernandez-CidpersName Alejandra Fernandez MendozapersName Maria FoordpersName Thomas H FordwohpersName Thomas Fox McKeepersName Deborah FraterpersName John Gallardo Sanchez Veronica GentpersName Nick GeorgioupersName Dominique GrovespersName Christopher J HallispersName Bassam HammondpersName Peter M HatchpersName Stephanie B. HarvalapersName Heli J HillpersName Jennifer HoosdallypersName Sarah J HorsingtonpersName Bryn HowarthpersName Alison JamespersName Tim JefferypersName Katie JonespersName Elizabeth JusticepersName Anita KarpepersName Fredrik KavanaghpersName JamespersName KimpersName David S KirtonpersName Richard KlenermanpersName Paul KnightpersName Julian C KoukouflispersName Leonidas KwokpersName Andrew LeuschnerpersName Ullrich LevinpersName RobertpersName LinderpersName Aline LockettpersName Teresa LumleypersName Sheila F MarinoupersName Spyridoula MarsdenpersName Brian D Martinez Jose Martins FerreirapersName Lucas MasonpersName Lara MatthewspersName Philippa C MentzerpersName Alexander J MobbspersName Alexander MongkolsapayapersName Juthathip MorrowpersName Jordan MukhopadhyaypersName Shubhashish M M NevillepersName MatthewpersName J OakleypersName Sarah OliveirapersName Marta OtterpersName Ashley PaddonpersName Kevin PascoepersName Jordan PengpersName Yanchun PerezpersName Elena PerumalpersName Prem K PetopersName Timothy E A PickfordpersName Hayleah PloegpersName Rutger J PollardpersName Andrew J RichardsonpersName Anastasia RitterpersName Thomas G RobertspersName David J RodgerpersName Gillian RollierpersName Christine S RowepersName Cathy RudkinpersName Justine K ScreatonpersName Gavin SemplepersName Malcolm G SienkiewiczpersName Alex Silva-ReyespersName Laura SkellypersName Donal T Sobrino DiazpersName Alberto StaffordpersName Lizzie StockdalepersName Lisa StoesserpersName Nicole nicole.stoesserpersName@ndm.ox.ac.uk * StreetpersName Teresa StuartpersName David I SweedpersName Angela TaylorpersName Adan ThravespersName Hannah TsangpersName Hoi P VerheulpersName Marije K VipondpersName Richard WalkerpersName Timothy M WareingpersName Susan WarrenpersName Yolanda WellspersName Charlie WilsonpersName Clare WithycombepersName Kate YoungpersName Rebecca K † * Correspondence to: Dr Nicole StoesserpersName, Department of Microbiology, John Radcliffe Hospital, Headington, Oxford OX3 9DU, UK nicole.stoesserpersName@ndm.ox.ac.uk † Group members listed in the appendix (pp 1–5) 12 2020 20 12 369547 1390 1400 01 12 2020 24 09 2020 17 02 2026 7511171 10.1016/S1473-3099(20)30634-4 1 23 09 2020 7689287 10.1016/S1473-3099(20)30634-4 2 01 12 2020 © 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license 2020 https://creativecommons.org/licenses/by/4.0/ This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Correction to Lancet Infect Dis 2020; 20: 1390–400 20 12 25 11 2020 e298 e298 The Lancet. Infectious Diseases PMC7709078 33248043 The complexities of SARS-CoV-2term serologyterm 20 12 23 9 2020 1350 1351 The Lancet. Infectious Diseases 10.1016/S1473-3099(20)30699-X PMC7511169 32979317 Summary Background Severe acute respiratory syndrome coronavirus 2term (SARS-CoV-2term) has caused a global pandemicterm in 2020. Testing is crucial for mitigating public health and economic effects. Serologyterm is considered key to population-level surveillanceterm and potentially individual-level risk assessmentterm. However, immunoassay performance has not been compared on large, identical sample sets. We aimed to investigate the performance of four high-throughput commercial SARS-CoV-2 antibody immunoassaysterm and a novel 384-well ELISAterm.
Methods We did a head-to-head assessmentterm of SARS-CoV-2term IgG assayterm (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2term S1/S2 IgG assayterm (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), SARS-CoV-2 Total assayterm (Siemens, Munich, Germany), and a novel 384-well ELISAterm (the Oxford immunoassayterm). We derived sensitivityterm and specificityterm from 976 pre-pandemic blood samplesterm (collected between Sept 4, 2014, and Oct 4, 2016) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infectionterm, collected at least 20 days post symptom onset (collected between Feb 1, 2020, and May 31, 2020). Receiver operating characteristic (ROC) curvesterm were used to assess assay thresholdsterm.
Findings At the manufacturers' thresholds, for the Abbott assay sensitivityterm was 92·7% (95% CI 90·2–94·8) and specificityterm was 99·9% (99·4–100%); for the DiaSorin assayterm sensitivityterm was 96·2% (94·2–97·7) and specificityterm was 98·9% (98·0–99·4); for the Oxford immunoassayterm sensitivityterm was 99·1% (97·8–99·7) and specificityterm was 99·0% (98·1–99·5); for the Roche assay sensitivityterm was 97·2% (95·4–98·4) and specificityterm was 99·8% (99·3–100); and for the Siemens assayterm sensitivityterm was 98·1% (96·6–99·1) and specificityterm was 99·9% (99·4–100%). All assays achieved a sensitivityterm of at least 98% with thresholds optimised to achieve a specificityterm of at least 98% on samples taken 30 days or more post symptom onset.
Interpretation Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2term serological testing to achieve sensitivityterm and specificityterm of at least 98%. The Siemens assayterm and Oxford immunoassayterm achieved these metrics without further optimisation. This benchmark study in immunoassay assessment should enable refinements of testing strategies and the best use of serological testing resourceterm to benefit individuals and population health.
Funding Public Health England and UK National Institute for Health Research.
pmc-status-qastatus 0 pmc-status-live yes pmc-status-embargo no pmc-status-released yes pmc-prop-open-access yes pmc-prop-olf no pmc-prop-manuscript no pmc-prop-legally-suppressed no pmc-prop-has-pdf no pmc-prop-has-supplement yes pmc-prop-pdf-only no pmc-prop-suppress-copyright no pmc-prop-is-real-version no pmc-prop-is-scanned-article no pmc-prop-preprint no pmc-prop-in-epmc yes pmc-license-ref CC BY Introduction Severe acute respiratory syndrome coronavirus 2term (SARS-CoV-2term) has emerged as a novel human pathogen, causing a global pandemicterm in 2020, with more than 25 million confirmed infections and more than 840 000 deaths to date. 1 Testing and case ascertainment have been crucial to controlling virus transmission and in developing public health and political strategies to mitigate the effect of this pathogen.
SARS-CoV-2term testing broadly takes two forms: first, direct detection of the virusterm in respiratory samples with real-time RT-PCRterm; and second, by using serologyterm to investigate the presence of antibodiesterm. 2 Immunoassays detect either specific types of antibodyterm (eg, IgM or IgG) or total antibodyterm. SARS-CoV-2term antibodies typically start to appear at least 5–7 days post infection 3 and are therefore an unreliable marker for early acute infectionterm. The degree and duration of immunity that antibodies confer are unclear. A prominent use for serological testing has therefore been at a population level, for informing the extent of population exposureterm. Other uses include assessing risk of infection at an individual level and to support research and development (eg, quantifying antibody responsesterm in vaccine trials). 4 Several manufacturers have developed immunoassays compatible with global laboratory infrastructuresterm, enabling widespread testing of hundreds to thousands of samples per day. However, the scale-up required for regular population-wide testingterm (eg, every few weeks or months) might exceed the capacity of commercial platforms. To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance, leading to a relaxation of typical assessment criteriaterm (eg, the US Food and Drug Administration's [FDA] Emergency Use Authorization programmeterm). 4 Research in context Evidence before this study Substantial global interest exists in the use of serologyterm to enable population-level surveillanceterm of severe acute respiratory syndrome coronavirus 2term (SARS-CoV-2term) infection and to inform individual-level management and risk stratification; however, it is unclear which widely available SARS-CoV-2term serological immunoassays perform to the standards required to meet these needs. The most widely used commercial assays that have obtained regulatory approvals for emergency useterm in the USA and the EU include the SARS-CoV-2term IgG assayterm (Abbott, Chicago, IL, USA), LIAISON SARS-CoV-2term S1/S2 IgG assayterm (DiaSorin, Saluggia, Italy), Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland), and the anti-SARS-COV-2 ELISA (IgG; EUROIMMUN, Lübeck, Germany). However, the sensitivityterm of the EUROIMMUN assayterm has been reported as being approximately 90%. We therefore replaced this with the SARS-CoV-2 Total assayterm (Siemens, Munich, Germany) in a head-to-head comparison. In addition to reviewing performance assessments undertaken by each manufacturer, we searched PubMed, BioRxiv, and MedRxivterm up to May 31, 2020, using the following search terms: (SARS-CoV-2term) AND ([ELISA] OR [EIA] OR [CLIA] OR [FIA] OR [IFA] OR [IgG]). Details and expanded PubMed search terms are given in the appendix (pp 7, 8) . We also reviewed any investigations of relevant assays by Public Health England (PHE) up to May 31, 2020. Of 423 studies assessed, full-text reviewsterm were done for 124 articles. Eight studies provided data on sensitivityterm and specificityterm for the Abbott (four studies) or DiaSorin (five) assays; PHE investigations were undertaken for the Abbott and Roche assays. For the Abbott assay, sensitivityterm ranged from 93·4% to 100% for samples taken at least 14 days post symptom onset (70 to 680 samples) and specificityterm from 95·1% to 100% (nine to 1020); for the DiaSorin assayterm, we found a single sensitivityterm estimate of 94·4% on samples taken at least 14 days post symptom onset (18 samples) and specificityterm ranged from 94·9% to 100% (69 to 1140 samples). For the Roche assay, we found a single sensitivityterm estimate of 87·0% on samples taken at least 14 days post symptom onset (n=77) and a single specificityterm estimate of 100% (n=472). No published studies assessed all four commercial platforms using the same sample setsterm.
Added value of this study In this study on a large, single sample set of more than 1500 samplesterm, we compared four commercial SARS-CoV-2term immunoassays (Abbott, DiaSorin, Roche, and Siemens assaysterm) with a global installed baseterm, and a novel 384-well ELISAterm (the Oxford immunoassayterm) that could scale to be used for population-level surveillanceterm. The Siemens assayterm and Oxford immunoassayterm achieved a sensitivityterm and specificityterm of at least 98% without optimisation (lower 95% CI ≥96%). The other assays could achieve sensitivities and specificities of at least 98% with assay threshold adjustmentterm or deployment on samples taken at least 30 days after symptom onset. However, differences observed in assay performance translate into thousands of additional incorrect diagnoses between the worst and best platforms if millions of tests are done in large populations. For example, at a 10% seroprevalence, the Siemens assayterm would generate an estimated 2800 total errors per million teststerm, versus the DiaSorin assayterm at a rate of 13 700 total errors per million teststerm.
Implications of all the available evidence Understanding the comparativeterm·mth performance of immunoassays is crucial to the development of appropriate individual and population-level SARS-CoV-2term testing strategies. We showed that although precise performance metrics varied between immunoassay platforms, all assays that we assessed could be usefully deployed with careful consideration of use caseterm, assay thresholdsterm, and by considering symptom-to-sample timingsterm, thus optimising the use of available serological testing resourceterm and enabling the most widespread rollout. We also showed that a novel 384-well format ELISA would potentially be scalable for population-level testing of hundreds of thousands of samples.
To facilitate national public health and health-care providers in choosing platforms that are appropriate for SARS-CoV-2term serological testing, we aimed to directly assess the performance of four commercial SARS-CoV-2term antibody immunoassays, with the primary aim of identifying the sensitivityterm and specificityterm of each assay. We also assessed the same samples using a novel 384-well format ELISA that targeted antibodies against trimeric spike proteinterm (the Oxford immunoassayterm).
Methods Study design To date, the UK has one of the few established standards for performance metrics for SARS-CoV-2term immunoassays internationally; no specific guidance exists on performance metrics from either the US FDA or the European Centre for Disease Prevention and Control. We established a blood (plasma and serum) sample collectionterm including pre-pandemic samples from healthy individuals aged at least 18 yearsterm, collected between Sept 4, 2014, and Oct 4, 2016, as part of the Oxford Biobank, 5 a population-based cohort of individuals in Oxfordshire, UK, who consented to participation in research studies, and samples from individuals aged at least 18 years with laboratory-confirmed SARS-CoV-2 infectionterm from cohorts of patients admitted to hospital or surveillance on health-care workers ( appendix p 9 ). These samples enabled us to assess the immunoassays in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) target product profile for enzyme immunoassays, 6 requiring known negative samplesterm to be taken more than 6 months before the known appearance of SARS-CoV-2term (ie, earlier than July 2019), and known positive samplesterm to be collected from individuals with a previous positive SARS-CoV-2term RT-PCR from a nose or throat swab and at least 20 days post symptom onset ( appendix pp 13, 14 ). The sample collection also enabled us to assess immunoassay performance on samples taken within 20 days after symptom onset as part of secondary analysesterm. In each case samples were de-duplicated by individualterm, and the latest sample from each individualterm was analysed. Two assays (SARS-CoV-2term IgG assayterm [Abbott, Chicago, IL, USA] and LIAISON SARS-CoV-2term S1/S2 IgG [DiaSorin, Saluggia, Italy]) were done in Oxford, UK (clinical biochemistry and microbiology laboratories, John Radcliffe Hospital), and two (Elecsys Anti-SARS-CoV-2 assay [Roche, Basel, Switzerland] and SARS-CoV-2 Total assayterm [Siemens, Munich, Germany]) at Public Health England (PHE) Porton Down.
Additionally, we analysed an eight-point 1:2 dilution seriesterm of three known high-volume plasma samples obtained from SARS-CoV-2-positive patients recruited by the UK National Health Service Blood and Transplant (NHSBT; RT-PCR positive, at least 20 days post symptom onset) with high (donor ID 10062), medium (10061), and low (10063) titre SARS-CoV-2 IgG antibodiesterm as assessed by the SARS-CoV-2term IgG ELISA (EUROIMMUN, Lübeck, Germany; ratio values of 33·33, 4·34, and 2·50, respectively; ratio is the optical densityterm of the sample divided by the optical densityterm of the calibrator [ratio ≥1·1 is positive]), and one known negative control (BD001). Samples were analysed at minimum in triplicate on the four commercial immunoassays, and singly on the Oxford immunoassayterm. The NHSBT samples were correlated with national SARS-CoV-2term reagents developed by the UK's National Institute for Biological Standards and Control (NIBSC; 20/130 [single donor, high-titre antibody], 20/120 [single donor, relatively high-titre antibody], 20/122 [pool of five donor samples, mid-titre antibody], 20/124 [low S1, high-nucleocapsid protein antibody titreterm], 20/126 [low-titre antibody], 20/128 [negative control] 7 , 8 ) on the Oxford immunoassayterm ( appendix p 10 ).
We drafted a protocol before implementing the studyterm, which was shared with the manufacturers and the UK Department of Health and Social Care.
Procedures The installed base of analysers running the investigated assays is global, making their widespread deployment feasible. Two assays targeted the nucleocapsid proteinterm (Abbott and Roche) and two had spike protein-based targetsterm (DiaSorin and Siemens); two were IgG assaysterm (Abbott and DiaSorin), and two were total antibody assaysterm (Roche and Siemens). Manufacturers' sensitivityterm estimates ranged from 96·8% to 100% on samples taken at least 14 days post symptom onset or RT-PCR, and specificityterm estimates ranged from 98·5% to 99·8% on pre-pandemic samples ( appendix p 14 ). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 Assays were done in accordance with the manufacturers' instructions by trained laboratory staffterm in laboratories that were accredited by the UK Accreditation Service, on appropriate analysers, and with the specified controls and calibrantsterm, using thresholds for calling positives and negatives set by the manufacturer for testing in the UK ( appendix p 14 ), but also reporting quantitative values. Samples were assigned unique study barcodesterm for this study, and laboratory staff doing the assays did not have access to sample metadataterm before running the assays.
The Oxford immunoassayterm is an indirect ELISAterm, measuring serum IgGterm against trimeric spike proteinterm, 19 using a horseradish peroxidase-linked anti-human IgG antibody as the secondary. We implemented this assay as previously described, 20 with minor modifications ( appendix p 10 ). Readoutsterm were measured as a fluorescent signalterm, normalised to standard units by calibrating to a dilution series of the NHSBT controls and a monoclonal antibody (CR3022), using a natural cubic spline-based regression modelterm to enable across-plate and across-batch comparisonsterm ( appendix p 10 ). We derived thresholds for the Oxford immunoassayterm using an independent set of serum samples from 44 individuals with acute SARS-CoV-2term infection; 99 convalescent individuals post SARS-CoV-2term infection; one Middle East respiratory syndrome coronavirus (MERS-CoV) antiserum; 23 individuals with other respiratory virus infections; and 1205 pre-pandemic samples ( appendix pp 10, 11 ). After at least 10 days post symptom onset, excluding the MERS-CoV antiserum (which was positive) but including the 23 samples from individuals with other viral infections, derivation sensitivityterm was 100% (120 of 120 [95% CI 97·0–100·0]) and specificityterm (at a normalised threshold of 8 million units) was 99·6% (1223 of 1228 [99·1–99·9).
The Oxford immunoassayterm runs batches of up to 12 384-well platesterm, each containing 3200 samples, over a period of 7 h to generate data on up to 3840 samples and controls per run. Samples and plates were handled using a Janus automated liquid handlerterm (Perkin Elmer, Waltham, MA, USA), and plates were read with an EnVision 2104 multilabel plate readerterm (Perkin Elmer). The assay requires low input sample volumesterm of the order of 2 μL serum sample, subsequently diluted 1:25 in sample buffer (1% milk in phosphate-buffered saline with Tween); however, dead volume requirementsterm of approximately 10 μL are required for adequate liquid handling function.
Sample aliquots of 0·5–2 mL were stored at −20°C before the study, and freeze-thaw cycles were minimised (fewer than five cycles). After thawing for processing, samples were spun for 10 min at 1844 g and the supernatant removed for subsequent work. No further freeze or thaw steps were taken during the study and samples were kept at 4°C for up to 7 days to complete all runs. For the Oxford immunoassayterm assessment, thawed samples were arrayed in 384-well platesterm and stored at −20°C with a single further thaw before processing.
Outcomes and statistical analysis Whereas sample numbers were partly decided by the availability of samples required to address the study aims, the sample sizeterm was prospectively calculated to provide 80% powerterm to detect a lower 95% CI of at least 96·0% if the actual assay sensitivityterm was 98·3% (which required ≥460 positive samples) and 80% powerterm to detect a lower 95% CI of at least 96·0% if the actual assay specificityterm was 97·7% (which required ≥901 negative samples). These metrics were in line with the manufacturers' performance characteristics.
The data were collated, cleaned, and locked before analysis. Sensitivityterm and specificityterm with exact binomial 95% CIsterm for each assay were compared with current UK MHRA guidanceterm 6 stipulating a requirement for sensitivityterm and specificityterm of at least 98%, with the lower bound of the 95% CI to be at least 96%. Analyses were carried out for samples for which results were available across all platforms as the primary analysisterm, and for all samples as part of secondary analysesterm. When more than one sample was available from the same individual, samples were de-duplicated by individualterm and only the last sample obtained was analysed, unless otherwise indicated.
Other secondary analysesterm were done; first, using receiver operator characteristic (ROC) curvesterm to define trade-offs in assay sensitivityterm and specificityterm. Assay performance was also assessed according to the sampling timepoint (days post symptom onset and post RT-PCR test; at intervals of ≥14 days [per original protocol], ≥20 days [per MHRA], and ≥30 days). Evidence of differences in immunoassay sensitivityterm with respect to symptom severityterm (defined as asymptomatic, mild, severe, critical or deathterm, in line with WHO criteria 21 ) for the subgroup of samples for which this information was available was assessed for each immunoassay. Positive and negative predictive valuesterm at population prevalences of 5%, 10%, 20%, and 50%term previous SARS-CoV-2term infection were modelled. The percentage of positive tests from SARS-CoV-2-positive individuals confirmed by RT-PCR over time and by serologyterm platform was assessed; for this analysis all samples were included. Trajectories of antibody titresterm with respect to days post symptom onset were modelled by fitting spline curvesterm (three to five knots; model selection based on the Akaike Information Criterionterm); for this analysis the first sample per patient was included. Statistical analyses and data visualisations were done in R (version 3.6.3) and Stata/IC (version 16.1). Analyses and results were overseen by an external review groupterm.
For the DiaSorin assayterm, for which the manufacturer specifies repeat testing in the event that results fall within an equivocal zone (12·0 ≤ × <15·0 AU/mL), we pre-specified that we were unable to do repeat testing because of insufficient sample volumesterm available. These samples were excluded from the primary sensitivityterm and specificityterm calculations for the DiaSorin assayterm, but subsequently included in the ROC analyses. STARD and PRISMA checklists are included in the appendix (pp 37–40) .
Role of the funding source The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Results From 1000 pre-pandemic samples (collected between Sept 4, 2014, and Oct 4, 2016), and 769 samples from 673 individuals with laboratory-confirmed SARS-CoV-2 infectionterm (collected between Feb 1, and May 31, 2020), assay sensitivityterm and specificityterm were assessed on 976 known negative pre-pandemic samplesterm and 536 samples from RT-PCR-confirmed COVID-19 patients, de-duplicated by individualterm for whom complete data across all platforms were available ( appendix p 24 ). No samples failed testing on any of the four commercial platforms. We found 18 liquid handling failuresterm on the Oxford immunoassayterm resulting in empty wellspersName. Sensitivityterm and specificityterm of each of the assays are shown in the table and figure 1 . Similar results were obtained analysing all available samples ( appendix p 27 ; table ). The clearest separation of known positive and known negative samplesterm was shown for the Siemens assayterm ( figure 2 ; appendix p 27 [all samples]). Table Sensitivityterm, specificityterm, and sample sizesterm for all groups and subgroups assessed Specificityterm Sensitivityterm Detected Equivocal Not detected Not available Total Specificityterm (95% CI) Detected Equivocal Not detected Not available Total Sensitivityterm (95% CI) Samples with complete data available Abbott 1 .. 975 .. 976 99·9% (99·4–100) 497 .. 39 .. 536 92·7% (90·2–94·8) DiaSorin 11 2 963 .. 976 98·9% (98·0–99·4) 509 7 20 .. 536 96·2% (94·2–97·7) Oxford immunoassayterm 10 .. 966 .. 976 99·0% (98·1–99·5) 531 .. 5 .. 536 99·1% (97·8–99·7) Roche 2 .. 974 .. 976 99·8% (99·3–100) 521 .. 15 .. 536 97·2% (95·4–98·4) Siemens 1 .. 975 .. 976 99·9% (99·4–100) 526 .. 10 .. 536 98·1% (96·6–99·1) Samples with complete data available (specificityterm set to ≥98%) Abbott 19 .. 957 .. 976 98·1% (97·0–98·8) 523 .. 13 .. 536 97·6% (95·9–98·7) DiaSorin 18 0 958 .. 976 98·2% (97·1–98·9) 523 0 13 .. 536 97·6% (95·9–98·7) Oxford immunoassayterm 19 .. 957 .. 976 98·1% (97·0–98·8) 533 .. 3 .. 536 99·4% (98·4–99·9) Roche 19 .. 957 .. 976 98·1% (97·0–98·8) 533 .. 3 .. 536 99·4% (98·4–99·9) Siemens 15 .. 961 .. 976 98·5% (97·5–99·1) 530 .. 6 .. 536 98·9% (97·6–99·6) All samples Abbott 1 .. 994 .. 995 99·9% (99·4–100) 500 .. 40 .. 540 92·6% (90·0–94·7) DiaSorin 12 2 981 .. 995 98·8% (97·9–99·4) 512 7 21 .. 540 96·1% (94·0–97·5) Oxford immunoassayterm 10 .. 967 18 977 99·0% (98·1–99·5) 535 .. 5 .. 540 99·1% (97·9–99·7) Roche 2 .. 993 .. 995 99·8% (99·3–100) 521 .. 15 4 536 97·2% (95·4–98·4) Siemens 1 .. 993 1 994 99·9% (99·4–100) 526 .. 10 4 536 98·1% (96·6–99·1) All samples (specificityterm set to ≥98%) Abbott 19 .. 976 .. 995 98·1% (97·0–98·8) 527 0 13 .. 540 97·6% (95·9–98·7) DiaSorin 19 0 976 .. 995 98·1% (97·0–98·8) 527 .. 13 .. 540 97·6% (95·9–98·7) Oxford immunoassayterm 19 .. 958 18 977 98·1% (97·0–98·8) 537 .. 3 .. 540 99·4% (98·4–99·9) Roche 19 .. 976 .. 995 98·1% (97·0–98·8) 533 .. 3 4 536 99·4% (98·4–99·9) Siemens 15 .. 979 1 994 98·5% (97·5–99·2) 530 .. 6 4 536 98·9% (97·6–99·6) Samples with complete data available taken ≥14 days post symptom onset Abbott 1 .. 975 .. 976 99·9% (99·4–100) 520 .. 41 .. 561 92·7% (90·2–94·7) DiaSorin 11 2 963 .. 976 98·9% (98·0–99·4) 529 8 24 .. 561 95·7% (93·6–97·2) Oxford immunoassayterm 10 .. 966 .. 976 99·0% (98·1–99·5) 554 .. 7 .. 561 98·8% (97·4–99·5) Roche 2 .. 974 .. 976 99·8% (99·3–100) 543 .. 18 .. 561 96·8% (95·0–98·1) Siemens 1 .. 975 .. 976 99·9% (99·4–100) 548 .. 13 .. 561 97·7% (96·1–98·8) Samples with complete data available taken ≥14 days post symptom onset (specificityterm set to ≥98%) Abbott 19 .. 957 .. 976 98·1% (97·0–98·8) 546 .. 15 .. 561 97·3% (95·6–98·5) DiaSorin 18 0 958 .. 976 98·2% (97·1–98·9) 544 0 17 .. 561 97·0% (95·2–98·2) Oxford immunoassayterm 19 .. 957 .. 976 98·1% (97·0–98·8) 557 .. 4 .. 561 99·3% (98·2–99·8) Roche 19 .. 957 .. 976 98·1% (97·0–98·8) 556 .. 5 .. 561 99·1% (97·9–99·7) Siemens 15 .. 961 .. 976 98·5% (97·5–99·1) 554 .. 7 .. 561 98·8% (97·4–99·5) Samples with complete data available taken ≥30 days post symptom onset Abbott 1 .. 975 .. 976 99·9% (99·4–100) 458 .. 32 .. 490 93·5% (90·9–95·5) DiaSorin 11 2 963 .. 976 98·9% (98·0–99·4) 468 6 16 .. 490 96·7% (94·7–98·1) Oxford immunoassayterm 10 .. 966 .. 976 99·0% (98·1–99·5) 487 .. 3 .. 490 99·4% (98·2–99·9) Roche 2 .. 974 .. 976 99·8% (99·3–100) 481 .. 9 .. 490 98·2% (96·5–99·2) Siemens 1 .. 975 .. 976 99·9% (99·4–100) 482 .. 8 .. 490 98·4% (96·8–99·3) Samples with complete data available taken ≥30 days post symptom onset (specificityterm set to ≥98%) Abbott 19 .. 957 .. 976 98·1% (97·0–98·8) 482 .. 8 .. 490 98·4% (96·8–99·3) DiaSorin 18 0 958 .. 976 98·2% (97·1–98·9) 481 0 9 .. 490 98·2% (96·5–99·2) Oxford immunoassayterm 19 .. 957 .. 976 98·1% (97·0–98·8) 488 .. 2 .. 490 99·6% (98·5–100) Roche 19 .. 957 .. 976 98·1% (97·0–98·8) 488 .. 2 .. 490 99·6% (98·5–100) Siemens 15 .. 961 .. 976 98·5% (97·5–99·1) 485 .. 5 .. 490 99·0% (97·6–99·7) Specificityterm was assessed using the known negative pre-pandemic samplesterm, and sensitivityterm using the known positive samples from patients who had previously had laboratory-confirmed SARS–CoV–2 infectionterm. Abbott=SARS-CoV-2term IgG assayterm (Abbott, Chicago, IL, USA). DiaSorin=LIAISON SARS-CoV-2term S1/S2 IgG assayterm (DiaSorin, Saluggia, Italy). Oxford immunoassayterm=a novel 384-well format ELISA (University of Oxford, Oxford, UK). Roche=Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland). SARS-CoV-2term=severe acute respiratory syndrome coronavirus 2term. Siemens=SARS-CoV-2 Total assayterm (Siemens, Munich, Germany).
Figure 1 Sensitivityterm and specificityterm for each assay on samples taken ≥20 days post symptom onset in patients with laboratory-confirmed SARS-CoV-2 infectionterm for positive samples, and >6 months before the first known COVID-19 cases for negative samples The UK MHRA performance target is shown (dark grey dashed line), including the required lower bound of the 95% CI (light grey dashed line) for sensitivityterm and specificityterm. Data are presented for 976 known-negative samples and 536 known-positive samples run on each assay. Equivocal results were excluded from the calculation of sensitivityterm and specificityterm for the DiaSorin assayterm (n=9). MHRA=UK Medicines and Healthcare products Regulatory Agency. Abbott=SARS-CoV-2term IgG assayterm (Abbott, Chicago, IL, USA). DiaSorin=LIAISON SARS-CoV-2term S1/S2 IgG assayterm (DiaSorin, Saluggia, Italy). Oxford immunoassayterm=a novel 384-well format ELISA (University of Oxford, Oxford, UK). Roche=Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland). SARS-CoV-2term=severe acute respiratory syndrome coronavirus 2term. Siemens=SARS-CoV-2 Total assayterm (Siemens, Munich, Germany).
Figure 2 Distribution of numerical results obtained for each commercial assay Results are represented as (A) histograms, to enable assessment of the frequency of values, and (B) dotplots, to review scatter of values, especially around thresholds. Prespecified assay thresholdsterm are shown as dashed lines. For the purposes of plotting values on a log scale, values of zero were set to the lowest non-zero value and results of greater or less than the largest or smallest values were truncated to the largest and smallest values. Data are presented for 976 known negative samplesterm and 536 known positive samplesterm run across all assays. Abbott=SARS-CoV-2term IgG assayterm (Abbott, Chicago, IL, USA). DiaSorin=LIAISON SARS-CoV-2term S1/S2 IgG assayterm (DiaSorin, Saluggia, Italy). Oxford immunoassayterm=a novel 384-well format ELISA (University of Oxford, Oxford, UK). Roche=Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland). SARS-CoV-2term=severe acute respiratory syndrome coronavirus 2term. Siemens=SARS-CoV-2 Total assayterm (Siemens, Munich, Germany).
Using the pre-defined assay thresholdsterm for calling test results positive or negative, only the Siemens assayterm and the Oxford immunoassayterm met the specificityterm and sensitivityterm target of at least 98%, achieving this threshold at all three timepoints assessed (≥14, ≥20, and ≥30 days post symptom onset; appendix pp 28, 29 ; figure 1 ; table ). The Roche assay also met the specificityterm and sensitivityterm target of at least 98% on samples taken at least 30 days post symptom onsetterm ( appendix p 29 ; table ). ROC curve analysis showed that the Roche assay could also potentially meet the sensitivityterm and specificityterm targets on samples taken at least 14 and at least 20 days post symptom onset through threshold adjustmentterm—eg, adjusting the threshold to at least 0·128 as opposed to the current threshold of more than 1·0 would result in a sensitivityterm and specificityterm of at least 98% (with a lower bound of the 95% CI of ≥96%) at 20 days or more post symptom onset ( figure 3 ; appendix p 30 for samples taken ≥14 days post symptom onset; table ). On samples taken at least 30 days post symptom onsetterm (but not ≥14 or ≥20 days post symptom onset), the Abbott and DiaSorin assayterm thresholds could be optimised to achieve sensitivityterm and specificityterm of at least 98% (with a lower bound of ≥96%; appendix p 31 ; table ). Overall, on samples taken at least 30 days post symptom onsetterm, optimisation of assay thresholdsterm to ensure a specificityterm of at least 98% (with a lower bound of ≥96%) would result in sensitivities of at least 98% (with a lower bound of ≥96%) for all assays assessed ( appendix pp 31, 32 ; table ). Figure 3 ROC curves for each assay at the specification of samples taken ≥20 post symptom onset The green shaded areas represent a target sensitivityterm and specificityterm of at least 98%. Dashed lines show the 98% sensitivityterm and specificityterm thresholds used as the standard. Assay thresholdterm values associated with ten exemplar points on the ROC curve are shown in each panel. Data are presented for 976 known negative samplesterm and 536 known positive samplesterm run on each assay. ROC=receiver operating characteristic. Abbott=SARS-CoV-2term IgG assayterm (Abbott, Chicago, IL, USA). DiaSorin=LIAISON SARS-CoV-2term S1/S2 IgG assayterm (DiaSorin, Saluggia, Italy). Oxford immunoassayterm=a novel 384-well format ELISA (University of Oxford, Oxford, UK). Roche=Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland). SARS-CoV-2term=severe acute respiratory syndrome coronavirus 2term. Siemens=SARS-CoV-2 Total assayterm (Siemens, Munich, Germany).
The NHSBT standards (10061-63) correlated wellpersName with the NIBSC reagents using the Oxford immunoassayterm ( appendix p 10 ). Replicate testing of serial dilutions of the NHSBT standards showed good reproducibility, albeit with some interesting differences observed—eg, for nucleocapsid proteinterm assays (Abbott and Roche), readings for the NHSBT medium control (10061) dilution series were lower than those of the NHSBT low control (10063), suggesting that antibodies targeting nucleocapsid and spike proteins might be present at different titres in certain samples ( appendix p 33 ). For the Roche and Siemens assaysterm the entire high control dilution series generated values above the assay thresholdsterm for calling positives. For the Abbott assay, all of the medium titre dilution series generated values below the threshold for distinguishing positives.
Although sensitivityterm and specificityterm are useful, positive and negative predictive valuesterm are more directly relevant to public health and clinical application. At 10% prevalence, the high specificityterm platforms (Abbott, Roche, and Siemens assaysterm) had lower numbers of false positive tests per million teststerm compared with other assays (DiaSorin and Oxford immunoassayterm; figure 4 ; appendix p 21 ). However, the number of false negative tests per millionterm was higher with the Abbott and DiaSorin assaysterm and lowest with the Oxford immunoassayterm ( figure 4 ; appendix p 21 ). Overall, the Siemens assayterm had the fewest total errors per million teststerm for prevalences of 20% or lowerterm ( figure 4 ; appendix p 21 ). Figure 4 False negatives, false positives, and total errors per 1 million tests False negatives (A), false positives (B) and total errors (C) per 1 million tests, using the unadjusted thresholds (manufacturers, and Oxford immunoassayterm), and sensitivityterm and specificityterm for all assays for samples tested after at least 20 days post symptom onset, at population SARS-CoV-2term seroprevalence of 5%, 10%, 20%, and 50%. Abbott=SARS-CoV-2term IgG assayterm (Abbott, Chicago, IL, USA). DiaSorin=LIAISON SARS-CoV-2term S1/S2 IgG assayterm (DiaSorin, Saluggia, Italy). Oxford immunoassayterm=a novel 384-well format ELISA (University of Oxford, Oxford, UK). Roche=Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland). SARS-CoV-2term=severe acute respiratory syndrome coronavirus 2term. SARS-CoV-2term=severe acute respiratory syndrome coronavirus 2term. Siemens=SARS-CoV-2 Total assayterm (Siemens, Munich, Germany).
Antibody responsesterm increased over the first 3–4 weeks after symptom onsetterm, with the maximum percentage of positive tests reached by day 27term across all assays ( appendix p 34 ). Antibody responsesterm were sustained up to 73 days post symptom onset and up to 82 days post positive RT-PCR result ( appendix p 35 ); however, any declines in titres could not be specifically ruled out given that, by this stage, most quantitative results were at near-maximum values for each assay.
For 158 (29%) of 536 individuals in the positive cohort for whom disease severity data were available, we found no evidence of a difference in immunoassay sensitivityterm by disease severity (asymptomatic n=13 [8%], mild n=122 [77%], severe n=16 [10%], critical or death n=7 [4%]) for any of the assays assessed ( appendix p 36 ); however numbers were small in the non-mild groups.
We also undertook an analysis of discordance between assaysterm ( appendix pp 22, 23 ). Most of the discordances were observed in samples taken before 20 days post symptom onset for SARS-CoV-2-positive patients confirmed by RT-PCR (37 [30%] of 124 samples taken <20 days post symptom onset vs 46 [9%] 538 samples taken ≥20 days post symptom onset; 27 [3%] of 976 pre-pandemic samples; p<0·0001). Three (<1%) of the samples classified as known positive (from two mild and one severe case) tested antibody-negative across all assays.
Discussion On a large panel of blood samples, we showed that the Siemens assayterm and the Oxford immunoassayterm both achieved sensitivityterm and specificityterm of at least 98% (with a lower bound of the 95% CI ≥96%) on samples taken at least 20 days post symptom onset without optimisation. However, all assays could potentially achieve these specifications through threshold adjustmentterm, or by assessing samples collected at least 30 days post symptom onset, consistent with the time-dependent nature of antibody responsesterm. Additionally, we highlighted the potentially scalable capacity of the Oxford immunoassayterm, which could enable centralised testingterm for large-scale projects (eg, national seroprevalence studies). This finding suggests that global serologyterm testing needs can be met using different assays, mitigating against the risk of shortages, and allowing immediate deployment in laboratories with different analysers already installed for other testing purposes. The optimal deployment of assays should reflect use caseterm—eg, an optimised assay measuring antibodies to nucleocapsid proteinterm might be most relevant for disease surveillanceterm in the context of a widespread rollout of vaccinesterm, which are mostly spike-protein basedterm.
Although sensitivityterm and specificityterm are important, estimating the negative and positive predictive values of each assay are perhaps more useful in deciding how tests can best be used and interpreted. Most published seroprevalence studies of non-health-care workers have estimated antibody prevalences of less than 15%. 22 , 23 From our estimates, all assays would have an error rate of less than 1% at a seroprevalence of 10%, except for the DiaSorin assayterm. Nevertheless, this error rate would still translate into several thousand errors per million tests done for any of the assays investigated, which could have important implications at an individual level.
In our study, antibody titresterm within individuals were largely sustained beyond 2 months of symptom onset for all assays investigated, suggesting that serosurveillance with these assays is likely to be accurate within months of exposure at least. We did not observe any significant difference in assay sensitivityterm by disease severity; notably, however, the numbers in our non-mild disease category, including asymptomatic cases, were small, and we were limited by the bounds determined by the dynamic range of the assaysterm and saturation. Inconsistencies have been observed in the association between antibody titresterm and disease severity in other studies, which might relate to assay target. 20 , 24 , 25 Further studies definitively assessing the longer-term trajectory of antibodies in larger longitudinal cohorts of individuals with different disease severities and taking the specific assay type into account would be of benefit.
Although all these assays can effectively detect SARS-CoV-2term antibodies, the durability and nature of immunity conferred by these antibodies remains unclear. Few data are available on immunity to seasonal human coronaviruses (alphacoronaviruses NL63 and 229E, and betacoronaviruses HKU1 and OC43). In two small, controlled, human infection models, re-challenge at 8–12 months with 229E (nine individuals) 26 and 229E-related strains (six) 27 showed protection against symptoms, but virus shedding was detected in six (67%) of nine individuals. 26 In an observational study of NL63 in Kenya, reinfections over a 6-month period occurred in 46 (28%) of 163 individuals, with reinfections within 80 days largely associated (>80% cases) with lower viral titres and fewer symptoms compared with those associated with reinfections after more than 80 days. 28 Data on Middle East respiratory syndrome (MERS) and severe acute respiratory syndrome coronavirus suggest that antibody titresterm wane over periods of approximately 2 years, with more severe infection resulting in more measurable and durable antibody responsesterm in MERS infection. 29 , 30 Animal data (re-challenge in four rhesus macaques) suggest protection from re-infection with SARS-CoV-2term extends to at least 28 days post infection. 31 The relationship between SARS-CoV-2term antibodies and neutralising activityterm remains an important research question. Because the spike protein is responsible for enabling cell entry, the spike protein appears to be the main antigen responsible for eliciting neutralising antibodiesterm. In SARS-CoV-2term, a correlation between anti-spike antibody titresterm when trimeric versions of spike are used as the antigen and neutralising capacity appears to be emerging. 24 , 32 If SARS-CoV-2term antibody titresterm are shown to clearly correlate directly or indirectly with immunity to infectionterm or attenuation of disease severity, then more accurately evaluating titres by optimising the dynamic range of assays might become relevant.
Notably, no clear gold standardterm exists against which to assess these antibody tests. RT-PCR positivityterm is a proxy for the expected presence of antibody, but negative antibody tests in RT-PCR-positive individuals could either reflect antibody test performance, or alternatively be explained by a failure to mount a measurable systemic IgM or IgG antibody responseterm (eg, in cases of immunocompromised individuals), effective infection clearance through other immune mechanisms (eg, T cells), an issue with the sample (eg, a major interferent), or through a false-positive RT-PCR test in individuals who have not had SARS-CoV-2term infection. The three samples that were negative by all assays in our study could plausibly have had a biological absence of antibodies.
Our study was limited by sample volumesterm, especially given the constraints imposed by dead volume requirements for liquid handlingterm, and we were unable to do repeat analyses. Subgroup analysis by timing of collection or disease severity was constrained by small numbers and larger studies would be of value. Neutralisation and pseudo-neutralisation assaysterm provide an in-vitro approach to investigating antibodies to help address the question as to whether antibody titresterm correlate with functionality; this work is ongoing. Longitudinal prospective clinical studiesterm are also needed to investigate whether repeat infections can arise in antibody-positive individuals. Our sample sets might under-represent some ethnic groups, did not include children, and captured insufficient clinical meta-dataterm to characterise features (eg, being immunocompromised), which might explain the presence of false-negative antibody results in individuals with RT-PCR-confirmed infection. Additional performance assessments in these groups would be warranted. Although assessments of precision and reproducibility have to some extent been carried out by the immunoassay manufacturers, a complete validation of the individual assays was beyond the scope of our study. We did not investigate our large negative sample set for the presence of antibodiesterm against seasonal coronaviruses, the presence of which might have accounted for some of the false-positive results. However, previous manufacturers' assessments for the Abbott and Roche immunoassays, and our assessment as part of this study for the Oxford immunoassayterm on small panels of sera taken from patients with respiratory virus infections including seasonal coronavirus infections, suggest good analytical specificityterm.
In conclusion, we characterised the performance of four commercial antibody platforms and a new ELISA from an academic research facility, all of which achieved the desired specificityterm, and with opportunities to optimise sensitivityterm for the Abbott, DiaSorin, and Roche assays through changes to thresholds and sample timing, although any post-hoc changes to thresholdsterm would require repeat verification of sensitivity and specificityterm. Assay selectionterm should carefully reflect use caseterm. This study represents a benchmark for future assessments of serological platformsterm. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level.
This online publication has been corrected. The corrected version first appeared at thelancet.com/infection on November 25, 2020 Data sharing Available metadata for each sample in the study, including raw immunoassay results and interpretation, are presented as online datasets .