You are here

Post-market evaluation of serology-based point of care tests

11 March 2021

In response to the COVID-19 pandemic, the Therapeutic Goods Administration (TGA) has expedited the approval (with conditions) of COVID-19 tests including tests intended for laboratory use and those intended for use by specified health professionals at the point of care (POC). Approval was based on the studies manufacturers were able to provide at the time of application.

Accurate identification of a COVID-19 infection based on serology results, particularly those obtained from tests used at the POC, requires an understanding of the antibody response profile which is currently not well defined. It is known that these tests can fail to detect COVID-19 if testing is performed in the acute phase of the infection prior to the development of detectable antibodies. If this 'negative' result is incorrectly interpreted the person may be told they haven't been infected when they may have been infected and may be infectious. Results need to be interpreted, in conjunction with clinical presentation, by a suitably qualified health care professional who can provide a patient with appropriate advice and treatment if required.

As part of a Post Market Review the TGA is now reviewing all approved serology-based COVID-19 POC tests to verify their ability to detect antibodies to SARS-COV-2 (the virus that causes COVID-19), taking into consideration the timeframes for an individual to develop detectable levels of antibody. The Peter Doherty Institute for Infection and Immunity (the Doherty Institute), has been engaged by the Department of Health to assist with the post-market verification process of these tests, to inform their best use. However reports by other Australian laboratories or comparable international regulators may also be taken into consideration.

To date, the Doherty Institute has completed its validation of twenty-three different serology-based POC tests approved by the TGA. Samples of the twenty-three POC tests were provided from the following manufacturers (note some devices have multiple sponsors):

Manufacturer Sponsor Device name

Atlaslink Beijing Technology Co Ltd (China)

Adelco Pharmaceuticals (Australia) Pty Ltd

NOVA Test® COVID-19 IgG/IgM Antibody Test (Colloidal Gold)

Beijing Wantai Biological pharmacy Enterprise Co Ltd (China)

Life Clinic Australia Pty Ltd

Wantai SARS-CoV-2 Ab Rapid Test kit

Cellex Inc (United States Of America)

MEDICISION PTY LTD

Cellex qSARS-CoV-2 IgG/IgM Cassette Rapid Test kit (lateral Flow Chromatographic Immunoassay)

CTK Biotech Inc

MD Solutions Australasia Pty Ltd

OnSite COVID-19 IgG/IgM Rapid Test

GenBody Inc (Korea - Republic of)

Biovent Consulting Pty Ltd

GenBody COVID-19 IgM/IgG

Guangzhou Wondfo Biotech Co Ltd (China)

  1. ALLSAFE MEDICAL PTY LTD
  2. Cellmid Limited
  3. Tayler Dental Consulting Pty Ltd- Agent Right Time Business Pty Ltd

SARS-CoV-2 Antibody Test (Lateral Flow Method)

Hangzhou Alltest Biotech Co Ltd (China)

AM Diagnostics

2019-nCOV IgG/IgM Rapid Test Cassette

HANGZHOU BIOTEST BIOTECH Co LTD (China)

Medsupply

COVID-19 lgG/IgM Rapid Test Cassette

Hangzhou Clongene Biotech Co Ltd (China)

  1. APAC Security Pty Ltd
  2. Ausliance Group Pty Ltd

COVID-19 lgG/IgM Rapid Test Cassette

Hangzhou Laihe Biotech Co Ltd (China)

Complementary Medicines Group Pty Ltd

LYHER Novel Coronavirus (2019-nCoV) IgM/IgG Antibody Combo Test Kit (Colloidal Gold)

Hangzhou Realy Tech Co Ltd (China)

Solasta Life Pty Ltd

2019-nCOV/COVID-19 IgG/IgM Rapid Test Device

Healgen Scientific Limited Liability Company (United States Of America)

Southwind International Pty Ltd

COVID-19 IgG/IgM Rapid Test Cassette (Whole Blood/Serum/Plasma)

Innovation Scientific Pty Ltd (Australia)

Innovation Scientific Pty Ltd

InnoScreenTM COVID-19 IgG/IgM Rapid Test

Innovita (Tangshan) Biological Technology Co Ltd (China)

  1. AM Diagnostics
  2. Aurora Future Education Group
  1. 2019-nCoV Ab Test (Colloidal Gold)
  2. 2019-nCoV Ab Test (Colloidal Gold) (IgM/IgG Whole Blood/ Serum/Plasma Combo)

Nantong Egens Biotechnology Co Ltd (China)

Lavinia Medical Pty Ltd

COVID-19 IgG/IgM Rapid Test Kit

Newscen Coast Bio-Pharmaceutical Co Ltd (China)

Kissun Pharmaceuticals Pty Ltd

COVID-19 IgG/IgM Rapid Cassette (S/P/WB)

NTBIO Diagnostics Inc (Canada)

Healtheguru Pty Ltd

One Step Rapid Test – COVID-19 IgG/IgM Antibody Test

PCL Inc (Korea - Republic of)

Haemokinesis Pty Ltd

PCL COVID19 IgG/IgM Rapid Gold

Qingdao Hightop Biotech Co Ltd (China)

Envon Pty Ltd

SARS-CoV-2 IgM/IgG Antibody Rapid Test

VivaChek Biotech (Hangzhou) Co Ltd (China)

  1. Endo X

VivaDiag™ COVID-19 IgM/IgG Rapid Test

Wuhan EasyDiagnosis Biomedicine Co Ltd (China)

Australian Trefoil Health Technologies Pty Ltd

SARS-CoV-2 IgM/IgG Antibody Test Kit

Zhejiang Orient Gene Biotech Co Ltd (China)

  1. Amandla China Pty Limited
  2. Expia Pty Ltd
  3. ONSITE DIAGNOSTICS PTY LTD

COVID-19 IgG/IgM Rapid Test Cassette

Zhuhai Livzon Diagnostics Inc (China)

  1. Emergence Technology Pty Ltd
  2. Marcel Equity Pty Limited/Avania PL

Diagnostic Kit for IgM/IgG Antibody to Coronavirus (SARS-CoV-2) (Lateral Flow)

Doherty Institute's validation

The validation performed by the Doherty Institute for these twenty-three devices suggests that manufacturers have claimed a better sensitivity compared to that observed in the Doherty Institute studies (when compared to a molecular-based method). The Doherty Institute studies did demonstrate that the sensitivity of most tests improved with increasing duration (i.e. longer time) between sample collection and symptom onset up to approximately 20-30 days post-symptom onset.

NOVA Test® COVID-19 IgG/IgM Antibody Test (Colloidal Gold) manufactured by Atlaslink Beijing Technology Co Ltd (China)

The overall total antibody (i.e. either IgM or IgG) sensitivity and specificity claimed by the manufacturer in the instructions for use (IFU) provided with the device is 92.4% sensitivity only specified at middle disease stage specified by the manufacturer of d14- 20.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 98.0% while the total antibody specificity was found to be 86.0%. For samples collected 15-21 days post onset of symptoms the overall sensitivity was 100%.

Wantai SARS-CoV-2 Ab Rapid Test kit manufactured by Beijing Wantai Biologicalpharmacy Enterprise Co Ltd, China

The overall total antibody sensitivity and specificity claimed by the manufacturer in the IFU is 94.70% and 98.89% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 82.0% while the total antibody specificity was found to be 98.0%. For samples collected 14 days post onset of symptoms the overall sensitivity improved to 84.2%.

Cellex qSARS-CoV-2 IgG/IgM Rapid Test kit manufactured by Cellex Inc, USA

The overall total antibody sensitivity and specificity (i.e. either IgM or IgG) claimed by the manufacturer in the IFU is 93.8% and 95.6% respectively. The sensitivity and specificity claimed by the manufacturer in the IFU for IgM is 82.0% and 98.4%, respectively; and for IgG is 90.6% and 97.6%, respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 76.0% while the total antibody specificity was found to be 97.0% For samples collected 14 days post onset of symptoms the overall sensitivity improved to 86.8%.

Onsite IgM/IgG Rapid Test manufactured by CTK Biotech Inc, USA

The overall total antibody (i.e. either IgM or IgG) sensitivity and specificity claimed by the manufacturer in the instructions for use (IFU) provided with the device is 96.86% and 99.39% respectively. The information provided in the IFU was silent on the relative sensitivity of the test at different time points post symptom onset.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 56.9% while the total antibody specificity was found to be 95.6%. For samples collected 14 days or longer post onset of symptoms the overall sensitivity improved to 84.6%.

GenBody COVID-19 IgM/IgG manufactured by GenBody Inc, Korea

The overall total antibody sensitivity (i.e. either IgM or IgG) claimed by the manufacturer in the IFU is 89.3% and 88.0% for samples collected 7 days from symptom onset. Overall claimed total antibody specificity was 95.9%.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 38.5% while the total antibody specificity was found to be 100%. For samples collected 14 days post onset of symptoms the overall sensitivity improved to 58.5%. Based on the Doherty Institute's findings, there was a significant difference in performance of this device in comparison to similar devices evaluated as part of the post-market review and the claims made in the IFU.

SARS-CoV-2 Antibody Test (Lateral Flow Method) manufactured by Guangzhou Wondfo Biotech Co Ltd, China

The overall total antibody sensitivity and specificity claimed by the manufacturer in the IFU provided with the device is 86.43% and 99.57% respectively (note this device does not distinguish between IgM and IgG results).

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 68.6% while the total antibody specificity was found to be 97.8%. For samples collected 14 days post onset of symptoms the overall sensitivity improved to 93.8%.

2019-n-CoV IgG/IgM Rapid Test Cassette manufactured by Hangzhou Alltest Biotech Co Ltd, China

The sensitivity and specificity claimed by the manufacturer in the IFU provided with the device for IgM is 85.0% and 96.0% respectively; and for IgG, 100% and 98.0% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 60.6% while the total antibody specificity was found to be 96.7%. For samples collected 14 days post onset of symptoms the overall sensitivity improved to 90.8%.

RightSign COVID-19 lgG/IgM Rapid Test Cassette manufactured by HANGZHOU BIOTEST BIOTECH CO LTD (China)

The overall total antibody (i.e. either IgM or IgG) sensitivity and specificity claimed by the manufacturer in the instructions for use (IFU) provided with the device is 91.4% and 100% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 96% while the total antibody specificity was found to be 94%. For samples collected 15-21 days post onset of symptoms the overall sensitivity was 100%.

Lungene COVID-19 IgG/IgM Rapid Test Cassette, (Hangzhou Clongene Biotech Co Ltd, China)

The overall total antibody sensitivity and specificity (i.e. either IgM or IgG) claimed by the manufacturer in the IFU is 91.06% and 96.48% respectively. The sensitivity claimed by the manufacturer in the IFU for IgM and IgG is 66.7% and 90.24%, respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 88.0% while the total antibody specificity was found to be 98.0% For samples collected 14 days post onset of symptoms the overall sensitivity was 97.4%.

LYHER Novel Coronavirus (2019-nCoV) IgM/IgG Antibody Combo Test Kit (Hangzhou Laihe Biotech Co Ltd, China)

The overall total antibody sensitivity and specificity claimed by the manufacturer in the IFU is 96.98% and 99.29% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 80.0% while the total antibody specificity was found to be 97.0% For samples collected 14 days post onset of symptoms the overall sensitivity improved to 86.8%.

Realy 2019-nCOV/COVID-19 IgG/IgM Rapid Test Device manufactured by Hangzhou Realy Tech Co Ltd (China)

The overall total antibody (i.e. either IgM or IgG) sensitivity and specificity claimed by the manufacturer in the instructions for use (IFU) provided with the device is 100% and 99.5% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 89.0 % while the total antibody specificity was found to be 93.9%. For samples collected 15-21 days post onset of symptoms the overall sensitivity was 87.5%.

COVID-19 IgG/IgM Rapid Test Cassette (Whole Blood/Serum/Plasma (Healgen Scientific Limited Liability Company (United States of America))

The overall sensitivity (i.e. either IgM or IgG) and specificity claimed by the manufacturer in the IFU is 93.8% and 98.5%, respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 100% while the total antibody specificity was found to be 91.0%. For samples collected 15-21 days post onset of symptoms the overall sensitivity was 100%.

InnoscreenTM COVID-19 IgG/IgM Rapid Test Device manufactured by Innovation Scientific Pty Ltd (Australia)

The sensitivity (from samples collected 7 days post symptom onset) and specificity (from samples collected within 7 days from symptom onset) claimed by the manufacturer in the IFU for IgM is 97.1% and 99.5%, respectively; and for IgG is 94.3% and 100%, respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 84.0% while the total antibody specificity was found to be 96.0%. For samples collected 14 days post onset of symptoms the overall sensitivity improved to 92.1%. For samples collected 14 days post onset of symptoms the sensitivity improved to 78.9% for IgM and 92.1% for IgG.

Innovita 2019-nCoV Ab Test manufactured by Innovita (Tangshan) Biological Technology Co Ltd, China

The overall total antibody sensitivity and specificity claimed by the manufacturer in the IFU is 85.0% and 97.4% respectively (compared to RT-PCR).

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 39.4% while the total antibody specificity was found to be 100%. For samples collected 14 days post onset of symptoms the overall sensitivity improved to 56.9%. Based on the Doherty Institute's findings, there was a significant difference in performance of this device in comparison to similar devices evaluated as part of the post-market review and the claims made in the IFU.

COVID-19 IgG/IgM Rapid Test Kit manufactured by Nantong Egens Biotechnology Co Ltd (China)

The overall sensitivity and specificity claimed by the manufacturer in the IFU for IgM is 61.2% and 100%, respectively; and for IgG is 83.7% and 100%, respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 97.0% while the total antibody specificity was found to be 91%. For samples collected 15-21 days post onset of symptoms the overall sensitivity was 87.5%.

COVID-19 IgG/IgM Rapid Cassette (S/P/WB) manufactured by Newscen Coast Bio-Pharmaceutical Co Ltd (China)

The overall sensitivity and specificity claimed by the manufacturer in the IFU is 95.8% and 99.3%, respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 90.9% while the total antibody specificity was found to be 96.0%. For samples collected 15-21 days post onset of symptoms the overall sensitivity was 87.5%.

NTBIO COVID-19 IgG/IgM Antibody Test manufactured by NTBIO Diagnostics Inc (Canada)

The overall total antibody (i.e. either IgM or IgG) sensitivity and specificity claimed by the manufacturer in the instructions for use (IFU) provided with the device is 85.7% and 100% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 94% while the total antibody specificity was found to be 92%. For samples collected 15-21 days post onset of symptoms the overall sensitivity was 100%.

PCL COVID19 IgG/IgM Rapid Gold manufactured by PCL Inc, Korea

The overall total antibody sensitivity and specificity claimed by the manufacturer in the IFU is 81.35% and 99.37% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 48.0% while the total antibody specificity was found to be 100.0% For samples collected 14 days post onset of symptoms the overall sensitivity was 50.0%.

SARS-CoV-2 IgM/IgG Antibody Rapid Test manufactured by Qingdao Hightop Biotech Co Ltd, China

The overall total antibody sensitivity and specificity claimed by the manufacturer in the IFU provided with the device is 94.15% and 93.91% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 61.0% while the total antibody specificity was found to be 100%. For samples collected 14 days post onset of symptoms the overall sensitivity improved to 93.8%.

VivaDiag COVID-19 IgM/IgG Rapid Test manufactured by VivaChek Biotech (Hangzhou) Co Ltd, China

The overall total antibody sensitivity claimed by the manufacturer in the IFU provided with the device is 81.25% between days 4 to 10 and 97.1% between days 11 to 24. Overall claimed total antibody specificity was 100%.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 51.8% while the total antibody specificity was found to be 97.8%. For samples collected 14 days post onset of symptoms the overall sensitivity improved to 78.5%.

EasyDiagnosis SARS-CoV-2 IgM/IgG Antibody Test Kit manufactured by Wuhan EasyDiagnosis Biomedicine Co Ltd (China)

The overall total antibody (i.e. either IgM or IgG) sensitivity and specificity claimed by the manufacturer in the instructions for use (IFU) provided with the device is 97.3% and 99.7% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 95% while the total antibody specificity was found to be 91%. For samples collected 15-21 days post onset of symptoms the overall sensitivity was 100%.

COVID 19 IgG/IgM Rapid Test Cassette manufactured by Zhejiang Orient Gene Biotech Co Ltd, China

The sensitivity and specificity claimed by the manufacturer in the IFU for IgM is 87.9% and 100.0% respectively; and for IgG, 97.2% and 100.0% respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 90.0% while the total antibody specificity was found to be 99.0% For samples collected 14 days post onset of symptoms the overall sensitivity improved to 97.4%.

Diagnostic Kit for IgM/IgG Antibody to Coronavirus (SARS-CoV-2) manufactured by Zhuhai Livzon Diagnostics Inc, China

The overall total antibody sensitivity and specificity (i.e. either IgM or IgG) claimed by the manufacturer in the IFU is 90.6% and 99.2% respectively. The sensitivity and specificity claimed by the manufacturer in the IFU for IgM is 79.0% and 99.7%, respectively; and for IgG is 84.3% and 99.4%, respectively.

The overall total antibody sensitivity of the device when tested by the Doherty Institute was found to be 58.0% while the total antibody specificity was found to be 99.0% For samples collected 14 days post onset of symptoms the overall sensitivity was 57.9%.

It should be noted that none of the manufacturers claim that these tests should be used as a sole basis for diagnosis of COVID-19 and advise that results need to be interpreted along with other clinical findings and consideration should be given to further testing.

Some of the variation observed between the different manufacturers' claims and the Doherty Institute's findings can be attributed to sample size and sampling bias and it is apparent that different patient populations and samples taken at different times post onset of symptoms can significantly influence the performance of the tests. The Doherty Institute also noted that a direct comparison with the manufacturers' claims was difficult as the performance information provided in the IFUs was limited. However four of the devices, the GenBody COVID-19 IgM/IgG, the Innovita 2019-nCoV Ab Test, PCL COVID19 IgG/IgM Rapid Gold and the Diagnostic Kit for IgM/IgG Antibody to Coronavirus (SARS-CoV-2), produced results that differed significantly from the manufacturer's claims and performed poorly in comparison to similar devices evaluated by the Doherty Institute.

The Doherty Institute's studies, along with other evaluations performed by Australian and international laboratories confirm that the sensitivity of these tests in the early stages of infection is poor and that they shouldn't be used for the diagnosis of acute COVID-19 infection. This conclusion is consistent with current advice from the Public Health Laboratory Network, the Royal College of Pathologists of Australasia and the TGA that these tests must be used with caution due to the potential for these tests to fail to detect COVID-19 during the acute phase of the illness, prior to the development of antibodies.

Based on the Doherty Institute's findings it is recommended that serology-based COVID-19 POC tests should not be used until at least two weeks post onset of symptoms. As yet not enough is known about the adequacy of the COVID-19 immune response or duration of immunity. Whether there is a role for these tests in determining immunity for return to work purposes or for population-level surveillance remains to be seen.

Post-market regulatory action

The TGA has requested the sponsors of these devices to provide further information on the clinical studies undertaken by the manufacturer to support the performance of these devices. This includes a re-evaluation of the clinical performance studies including stratification of results according to different periods post onset of symptoms and will provide a better basis for comparing the manufacturer's claims against the Doherty Institute's findings. This will inform the need for any subsequent regulatory action which may include changes to the IFUs to include more detail about the test sensitivity for samples collected at different time points post onset of symptoms; and additional warnings to inform the user of the limitations of test. Other regulatory action includes suspension or cancellation of non-compliant or poor performing tests from the ARTG. Further information can be found via the post market review of COVID-19 point-of-care tests page.