Clinical performance requirements and risk mitigation strategies for HIV tests

On this page: 1. Purpose | 2. Clinical performance characteristics and risk mitigation strategies for HIV tests | Version history

1. Purpose

The purpose of this document is to provide manufacturers and sponsors with guidance on TGA's expectations in regard to clinical performance requirements (i.e. clinical sensitivity and specificity) and risk mitigation for in vitro diagnostic medical devices (IVDs) intended to be used to screen or diagnose human immunodeficiency virus (HIV) infection.

This guidance applies to Class 4 HIV antibody/antigen based tests that are intended to be used:

• in the laboratory for diagnostic and/or donor screening or reference testing; and
• at the point of care (PoCT) or in the self-test environment for presumptive screening.¹

This document identifies key risks that must be mitigated and identifies conditions that may be imposed on HIV test kits when they are included in the Australian Register of Therapeutic Goods (ARTG). Additional risks and mitigation strategies, including conditions of approval, may apply to individual devices on a case-by-case basis.

The performance requirements for HIV nucleic acid tests (NAT) intended for use in the laboratory setting are only briefly described.

Other aspects of demonstrating safety and performance, such as analytical performance studies or stability studies for HIV tests, are not addressed in this document.

The Australian sponsor of a Class 4 HIV test must apply to include the device in the ARTG. Sponsors can support the application using a conformity assessment document issued to the test manufacturer by a European notified body under the European IVD Directive 98/79/EC or EU IVD Regulations (2017/746). Alternatively, the sponsor or test manufacturer may apply to the TGA for a TGA Conformity Assessment Certificate.

Manufacturers are required to hold full technical documentation to demonstrate that their device complies with the essential principles. For further information on the Use of market authorisation evidence from comparable overseas regulators/assessment bodies for medical devices (including IVDs) and the technical documentation requirements to be included in the Summary Technical File (STED) or design dossier for IVDs, please see the guidance material provided on the TGA website.

2. Clinical performance characteristics and risk mitigation strategies for HIV tests

All tests for HIV should demonstrate the highest possible standard of clinical performance relative to the intended purpose of the test (essential principles 14 and 15). Different clinical performance requirements and risk mitigation strategies, including conditions of approval, are applicable depending on the nature of the test and take into consideration:

• the intended purpose of the test (e.g. presumptive screening test versus donor screening or confirmatory testing)
• the format of the test (e.g. simple rapid versus automated tests)
• the intended user of the test (e.g. whether it is laboratory-based, a PoCT or a self-test) and the environmental conditions under which the test would be conducted
• the specimen type (e.g. oral fluid versus finger-stick whole blood).

The purpose of this stratified approach is to balance the need for high quality tests with clinical characteristics that are fit for purpose.

Overall acceptability of any test for the purposes of inclusion in the ARTG depends on compliance of the test device with the essential principles and in particular, a demonstration that the test does not compromise health and safety, is suitable for the intended purpose and the benefits of the test outweigh any residual risks associated with its use (essential principles 1, 3 and 6).

2.1 Laboratory tests

Laboratory tests include donor and diagnostic screening tests and reference tests intended to be used in a laboratory environment by trained laboratory professionals. Samples that are positive (i.e. reactive) when tested with a screening test then undergo confirmatory or additional supplementary testing in accordance with a validated testing algorithm to confirm the positive status before the result is accepted as a true positive.

Clinical characteristics

TGA will be guided by the European Union (EU) Common Technical Specifications (CTS)² as an appropriate standard for laboratory-based screening tests. Manufacturers are expected to comply with these requirements, but can adopt solutions of a level equivalent to the EU CTS for appropriately justified reasons. This reflects the importance of these tests in relation to screening the blood supply and diagnostic testing strategies. Laboratory-based HIV tests intended for donor or diagnostic screening are required to have the following performance characteristics in relation to the detection of HIV antibodies:

• 100% sensitivity for true HIV positive samples and ≥ 99.5% specificity based on a direct comparison with an established state-of-the-art device (e.g. a fourth-generation enzyme immunoassay (EIA)).
• 100% sensitivity and ≥ 99.5% specificity for HIV seroconversion samples based on a direct comparison with an established state-of-the-art device. A seroconversion sample is defined in the EU CTS as p24 antigen and/or HIV RNA positive, recognised by all of the antibody screening tests, and positive or indeterminate confirmatory assays.
• At least 40 early HIV seroconversion samples should be tested and performance should be comparable to an established state-of-the-art device. Early seroconversion samples are defined in the EU CTS as p24 antigen and/or HIV RNA positive, not recognised by all of the antibody screening tests, and indeterminate or negative confirmatory assays.

The EU CTS provides further guidance on other performance evaluation requirements such as appropriate specimen numbers and sample selection. Additional requirements are outlined with regard to the detection of HIV-1 antigen in combined antibody/antigen tests (i.e. fourth generation EIA). True positive HIV samples are expected to be characterised and discordant results investigated using testing algorithms in accordance with a well-established laboratory case definition for HIV.³

Although laboratory tests are required to perform to the highest possible standard, it is recognised that no test is necessarily 100% sensitive in all circumstances and these tests should be evaluated in the context of:

• their performance in a population equivalent to the Australian population (i.e. similar prevalence rate as that in Australia)
• whether the benefits of performing the test outweigh any potential risks.

HIV reference tests (e.g. western blot) must have a demonstrated high level of clinical sensitivity, but it is recognised that these tests may not necessarily perform to the same standard as established screening tests particularly for the detection of HIV during seroconversion. Therefore reference tests, such as the western blot, will be evaluated primarily on the basis of test specificity which reflects the key role these tests have in confirming true positive status by distinguishing true from false positive results.

For laboratory-based rapid tests intended for screening or confirmatory diagnosis, the clinical performance requirements are the same as for HIV PoCT described below.

The performance evaluation criteria for laboratory-based NAT tests will also be guided by the EU CTS.

Risk mitigation strategies

In the laboratory setting, residual risks associated with use of the product are mitigated by the fact that these tests are performed by qualified staff in medical testing laboratories accredited by the National Association of Testing Authorities (NATA) or laboratories that hold a TGA issued Good Manufacturing Practice (GMP) licence with appropriate quality control and quality assurance procedures in place to continually monitor the performance of the test and quality of the results.

2.2 HIV Point of Care Tests (PoCTs)

HIV PoCTs are screening tests for HIV that are intended to be performed in a clinical setting (i.e. outside the laboratory environment), near to or at the side of the patient by a health professional or appropriately trained user who can interpret the test and provide appropriate clinical support. Confirmation of positive results is required using a diagnostic laboratory test.

Clinical characteristics

HIV PoCTs must demonstrate a high level of clinical sensitivity and specificity, but it is recognised that these tests may involve the use of alternative specimen types that are more convenient to the user in a point-of-care setting (e.g. fingerstick whole blood, oral fluid) and may not necessarily perform to the same standard as laboratory tests that are intended for professional use (e.g. third or fourth-generation EIA on serum/plasma). There are grounds for a more flexible approach where it can be demonstrated that:

• the manufacturer has clearly identified the limitations of the test and provided acceptable evidence of risk mitigation; and
• the benefits of the test outweigh any potential risks associated with use of the product.

HIV PoCTs are required to demonstrate the following minimum clinical performance requirements in relation to the detection of HIV antibodies (in the context of their performance in a population with HIV prevalence similar to Australia):

• a clinical sensitivity of at least 99.5% for whole blood, serum or plasma and 99% for oral fluid (based on testing performed outside the window period for the device)⁴; and
• a clinical specificity of at least 99% for detection of HIV infection.⁵

Clinical performance evaluations should be carried out in direct comparison with an established state-of-the-art device (e.g. a third or fourth-generation enzyme immunoassay (EIA)). The TGA will continue to be guided by the EU CTS with regard to appropriate specimen numbers and sample selection for the evaluation of HIV rapid tests (e.g. 400 HIV-1 antibody positive specimens and 1,000 negative specimens).

True positive HIV samples are expected to be characterised and discordant results investigated using testing algorithms in accordance with a well-established laboratory case definition for HIV (as previously described).

Manufacturers claiming that their HIV test kit identifies HIV in oral fluid should conduct parallel testing with a state-of-the-art serological assay using serum or plasma from the same individuals.

Manufacturers are required to provide studies that demonstrate the performance of the test with seroconversion panels in order to establish the limitations of the tests with regard to the window period for the device. The performance of the device should be comparable to an established state-of-the-art rapid test.

Manufacturers are also expected to provide clinical evidence in the form of usability studies and/or a review of relevant published literature that support the performance of the device in the point of care setting.

These performance requirements ensure that tests used at the point-of-care are of a high quality while also reflecting the fact that rapid HIV tests intended for use at the point of care are intended for presumptive screening for HIV rather than as part of a laboratory-based diagnostic/confirmatory testing strategy.

Risks

False negative results are more likely to occur if lower levels of clinical sensitivity are accepted (i.e. less than 100%). There will be a greater 'window period' resulting in a higher number of false negative results if testing is performed during the acute phase of infection, and prior to seroconversion. There is also an increased likelihood of false positives if lower levels of specificity are accepted and if HIV PoCTs are used in low-prevalence populations.

Despite these limitations it has been accepted that the benefits to be gained from the use of HIV PoCT (i.e. increased testing rates) at sensitivities and specificities below 100% outweigh any undesirable effects arising from its use (i.e. false negative and false positive results).

Mitigation strategies

HIV PoCTs differ from laboratory tests and self-tests in that a health professional or appropriately trained user is responsible for performing or supervising all aspects of the testing process from sample collection to test interpretation. Unlike a self-test, a health professional or trained user is available to obtain informed consent from the person being tested and can provide information on the limitations of the test, the risk of false positives and the risk of false negative results, particularly if testing is done soon after possible exposure to the virus.

The overall acceptability of a HIV PoCT will depend on the mitigation strategies the manufacturer has in place to offset any potential risks associated with the use of the product. For example:

• the test must be easy to perform with minimal operator intervention or procedural steps
• the instructions for use (IFU) must be clear and easy to understand
• the sensitivity and specificity of the test must be clearly identified in the IFU
• the IFU should clearly state the limitations of the procedure such as:
  • false negative results can be obtained if testing is performed after recent exposure to HIV (i.e. during the early stages of infection prior to seroconversion when antibody is below detectable limits of the test) and repeat testing within 3 months should be recommended to confirm the initial negative result
  • positive results require confirmation using another test method
  • results should be evaluated in light of the overall clinical assessment.

Other requirements for the IFU and information provided with a device, including product labelling, are detailed in essential principle 13.

Evidence must also be provided that demonstrates the stability and reliability of the product across a range of operational and environmental conditions.

Depending on the performance of the test and the information provided in the IFU, TGA may impose conditions on the supply of a particular PoCT to ensure that the product is only supplied in an appropriate point-of-care setting. These are likely to include a requirement that the sponsor:

• supply the device only for use by:
  • NATA accredited laboratories or organisations that employ health professionals (e.g. medical practitioners, registered nurses) who perform or supervise⁶ the performance of testing by appropriately trained staff⁷; and
  • that participate in an HIV point of care quality assurance program
• make available training in the correct use of the device and interpretation of results (the sponsor is not expected to provide training in the delivery and administration of HIV PoCTs)
• provide the TGA with regular reports on:
  • the distribution of the product; and
  • numbers of any false positive or false negative results or problems with the test.

These conditions will ensure that high quality tests are used at the point-of-care and are conducted by health care professionals and other appropriately trained users in an environment where the individual can be provided with appropriate counselling and follow-up testing and treatment if required. Additional conditions may be considered on a case-by-case basis and will depend on the individual product.

2.3 HIV self-tests

HIV self-tests are rapid presumptive screening tests for HIV intended to be used in the home or similar environment by a person who does not have formal training in a medical field or discipline related to HIV testing. Confirmation of positive results is required using a diagnostic laboratory test.

Clinical characteristics

Because HIV self-tests and HIV PoCTs are both presumptive rapid HIV screening tests (and generally the technology is the same), the expected "benchmark" level for the inherent clinical sensitivity and specificity for a self-test across the relevant specimen types such as whole blood (fingerstick) and oral fluid would be the same as that expected for HIV PoCTs:

• a clinical sensitivity of at least 99.5% for whole blood and 99% for oral fluid
• a clinical specificity of at least 99% for detection of HIV infection.

This would reflect the expected performance of the test with clinical specimens in comparison to a currently accepted state of the art device (e.g. third or fourth generation EIA). The TGA will continue to be guided by the EU CTS with regard to appropriate specimen numbers and sample selection for the evaluation of HIV rapid tests (e.g. 400 HIV-1 positive specimens and 1,000 negative specimens). True positive HIV samples are expected to be characterised and discordant results investigated using testing algorithms in accordance with a well-established laboratory case definition for HIV.

As self-tests would predominantly be performed by inexperienced users, usability studies would be required to establish performance of the test in the hands of a layperson (i.e. in the hands of an untrained/inexperienced user). However, it is recognised that the same level of sensitivity and specificity may not be achieved in a self-testing environment.

The manufacturer is not required to provide Australian specific usability studies, but it is expected that studies will reflect the performance of the test in a comparable setting and prevalence to the Australian population.

Usability studies are expected to address:

• Inter-reader variability
• An inter-reader variability study should take into account individual’s ability to interpret pre-determined results (e.g. positive results, weakly positive results, negative and invalid results).
• A significant inter-reader variability (e.g. ≥ 5%) for clearly positive or negative results implies that the device is not easy to use or may be difficult to interpret resulting in an increased rate of false negative or positive results.
• Invalid test rate
• The incidence of operational errors and test system failures that lead to an invalid result should be determined. This will provide an indication of the reliability and robustness of the test. Ideally the invalid test rate would be expected to be ≤ 2% of the total tested (this includes defective tests or components).
• 'User' sensitivity/specificity studies
• Studies should be performed to establish the 'user' sensitivity and specificity of the test in hands of an untrained/inexperienced user in the self-testing environment. The 'user' sensitivity and specificity should be estimated in comparison to the true HIV status of the individual as determined by laboratory testing (e.g. EIA). Studies should include participants from high and low prevalence setting.
• A 'user' sensitivity of < 90% would not be considered acceptable. A 'user' sensitivity of 90-95% could be considered acceptable where evidence of significant public health benefits can be demonstrated and where thorough risk mitigation strategies have been put in place to minimise the risk of false negative and false positive results.

The suitability of these studies will be assessed on a case-by-case basis and will depend on how well the manufacturer has mitigated any risks and demonstrated that the overall benefits of the product outweigh any residual risks associated with its use. Demonstration of the benefit of a test and effectiveness of risk mitigation measures in the self-testing environment may be supported by a documented review of relevant published literature.⁸

It is not a requirement that a HIV self-test would need to be evaluated and registered for use at the point of care before it could be evaluated as a self-testing device.

Risks

As is the case for HIV PoCTs, false negative results are more likely to occur if a test has a lower level of sensitivity and if testing is performed during the 'window' period for the device. False positive results are also more likely to occur if these tests are used in lower prevalence populations. Although these limitations also apply to PoCTs, they are exacerbated in a self-testing environment due to individual user variability in the correct performance and interpretation of the test (i.e. the risks are predominantly user focussed). Additionally, in the self-test environment, follow up testing is not easily able to be encouraged or implemented.

Mitigation strategies

The proposed mitigating strategies recognise that self-tests differ from laboratory-based tests and PoCT in that the user is responsible for all aspects of the testing process from sample collection to test interpretation.

Some of the mitigation strategies for HIV PoCTs are equally relevant to self-tests, that is:

• The specimen collection process must be straightforward and the specimen able to be collected safely in the home testing environment.
• The test must be easy to perform with minimal operator intervention or procedural steps. Extensive usability studies would be expected (e.g. device interpretation study, label comprehension study and observed self-testing studies).
• The stability of the product should be demonstrated across a range of operational and environmental conditions.

In addition, the manufacturer/sponsor of a HIV self-test is also expected to clearly outline the limitations of the test and provide clear advice, in the IFU and/or other information provided with the test, including the following:

• clear and simple instructions on how to perform and interpret the test
• the 'user' sensitivity and specificity of the test (i.e. in a self-testing environment) must be clearly identified
• clear warnings on the risk of false negative results if testing is performed in the 'window period' (and a clear explanation of what the window period is)
• warning that negative results obtained within three months of a high risk event should be repeated at three months to confirm the initial negative result
• clear indication that HIV self-testing is for presumptive screening only and the need to consult a medical practitioner for confirmatory testing of positive results by a laboratory test
• information should be included to identify groups at high risk
• information on behaviour that may place an individual at an increased risk for HIV infection and the need to test frequently if there is an ongoing risk, including a warning that a negative result does not indicate that engaging in high risk behaviour is safe
• how to contact locally available support and counselling services including phone lines and websites.

It is also recommended that the IFU contain information to promote safe sex and safe injecting practices and the need for individuals engaging in high risk behaviours to undergo testing for other sexually transmitted infections and blood borne viruses.

Other requirements for the IFU and information provided with a device, including product labelling, are detailed in essential principle 13.

Depending on the performance of the test and the information provided in the IFU and robustness of the test, TGA may impose conditions on the supply of a particular HIV self-test to encourage the effective and safe use of the product. These are likely to include a requirement that the sponsor:

• provide additional support for users of the test through provision of information that will direct users to on-line support services and/or 24/7 phone line
• provide the TGA with regular reports on the distribution of the product, numbers of tests sold and numbers of any reported false positive or false negative results or problems with the test.

Additional conditions would be applied on a case-by-basis and would depend on the evaluation of an individual product, the overall benefits and how well any risks have been mitigated.

Footnotes

1. This guidance does not apply to Class 3 HIV test kits which are intended for patient management and monitoring.
2. Commission Decision of 27 November 2009 amending Decision 2002/364/EC on common technical specifications for in vitro diagnostic medical devices (notified under document C(2009) 9464) (Text with EEA relevance) (2009/886/EC).
3. Accepted laboratory case definitions would include, as the manufacturer prefers, Australian Government Department of Health Human immunodeficiency virus (HIV) (unspecified) case definition; World Health Organisation WHO case definitions of HIV for surveillance and revised clinical staging and immunological classification of HIV-related disease in adults and children; United States Centres for Disease Control and Prevention (CDC) Revised Surveillance Case Definition for HIV Infection.
4. It has been reported that rapid testing using oral fluid has a lower sensitivity compared with whole blood and that this may be attributable to a lower quantity of HIV antibodies in oral fluid rather than a variation in the inherent performance characteristics of the test itself, (Pai, NP et al 2012, 'Head-to-Head Comparison of Accuracy of a Rapid Point-of-Care HIV Test with Oral Versus Whole-Blood Specimens: A Systematic Review of Meta-Analysis' Lancet Infectious Disease, vol. 12, no. 5, pp. 373-380).
5. This requirement is guided by the EU CTS.
6. Supervision in this context can encompass direct supervision or indirect supervision providing appropriate protocols and procedures are in place to ensure that immediate access to advice from health professionals is available when required.
7. Training in regard to the delivery and administration of HIV PoCTs.
8. The literature review may include data for devices used for similar intended purpose(s) as the device under assessment.

Version history

Version	Description of change	Author	Effective date
V1.0	Original publication	Devices Authorisation Branch	March 2015
V1.1	Addition of information on conformity assessment documents issued by European notified bodies that can be used to support an application to include an HIV test in the Australian Register of Therapeutic Goods (ARTG).	Devices Authorisation Branch	April 2023