Evaluating seasonal influenza in-vitro diagnostic (IVD) self-tests – clinical performance and risk mitigation

The Therapeutic Goods Act 1989 (the Act) provides a system of controls for the regulation of therapeutic goods in Australia. 

Home-use tests (also known as self-tests) are therapeutic goods.

Self-tests for serious diseases (e.g. notifiable infectious diseases, sexually transmitted infections, cancer, genetic markers of disease) have been prohibited from supply in Australia since 1 July 2010 under the Therapeutic Goods (Excluded Purposes) Specification 2010 (the Excluded Purposes Specification 2010). 

The exception was self-tests for Human Immunodeficiency Virus (HIV) which have been permitted since 2014.

In accordance with the Legislative Instruments Act 2003, legislative instruments are automatically repealed after a fixed period of time (subject to some exceptions). 

The Excluded Purposes Specification 2010 sunset on 1 October 2020. Before the remaking of the instrument, it was a legal requirement to perform a review.

On 4 September 2020, after formally consulting stakeholders and following a review of self-testing regulations, the Therapeutic Goods (Medical Devices – Excluded Purposes) Specification 2020 (the Excluded Purposes Specification 2020) was made, and came into effect on 1 October 2020. 

Sponsors and manufacturers will now be able to apply to the TGA for the inclusion in the ARTG of specified IVD self-tests; including self-tests for influenza. 

Any tests allowed under the Excluded Purposes Specification 2020 can only be supplied following evaluation of individual products by the TGA to ensure appropriate clinical performance requirements are met and risk mitigations are in place.

This guidance refers to seasonal influenza only and does not include self-tests to detect specific influenza strains that are novel or emerging (e.g. pandemic strains) which will remain prohibited.

Seasonal Influenza

Influenza, or 'the flu', is a viral infection of the nose, throat and lungs (the respiratory system). In Australia, it usually affects people during the winter months from June to September. The flu viruses that circulate every winter are often similar to those from the preceding winter, so there is already a level of immunity (body defences) in the community. Seasonal influenza most commonly affects the very young or the elderly. ^[1]

Seasonal influenza strains (usually A (e.g. H1N1 and H3N2) and B) generally follow predictable seasonal patterns and occur annually. Only influenza type A viruses are known to have caused pandemics^[2][3]

Influenza Pandemic

An influenza pandemic is when a new influenza A virus emerges that is very different from current and recently circulating human seasonal influenza A viruses³. New (novel) influenza A viruses infect people easily and spread from person to person in an efficient and sustained way^[3]

Influenza is a serious global health threat that affects all countries: every year, there are an estimated 1 billion cases, 3-5 million severe cases, and 290 000-650 000 influenza-related respiratory deaths worldwide.^[4]

The Global Influenza Strategy for 2019-2030 provides a framework for WHO, countries and partners to approach influenza holistically through robust national programmes - from surveillance to disease prevention and control - with the goal of strengthening seasonal prevention and control and preparedness for future pandemics.^[5]

Australian seasonal influenza infection rates fluctuate annually, depending on the circulating strains and does pose a serious health risk to priority/vulnerable populations. 

The appropriate use of anti-viral medication along with growing public awareness of adequate infection control procedures is aimed at reducing the incidence of circulating seasonal influenza within the Australian community.

The Excluded Purposes Specification 2020 allows manufacturers and sponsors to apply for inclusion of seasonal influenza self-tests in the ARTG, allowing for the legal supply of these self-tests in Australia (subject to TGA evaluation and approval). 

Legal supply of influenza self-tests may improve access for testing among priority populations and allow self-test kits to be safely and legally distributed in Australia. 

Reduced delays in testing may support earlier access to intervention and treatment, including antiviral medications that could assist in the reduction of transmission rates and the severity of complications associated with seasonal influenza.

All self-tests for serious diseases should demonstrate the highest possible standard of clinical performance relative to the intended purpose and classification of the test (Essential Principles 14 and 15 of the Therapeutic Good (Medical Devices) Regulations 2002 (the Medical Device Regulations)).

Different clinical performance requirements and risk mitigation strategies, including imposing conditions of inclusion on the ARTG, may be applicable depending on the nature of the test and take into consideration:

• the intended purpose of the test (e.g. presumptive screening test)
• the format of the test (e.g. antigen or antibody)
• the environmental conditions under which the test would be conducted by a lay person (i.e. in the hands of an untrained/inexperienced user); and
• the specimen type (e.g. finger stick whole blood, or oropharyngeal swab).

Note
For an IVD medical device for self-testing, a lay person is defined as an individual who does not have formal training in a medical field or discipline to which the self-testing relates.

For the full definition, refer to the Therapeutic Goods (Medical Devices) Regulations 2002.

The purpose of this approach is to balance the need for high quality tests with clinical characteristics that are fit for purpose.

Manufacturers must also meet the specific requirements for self-tests in accordance with Essential Principle 15:

• An IVD medical device for self-testing must be designed and manufactured so that it performs appropriately for its intended purpose, taking into account the skills and the means available to users and the influence resulting from variation that can reasonably be anticipated in the user's technique and environment
• The information and instructions provided by the manufacturer of an IVD medical device for self-testing must be easy for the user to understand and apply; and
• An IVD medical device for self-testing must be designed and manufactured in a way that reduces, to the extent practicable, the risk of error in the use of the device, the handling of the sample and the interpretation of results.

Overall acceptability of any test for the purposes of inclusion in the ARTG depends on compliance of the test device with the Essential Principles and in particular, a demonstration that the test does not compromise health and safety, is suitable for the intended purpose and the benefits of the test outweigh any residual risks associated with its use (Essential Principles 1, 3 and 6).

Influenza self-tests are rapid presumptive screening tests for seasonal influenza (usually type A and B) intended to be used in the home or similar environment by a lay person. 

Confirmation of positive results is required using a laboratory test. 

A negative result does not mean a person does not have influenza, and if symptoms persist, the person should seek medical attention and further testing if required.

Clinical characteristics and clinical performance requirements for seasonal influenza self-tests

Influenza self-tests are expected to demonstrate a high level of clinical sensitivity and specificity. 

It is anticipated that these will be predominantly antigen tests performed on nasal or throat swabs, however these may involve alternative specimen types, and may not necessarily perform to the same standard as laboratory tests that are intended for professional use. 

Serologic (antibody detection) testing is not suitable for diagnosis of influenza (as recent infection can only be reliably diagnosed by demonstrating a significant rise in influenza-specific antibody titres) and provides retrospective information only.

Influenza self-tests must meet minimum clinical performance requirements for clinical sensitivity and specificity for a self-test across the claimed specimen types such as nasopharyngeal, nasal or throat swab:

• an overall clinical sensitivity of at least 85%; and
• an overall clinical specificity of at least 95% for the detection of influenza infection ^[6].

The clinical sensitivity and specificity on samples taken from patients at the peak time for viral shedding (i.e. within the first 4 days following onset of symptoms) would be expected to be >95%. 

Clinical performance studies should include information on when samples were taken and tested (i.e. days post symptom onset) and clearly demonstrate the optimal days for testing.

The clinical sensitivity and specificity reflect the expected performance of the test with clinical specimens in comparison to the currently accepted reference standard, a reverse transcriptase polymerase chain reaction (RT-PCR) test. 

It is expected that the self-test shall have an overall performance equivalent to that of the established device.

The TGA will expect statistically appropriate specimen numbers and sample selection appropriate to each strain or strains of influenza detected, for the evaluation of an influenza self‑test. 

Positive samples are expected to be characterised and discordant results investigated using testing algorithms in accordance with a well-established laboratory case definition for influenza ^[7][8]. 

Manufacturers claiming that their kit identifies influenza in nasopharyngeal, nasal or throat swabs should conduct parallel testing to demonstrate specimen equivalence. 

Contrived samples would not be acceptable in determining clinical sensitivity and specificity. 

It is expected that sample stability will be demonstrated for all claimed specimen types intended for use with the test.

As self-tests would predominantly be performed by a lay person, clinical evidence in the form of usability studies would be required to establish performance of the test in the hands of these users. 

It is expected that the clinical performance studies would include clinical patient samples in their usability studies.

The manufacturer is not required to provide Australian specific usability studies, but it is expected that studies will reflect the performance of the test in a comparable setting and relevant to the Australian population.

Usability studies are expected to address:

• User comprehension

A usability study should take into account the ability of the user to interpret the instructions for use (IFU), and to ensure that the labelling is clear and easy to follow (e.g. a questionnaire to assess the ability of users to correctly comprehend instructions for use, limitations, diagrams, result interpretation and access to follow-up services).

• Inter-reader variability

An inter-reader variability study should take into account an individual's ability to interpret pre-determined results (e.g. positive results, weakly positive results, negative and invalid results).

A significant inter-reader variability (e.g. ≥ 5%^[9][10]) for clearly positive or negative results implies that the device is not easy to use, the IFU is not clear enough, or the test may be difficult to interpret resulting in an increased rate of false negative or positive results.

• Invalid test rate

The incidence of operational errors and test system failures (e.g. failure to sample correctly resulting in an unreadable result or user not being able to interpret the result) that lead to an invalid result should be determined. This will provide an indication of the reliability and robustness of the test. Ideally the invalid test rate would be expected to be ≤ 2%^[10][11] of the total tested (this includes defective tests or components).

• User sensitivity/specificity studies

Studies should be performed to confirm the clinical sensitivity and specificity of the test in hands of a lay person in the self-testing environment. The user sensitivity and specificity should be estimated in comparison to the true influenza status of the individual as determined by laboratory testing (e.g. RT-PCR). Where possible studies should include participants from high and low prevalence setting.

Preferably, the user sensitivity should be ≥85%. A user sensitivity of <85% may be considered acceptable where evidence of significant public health benefits can be demonstrated and where thorough risk mitigation strategies have been put in place to minimise the risk of false negative and false positive results.

The suitability of these studies will be assessed on a case-by-case basis and will depend on how well the manufacturer has mitigated any risks and demonstrated that the overall benefits of the product outweigh any residual risks associated with its use. Demonstration of the benefit of a test and effectiveness of risk mitigation measures in the self-testing environment may be supported by a documented review of relevant published literature ^[13].

False negative results are more likely to occur (for antigen tests) if a test is performed outside the window of highest viral shedding of the influenza virus (which is usually within the first 4 days of onset of symptoms). 

The influenza antigen self-test has a lower level of sensitivity if testing is performed outside of this period. Self-tests may have different specifications for different specimen types (nasopharyngeal, nasal, throat swab) and the quality of the specimen collected may also affect results. 

It is expected that the majority of seasonal influenza self-tests will be antigen based.

False positive results are more likely to occur when influenza prevalence in the community is low, which is generally at the beginning and end of the influenza season or periods in which influenza viruses are not circulating (e.g. summertime). 

False negative results are more likely to occur when influenza prevalence is high in the community, which is typically at the peak of the influenza season ^[14].

The above risks are exacerbated in a self-testing environment due to individual user variability in the correct performance and interpretation of the test and adequate specimen collection (i.e. the risks are predominantly user focussed). 

Additionally, in the self-test environment, follow up testing is not easily encouraged or implemented.

The proposed mitigating strategies recognise that self-tests differ from laboratory-based tests and point-of-care tests in that the user is responsible for all aspects of the testing process from sample collection to test interpretation.

Self-tests for influenza will be subject to mandatory application audits prior to entry in the ARTG.

The mitigating strategies for influenza self-tests are:

• The specimen collection process must be straightforward, the instructions for specimen collection must be clear and easy to understand, and the specimen able to be collected safely in the home testing environment
• The test must be easy to perform with minimal operator intervention or procedural steps. Extensive usability studies would be expected (e.g. device interpretation study, label comprehension study and observed self-testing studies)
• The stability of the product should be demonstrated across a range of operational and environmental conditions; and
• The strains of virus that can be detected by the influenza self-test should be appropriate for the detection of seasonal strains of virus that are circulating, or prevalent within Australia.

Requirements for the instructions for use (IFU)

In addition, the manufacturer/sponsor of an influenza self-test is also required to clearly outline the limitations of the test and provide clear advice, in the IFU and/or other information provided with the test, including the following:

• clear and simple instructions on how to perform and interpret the test (this may involve images or visual representation of the instructions, flow diagrams or QR codes linking to online demonstrations)
• available either in print or online in multiple languages (e.g. including local languages)
• which strains of influenza the test covers
• the clinical sensitivity and specificity of the test (i.e. in a self-testing environment) must be clearly identified (including information on the clinical sensitivity/specificity of the test at various time points post symptom onset)
• clear information on when testing should be performed (e.g. within the first 4 days of symptom onset when viral shedding is highest)
• clear warnings on the risk of false negative results
• clear indication that influenza self-testing is for presumptive screening only and the need to consult a medical practitioner for confirmatory testing of positive results by a laboratory test and for follow-up clinical care
• negative results may not mean that a person is not infectious and if symptoms persist to seek medical assistance
• a negative result does not rule out infection with another type of respiratory virus
• information on other limitations of the test such as a positive result cannot necessarily determine whether a person is infectious
• a statement to the user that the test can only be used once
• information on how to safely dispose of the kit and its contents
• how to contact locally available support services including phone lines and websites; and
• how to contact the TGA to report poor performance or usability issues in the self-test environment (report an issue via the Users Medical Device Incident Report, email iris@tga.gov.au or call 1800 809 361).

It is also recommended that the IFU contain information to promote good infection control procedures of individuals to reduce the spread of viruses to the general population.

Other requirements for the IFU and information provided with a device, including product labelling, are detailed in Essential Principle 13 of the Medical Device Regulations.

Post-market monitoring and standard conditions of inclusion

All sponsors of self-tests included in the ARTG have ongoing responsibilities under the Act, the Medical Device Regulations and the Therapeutic Goods Advertising Code the Advertising Code), including conditions that apply automatically to all ARTG entries as described in the Australian Regulatory Guidance for Medical Devices. 

These conditions facilitate post-market monitoring and include, but are not limited to, the following:

• allowing entry and inspections of premises
• delivery of device samples upon request
• availability of information, such as facilitating access to technical documentation that demonstrates compliance with the Essential Principles
• ensuring any advertising material relating to the medical device complies with regulatory requirements; and
• reporting details of certain incidents and performance issues to the TGA, and any overseas regulatory actions to the TGA if the product involved is from the same batch or production run that was supplied in Australia.

All sponsors are also required to report adverse events to the TGA.

Additional conditions that may be applied

Depending on the performance of the test, the information provided in the IFU and robustness of the test, the TGA may impose additional non-standard conditions to mitigate any residual risk identified relating to the effective and safe use of the product or to facilitate the monitoring of potential trends. 

These are likely to include a requirement that the sponsor:

• provide additional support for users of the test through provision of information that will direct users to on-line support services and/or 24/7 phone line
• provide the TGA with regular annual reports on the distribution of the product, numbers of tests sold and numbers of any reported false positive or false negative results or problems with poor performance of the test in Australia and worldwide (this may be a combination of monthly and annual reporting requirements)
• may potentially only supply the device through specified distribution channels that allow relevant information/education to be provided to users at the time of purchase. This will be considered on a case-by-case basis and will depend on what risks need to be mitigated.

Any further conditions would be applied on a case-by-case basis and would depend on the evaluation of an individual product, the overall benefits, and how well any risks have been mitigated.

Post-market review

The TGA can conduct a post-market review of certain kinds of devices included in the ARTG. 

ARTG entries for influenza self-tests may also be subject to a post-market review and sponsors may be asked to provide a number of test kits for independent laboratory evaluation of the clinical sensitivity and specificity to verify their performance.

Note
Advertisements for IVDs, including self-tests, are subject to the requirements of the Act. For advertising to consumers, this includes the requirement to comply with the Advertising Code.

The Advertising Code requires advertising to consumers to be accurate and not misleading (including misleading through the omission of important information, like the limitations of an IVD).

Consumer advertising for IVDs for detecting or diagnosing a serious disease, condition, ailment or defect is likely to contain a restricted (e.g. influenza) or prohibited (e.g. HIV) representation. 

Under the Act, the TGA must authorise these types of representations prior to their use in consumer advertising. 

1. Department of Health - types of influenza
2. Adapted from World Health Organisation (WHO) - influenza (seasonal) fact sheet
3. Adapted from Centers for Disease Control and Prevention (CDC) - pandemic influenza
4. WHO - Global Influenza Strategy 2019-2030
5. WHO - Global Influenza Strategy 2019-2030
6. Based on current clinical sensitivity/specificity of influenza point-of-care tests included in the ARTG.
7. Public Health Laboratory Network - influenza laboratory case definition
8. European Union - case definition of influenza
9. WHO - HIV assays: operational characteristics
10. TGA - clinical performance requirements and risk mitigation strategies for HIV tests
11. WHO - HIV assays: operational characteristics
12. TGA - clinical performance requirements and risk mitigation strategies for HIV tests
13. the literature review may include data for devices used for similar intended purposes as the device under assessment
14. Centers for Disease Control and Prevention (CDC) - information for clinicians on rapid diagnostic testing for influenza.