Meeting clinical evidence requirements for in-vitro diagnostic (IVD) medical devices

To be supplied in Australia, all in vitro diagnostic (IVD) medical devices need to be supported by clinical evidence appropriate for their intended use and risk classification, demonstrating that the device complies with the applicable provisions of the Essential Principles.

From time to time, the TGA may request the clinical evidence that a manufacturer holds for an IVD medical device is presented in the form of a clinical evaluation report (CER). This will be requested in a letter sent under s41FH or s41JA of the Therapeutic Goods Act 1989. The Clinical evidence guidelines supplement: In vitro diagnostic (IVD) medical devices provides guidance for compiling a CER for an IVD medical device.

The scope of this document includes all devices meeting the definition of an IVD medical device as defined in the Therapeutic Goods (Medical Devices) Regulations 2002. This includes devices that are considered to be software as an IVD (SaIVD), IVD medical devices that are included in systems and procedure packs and IVD companion diagnostics (CDx).

This document gives specific, example-based guidance to devices in the following categories:

• IVD medical devices with established clinical utility;
• IVD medical devices with established clinical utility now intended to be used in a novel or expanded context;
• IVD medical devices that are entirely new or novel.

This guidance does not specifically address regulatory requirements for those producing in-house IVD medical devices.

This document supplements existing guidance found within the documents Clinical evidence guidelines: Medical devices and Application audit (technical file review) of IVD medical device applications. The documents should be reviewed in conjunction with each other.

This document comprises 3 parts:

In Part 1, key concepts in clinical evidence for IVD medical devices are described. These constitute concepts that manufacturers should consider when they are gathering, analysing and presenting clinical evidence for an IVD medical device.

In Part 2, the concept of a clinical evaluation report is discussed. This includes an overview of how to construct a CER for an IVD medical device.

In Part 3, case studies are used to illustrate the expectations of the TGA when it comes to the type of clinical evidence provided for an IVD medical device. These case studies are a guide only, and the manufacturer of an IVD medical device should take into consideration the guidance found in Part 1 when they are reviewing the case studies.

In each part, corresponding sections of the Clinical evidence guidelines: Medical devices are referenced to help the reader understand how the two documents fit together. However it is recommended that applicants review the Clinical evidence guidelines: Medical devices in full to understand the requirements to have their IVD medical device included on the Australian Register of Therapeutic Goods (ARTG) and their ongoing obligations for registered IVD medical devices.

The Essential Principles

This section should be read with section 1. The Essential Principles of the Clinical evidence guidelines: Medical devices and Schedule 1 of the Therapeutic Goods (Medical Devices) Regulations 2002- external site.

The Essential Principles (EPs) are the foundation upon which the TGA regulates all medical devices- including IVD medical devices - to be supplied in Australia. There are 15 EPs in total, divided into 'General' principles applying to all medical devices and 'Specific' principles that may apply to a particular medical device. EP 14 is the overarching principle for clinical evidence. EP 14 states:

'Every medical device requires clinical evidence, appropriate for the use and classification of the device, demonstrating that the device complies with the applicable provisions of the Essential Principles.'

To conform to EP 14, the manufacturer of an IVD medical device will need to review the EPs - both 'General' and 'Specific' - and consider which relate to the clinical aspects of the device. Clinical evidence must be provided that demonstrates how the device conforms to each of the EPs identified as clinically relevant.

One 'Specific' principle - EP 15 - relates only to IVD medical devices and must always be considered by the manufacturers of these devices.

Essential Principle 15: Principles applying to IVD medical devices only

1. An IVD medical device must be designed and manufactured in a way in which the analytical and clinical characteristics support the intended use, based on appropriate scientific and technical methods.
2. An IVD medical device must be designed in a way that addresses accuracy, precision, sensitivity, specificity, stability, control of known relevant interference and measurement of uncertainty, as appropriate.
3. If performance of an IVD medical device depends in whole or part on the use of calibrators or control materials, the traceability of values assigned to the calibrators and control material must be assured through a quality management system.
4. An IVD medical device must, to the extent reasonably practicable, include provision for the user to verify, at the time of use, that the device will perform as intended by the manufacturer.
5. An IVD medical device for self-testing must be designed and manufactured so that it performs appropriately for its intended purpose, taking into account the skills and the means available to users and the influence resulting from variation that can reasonably be anticipated in the user's technique and environment.
6. The information and instructions provided by the manufacturer of an IVD medical device for self-testing must be easy for the user to understand and apply.
7. An IVD medical device for self-testing must be designed and manufactured in a way that reduces, to the extent practicable, the risk of error in the use of the device, the handling of the sample and the interpretation of results.

Below are some examples of clinical evidence considerations for EP 15 that a manufacturer may need to make. Examples of clinical evidence considerations that a manufacturer may need to make for other EPs can be found in the section 1 of the Clinical evidence guidelines: Medical devices.

EP 15(1)

EP 15(1) explicitly states that the clinical characteristics of the device must support the intended purpose. Questions that a manufacturer may need to ask include- is there a known association between the target analyte and the condition to be diagnosed? Who is the target population, and is there any variation that may be inherent to that population? What sort of sample is intended to be used? What algorithms are the test results incorporated into?

EP 15(2)

Analytical and clinical sensitivity and specificity go hand in hand, so while EP 15(2) has more to do with analytical elements of the device, it is important that these are tied back to the clinical question to be answered through use of the device. Key considerations for EP 15(2) from a clinical perspective include how the target values and limits for each of the parameters listed relate to the clinical question to be answered. For example, a device with a high limit of quantitation may not provide adequate clinical sensitivity to detect patients with low levels of the target analyte. How has this been considered in the development of the device, and in the information to be supplied with it? What device design measures facilitate true result determination?

EP 15(4)

IVD medical devices are not just used in laboratories, and with more and more point-of-care and self-testing IVD medical devices emerging, it is important that the results of these devices can be appropriately validated by the intended users. An example could be the control line on a pregnancy test; if the control line does not appear, the information provided with the device should clearly state that the test is invalid and that the user should re-test with a different device. Key considerations for EP 15(4) from a clinical perspective include how easy it is for the user to verify the test performance. For example, is the validation step in-built or is it a separate step? If it is a separate step, when does it occur and, how likely are people to do it? Is there a way for the user to be reminded to perform the validation? Are controls provided with the device, or are details given on what to use as controls? Do the instructions for use include clear instructions for performing the validation?

EPs 15(5, 6 & 7)

These three elements of EP 15 all deal with self-testing devices. Key clinical considerations include the anticipated health literacy of the intended user, the validity of performance claims that are being presented direct to the consumer, the level of contact that the user has with a health professional that may be able to provide assistance should an error occur, or an unexpected result be produced. Are there any harms that could be reasonably anticipated for the user or public should an error occur or the results be misinterpreted? How can the risk of these harms be mitigated, and is this well-documented in a risk management report? Is there evidence that the device can be consistently used, read and interpreted correctly by the intended user?

What kind of clinical evidence is needed for an IVD medical device?

This section should be read with sections 2.2.1. Clinical investigation data, 2.2.2. Literature review and 2.2.3. Post-market data of the Clinical evidence guidelines: Medical devices

Clinical evidence for an IVD medical device is made up of all of the data and information that supports the validity and performance of the device when it is used as intended by the manufacturer. This data and information can be gathered through investigations of the device and through review of existing clinical and scientific literature. It becomes clinical evidence when it is evaluated and given context; for example, when it is compiled into a CER. Clinical evidence is an important component of the technical documentation for an IVD medical device. Along with other design verification and validation documentation, the device description, labelling, risk analysis and manufacturing information, clinical evidence is needed to demonstrate conformity to the EPs.

The Global Harmonization Task Force (GHTF) - now succeeded by the International Medical Device Regulators Forum (IMDRF) - produced guidance around concepts relevant to clinical evidence for IVD medical devices. The TGA has adopted the following definitions of clinical evidence for IVD medical devices from the GHTF¹.

Note that if the IVD medical device is intended to be used by patients in self-testing or at the point-of-care, the TGA expects that usability and human factors testing data derived from a study of the device within the applicable user groups would be included as part of the clinical evidence. Applicable user groups may be from different locations provided that relevant factors are equivalent and a justification provided.

Scientific validity

Scientific validity is defined as the association of an analyte to a clinical condition/physiological state. Scientific validity is often identified from academic research, and supported by studies evaluating the analyte for potential clinical applications. Literature review and where applicable, feasibility and/or scientific validity studies performed by the manufacturer will help to establish the potential scientific validity. For many analytes, scientific validity is well established. An example would be calcium in serum. The scientific validity for this analyte is well established as being linked to the diagnosis and treatment of parathyroid disease, a variety of bone diseases, chronic renal disease and tetany. However, some IVD medical devices are developed when the scientific validity of the analyte is still emerging. An example would be a newly characterized biomarker that is potentially useful in monitoring recurrence or progressive disease in patients with cancer. As the scientific and medical knowledge further develops, the initially established scientific validity might change and/or expand. An example of such evolving scientific validity would be C-reactive protein (CRP). This analyte was initially established as being linked in the detection and evaluation of infection, tissue injury and inflammatory disorders. Subsequently CRP was found to be linked to the risk of cardiac disease if the performance of the assay is appropriate for this clinical application.

Analytical performance

Analytical performance is defined as the ability of an IVD medical device to detect or measure a particular analyte. The demonstration of analytical performance supports the intended use of the IVD medical device. Analytical performance is determined by the collection of testing results (analytical performance data) from analytical performance studies used to assess the ability of the IVD medical device to measure a particular analyte. Analytical performance should include analytical sensitivity (e.g. limit of detection or limit of quantitation, inclusivity), analytical specificity (e.g. interference, cross-reactivity), accuracy (derived from trueness and precision) and linearity (as applicable).

Clinical performance

The clinical performance of an IVD medical device is defined as the ability of that device to yield results that are correlated with a particular clinical condition/physiological state in accordance with target population and intended user. The demonstration of clinical performance supports the intended use of the IVD medical device. Clinical performance demonstrates that the IVD medical device, depending on its test purpose, identifies an individual's current or future state or evaluates changes in an individual's state. Clinical performance may include expected values, diagnostic sensitivity and diagnostic specificity based on the known clinical/physiological state of the individual, and negative and positive predictive values based on the prevalence of the disease. Clinical performance data can be derived from multiple sources such as clinical performance studies, literature review, or experience gained by routine diagnostic testing. It is important that the determination of clinical performance is based on representative and appropriately varied populations.

Clinical utility

Clinical utility is defined as the usefulness of the results obtained from testing with the device and the value of the information to the individual being tested and/or the broader population. The clinical utility of a device is also informed by the information that supports the scientific validity and performance of that device, but should also take into consideration Australian clinical practice standards and accepted or proposed diagnostic algorithms. Clinical utility is therefore a key element of clinical evidence in the regulatory space.

Data and information related to all four of these elements need to be evaluated in order for the risk/benefit profile of the device to be understood. It is recognised that IVD medical devices typically carry different risks to other medical devices. To achieve any benefit, the information provided by an IVD medical device must result in a change in the way a patient's healthcare is managed, or in a change in the user's behaviour; this is where the risks for IVD medical devices typically lie. Evaluating the data and information gathered regarding an IVD medical device will allow for an understanding of how certain we can be about its risk/benefit profile. A manufacturer may use multiple sources of data and information to strengthen their evidence- for example, bench testing data can be combined with a literature review and clinical experience or post-market data.

Post-market data gives some insight into how the device is performing in the real world and may help support the conclusions drawn from the clinical evidence. Information about what kind of information should be included in the post-market data to be supplied to the TGA as part of a clinical evaluation can be found in section 2.2.3. Post-market data of the Clinical evidence guidelines: Medical devices.

How much clinical evidence is needed for an IVD medical device?

Generally speaking, the level of evidence required is that which allows the manufacturer to demonstrate that the device conforms to the relevant provisions of the EPs. This will be influenced by several factors, including but not limited to:

• The risk classification of the device;
  • Generally, the higher the risk classification, the stronger and more robust the clinical evidence is expected to be. Information on how the TGA classifies IVD medical devices is available in Classification of IVD medical devices.
• What the device is detecting/measuring and how it does so;
  • Is the device detecting/measuring a novel or established analyte? How established is the technology that the device employs? Has the technology that the device employs repeatedly delivered expected analytical and/or clinical performance results according to scientific testing? Note that novel analytes and technologies generally require more robust clinical evidence.
• The intended purpose of the device;
  • Is the device for screening, monitoring or diagnosis? Is the test alone confirmatory or is it performed in conjunction with other testing or clinical information? What is the intended specimen type? Who is the target population? How routinely would the test be performed?
• The context in which the device is used;
  • Who are the users of the device? Where are they using it? What are potential sources of error specific to that group of users or that environment? Does the device come into contact with patients?
• How the results are interpreted; and
  • Do the instructions for use (IFU) contain clear instructions on how to interpret the test results? What units are the results provided in? Are these the standard units used in Australia? Is there variability in the subject population and/or disease state that should be taken into account? Is there any specific clinical information required to accurately interpret the test results?
• The clinical utility i.e. the impact the results will have on patient management.
  • Does the analyte have an established use in clinical practice? What are the risks to the patient of an incorrect result?

The TGA recognises that IVD medical devices differ from other medical devices as the majority do not come into direct contact with the patient. The risks and benefits they pose are therefore related to their impact on clinical management, rather than any direct interaction between the device and the patient.

The manufacturer of an IVD medical device should consider these and any other relevant factors when gathering, evaluating and compiling clinical evidence for that device.

The TGA may request additional information- both before and after inclusion onto the ARTG- about the clinical evidence presented by the manufacturer through a letter to the applicant, as specified under s41JA of the Therapeutic Goods Act 1989. The TGA may also request review of the clinical evidence by the Advisory Committee on Medical Devices (ACMD).

Standards

The Therapeutic Goods (Medical Devices) Regulations 2002 state that, 'The manufacturer must ensure that the clinical data obtained takes account of any medical device standard or conformity assessment standard'.

Standards can help to establish the state of the art in a particular field by clearly communicating known hazards relevant to the types of devices they describe and how their potential for impact can be determined. Clearly communicating the state of the art allows for easy comparison of the device and may improve the robustness of the evidence presented.

Examples of IVD medical device standards that the TGA considers relevant to clinical evidence include but are not limited to:

• The EU Common Technical Specifications 2009/886/EC;
• ISO 20916:2019 In vitro diagnostic medical devices - Clinical performance studies using specimens from human subjects - Good study practice;
• CLSI EP19-ED2:2015 A Framework for Using CLSI Documents to Evaluate Clinical Laboratory Measurement Procedures, 2nd Edition; and
• CLSI Measurement Procedure Comparison and Bias Estimation Using Patient Samples. 3rd ed. CLSO guidelines EP09c. Wayne, PA: Clinical and Laboratory Standards Institute, 2018.

For Class 4 IVD medical devices it is expected that robust references to relevant technical standards and specifications which are considered state-of-the-art will be made.

This is by no means a complete list, and manufacturers of IVD medical devices will need to review recognised standards in order to select what is most relevant and appropriate to their own devices. Once standards have been identified, the manufacturer of an IVD medical device will need to justify the approach taken to meet the standards. The manufacturer should not merely state that they met the standard during design and testing, but demonstrate how it is that the objects of the standard are met. If the manufacturer chooses not to follow the specifications of a recognised standard, a strong justification for the approach taken must be provided.

It may be useful to read this section with section 3. Clinical evaluation report and supporting documents of the Clinical evidence guidelines: Medical devices.

The strengths and limitations of clinical evidence can only be identified through a process of evaluation. The evaluation should be compiled into a report in the form of a CER. The purpose of the CER is to present the evaluated information from a clinical perspective in a way that allows for the risk/benefit profile, when used for its intended purposes as stated by the manufacturer, to be clearly understood.

The structure of a CER for an IVD medical device

The CER acts as a roadmap to the clinically-relevant elements of a submission. The recommended structure of the CER for an IVD medical device is similar but not identical to that represented in section 3.1. Content and format of the report of the Clinical evidence guidelines: Medical devices.

Below is the suggested format for a CER for an IVD medical device. The following structure is highly recommended, and it may improve the efficiency of the review process if it is received in this format as assessors will know where to find the information that they need.

1. Device description, information around the lineage and version (if applicable)
2. Intended purpose, indications for use and claims
3. Regulatory status in other countries
4. Description of the clinical algorithms used to interpret the test (if applicable)
5. Demonstration of substantial equivalence or comparable performance (if applicable)
6. Overview and appraisal of evidence of scientific validity, analytical and clinical performance and clinical utility
7. Summary of relevant analytical and clinical performance data
8. Usability data (if applicable)
9. Post-market data
10. Critical evaluation of clinical evidence
11. Risk/benefit analysis
12. Conclusions
13. Clinical expert's endorsement
14. The name, signature and curriculum vitae (CV) of the clinical expert

A CER template following the above format has been included as Appendix 1 to help guide manufacturers in putting this information together.

The Clinical evidence guidelines: Medical devices make reference to the term substantial equivalence. Note that the concept of substantial equivalence may not always be relevant when discussing IVD medical devices. Manufacturers of IVD medical devices may instead seek to establish comparable performance with a validated assay.

The Therapeutic Goods (Medical Devices) Regulations 2002 require that the manufacturer of a kind of medical device must ensure that the clinical data is evaluated by competent clinical experts. The competent clinical expert should have current clinical experience using information generated by the type of device- for example an infectious disease specialist for a viral assay or a geneticist for a genetic assay.

Supporting documents

A checklist is available in Appendix 1: CER and supporting documents of the Clinical evidence guidelines: Medical devices to help manufacturers ensure they have compiled the expected supporting documents.

The TGA will review the contents of the CER for an IVD medical device alongside any device labelling, information to be supplied with the device and risk management documents to ensure that the information communicated to users is supported by clinical evidence and sound risk-management processes. Full technical reports - including raw data - should also be submitted as attachments and clearly referenced, as this may allow for confirmation of the conclusions drawn in the CER.

In this section, clinical evidence requirements for IVD medical devices are outlined according to whether or not the kind of device already has an established role in clinical practice i.e. clinical utility. Case studies of hypothetical IVD medical devices have been included to illustrate the expectations that the TGA has of device manufacturers.

IVD medical devices are classified in Australia according to the level of risk an incorrect result could pose to personal and/or public health. For information about how the TGA classifies IVD medical devices see the guidance document Classification of IVD medical devices.

The examples below illustrate general expectations, however the strength of evidence - for example, the minimum number of samples included in a study of analytical and clinical performance - should be proportionate to the risk classification of the device in question.

IVD medical devices with established clinical utility

The clinical utility of an IVD medical device is established if:

• The scientific validity of the analyte is already well established; and
• The clinical performance of the device is already well established when used for its intended purpose as stated by the manufacturer.

A note on non-assay specific quality control (QC) materials and general laboratory equipment: Non-assay specific QC materials are those that do not have assigned analyte values and are not intended for use with any specific assay. General laboratory equipment is that which is not intended for use with any specific assay and which may or may not be identified as an IVD medical device by the manufacturer. Their clinical utility is generally considered to be established.

If a manufacturer considers that the clinical use of an IVD medical device is already established, this needs to be documented and clearly demonstrated in the CER.

The manufacturer of such a device should hold clinical evidence that:

• Establishes the current standard of care relevant to the intended purpose and indications for use of the device through review of information related to both scientific validity and clinical performance; and
• Demonstrates that the analytical performance of the device is comparable to the established standard of care.

The clinical evidence requirements outlined above apply to equally IVD CDx as they do for other IVD medical devices. The significant additional requirement for IVD CDx is the appraisal of clinical evidence for the associated medicine or biological, and clear consideration of the associated medicine or biological in the risk/benefit profile of the device.

If the IVD medical device is (1) intended by the manufacturer to be used as an IVD CDx and (2) the IVD medical device is not the device used in the pivotal trial for the approved medicine or biological, then this needs to be clearly documented and data from concordance studies must be provided. The concordance data must demonstrate that the IVD CDx has comparable performance to the clinical trial assay, and that there is no significant difference in the risk/benefit profiles of the two devices.

The manufacturer of an IVD medical device with established clinical utility may be able to sufficiently establish the scientific validity and clinical performance of a device through a literature review and/or claims of substantial equivalence to a predicate or similar marketed device. Note that the definition of a predicate or similar marketed device may be different in Australia to what they are in other jurisdictions. Definitions of these terms can be found in sections 4.2.1. What is a predicate device? and 4.2.2. What is a similar marketed device? in the Clinical evidence guidelines: Medical devices.

Refer to sections 2.2.2. Literature Review, 4.2. Predicate and similar marketed devices and 4.3. Substantial equivalence in the Clinical evidence guidelines: Medical devices for further information on providing a literature review and/or indirect evidence of scientific validity and clinical performance to the TGA.

Case study

Bean's Devices Pty Ltd (Bean's) has developed an IVD medical device intended to be used for the qualitative detection of an infectious organism from nasopharyngeal swabs using a multiplex, real-time PCR assay.

To determine their approach to providing clinical evidence for the device, Bean's considers the following:

• The risk classification of the device;
  • They review the classification rules for IVD medical devices and identify that their device is a Class 3 device in Australia. They recognise that there is a moderate public health risk and high personal risk associated with a failure of the device to accurately identify the presence of the infectious organism as an unresolved infection can lead to significant respiratory compromise.
• What the device is detecting/measuring and how it does so;
  • The device is qualitatively detecting the presence of a highly infectious organism in nasopharyngeal swabs. The device detects this through a RT-PCR of a well-defined genome. Neither the target analyte nor the technology are new; both are well characterised. They determine that the test requires the additional use of general laboratory equipment only.
• The intended purpose of the device;
  • Bean's intend for the device to be used for diagnosis in symptomatic patients, and the organism is always pathogenic when present in the nasopharynx. The assay is confirmatory as the clinical presentation of patients with this infection is fairly uniform and there is a high positive predictive value associated. However, the pathogen is known to harbour resistance to several commonly-used antibiotics. As such, an incorrect result could have the potential to delay appropriate treatment.
• The context in which the device is used;
  • Bean's intend that patient samples for testing are collected using nasopharyngeal swabs, which are readily available. The swabbing technique does not differ from the standard technique and does not require special training of health professionals who would be performing the sample collection. The test is expected to be performed relatively routinely, particularly over the winter months. The device itself is used by laboratory staff in a molecular pathology laboratory. It does not differ from standard PCR techniques or practices, and does not require special training of laboratory staff. Potential sources of error include contamination at collection, transport or in the laboratory; this is reduced by the relative rarity of the pathogen in the normal environment. The device comes as a two-step kit - a primer to be combined with an already commercially available mastermix; this reduces the risk of errors from use (e.g. pipette under-delivers polymerase), however may lead to a risk of contamination. The use of the device should not pose any risks to the health or safety of the user, provided appropriate personal protective equipment (PPE) is worn and handling protocols followed.
• How the results are interpreted;
  • The presence of the target nucleic acid sequence in a sample taken from a symptomatic patient is confirmative; this is standard for this type of device and laboratory staff would not require specialist training to interpret.
• The clinical utility i.e. the impact the results will have on patient management.
  • The disease course is well established, the analyte is well established, the role of the pathogen in disease is well established. A patient infected with the organism must be treated with a specific antimicrobial and for a very specific length of time. An incorrect result could delay treatment; this is mitigated to a degree by the fact that the test is confirmatory and not pivotal.

Bean's review the EPs and note that EPs 1, 3, 6, 13, 14 and 15 all contain elements that are relevant to the clinical evidence requirements for their device. They determine that both the scientific validity and clinical performance of the device are well established; therefore, the device has established clinical utility. Bean's decide to demonstrate conformity to EP14 in the following ways:

• They decide to conduct a review of current scientific literature, clinical and technical standards to establish and scientific validity of the device, as well as to inform the expected clinical performance of the device.
• To further support their claims regarding the clinical performance of the new device, Bean's demonstrate comparable performance between the new device and a predicate device that they intend to phase out of production.
• Analytical performance is summarised in a series of tables, accompanied by a clear explanation of how the testing was conducted. Where standards relating to the testing are available, they are clearly referenced.
• Bean's does not hold any post-market data regarding the new device, however they supply post-market data for the predicate device that demonstrates that it is performing as expected in the real world. They also provide details of corrective and preventative actions (CAPAs), demonstrating that their risk management and quality control processes are established and functional.
• The entirety of the data - literature review, equivalence claims, analytical testing and post-market data - are reviewed and critically evaluated. Bean's tells the clinical story of the device by relating the analytical performance back to the clinical performance and clinical utility. They discuss uncertainty and limitations in the data provided, and how these are managed through their risk management processes.
• Bean's summarises the critical evaluation of the data in a brief risk/benefit analysis.
• Bean's selects a clinical microbiologist as the clinical expert to review the clinical evidence. A clinical microbiologist is selected as they have current knowledge and experience in the use of devices of the same kind as the new device. The clinical microbiologist provides a brief statement of support, explaining how it is that the evidence compiled by Bean's is sufficient to establish an acceptable clinical risk/benefit profile.

Bean's compiles all of the above information into a CER, using the template in Appendix 1 of this document. They also provide supporting documents as attachments to the CER as outlined in Appendix 1 of the Clinical evidence guidelines: Medical devices, including full technical reports for the bench testing conducted, a copy of the IFU and the risk management report for the device. Some of this information was already included as part of their submission; these are clearly referenced by name.

IVD medical devices with established clinical utility now intended to be used in a novel or expanded context

A manufacturer of a clinically-established IVD medical device may intend to expand the context in which the device is used. This may include-

• A change to the intended purpose or indications for use of the device;
  • For example, from diagnostic use in symptomatic patients to screening in asymptomatic patients;
• A change to the intended user of the device; or
  • For example, from laboratory-only use to health professional testing at the point-of-care or to patient self-testing.
• A new analytical method to detect an established biomarker.
  • E.g. a novel technology to replace IHC in assessing protein expression.

The manufacturer of such a device should hold clinical evidence that:

• Establishes the scientific validity and clinical performance of the device when it is used in the novel context; and
• Demonstrates that the device has comparable performance to the current standard-of-care through analysis of information derived through analytical and clinical performance testing - including usability testing - of the device.

The clinical evidence requirements outlined above apply to equally IVD CDx as they do for other IVD medical devices. The significant additional requirement for IVD CDx is the appraisal of clinical evidence for the associated medicine or biological, and clear consideration of the associated medicine or biological in the risk/benefit profile of the device.

If the IVD medical device is (1) intended by the manufacturer to be used as an IVD CDx and (2) the IVD medical device is not the device used in the pivotal trial for the approved medicine or biological, then this needs to be clearly documented and data from concordance studies must be provided. The concordance data must demonstrate that the IVD CDx has comparable performance to the clinical trial assay, and that there is no significant difference in the risk/benefit profiles of the two devices.

Case study

Bean's Devices Pty Ltd (Bean's) has developed an IVD medical device self-test intended to be used for the qualitative detection of an inflammatory marker in patients with chronic obstructive pulmonary disease (COPD) to identify viral or microbial-associated exacerbations

To determine their approach to providing clinical evidence for the device, Bean's considers the following:

• The risk classification of the device;
  • Bean's review the classification rules for IVD medical devices and identify that their device is a Class 2 device in Australia. They recognise that there is moderate public and personal health risk associated with a failure of the device to accurately identify the presence of the virus.
• What the device is detecting/measuring and how it does so;
  • The device is qualitatively detecting the presence of a marker in whole blood from human patients. The device detects the marker using an immunoassay method. Neither the target analyte nor the technology for its detection are new however no other device exists in the market that allows for a patient to self-test for the marker.
• The intended purpose of the device;
  • Bean's intend for the device to be used for the detection of the marker in whole blood of patients with diagnosed COPD who are experiencing symptoms of an exacerbation. The IFU states that the assay is not confirmatory, and that positive results and/or negative results in symptomatic persons should be followed up with a health professional for confirmation by laboratory testing. The test sample is collected using a lancet that is packaged with the IVD medical device.
• The context in which the device is used;
  • Bean's intend that the device to be made available through pharmacies and so it is assumed that the device will frequently be used by people without medical qualifications. Owing to this, they recognise that there are potential sources of error related specifically to the context in which the device is used; these include contamination of the sample during collection, misuse of the test leading to a false negative result and misinterpretation of the test leading to either a false negative or false positive result. If not properly disposed of, there is a risk that the device could have the potential to infect others with any blood-borne viruses they may carry.
• How the results are interpreted;
  • The results are interpreted in a similar fashion to a blood glucose test- the device will self-validate, and then display a number representing the patient's titer. The device runs a series of self-tests during set-up and testing and will alert the user if any errors are detected.
• The clinical utility i.e. the impact the results will have on patient management.
  • The disease course is well established, the analyte is well established, the role of the marker in identifying the disease course is well established. A false positive result would likely be followed up and resolved during lab-based testing. A false negative result, however, could result in a significant delay in commencement of treatment that could significantly impact upon the health of the user.

Bean's review the EPs and note that EPs 1, 3, 6, 13, 14 and 15 all contain elements that are relevant to the clinical evidence requirements for the device. They note that EPs 15(5, 6 & 7) specifically discuss requirements IVD medical devices that are intended for self-testing. Bean's determine that the scientific validity of the device is established, however the clinical performance is only partially established. Clinical utility of the test is established however further clinical performance considerations must be made due to the change in user and usage environment. Bean's decide to demonstrate conformity to EP 14 in the following ways-

• They decide to conduct a review of current scientific literature, clinical and technical standards to establish the scientific validity of the device, as well as to inform some of the technical elements of the clinical performance of the device.
• They demonstrate partial equivalence between the new self-test device and a predicate laboratory-only device, acknowledging that there are clinical differences between the two owing to the change in user.
• Clinical performance in the specific user group (i.e. lay patients) is determined through user testing in a clinical study. The user testing is performed in accordance with existing standards, and Bean's performs an a priori power analysis to determine how many participants are needed. They select a range of subjects representing different ages and backgrounds to participate in the user testing, representing the anticipated users of the device. Bean's especially considers EPs 15(5, 6 & 7) when compiling this information.
• Analytical performance is summarised in a series of tables, accompanied by a clear explanation of how the testing was conducted. Standards to which the testing was performed are clearly referenced.
• Bean's holds limited post-market data regarding the new device as it is currently approved in a single jurisdiction only and has been marketed for less than five years. They provide what data is held however, including the volume of sales, the volume of complaints received since market entry to the current year and details of the nature of those complaints. Bean's also provide post-market data for the predicate laboratory-only device that demonstrates there is currently no concern raised by real-world use regarding the analytical characteristics of that device. They also provide details of corrective and preventative actions (CAPAs) undertaken for both devices, demonstrating that their risk management and quality control processes are established and functional.
• The entirety of the data - literature review, equivalence claims, user testing, analytical testing and post-market data- are reviewed and critically evaluated. Bean's tell the clinical story of the device by relating the analytical performance back to the clinical performance and clinical utility. They discuss uncertainty and limitations in the data provided, and discuss how these are managed through their risk management processes.
• Bean's summarise the critical evaluation of the data in a detailed risk/benefit analysis.
• Bean's select a respiratory specialist as the clinical expert to review the clinical evidence. The respiratory specialist is selected as they are a consultant physician with current knowledge and experience in the use of devices similar to the new device. They also hold knowledge of the epidemiology of COPD in Australia specifically. The clinical expert provides a statement of support, explaining how it is that that the evidence compiled by Bean's is sufficient to establish an acceptable clinical risk/benefit profile.

Bean's compiles all of the above information into a CER, using the template in Appendix 1. of this document. They also provide supporting documents as outlined in Appendix 1. of the Clinical evidence guidelines: Medical devices, including full technical reports for the bench testing conducted, labelling for the device, a copy of the IFU and the risk management report for the device. Some of this information was already included as part of their submission; these are clearly referenced by name.

Entirely new or novel IVD medical devices

An IVD medical device is considered to be novel any time the scientific validity and/or clinical performance are not already established in clinical practice. This includes diagnostic devices intended to be used with novel specimen/sample types (e.g. circulating tumour cells) or analytes (e.g. a newly-described tumour marker).

The manufacturer of such a device should hold clinical evidence that:

• Establishes the current standard of care in the screening, diagnosis or prognosis of the disease or condition where applicable;
• Clearly derives analytical and clinical performance standards for the device that are relevant to the scientific validity and clinical performance; and
• Establishes the scientific validity and demonstrates the clinical and analytical performance of the device.

The clinical evidence requirements outlined above apply to equally IVD CDx as they do for other IVD medical devices. The significant additional requirement for IVD CDx is the appraisal of clinical evidence for the associated medicine or biological, and clear consideration of the associated medicine or biological in the risk/benefit profile of the device.

Case study

Bean's Devices Pty Ltd (Bean's) has developed an IVD medical device intended to be used at the point-of-care to help first responders and emergency room staff identify patients who are suffering from a potentially catastrophic cardiovascular event. The device qualitatively detects the presence of an analyte in patient saliva.

To determine their approach to providing clinical evidence for the device, Bean's consider the following:

• The risk classification of the device;
  • Bean's review the classification rules for IVD medical devices and identify that their device is a Class 3 device in Australia. They recognise that there is high personal risk associated with a failure of the device to provide an accurate diagnosis as it could lead to treatment delay and unfavourable outcomes. Also, if the device gives a false positive result, Bean's recognise that the patient could be treated inappropriately.
• What the device is detecting/measuring and how it does so;
  • The device is quantitatively detecting the presence of an analyte in human saliva. The device detects the analyte using an immunoassay method. While the technology employed in the device is already well established, the analyte is entirely novel and no other tests are available in the market that test for the analyte for this purpose. The test is performed by placing a small, white, plastic strip under the patient's tongue- in the presence of the analyte, the colour of the test window will change from white to blue.
• The intended purpose of the device;
  • Bean's intend for the device to be used by health professionals only for the diagnosis of the cardiovascular event in acutely symptomatic adult patients over the age of 18 years. Bean's that the device be used to screen and triage symptomatic patients quickly as they present, and that it is supportive; the results should be followed up with radiographic imaging and laboratory-based testing for other markers.
• The context in which the device is used;
  • Bean's intend that the device would be used by first responders (e.g. ambulance officers) and other health professionals working in emergency and/or acute medicine. They recognise that these groups represent highly trained specialists. However, as no other devices are currently marketed that test for the analyte for this purpose, potential sources of error include misuse of the device and misinterpretation of the test results. Bean's recognise that both could lead to serious adverse outcomes for patients.
• How the results are interpreted;
  • The results are interpreted in a similar fashion to a pregnancy test- two lines is positive, absence of colour in the test line with colour in the control line is negative. Absence of colour entirely is a test failure and the user is advised to re-test using another device. The higher the titre the darker blue the test window will be.
• The clinical utility i.e. the impact the results will have on patient management.
  • The pathophysiology and clinical course of the cardiovascular event is well established however using the presence of the analyte to inform screening and/or management of patients is entirely novel. A false positive result could result in inappropriate treatment being administered. A false negative result, however, could result in a significant delay in commencement of treatment that could significantly impact upon the health of the user. The cardiovascular event requires significant and timely medical intervention - both pharmacologic and surgical- in order to improve survival outcomes. Clinical utility will need to be established.

Bean's review the EPs and note that EPs 1, 3, 6, 13, 14 and 15 all contain elements that are relevant to the clinical evidence requirements for the device. They determine that the scientific validity and clinical performance of the device have not been previously established; therefore, the device does not have established clinical utility and is considered to be entirely novel. Bean's decide to demonstrate conformity to EP 14 in the following ways:

• They conduct a review of current scientific literature, clinical and technical standards to establish the current standard of care for patients who suffer the cardiovascular event.
• Once a prototype of the device has been developed, Bean's conduct a feasibility study in patients presenting with symptoms of the cardiovascular event who are treated as per the current standard of care. They perform a post hoc power analysis and determine that data from this study is able to sufficiently establish the scientific validity and clinical performance of the test such that a pivotal trial can be performed.
• Bean's then uses the lessons learnt in the feasibility study to conduct a pivotal study in a larger cohort of patients. They conduct an a priori power analysis to determine the minimum number of patients that must be enrolled. The results of the study further support the scientific validity and clinical performance of the test.
• Bean's finalise the final test design and conducts both bench testing and user testing to demonstrate that the analytical performance of the final test design is equivalent to that of the prototype design.
• Bean's does not hold any post-market data for the new device as it has not yet been brought to market in any jurisdiction. As no other comparable devices are available in the market, Bean's provide details of a robust post-market clinical follow-up plan. They explain how it is that they intend to proactively monitor the performance and safety of the device in the real world.
• The entirety of the data - the literature review, clinical studies, analytical testing and post-market plan - are reviewed and critically evaluated. Bean's tells the clinical story of the device by relating the analytical performance back to the scientific validity and clinical performance. They discuss uncertainty and limitations in the data provided, and how these are managed through their risk management processes.
• Bean's summarises the critical evaluation of the data in a detailed risk/benefit analysis.
• Bean's selects an interventional cardiologist as the clinical expert to review the clinical evidence. An interventional cardiologist is selected as they have current knowledge and experience in the standard of care for patients who suffer from the cardiovascular event. The clinical expert provides a detailed explanation of how it is that the evidence compiled by Bean's is sufficient to establish an acceptable clinical risk/benefit profile.

Bean's compiles all of the above information into a CER, using the template in Appendix 1. of this document. They also provide supporting documents as outlined in Appendix 1. of the Clinical evidence guidelines: Medical devices, including full study reports for the two clinical investigations performed, full technical reports for the bench and user testing conducted, labelling for the device, a copy of the IFU and the risk management report for the device. Some of this information was already included as part of their submission; these are clearly referenced by name.

Device description, lineage and version (if applicable)

Describe the device- what is it made up of? Is it used as part of a system and/or with any other devices and accessories? Is it the same version that underwent the testing described further in the report?

Please note if the manufacturer lists an intended purpose, indications for use or claims in this section of the CER that differ from what is stated in the IFU then the TGA will hold what is stated in the IFU to be the correct version of the information.

Intended purpose/indications for use and claims

What is the device intended to do? When should it be used? Is it intended for use as an IVD companion diagnostic in the selection of patients for a targeted therapy? What specimen types are suitable for use with the device? Are any specific performance claims associated with it - is it faster? More accurate?

Please note if the manufacturer lists an intended purpose, indications for use or claims in this section of the CER that differ from what is stated in the IFU then the TGA will hold what is stated in the IFU to be the correct information.

Regulatory status in other countries

Which countries is the device currently approved for marketing in? Do the intended purposes, indications for use and claims differ in those countries from what is seeking approval in Australia? Have there been any rejections or withdrawn applications as a result of regulatory review?

Description of the clinical algorithms used to interpret and apply the results of the test (if applicable)

Outline the algorithms used in clinical interpretation of the test output. This includes any supplementary test results or clinical information to be considered.

Demonstration of substantial equivalence or comparable performance (if applicable)

Outline the clinical, technical and biological characteristics of the device and any devices you want to claim equivalence to. Demonstrate that the differences between them do not impact upon the safety and performance of the device, i.e. that the subject device is non-inferior to the similar marketed or predicate device. This section is always applicable to IVD companion diagnostics if the device differs from the clinical trial assay (CTA) used in the clinical trials of the medicine or biological. Analytical and clinical concordance of the device with the CTA must be addressed in the CER.

Please note that concordance studies (or bridging studies) for an IVD CDx device that is not the device used in the clinical trials of the approved medicine/biological should be described here.

Overview and appraisal of evidence of scientific validity, clinical performance and clinical utility

State the approach taken to gathering evidence i.e. the search protocol for a literature review or the study protocol for a clinical study. Describe the approach taken to gathering the evidence, including a justification of any search or study protocols. Outline the results - an annotated bibliography may be used for a literature review, while an overview of key findings related to safety and performance should be provided for a clinical study. If a clinical study has been performed by the manufacturer, any available study reports - including any publications about the study - should be provided as attachments. The strength of the data and information presented should be discussed and the level of evidence they areconsidered to represent clearly stated and justified.

Summary of relevant analytical performance data

Provide details of the approach taken to gathering analytical performance data, including the standards referenced. Provide summary tables of the data gathered during testing. Describe how the data in each table demonstrate conformity to the relevant referenced standards. Full bench-testing reports should be provided as attachments or elsewhere in the regulatory submission as appropriate.

Usability data

Particularly for self-testing devices, provide details of the approach taken to gathering usability testing data, including the standards referenced. Provide summary tables of the data gathered during usability testing. Describe how the data in each table demonstrate that the device has an acceptable usability profile. Full usability testing reports should be provided as attachments.

Post-market data

Provide details - where available - on the post-market performance of the device. This should include, but not be limited to the number of sales made and in which countries, the number of complaints received, a table summarising the nature of the complaints received, the number and nature of any vigilance reports made to relevant notifiable bodies in each jurisdiction. Details of any corrective and preventative actions (CAPAs) and recalls should also be provided, where applicable. If you have chosen to claim substantial equivalence to another device, then information available regarding the equivalent device should also be provided (for example, from US FDA's MAUDE database). The applicant may also wish to demonstrate how it is that the device performs compared to other devices of the same kind using information available through adverse events databases. It is preferred that this data capture the entire market life of the device, however it should cover at minimum a 5 year period ending no more than 2 years prior to the applicant's submission to the TGA.

Critical evaluation of clinical evidence

Bring together and critically evaluate all of the evidence presented in the report (sections 4, 5, 6, 7 and 8). Explain how it is that the bench testing and post-market data demonstrate that the device meets the requirements set out in the discussions of scientific validity and clinical utility. Describe the strengths and limitations, and justify why it is that the manufacturer considers it sufficient to assure them that the device performs as intended.

Risk/benefit analysis

Clearly outline the observed and expected benefits and potential risks to patient management to be gained through use of the device, referencing the evaluated clinical evidence. Use this information to provide a clear and well-reasoned justification for believing the risk/benefit profile of the device to be acceptable.

Conclusion

Provide a brief summary of key findings from the CER.

Clinical expert endorsement

The endorsement needs to include the name, signature and CV of the clinical expert.

Where information relevant to a particular section of the CER is provided elsewhere in a submission (e.g. bench testing reports), the manufacturer should ensure it is clearly cross-referenced and easy to find.

1. GHTF/SG5/N6:2012 Clinical Evidence for IVD medical devices - Key Definitions and Concepts