You are here

Clinical evidence guidelines: Medical devices

24 February 2017

Book pagination

Part 1 - General requirements

1. The Essential Principles

For a medical device to be supplied in Australia it must be demonstrated that the relevant EPs have been met to ensure the device is safe and performs as intended. Schedule 1 of the MD Regulations outlines the EPs which discuss safety and performance.

There are six general and eight specific principles and one for IVDs, paraphrased below.

General:

  • Principle One: Use not to compromise health and safety
  • Principle Two: Design and construction to conform with safety principles
  • Principle Three: Must perform the way the manufacturer intended
  • Principle Four: Must be designed and manufactured for long-term safety
  • Principle Five: Must not be adversely affected by transport or storage
  • Principle Six: Benefits must outweigh undesirable effects

Specific:

  • Principle Seven: Chemical, physical and biological properties
  • Principle Eight: Infection and microbial contamination
  • Principle Nine: Construction and environmental properties
  • Principle Ten: Principles for devices with a measuring function
  • Principle Eleven: Protection against radiation
  • Principle Twelve: Medical devices connected to or equipped with an energy source
  • Principle Thirteen: Information to be provided with a medical device
  • Principle Fourteen: Clinical evidence
  • Principle Fifteen applies to IVDs only.

1.1 Compliance with Essential Principles

Compliance with the EPs is required for all devices included on the ARTG; however the principles do not set out categorically how manufacturers should comply. They leave some room for flexibility according to the intended use and risk profile or class of the device.

The following EPs (1, 3, 4, 6, 13 and 14) are particularly relevant to the clinical evidence, with EP 14 being the overarching principle.

Principle fourteen: Clinical evidence

EP 14 states:

"Every medical device requires clinical evidence, appropriate for the use and classification of the device, demonstrating that the device complies with the applicable provisions of the Essential Principles."

In addition to other procedures, manufacturers must apply clinical evaluation procedures to the medical devices they supply12.

These clinical evaluation procedures must be implemented in accordance with the requirements specified in Schedule 3 Part 813 of the MD Regulations.

Specifically Part 8 requires the manufacturer to:

  • obtain clinical data, in the form of 'clinical investigation data' (clause 8.4) and/or 'literature review' (clause 8.5)
  • ensure that the clinical data held in relation to the device is critically evaluated by competent clinical experts in the relevant field, and that the clinical evidence demonstrating that the device complies with the applicable provisions of the EPs is documented in writing.

The clinical evidence must primarily demonstrate that the device complies with the EPs 1, 3, and 614 as outlined below.

Other EPs also should be considered in the context of the clinical evidence available for the device, for example, the period within which the manufacturer claims the device can be safely used must be supported by the relevant evidence15; the warnings and precautions stated on the labelling and instructions for use for the device must clearly reflect hazards and known side effects associated with the use of the device16.

Principle one: Use not to compromise health and safety

Key considerations from a clinical perspective include the context of how the device is used, for example, whether it is used by specialist medical practitioners only, or by the general public. This will impact on the safety assessment for many devices. How the device is used, for example, the type of treatment administered, or procedure or testing undertaken and if there are any inherent dangers in this all have implications on the safety of the device. Any inherent dangers in the proposed treatment setting rather than the treatment itself should also be taken into account. The patient, user and any other person in the vicinity of the device may need to be considered.

Principle three: Must perform the way the manufacturer intended

The purpose(s) for which the device is intended to be used (intended purpose) is ascertained from the labelling, instructions for use, any advertising material relating to the device and/or technical documentation describing the mechanism of action of the device17.

The assessor will examine whether there is sufficient clinical evidence to demonstrate that the device performs as intended. Each of the intended uses proposed should be substantiated by the clinical evidence submitted, and the evidence must be a true and complete account of available scientific knowledge. When the range of indications is broad and diverse it may be reasonable to provide evidence of safety and performance for the higher risk and most common indications with a justification as to why these were selected as 'worst case scenarios' and/or common indications and an explanation provided as to how these results can be justifiably extrapolated to other indications.

Principle six: Benefits must outweigh undesirable side-effects

Under the regulatory framework medical devices must have clinical evidence which provides assurance of safety and performance. The level of 'assurance' required will vary with the risk of the device. Any likely benefits to health from the use of the device should be weighed against any risks of injury or illness from such use; essentially the greater the risk, the greater the benefit that needs to be demonstrated to balance the risk18. In developing the device all possible methods to minimise hazards identified in the risk assessment should have been incorporated into the device design. The residual risk then needs to be demonstrated to be acceptable.

Clinical investigations should be appropriately designed to provide an assessment of the benefit-risk profile for the medical device when it is used for its intended purpose(s). A safety profile can be established via clinical investigations, literature reviews and clinical experience (from post-market data, adverse event data and special access use). It may also be appropriate, on occasion, to argue for safety based upon data for a predicate or similar marketed device.

Other EPs that manufacturers are expected to consider in the context of the clinical evidence available for the device include:

Principle four: Must be designed and manufactured for long-term safety

The clinical assessor will take note of the intended purpose of the device and therefore its likely lifetime. The clinical evidence must demonstrate that the device performs as intended for the length of time appropriate to the intended purpose without adversely affecting characteristics and performances mentioned in EP 1, 2 and 3. However, for many devices, this is difficult to demonstrate in pre-market clinical investigations. In this case surrogate markers and post-market data from jurisdiction(s) where the device is already in use may be used to provide evidence of long-term safety.

Principle thirteen: Information to be provided with a medical device

The intended purpose is ascertained from all documentation provided with the device, and therefore any claims/statements in relation to the performance and safety of the device provided on the labelling and/or packaging, instructions for use, patient or clinician cards, leaflets, manuals, brochures etc., must be supported by the clinical evidence available for the device. During assessment of the clinical evidence clinical assessors review compliance of the device with some or all aspects (items) of EP 13.

The substantiation of the intended purpose is required and the patient groups for whom the device has a positive benefit-risk balance need to be well defined. Manufacturers should bear these uppermost in their mind when deciding upon the wording of the intended purpose(s) of the medical device and the patient group(s) in which it can be used. Other information provided with the device must also be consistent and supported by the evidence.

Information should explain how to use the device safely, and very clearly highlight any potential hazards, with appropriate contraindications, warnings or precautions indicating who may or may not safely use the device with directions on how it is to be inserted, implanted or used. For example, the following information must be provided on the labelling/packaging and/or instructions for use: device-related and/or procedure-related adverse events expected and/or reported19; for implantable devices - information about any risks associated with the implantation20; warnings, restrictions or precautions that may apply to the use of the device (including clinical or environmental)21, requirements for handling or storage22, risks (if any) associated with the disposal of the device23. These are risk minimisation tools.

These principles have a significant impact on the clinical assessment and manufacturers (and sponsors) should be mindful of this when compiling their clinical data.

The clarity and comprehensiveness of the information provided with a medical device has an impact on the risks and therefore the safety of the device. Unclear or ambiguous terms, poor grammar and spelling, foreign words or poor diagrams can all negatively impact on the ability of a patient or person to safely use the device and therefore negatively affect the benefit versus harm ratio of the device.

1.2 Standards

Compliance with recognised standards published by an Australian or International Standards Agency may be used to satisfy the clinical evidence requirements and the relevant EPs for devices based on technologies with well-established safety and performance characteristics. Conformity with such standards is not mandatory in Australia, but if they are not followed, adequate justification must be provided. If a manufacturer chooses to use other standards they must demonstrate that the application of the standard satisfies the requirements of the regulations. There are three main International Standards Organization (ISO) documents relevant to clinical evidence requirements for medical devices:

  • ISO13485:2016 - Quality Management Systems (QMS)24
  • ISO14155:2011 - Good Clinical Practice25
  • ISO 14971:2007 - Application of risk management to medical devices26
ISO 13485:2016 Quality Management Systems

The primary objective of this standard is to facilitate harmonised medical device regulatory requirements for QMS. The standard is based on ISO 900127, and

"...specifies requirements for a QMS where an organisation needs to demonstrate its ability to provide medical devices and related services that consistently meet customer requirements and regulatory requirements applicable to medical devices and related services." 24

Manufacturers are expected to continue to monitor the performance and safety of devices, including IVDs, via a surveillance program as part of their QMS once the device is marketed. These programs should be appropriate to the use and risks of the device. The data generated from safety and adverse event reports and complaints, newly identified risks, literature, any updated or new clinical investigations, significant regulatory actions and formal surveillance activities such as registries should be used to review the performance, safety and benefit-risk assessment of the device. This data should be evaluated and the CER updated in line with this new information. As a minimum the CER should be updated every 1-5 years depending on the novelty of the device and risk, as per MEDDEV 2.7/1 revision 428 (page 12) . As this information is incorporated into the ongoing risk analysis, it may result in changes to the 'Instructions For Use' (IFU) and other information supplied with the device.

Compliance with ISO 13485:2016 is not mandatory in Australia, however, under the Conformity Assessment Standards Order (Standard for Quality Management Systems and Quality Assurance Techniques) 200829 compliance with ISO 13485:2016 is considered to satisfy the Quality Management System requirements specified in the legislation.

ISO 14155:2011 Good clinical practice

ISO 14155:2011 provides guidance on the design and conduct of clinical investigations involving medical devices. It can also be used by regulatory bodies and ethics committees when reviewing clinical investigational plans. Thirteen principles are included such as adherence to ethical principles (as per the Declaration of Helsinki 30), subjects' rights, a determination that benefits outweigh risks and oversight by an independent ethics committee.

Compliance with ISO14155 is not mandatory in Australia, however the Therapeutic Goods (Medical Devices) Regulations 2002 state in 8.4 (5) that:

If clinical investigation data is collected outside Australia, the investigation must have been conducted in accordance with the principles of the Declaration of Helsinki, as in force at the time and place where the investigation was conducted.

The manufacturer must additionally ensure that any further standards that apply to the device are taken into account.

ISO 14971:2007 Application of risk management to medical devices

ISO 14971:2007 specifies a process for a manufacturer to identify the hazards associated with medical devices, including IVDs, to estimate and evaluate the associated risks, to control these risks and to monitor the effectiveness of the controls. The requirements of ISO 14971:2007 are applicable to all stages of the life-cycle of a medical device.

Examples of device types which have specific ISO standards outlining requirements for demonstrating clinical evidence are the current editions of the series of standards:

  • ISO 11979-7- Ophthalmic implants - intraocular lenses 31
  • ISO 5840-1; ISO 5840-2 and ISO 5840-3 - Cardiovascular implants- cardiac valve prostheses 32

and

  • ISO 14708 - Implants for surgery - Active implantable medical devices
  • ISO 14117 - Electromagnetic compatibility test protocols for active implantable medical devices

In addition, there is a technical specification ISO/TS 10974 entitled 'Assessment of the safety of magnetic resonance imaging for patients with an active implantable medical device' 33 which refers to non-clinical testing of AIMDs in an MR environment.

2. Clinical evidence

This section outlines the sources and types of clinical evidence and how these may be used to demonstrate compliance with the EPs to establish the safety and performance of the medical device for its intended purpose(s). This document is based on publicly available GHTF and related MEDDEV documents in particular MEDDEV 2.7/1 revision 428.

2.1 Key definitions and concepts

The following definitions used in this section are consistent with those provided in the GHTF document, Clinical Evidence - Key Definitions and Concepts, SG5/N1R8:2007 34 and MEDDEV 2.7/1 revision 4 28 (apart from addition of 'substantial').

Clinical investigation: systematic investigation in one or more human subjects, undertaken to assess the safety or performance of a medical device.

Note: 'clinical trial' or ' clinical study' is synonymous with ' clinical investigation'(these terms are used interchangeably in this document).

Clinical data: Safety and/or performance information that is generated from the clinical use of a device.

Clinical data are sourced from:

  • clinical investigation(s) of the device concerned; or
  • clinical investigation(s) or other studies reported in the scientific literature, of a device for which substantial equivalence to the device in question can be demonstrated; or
  • published and/or unpublished reports on other clinical experience of either the device in question or a similar device for which substantial equivalence to the device in question can be demonstrated.

Clinical evaluation: a methodologically sound ongoing procedure to collect, appraise and analyse clinical data pertaining to a medical device and to evaluate whether there is sufficient clinical evidence to confirm compliance with relevant essential principles (essential requirements in EU) for safety and performance when using the device according to the manufacturer's Instructions for Use.
Note: in exceptional cases where an instruction for use is not required, the collection, analysis and assessment are conducted taking into account generally recognised modalities of use.

Clinical evidence: The clinical data and the clinical evaluation report pertaining to a medical device.

Clinical use: use of a medical device in or on living human subjects. Note: Includes use of a medical device that does not have direct patient contact.

Definitions of additional terms used throughout this document:

Competent clinical expert: someone with relevant medical qualifications and direct clinical experience in the use of the device or device type in a clinical setting.

Critical analysis: the process of the careful and systematic examination, appraisal and evaluation of both favourable and unfavourable data.

Predicate: A previous iteration of the device, within the same lineage of devices, with the same intended purpose and from the same manufacturer, in relation to which a manufacturer is seeking to demonstrate substantial equivalence with that device.

Similar marketed device: An existing marketed device with a similar structure and design and the same intended purpose as the device but not a predicate of the device in relation to which a manufacturer is seeking to demonstrate substantial equivalence. Such a device may not be manufactured by the same manufacturer.

Substantial equivalence: Substantial equivalence confirms that the new device is as safe as and performs as well as the predicate or similar marketed device. This determination is based on a review of the new device's intended purpose and clinical, technical and biological characteristics.

There are two types of clinical evidence, direct and indirect:

  • Direct clinical evidence is derived from an evaluation of clinical data pertaining to the device.
  • Indirect clinical evidence is derived from an evaluation of clinical data pertaining to a predicate or similar marketed device with which the manufacturer seeks to establish substantial equivalence.

The requirement for clinical evidence drives the process of data generation and clinical evaluation, producing clinical data and clinical evidence, respectively. Clinical evidence is needed to satisfy the EPs specifically that the device continues to be safe and to perform as intended and the benefits outweigh the undesirable effects while the device is included on the ARTG. Generating clinical evidence is therefore an ongoing process of monitoring for new data and the evaluation of this data by a competent clinical expert. 35 This clinical evidence is used to compile the clinical evaluation report (CER). The CER should be updated as new evidence is generated once the device is marketed.

2.2 Clinical data

Clinical data (meaning data relating to use of the device in or on living humans) may be generated for either the device or the predicate/similar marketed device. It includes:

  • Clinical investigations (synonymous with trials and/or studies)
  • Literature reviews
  • Clinical experience, usually post market data

Each is described below.

2.2.1 Clinical investigation data

Clinical investigation data as referred to in Schedule 3 Part 8 of the MD regulations includes:

  1. documentation in relation to the design, approval, conduct and results of each investigation carried out by the manufacturer of the device in relation to the use of the device in or on a human body; and
  2. a record of qualitative or quantitative information obtained through observation, measurement, tests or any other means used to assess the operation of the device; and
  3. a written report by an expert in the relevant field, being a report that contains a critical evaluation of all the clinical investigation data held in relation to the device. 35

Clinical investigation data sourced directly from the device produces a higher level of confidence in its relevance and capacity to inform the safety and performance characteristics of the device and is the preferred option for fulfilling clinical evidence requirements. It should be clearly indicated if the device has been modified since the clinical data were gathered and, if so, to clarify the device version and the nature of the changes. It is acknowledged that in some circumstances clinical investigation data are not available for the device or are insufficient in quantity or quality. In this situation clinical investigation data from a 'substantially equivalent' device such as a predicate or similar marketed device may be used to support the safety and performance of the device under assessment. The substantial equivalence decision making process is described in Section 4: Demonstrating Substantial Equivalence.

As per MEDDEV 2.7/1 revision 4, June 2016, the manufacturer should perform a detailed gap analysis to decide if additional clinical investigations need to be carried out:

The gap analysis should determine whether the existing data are sufficient to verify that the device is in conformity with all the EPs (corresponding to ERs in the EU) pertaining to clinical performance and clinical safety.

Special attention should be given to aspects such as:

  • new design features, including new materials,
  • new intended purposes, including new medical indications, new target populations (age, gender, etc.),
  • new claims the manufacturer intends to use,
  • new types of users (e.g. lay persons),
  • seriousness of direct and/or indirect risks,
  • contact with mucosal membranes or invasiveness,
  • increasing duration of use or numbers of re-applications,
  • incorporation of medicinal substances,
  • use of animal tissues (other than in contact with intact skin),
  • issues raised when medical alternatives with lower risks or more extensive benefits to patients are available or have become newly available,
  • issues raised when new risks are recognised (including due to progress in medicine, science and technology),
  • whether the data of concern are amenable to evaluation through a clinical investigation,
  • etc.

Data on the safety and performance of other devices and alternative therapies, including benchmark devices and equivalent devices, should be used to define the state of the art or identify hazards due to substances and technologies. This will allow the clinical data requirements to be established more precisely in relation to the intended purpose of a device. Precision in this analysis and the choice of selected medical indications and target populations may reduce the amount of clinical data needed from additional clinical investigations28.

Conducting clinical trials

Clinical investigations (synonymous with trials or studies) may be undertaken in Australia or outside of Australia36. When clinical trial data is collected in Australia, it is subject to the National Health and Medical Research Council's (NHMRC) National Statement of Ethical Conduct in Human Research37. Trials should comply with both the International Conference on Harmonisation's Note for Guidance on Good Clinical Practice38 and ISO 14155:2011 regarding clinical investigation in human subjects25. When clinical investigations are undertaken outside of Australia, the investigation must have been conducted in accordance with the principles of the Declaration of Helsinki30, as it is observed at the time and place the investigation is conducted39. The investigation report should note that the clinical investigation was carried out in accordance with such standards (and name the relevant regulatory authority or ethics committee/s giving approval) or indicate if it was not.

Since July 1, 2005, the International Committee of Medical Journal Editors (ICMJE) has required (and recommended that all medical journal editors require) registration of clinical trials in a public trials registry at or before the time of first patient enrolment as a condition of consideration for publication40. Registries include clinicaltrials.gov or any registry participating in the WHO International Clinical Trials Registry Platform, 41 such as the Australian New Zealand Clinical Trials Registry (ANZCTR). 42 Registration is currently not mandatory in Australia for regulatory purposes.

Clinical trials can be conducted within Australia under either the Clinical Trial Notification (CTN) or Clinical Trial exemption (CTX) schemes for devices not currently included on the ARTG, or to extend the use of a medical device beyond the conditions of current market approval43.

Reporting standards for clinical trials

International guidance on reporting standards for clinical trials can be found in International Standard Order (ISO) 14155: 2011 - Clinical investigation of medical devices for human subjects - Good clinical practice.25 Annex D of this ISO provides useful information on what should go into a clinical trial report.

In addition to high-level guidance on how to structure a full clinical trial report, the reporting requirements for specific trial designs are also included, outlined below.

Note

The following checklists are intended to inform reporting standards for peer-reviewed publications, and should be viewed as minimum requirements only for full clinical trial reports.

Reporting standards for randomised controlled trials

The Consolidated Standards of Reporting Trials (CONSORT)44 statement provides an evidence-based set of minimum guidelines for reporting parallel group randomised-controlled trials. The statement provides a 25-item checklist and flow diagram displaying the progress of all participants through randomised clinical trials. The focus is on transparent reporting of how the trial was designed, analysed and interpreted.

Reporting standards for observational studies

The Strengthening the Reporting of Observational studies in epidemiology (STROBE)45 statement is used for reporting observational studies, including case series and surveys. The statement provides a 22-item checklist for reporting criteria, and the use of a flow diagram is suggested but no official format is given. The STROBE statement provides guidance on how to report observational research well, and is endorsed by leading journals.

Reporting standards for diagnostic accuracy studies

The Standards for the Reporting of Diagnostic accuracy studies (STARD)46 statement is used for the reporting of in vivo diagnostic accuracy studies. The statement provides a 25-item checklist and flow diagram describing the design of the study and the flow of patients through the study. The focus of the statement is on identifying the quality of reporting.

Reporting standards for systematic literature reviews

Guidelines for reporting systematic literature reviews are outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement47. It is recommended that the PRISMA be followed closely when compiling a literature review as part of a submission for pre- and post-market reviews. The statement includes a 27-item checklist and flow diagram describing the study selection process in systematic literature reviews. Guidelines for Meta-analysis of Observational studies in Epidemiology (MOOSE)48 may also be used for meta-analyses of observational studies.

2.2.2 Literature review

A literature review may be presented in addition to clinical investigation data described above, or on its own. If the literature review does not pertain directly to the device under evaluation, a reasoned justification is necessary as to why any data obtained for another device may be used to support the safety and performance of the device under review.

Similarities and differences in clinical, technical and biological characteristics must be compared and substantial equivalence demonstrated to make an argument as to why the data supports the device under review, as recommended in Section 4: Demonstrating substantial equivalence. A literature review relating to the specific device or a device demonstrated to be substantially equivalent provides direct or indirect clinical data for the device. A literature review may also be presented that does not relate to the specific device or device demonstrated to be substantially equivalent, but only to the same kind of device. In this case, it may be used to present the state of the art and to identify risks and adverse events associated with that kind of device, but not to provide clinical data for the device. The purpose of the literature review (i.e. to present clinical data or to present the state of the art for the kind of device) should be clearly identified, and literature reviews for different purposes should not be combined.

In line with the MD Regulations Schedule 3 Part 8, a literature review in relation to a medical device includes a compilation, prepared using a documented methodology, of published and unpublished scientific literature, both favourable and unfavourable, relating to the medical device. This includes expert opinion, information about the hazards and associated risks arising from the use of the device for its intended purpose, and the foreseeable misuse of the device and information about the performance of the device, including a description of the techniques used to examine whether the device achieves its intended purpose.

A written report must be prepared by an expert in the relevant field containing a critical evaluation of the compilation of the literature. The manufacturer of the medical device must ensure that the clinical data is evaluated by competent clinical experts and must ensure that clinical evidence demonstrating that the device complies with the applicable provisions of the essential principles is documented in writing.49

A literature review involves the systematic identification, synthesis and analysis of the literature on the device when used for its intended purpose. The highest standard of literature review is a systematic review with meta-analysis. Such a systematic review is usually required for assessment for both pre- and post-market TGA reviews50. It is critical that the methods used to conduct the literature review are comprehensive and transparent in order for the clinical assessor to evaluate objectivity (lack of bias) and quality.

A literature review consists of the following key components:

Search protocol

Prior to conducting a literature review a protocol should be developed to identify, select and collate relevant literature. The protocol should include the search aim(s) and outline the population, intervention, comparator(s) and outcome(s) (PICO) criteria for the review. A record must be kept of databases searched with justification, search terms used (including key words and MeSH headings), date searched, period covered by the search, search limits applied (including language, study design, etc.) and inclusion and exclusion criteria. This must contain enough detail for a clinical assessor to reproduce the search. The search protocol should describe the method used to extract data from included studies and any processes for confirming data extracted by investigators.

Selection strategy

The selection criteria applied to the resulting list of studies should be clearly defined in enough detail to enable the clinical assessor to understand exactly how the final list of studies included in the review was compiled.

When selecting papers, the study design, quality of the data reported, quality of analysis and the clinical significance of the results should be considered. Any weighting criteria applied to the included studies should be clearly detailed. Variables for which data are extracted should be listed and defined.

A flow diagram should detail each step in the screening process, including total numbers of studies screened, assessed for eligibility and included in the review. Objective, non-biased, systematic search and review methods should be used such as PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses)47 or Meta-analysis of Observational Studies in Epidemiology (MOOSE)48 guidelines in accordance with the section: Reporting standards for clinical trials. The report should also summarise how each citation did or did not fit the selection criteria for inclusion in the review. This may be presented as an appendix of excluded studies with justification for the decision.

Review and critical analysis

It is preferred that the study characteristics and results of individual studies are summarised in tabular format. This should include, for all outcomes considered (including safety and performance measures), an effect size estimate and confidence interval for each study. Where relevant, the range found across all studies for outcomes (e.g. adverse event rates for different types of adverse events) should be presented. Then critical analysis of the literature should be undertaken. This is not a simple summary of the individual study results, but a critique and discussion of the study method, results and outcomes and how these apply to the device.

Literature report

A report must be provided, analysed and endorsed (evidenced by signature and date) by a competent clinical expert, containing a critical appraisal of this compilation, as per the legislative requirements49. It is recommended that reviews are prepared by researchers skilled in systematic review methods in conjunction with a clinical expert.

Where the review relies in part or wholly on literature for a predicate or similar marketed device, the report should also clearly justify how the device described in the compiled literature is relevant to the safety and performance of the device under review. It is important that the published literature be able to establish the clinical performance and safety of the device, and demonstrate a favourable risk profile.

For further guidance on performing a literature review see MEDDEV 2.7/1 revision 4 (section 9 and appendices 5 and 6)28.

2.2.3 Post-market data

Data from clinical experience, generally post-market data, can support the substantiation of the safety and performance claims of the device, help in identifying risks and guide risk assessment and risk management plans. Clinical experience data should be provided for pre- and post-market TGA reviews, and are particularly important where there may be a paucity of clinical data from other sources or when the data is not sufficiently robust to establish a favourable benefit-risk profile of the device. Post-market data may consist of investigation of complaints and individual vigilance reports, recalls or cancellations, registry data and literature reports or reviews.

Adverse events and complaints

Adverse events are required to be reported to the governing body in the country the device is in use when the event leads to or may lead to death or serious injury. Data from extractions of the Manufacturer's own internal complaint handling log should be provided. In the case of a similar marketed device from a different manufacturer, publicly available data such as that from FDA's Manufacturer and User Facility Device Experience (MAUDE) database or TGA Incident Reporting and Investigation Scheme (IRIS) should be submitted. However, it is noted that one of the serious limitations of post-market adverse event and complaint reports is under-reporting by end-users. More information on reporting adverse events (and complaints) can be found on the Database of Adverse Event Notifications - medical devices50 page on the TGA website.

Product recalls and cancellations

Recall and cancellation information is also valuable. A recall takes place to resolve a problem with a device for which there are deficiencies or any other issues concerning safety, quality or performance. There are two key types of recall (a) correction, which may involve temporary removal from use for example, for changes to the IFU, and (b) permanent removal of deficient, defective or unsafe medical devices from use. More information about product recalls can be found on the System for Australian Recall Actions (SARA)51 page on the TGA website. Suspensions, removals, withdrawals, cancellations or other corrective actions in any jurisdiction with the reasons for these should also be reported.

Device registries

Registries, systematic collections of data of medical outcomes, play a unique and important role in medical device surveillance. These can provide additional detailed information about patients, procedures, and devices not routinely collected by other means. Registries can provide valuable comparative information on the performance in terms of functional outcomes and quality of life of patients. Use of registries should take appropriate account of data limitations, variation across registries with respect to data structure and analysis and populations covered. Examples of Australian device registries include the Australian Breast Device Registry52, the Australian National Orthopaedic Association National Joint Replacement Registry (ANOANJRR)53 and the Victorian Cardiac Outcomes Registry54.

Published literature

To ascertain if any post-market data exists particularly if the above tools are not fruitful, a targeted literature search of biomedical databases, e.g. PubMed55, can be conducted to source post-market information. Keywords might include: brand name/ product name/ generic device description AND adverse events/ recall/ registry/ post-market surveillance.

Regulatory approval in other jurisdictions

If the device is approved for use in another jurisdiction the manufacturer should provide regulatory status, including the certificate number, date of issue and name under which the device is marketed. The exact wording of the intended purpose and any specific conditions in other jurisdictions should be provided. If MRI designation in other jurisdictions is provided, this will improve the efficiency of the assessment.

Post-market data to be provided

Post-market data is useful for identifying less common but serious device-related adverse events and it provides long-term information about the safety and performance of a device. All post-market data should be reported where possible including:

  • The number of units sold (or unit demand) worldwide since launch stratified by country (particularly if numbers are small) or geographic region. Note: this may not always be appropriate for high use devices, those with several components or those on the market for many years.
  • The number and types of complaints to the manufacturer regarding the device, both as reported and as confirmed on analysis and, in the case of new devices, stratified by year of occurrence of complaint.
  • The total number of adverse events and vigilance data reported to regulatory agencies, both as reported and as confirmed on analysis and categorised by type (e.g. device malfunction, use error, inadequate design or manufacture) and clinical outcome (e.g. death, amputation, surgical procedure required, no harm to patient). These should be stratified by year of supply and/or year of occurrence of event.
  • Any regulatory actions such as voluntary or mandatory recalls, including recalls for product correction, removals, suspensions, withdrawals or other corrective actions occurring in the market for IFU changes or other reasons and cancellations of the device anywhere in the world.

Together this data should be compiled into an adverse event, vigilance report and a device complaint rates which will allow the clinical assessor to better evaluate the benefit-risk profile of the device. The CER should include an analysis and commentary on the profile, severity and frequency (rate) of events reported. Adverse event and complaint data and rates should be discussed and critiqued to enable an understanding of the safety profile of the device in a 'real-world' setting. If the manufacturer chooses to use indirect clinical evidence to demonstrate compliance with the EPs, post-market data for the predicate or similar marketed device should be presented. As the time since first approval worldwide lengthens, the relevance of predicate data diminishes and should be replaced by data for the device itself. The manufacturers should clearly indicate whether the data reported is for the device or a predicate/similar marketed device.

2.3 Evaluation of the clinical data

The clinical data should be evaluated to identify potential sources of bias that may influence the results of the clinical investigations and information sourced from the literature review. It is important to describe the methods used for assessing risk of bias and to mention how this information will be used in data synthesis. Several quality appraisal tools are available for assessing trials reported in literature. A reviewer must pick a tool which is appropriate for the study design of every study in the literature sourced. Some tools can be used for multiple study designs but more often than not, more than one tool will be used. Commonly used quality appraisal tools include:

Table 1: Commonly used quality appraisal tools
Tool Applicable study designs Source
Jadad Score Randomised studies https://www.ncbi.nlm.nih.gov/pubmed/8721797
Downs & Black Randomised & non-randomised studies https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1756728/
QUADAS Studies of diagnostic accuracy http://www.bris.ac.uk/quadas/
AMSTAR Systematic reviews http://amstar.ca/

Additional guidance on critical appraisal tools is provided by the Scottish Intercollegiate Guidelines Network (SIGN)56, the Centre for Evidence-Based Medicine (CEBM)57, and the Cochrane Collaboration's Handbook for Systematic Reviews of Interventions58. It is preferable to use a tool that has been validated. Indicate in the report which tool was used and present checklists and other information about the tool in Appendices.

An important part of clinical evaluation is determining the overall strength of the evidence presented. A widely accepted tool for ranking different types of study design is the National Health and Medical Research Council's (NHMRC) levels of evidence59. The levels of evidence rank different study designs into a hierarchy according to their potential to adequately answer a particular research question (e.g. diagnostic, intervention, screening etc.). The hierarchy is based on the level of bias inherent in the study design. Using this hierarchy, systematic reviews of randomised controlled trials represent the highest level of evidence, followed by individual randomised controlled trials, pseudo randomised controlled trials, non-randomised comparative trials, and case series. The level of evidence ultimately affects the confidence that can be placed in the study results. Manufacturers should source the highest level of evidence available that demonstrates the safety and performance of the device for the intended purpose(s).

A summary of the evaluation conducted should be reported in the CER. Results of an evaluation usually take the form of a table showing a quality assessment on different aspects of the study, for all studies appraised. The layout and presentation of this information will vary depending on the tool used for evaluation. The manufacturer should present data on risk of bias of each study and outcome level assessments. The results of any assessment of risk of bias across studies (e.g. publication bias, selective reporting within studies) should also be presented where such information is available. Funding sources should be included if it is one of variables for data extraction.

Manufacturers are referred to MEDDEV 2.7/1 Rev 4 appendix 6, Appraisal of clinical data, for examples of studies that lack scientific validity for demonstration of adequate clinical performance and/or clinical safety:

  1. Lack of information on elementary aspects

    This includes reports and publications that omit disclosure of
    • the methods used
    • the identity of products used
    • numbers of patients exposed
    • what the clinical outcomes were
    • all the results the clinical study or investigation planned to investigate
    • undesirable side-effects that have been observed
    • confidence intervals/ calculation of statistical significance
    • if there are intent-to-treat and per protocol populations: definitions and results for the two populations
  2. Numbers too small for statistical significance

    Includes publications and reports with inconclusive preliminary data, inconclusive data from feasibility studies, anecdotal experience, hypothesis papers and unsubstantiated opinions.
  3. Improper statistical Methods

    This includes
    • results obtained after multiple subgroup testing, when no corrections have been applied for multiple comparisons.
    • calculations and tests based on a certain type of distribution of data (e.g. Gaussian distribution with its calculations of mean values, standard deviations, confidence intervals, t-tests, others tests), while the type of distribution is not tested, the type of distribution is not plausible, or the data have not been transformed. Data such as survival curves, e.g. implant survival, patient survival, symptom-free survival, are generally unlikely to follow a Gaussian distribution.
  4. Lack of adequate controls

    In the following situations, bias or confounding are probable in single arm-studies and in other studies that do not include appropriate controls:
    • when results are based on subjective endpoint assessments (e.g. pain assessment).
    • when the endpoints or symptoms assessed are subject to natural fluctuations (e.g. regression to the mean when observing patients with chronic diseases and fluctuating symptoms, when natural improvement occurs, when the natural course of the disease in a patient is not clearly predictable).
    • when effectiveness studies are conducted with subjects that are likely to take or are foreseen to receive effective co-interventions (including over-the-counter medication and other therapies).
    • when there may be other influencing factors (e.g. outcomes that are affected by variability of the patient population, of the disease, of user skills, of infrastructure available for planning/intervention/ aftercare, use of prophylactic medication, other factors).
    • when there are significant differences between the results of existing publications, pointing to variable and ill controlled influencing factors.
    In the situations described above, it is generally not adequate to draw conclusions based on direct comparisons with external or historic data (such as drawing conclusions by comparing data from a clinical investigation with device registry data or with data from published literature).

    Different study designs may allow direct comparisons and conclusions to be drawn in these situations, such as randomised controlled design, cross-over design, or split-body design.
  5. Improper collection of mortality and serious adverse events data

    Demonstration of adequate benefits and safety is sometimes based on mortality data or occurrence of other serious outcomes that limit a subject's ability to live in his home and be available for follow-up contacts. In this type of study,
    • consent of the subjects for contacting reference persons/ institutions for retrieval of medical information should be obtained during recruitment; when subjects can no longer be found, outcomes should be investigated with the reference persons/ institutions;
    • the consequences of missing data on the results should be analysed (e.g. with a sensitivity analysis); alternatively, when patients can no longer be found and their outcomes cannot be identified, they should be considered to meet the SAE endpoint under investigation (e.g. the mortality endpoint of a study).
    In mortality studies (and other studies addressing serious outcomes) procedures for investigating serious patient outcomes, numbers of subjects lost to follow-up, reasons why subjects leave the study, and the results of sensitivity analysis should be fully disclosed in reports and publications.
  6. Misinterpretation by the authors

    Includes conclusions that are not in line with the results section of the report or publication, such as
    • reports and publications not correctly addressing lack of statistical significance/ confidence intervals that encompass the null hypothesis.
    • effects too small for clinical relevance.
  7. Illegal activities

    Includes clinical investigations not conducted in compliance with local regulations. Clinical investigations are generally expected to be designed, conducted and reported in accordance with ISO 14155 or to a comparable standard, and in compliance with local regulations and the Declaration of Helsinki.

3. Clinical evaluation report and supporting documents

  • Clinical evaluation is an ongoing process conducted throughout the lifecycle of a medical device. Manufacturers must periodically review the performance, safety and risk-benefit profile of the device and update the clinical evidence accordingly.
  • Over the life cycle of the device the clinical evaluation will change. For instance when the device has been on the market for a number of years, the relevance of comparisons to predicates or similar marketed devices is less significant, and post-market data is likely to be of greater relevance.

After the completion of the clinical evaluation process a report should be compiled outlining the scope and context of the evaluation; the clinical data, analysis and conclusions reached about performance, safety and presentation (including labelling, patient information and IFU) of the medical device when used for the intended purpose(s). This section provides an overview of the recommended content and format of the clinical evaluation report (CER) which may be requested by the TGA for pre- and post-market reviews. The CER should be updated as needed through the lifecycle of the device to incorporate new evidence including post-market data and updated risk/benefit analyses. A record of reviews and amendments should be kept, and the CER submitted with the device application or for post market review must be up to date (usually within at least two years).

3.1 Content and format of the report

Standardising the content and format of these submissions will allow the TGA to assess applications and undertake post-market reviews of medical devices more effectively and efficiently. Manufacturers should refer to the GHTF document Clinical Evaluation SG5/N2R8:200760 and MEDDEV 2.7/1 revision 428. These have been adapted for this section.

The recommended structure of the CER is provided below, and checklists for these are in Appendix 1:

  1. Device description, lineage and version if applicable
  2. Intended purpose/indications and claims
  3. Regulatory status in other countries
  4. Summary of relevant pre-clinical data
  5. Demonstration of substantial equivalence (if applicable)
  6. Overview and appraisal of clinical data
  7. Critical evaluation of clinical data including post market data
  8. Risk-benefit analysis
  9. Conclusions
  10. The name, signature and curriculum vitae of the clinical expert and date of report
Device description, lineage and version (if applicable)

Manufacturers should identify the device by its proprietary name, and any code names assigned during its development and provide a description of the device including the models, sizes and device group to which the device belongs (e.g. biological artificial aortic valve).The description should also include the materials used, whether it incorporates a medicine (new or existing), biological tissues and/or blood products, the device components (including software and accessories), the mechanical characteristics, how the device functions and other relevant information relating to the device such as sterility and radioactivity. Diagrams or photographs of the device including steps for assembly and use are helpful. This information should be cross-referenced and linked to the manufacturer's technical information. The description should be detailed enough to allow for a valid evaluation of compliance with EPs, retrieval of meaningful literature and, if applicable, assessment of equivalence to other devices described in the literature, or alternatively, the novelty of the design, features or mechanism of the device. If the application is for a multi-component procedure pack, each component in the system must be adequately described.

Intended purpose/indications and claims

Defining the indications for use, performance claims, contraindications and warnings relating to the device is a core requirement for medical device assessment. In this context, a description of the 'indications for use' should include the clinical condition being managed, intended patient population, the severity and stage of disease, the site and nature of interaction with the body and the intended application of the device; that is whether single use/reusable; invasive/non- invasive, implantable. In addition the magnetic resonance (MR) status of the device (MR unsafe, MR conditional, MR safe or 'safety in magnetic resonance environment not evaluated') should be provided for all implantable devices (and components of these devices which may be taken into the MR scanner room). Consideration should be given to the duration of use or contact with the body. Outline any safety or performance claims made for the device. Particular attention should be paid to whether the intended purpose claimed by the manufacturer is supported by the clinical data provided.

Regulatory status in other countries

The CER should clearly describe the regulatory history of the device, including a list of countries in which the device has been marketed, the dates of introduction into each country and information about the quantity of product distributed in each country. Any countries in which the device has been recalled, including for product correction, withdrawn, suspended, removed or cancelled should be listed including the reasons for the action.

The exact wording of the intended purpose in other jurisdictions should be provided. It is preferable that certificates of conformity in other regulatory jurisdictions (e.g. CE marking, FDA, Health Canada) be provided including the number and date of issue of international certificates, as these allow verification of post-market data (e.g. through search of FDA's Manufacturer and User Facility Device Experience (MAUDE)), and may increase confidence in performance and safety claims. The trade name(s) of the device in other regulatory jurisdictions should also be clearly stated, if different from the name used in Australia. If the device has evolved from predicate/s over time the number and dates of certificates for these may be useful in exploring the history of the device. Information on concurrent applications for registration in the other jurisdictions, particularly Europe, the USA and Canada, is helpful if available.

Summary of relevant pre-clinical data

The clinical expert should comment on any potential safety and performance issues highlighted by pre-clinical testing and any potential risks for which testing has not been done. The summary may include an assessment of the adequacy of pre-clinical testing (e.g. bench testing including verification and validation, animal testing) to verify safety and performance for any claims made in the device labelling not adequately substantiated by the clinical data.

Demonstration of substantial equivalence

In some circumstances, the safety and performance of the device may be substantiated by presenting evidence from a predicate or similar marketed device (indirect clinical evidence). Information to help manufacturers determine whether clinical evidence from a predicate or similar marketed device may be suitable, and the steps involved in demonstrating substantial equivalence are provided in Section 4: Demonstrating Substantial Equivalence. Where indirect evidence is presented, the clinical expert must carefully and comprehensively critically evaluate whether there is potential for an adverse impact attributable to the differences between the device and any chosen comparators and include the conclusion based on this analysis in the CER. In order to demonstrate this, any differences between the clinical, technical and biological characteristics of the device should be clearly stated, including a comparison between the materials, design, function, energy source and any other device features that may alter the safety or performance of the device. This may be presented in a summary table clearly identifying and demonstrating the impact of any differences between the device and the predicate or similar marketed device.

When claiming substantial equivalence with a predicate as a means of establishing the safety and performance of a new iteration of a device, the applicant must provide a detailed analysis of the clinical data they have generated and undertake a literature search and review for the predicate and/or provide clinical investigation data for the predicate. Overall this will establish that the safety and performance of the predicate is acceptable before any comparisons are made. An analogous process is required for comparisons with a similar marketed device although access to a detailed analysis of the clinical data may not always be available. Manufacturers should always consider the age of the initial data generated to support the product and decide whether these data still present an accurate, current picture of medical knowledge for this product or treatment method. The literature review should be up-to-date to identify any new safety issues that have been identified since the clinical data for a predicate or similar marketed device were generated.

Note

The CER should clearly specify whether the clinical data being reported relate to the device or a predicate/similar marketed device that is claimed to be substantially equivalent to the device.

Overview and appraisal of clinical data

What constitutes appropriate clinical data will vary depending on the type of device under assessment and its state of development, but this should include clinical investigation(s) data, a literature review and/or post-market data (clinical experience) with the device or predicate/similar marketed device with which the manufacturer is claiming substantial equivalence. The CER should include a summary of all the clinical data, including post-market data, with the full clinical investigation reports, literature search and selection strategy provided in the supporting documents. For further information refer to section 2.2.4 and MEDDEV 2.7/1 Rev 4 appendix 6, Appraisal of clinical data.

Critical evaluation of clinical data including post market data

A competent clinical expert should evaluate all the clinical data and provide a reasoned argument as to how the clinical data constitutes valid clinical evidence, demonstrates the safety and performance of the device and establishes a satisfactory benefit- risk profile for the device when used for the intended purpose(s). This evaluation seeks to explain and justify the clinical data and typically involves a discussion of the quality of the clinical data, the relative strengths and weaknesses of the investigations and/or literature presented, the appropriateness of the inclusion and exclusion criteria, the appropriateness of the outcome measures, efforts to minimise bias, presence of confounders, length of follow-up, sample size, generalisability for example. Particular emphasis should be placed on explaining in detail the links between the clinical data and the contraindications, warnings and precautions and actual and potential adverse effects of the device on health. This enables the clinical experience to be adequately conveyed to users of the device.

The applicant should objectively link the medical claim(s) for the device to the hypotheses tested and conclusions drawn from all the clinical data including those presented in the literature. There are many tools available to guide the evaluation of clinical data that are specific to different study methods. Guidance on the recommended reporting requirements for clinical studies and examples of validated tools that can be used to guide the quality appraisal of both clinical investigations and literature reviews are provided in sections 2.2.1 and 2.3.

  • It cannot be over-emphasised that a CER as required by the legislation is not simply a summary of the data, followed by a statement that the data demonstrate safety and performance. This approach is commonplace, but does not represent an adequate clinical evaluation.
  • It must also be explicitly clear to the clinical assessor whether direct (pertaining to the device) or indirect (pertaining to a predicate or similar marketed device) data are provided. It is important to clarify if any changes have been made to the device since the clinical data were gathered and if so to document the changes and to clarify the exact version of the device.

The CER should include an evaluation of the post-market data presented in the submission and any other data from clinical experience (special access schemes etc.) and comment on its clinical significance. The detailed data can be provided in the supporting documents. In assessing the post-market data, the clinical expert should objectively comment on adverse events, vigilance reports and complaint rates and any recalls, withdrawals, removals, suspensions and cancellations for any reason in any jurisdiction and discuss the implications for the safety of the device. The evaluation of the post-market data should clearly indicate whether the data reported is for the device or a predicate/similar marketed device.

Risk-benefit analysis

Following the evaluation of all the clinical data, the manufacturer should provide a well-reasoned and documented analysis of the foreseeable risks that could occur with the use or misuse of the device, and compare these with an analysis of the expected benefits that may be provided to the end user. The nature, extent, probability and duration of benefits should be considered. This analysis should be clearly supported by evidence, including appropriate references. In demonstrating whether the expected benefits of the device outweigh the undesirable effects, the analysis may consider (but should not be limited to) the following criteria:

  • The strengths and limitations of the clinical data presented in support of the safety and performance of the device for the intended purpose(s) e.g. level of evidence, bias, confounders, length of follow-up
  • The clinical significance of the benefits of the device for the intended purpose(s) as demonstrated by the clinical data
  • Based on the clinical data provided and on a sound statistical approach, a reasonable prediction of the proportion of "responders" out of the target group or subgroups should be made
  • The safety issues identified in the clinical investigation data and/or literature review and post-market data (clinical experience) for the intended purpose(s), as well as reasonably foreseeable hazards associated with the clinical use of the device that the data may not have captured e.g. misinterpretation or misuse of the device
  • The probability of patients experiencing a harmful event, that is, the proportion of the intended population that would be expected to experience a harmful event and whether an event occurs once or repeatedly may be factored into the measurement of probability
  • The duration and severity of adverse events caused by the device or the procedure
  • Whether there are mitigation strategies that have been implemented to address real or theoretical safety issues i.e. risk management documentation and IFU/labelling
  • Any issues of uncertainty surrounding the application of the device for its intended purpose, e.g. limitations in the statistical analysis, generalizability of results to an Australian population.

The clinical expert should comment on the risk analysis and risk management approach by the manufacturer and make a determination of the benefit-risk profile of the use of the device in the intended target groups for the indications sought. The CER should clearly demonstrate a favourable profile based on current knowledge and the state of the art in the relevant medical fields, considering the totality of the clinical data on the device.

Conclusions

Essential Principle 14 states that the manufacturer must hold clinical evidence that demonstrates compliance with the other EPs. The conclusion of the CER should outline key supporting clinical data and evaluation findings supporting the safety and performance of the medical device. This should be based on the following:

  • Clinical data on the device and/or predicate/similar marketed device demonstrated to be substantially equivalent which is supportive of the safety and performance of the device
  • Confirmation that any differences between the device and the predicate/similar marketed device used for comparison will not adversely affect the benefit risk profile
  • Clinical evidence demonstrates the device performs as intended
  • Post-market data shows an acceptable level of adverse events
  • The residual risks have been mitigated with appropriate justification, for example, inclusion of relevant statements in the IFU documentation
  • The benefits outweigh the undesirable effects of the device.

Therefore the device is safe and performs as intended when used for its intended purpose.

Name, signature and curriculum vitae of clinical expert and date of report

As stated in Schedule 3 Part 8.6 of the MD regulations:61

Evaluation of clinical data

  1. The manufacturer of a kind of medical device must ensure that the clinical data is evaluated by competent clinical experts.
  2. The manufacturer must ensure that clinical evidence demonstrating that the device complies with the applicable provisions of the essential principles is documented in writing.

The name and signature of the clinical expert and the date of signing should be provided clearly demonstrating that he/she has evaluated all the clinical data and endorses all of the CER. A 'competent clinical expert' is someone with relevant medical qualifications and direct clinical experience in the use of the device or device type in a clinical setting. For novel devices, the clinical expert's clinical experience with the device type is expected to be current or recent (preferably within the past two years), to provide confidence in their experience with the current iterations of medical devices that often evolve rapidly with significant changes in functional characteristics and implantation procedures. The selection of a clinical expert will therefore depend on the type of device under assessment, and its intended purpose(s). For example, for a coronary stent submission the clinical expert should be an interventional cardiologist. In order for the clinical assessor to determine whether an appropriate clinical expert has been chosen, the full curriculum vitae of the clinical expert should be included with any convergence of interests or potential for conflict with the manufacturer or sponsor noted. For lower class devices which are not typically used by medical practitioners, another health practitioner who uses the device or similar devices in a clinical setting may be deemed, on a case by case basis, as an appropriate clinical expert who is able to critically evaluate all the clinical data and endorse the CER (evidenced by signature and date).

3.2 Constructing the CER

The following flow-chart outlines the components that comprise clinical evidence for a medical device and the process to compile a CER.

Figure 1: Flow chart for constructing a CER

Image of a flow-chart which outlines the components that comprise clinical evidence for a medical device, and the process to compile a clinical evaluation report. The process is outlined in the text of this guidance.

* Source documents for clinical data may not initially be required for a clinical assessment requested as part of an audit of an application for inclusion based on EU certification, provided that the CER contains sufficient detail for the TGA assessor to appreciate how the clinical expert was able to demonstrate compliance with the Essential Principles.

3.3 Supporting documents

The following information on the device must also be provided for pre-market (conformity assessment reviews and applications for inclusion) and post-market reviews in addition to the CER:

  • risk assessment and management documents
  • IFU, labelling, product manual and all other documents supplied with the device. These must highlight the risks and ensure that they are appropriately communicated to user.
Risk analysis and management documents

A well-reasoned and comprehensively documented risk analysis outlining the potential hazards related to the device is necessary in order to demonstrate compliance with the EPs. The manufacturer should ensure that all risks identified in the clinical data are included in the risk assessment, that is, all risks relating to patient treatment, method of operation of the device including potential device failures, and risks relating to usability i.e. harm to the patient that results from use of the device but is not caused by the device itself.

Device-related hazards include, but are not limited to, chemical, mechanical, thermal, electrical, radiation, and biological hazards. Use-related hazards62 refer to hazards associated with user interactions with the device, and include but are not limited to hazards that occur when the device is used as intended by appropriately trained clinicians but there are inherent risks associated with the procedure or use of the device, when the device is not used as intended, users are not suitably trained or equipped to use the device, users are not capable of using the device, or when the user's expectations about the device are not consistent with the intended use of the device.

All ongoing safety concerns (risks) should be specified as to potential causes, the nature, probability, extent, duration, frequency and severity of occurrence. This type of analysis should commence before beginning product development as it generates the safety requirements for the design specification. Once all potential hazards arising from the use of the device for its intended purpose(s) in the target population have been identified, the manufacturer is expected to implement a Quality Management System (QMS) to mitigate and monitor these undesirable effects and hazards.

Strategies to mitigate and minimise these risks such as contraindications or warnings in the IFU, check lists, educational initiatives, patient cards and any others documents supplied with the device should be discussed, including the expected impact of these risk mitigation and minimisation strategies. The manufacturer should discuss the adequacy of the documentation of the risks and address the clinical significance of risks that remain after the implementation of risk mitigation strategies. Sufficient details of the QMS should be provided so that the clinical expert who reviews and endorses the CER can determine whether the potential hazards and undesirable effects associated with the device are being minimised and mitigated adequately. A discussion highlighting how this has been done should be included in the CER.

Comprehensive documentation of the risk analysis and QMS is necessary to allow the clinical expert to comment on the overall benefit-risk profile of the device. ISO 14971: 200763 can provide further guidance on this.

Instructions for use, labelling and documents supplied with the device

Comments on any issues relating to the IFU, product manuals, patient cards, labelling and promotional materials or other documents supplied with the device should be provided in the CER with an assessment by the clinical expert as to whether these are consistent with the clinical data, with particular attention paid to indications for use, target population, contraindications and adverse events. The IFU should include all identified hazards and other clinically relevant information that may impact on the use of the device and sufficient warnings to mitigate risks where possible. Foreseeable safety or performance concerns that may arise from the hazards identified in the IFU, labelling and other documents should have been flagged and incorporated into the overall benefit-risk analysis, and the content of the IFU should take into account who may use the device. For example, self-use devices may require an IFU that is aimed at a different audience compared with devices intended to be used by a medically qualified person.

Additional information

Additional information should be provided as applicable. This may include (but is not limited to):

Note

When available, the clinical assessment report from acceptable European notified bodies may aid timely clinical review of the submission.

Additional information on the device

The description of the device should include sufficient detail to satisfying the requirements of Appendix 3 of MEDDEV 2.7.1 Rev 428 on "Device description - typical contents".

Preclinical data (if relevant)

Medical devices may contain elements that cannot be assessed solely through clinical testing, but which are critical to the safety or performance of the device. In such cases, a concise summary of the preclinical data may be required to establish the safety and performance profile for the device. In some cases it may be relevant to include a summary of the following in the supporting documents when recommended for the device type by relevant ISO technical specifications, standards or by other international regulatory agencies such as the US FDA:

  • physical and chemical analyses
  • engineering assessment
  • sterilisation and stability
  • microbiology
  • in vivo and in vitro testing
  • engineering studies under simulated conditions of use
  • modelling data
  • Good Laboratory Practices testing

Note

For applications for inclusion based on EU certification which are selected for clinical audit only, the TGA clinical assessor is not qualified to review the preclinical information. Such data should be assessed by a suitably qualified expert provided by the manufacturer, and where preclinical data is referenced, details on the type of preclinical testing performed on the device and the relevant standards to which the test adheres must be provided.

Full clinical investigation reports

Full reports for the investigation(s) on the device should include significantly more detail than peer-reviewed publications or journal articles that report results from investigations. The investigation reports should include the design, subject selection and inclusion/exclusion criteria, population demographics, duration, safety and performance data, adverse reactions and complications, patient discontinuation, device failures and replacements, tabulations of data from all individual subject reporting forms and copies of such forms for each subject who died during a clinical investigation or who did not complete the investigation, results of statistical analyses of the clinical investigations, contraindications and precautions for use of the device, and other information from the clinical investigations, as appropriate.

Literature search and selection strategy

It is recommended that the full electronic search strategy for at least one database searched, and the strategy for selecting studies which were included in the review are covered in this section of the supporting documents as a way to demonstrate the rigour of the search and selection strategy. The search strategy should include a summary justification as to how each citation did or did not fit the selection criteria for inclusion.

Pivotal articles from the literature review

The full text of pivotal articles in the literature review contributing to the clinical evidence should be provided.

3.4 Common errors in the CER

There are a number of common errors or deficiencies in clinical submissions that can be avoided, which include (but are not limited to):

  • Absence of the required components of the CER and/or referenced attachments and appendices missing
  • Intended purpose(s), indication and claims inconsistent between documents i.e. application, IFU and CER list different intended purpose(s)
  • Intended purpose(s), indication and claims not supported by clinical data
  • Lack of information about the regulatory history of the device in other countries, for example recalls, withdrawals, removals from market, suspensions and cancellations and the reasons for these in any jurisdiction
  • Information on predicate or previous related devices not included and/or substantial equivalence not demonstrated (if relevant)
  • Insufficient or incomplete clinical investigation(s) data, literature and post-market data with the device or predicate/similar marketed device if relevant.
  • In submissions where a literature review is provided there is:
    • No documented method and/or no demonstrated comprehensive literature review
    • Insufficient information and/or poor quality search protocol that result in inability to reproduce or understand the literature review strategy
    • Provision of a multitude of publications with little or no explanation as to why they are of relevance
    • No identification of device used or indication for use in articles reviewed
    • No summary of study characteristics and findings for each included article
  • Little or no synthesis and critical evaluation of the clinical investigation data, results of the literature review and post-market data:
    • No discussion of relative strengths of the data, for example randomised controlled trials, case control studies, case series
    • Substantial equivalence covering technical characteristics, biological characteristics and clinical use not established to validate the data for a different device (i.e. predicate or similar marketed device) to the device under review
    • Lack of discussion of the validity or otherwise of outcome measures used
    • No endorsement by the clinical expert that the differences will not adversely affect the safety or performance of the device
  • Inadequate critique and summary of the totality of evidence provided for the device
  • No post-market data including adverse events, vigilance reports, complaints, failures in cases where this information is available
  • More than one CER
  • Author of CER not included, totality of clinical data not evaluated by competent clinical expert, CER not endorsed/signed by clinical expert and/or CER not dated or out-dated
  • Inappropriate selection of clinical experts. The clinical expert who critically evaluates the clinical data and endorses/signs (with date) the CER is expected to be someone with relevant medical qualifications and direct (and for newly developed devices recent) clinical experience in the use of the device or device type in a clinical setting. For some lower class devices which are not typically used by medical practitioners, another health practitioner who uses the device or similar devices in a clinical setting may be deemed, on a case by case basis, as an appropriate clinical expert who is able to critically evaluate all the clinical data and endorse the CER (evidenced by signature and date).
  • CV of clinical expert(s) is not provided

It cannot be over-emphasised that a CER as required by the legislation is not simply a summary of the data, followed by a statement that the data demonstrate safety and performance. This approach is commonplace, but does not represent an adequate clinical evaluation.

It must be explicitly clear to the clinical assessor whether direct (pertaining to the device) or indirect (pertaining to a predicate or similar marketed device) data are provided for assessment. It is important to clarify if any changes have been made to the device since the clinical data were gathered and if so to document the changes and to clarify the exact version of the device.

The clarity and comprehensiveness of the information provided with a medical device has an impact on the risks and therefore the safety of the device. Unclear or ambiguous terms, poor grammar and spelling, foreign words or poor diagrams can all negatively impact on the ability of a patient or another person to safely use the device and therefore negatively affect the benefit versus harm ratio of the device.

Avoidance of these common errors and deficiencies will help to ensure that submissions for pre-and post-market clinical assessments are processed efficiently, thereby reducing the time required to report back to the applicant.

4. Demonstrating substantial equivalence

4.1 Clinical evidence requirements

Essential Principle 14

From the Therapeutic Goods (Medical Devices) Regulations 2002 - Schedule 1, Part 2

All medical devices require clinical evidence, appropriate for the use and classification of the device, demonstrating that the device complies with the applicable provisions of the essential principles.

Many devices are developed or modified by incremental changes and therefore are not completely novel. In such cases it may be possible to draw on the clinical experience of safety and performance from predicates of the device or similar devices. This may reduce the need for clinical data for the device under review64. In some instances it may be difficult to collect clinical data for a device prior to inclusion on the ARTG due to very small numbers of eligible patients or particularly high risk procedures limiting use. If there are no clinical data for the specific device, depending upon the nature of the device, it may be possible to provide a full clinical justification for why direct clinical evidence is either not required or only partially required. This involves referencing the performance and safety of a predicate or similar marketed device (as described below) and critically examining each change or difference in terms of materials, design, clinical use and their likely impact on safety and performance.

If it can be established via contention and/or additional data that the differences should not have any adverse impact on safety and performance, then the predicate/similar marketed device may be considered 'substantially equivalent' to the device. In this case, a clinical justification in addition to the clinical evidence for the predicate/similar marketed device can, in some circumstances, suffice for clinical evidence for the device. Equivalence should be based on a single device. If this is not possible substantial equivalence of every single device to the device under evaluation should be fully investigated, demonstrated and described in the CER. Reference MEDDEV 2.7/1 (Rev 4)28.

4.1.1Intended purpose

The predicate or similar marketed device should have the same intended purpose as the device in question. The only reference to intended purpose is in the Dictionary of MD Regulations65:

intended purpose of a kind of medical device means the purpose for which the manufacturer of the device intends it to be used, as stated in:

  • the information provided with the device; or
  • the instructions for use of the device; or
  • any advertising material applying to the device.

The GHTF makes the following references to intended use:

"The intended use of a device relates to the clinical condition being treated, the severity and stage of disease, the site of application to/into the body and the patient population." 66

The GHTF also infers that the intended purpose of a device should be based on the condition being treated and, where relevant, the patient population in whom the device should be used.

European Medical Device Directive guidance 67 states that 'intended purpose' means the use for which the device is intended according to the data supplied by the manufacturer on the labelling, in the IFU and/or in promotional materials.

Given the overlapping concepts across existing documentation, TGA has interpreted intended purpose as being interchangeable with intended use and expects that information defining the condition being treated and the patient population will be included in the IFU for medical devices where such instructions are required.

4.1.2 When is the use of clinical evidence for a predicate or similar marketed device considered inappropriate?

The GHTF identified certain situations where devices are likely to require direct clinical evidence:

"Clinical evaluation of medical devices that are based on existing, established technologies and intended for an established technology is most likely to rely on compliance with recognised standards and/or literature review and/or clinical experience of comparable devices. High risk devices, those based on technologies where there is little or no experience, and those that extend the intended purpose of an existing technology (i.e. a new clinical use) are most likely to require clinical investigation data."66 (emphasis added)

4.2 Predicate and similar marketed devices

4.2.1 What is a predicate device?

A device may be regarded as a predicate device in relation to a device for the purposes of demonstrating substantial equivalence if all of these apply:

  • It is a previous iteration of the device
  • has the same intended purpose as the device
  • is within the same lineage of devices as the device
  • is from the same manufacturer as the device.

Note

This differs from the definition used by the FDA where the manufacturer of the predicate device is not required to be the same as the device under assessment.

4.2.2 What is a similar marketed device?

A device may be regarded as a similar marketed device in relation to a device for the purposes of demonstrating substantial equivalence if all of these apply:

  • It is an existing marketed device
  • has a similar structure and design as the device
  • has the same intended purpose as the device
  • is not a previous iteration of the device by the same manufacturer.

The identification of an appropriate predicate or similar marketed device is contingent on the characteristics of the device and the requirement to substantiate their equivalence to the predicate or similar marketed device. A predicate/similar marketed device should have clinical evidence available to support its safety and performance.

4.3 Substantial equivalence

If a device can be demonstrated to be substantially equivalent to an existing device then the clinical evidence for the existing device demonstrating compliance with the EPs can be used to demonstrate compliance with the EPs for the device. A determination of substantial equivalence is based on a detailed review of the clinical (intended purpose and patient population), technical and biological characteristics of the device.

The GHTF identifies when a predicate or similar marketed device can be used to support the safety and performance of a device:

"The devices should have the same intended use and will need to be compared with respect to their technical and biological characteristics. These characteristics should be similar to such an extent that there would be no clinically significant difference in the performance and safety of the device."66 (emphasis added)

4.3.1 Steps to demonstrate substantial equivalence

Each of the steps in the process is explained below. In each step a device that addresses each requirement may be found to be substantially equivalent to its predicate or similar marketed device. Devices that do not address these requirements will only be suitable for inclusion on the ARTG if direct clinical evidence on the device is provided which demonstrates compliance to the EPs.

Any application that uses the substantial equivalence process should include a justification from a person with appropriate expertise relevant to the device under assessment to substantiate the proposal put forward by the manufacturer at each step of the substantial equivalence process. In addition it is critical that a clinical expert, specifically someone with relevant medical qualifications and direct clinical experience in the use of the device or device type in a clinical setting, justifies that any differences between the device and the device(s) claimed to be substantially equivalent will not have an adverse effect on the safety, quality and performance of the device. A full curriculum vitae of the clinical expert should be provided.

The following flowchart provides guidance to manufacturers on how to demonstrate substantial equivalence with a predicate device or similar marketed device.

Figure 2: Demonstrating substantial equivalence

Image of a flowchart on how to demonstrate substantial equivalence with a predicate device or similar marketed device. The process is described in the text of the guidance.

Step 1: Identification of a predicate or similar marketed device

The manufacturer should clearly identify if a predicate or similar marketed device is to be used to demonstrate substantial equivalence to the device under review. The predicate or similar marketed device should meet the definition of these devices outlined earlier in the document including having the same intended purpose as the device under review.

Devices that had been approved for marketing but have been removed from the ARTG or from a market under the jurisdiction of a comparable regulatory authority due to safety concerns (or when a proposal to cancel due to safety concerns prompts a voluntary recall) are not suitable comparisons. It is incumbent upon manufacturers to be aware of any safety concerns related to the predicate or similar marketed device. If the manufacturer nominates a predicate or similar marketed device that has related safety issues, then it is unlikely that the application would be successful. There may be exceptions when the applicant claims that design flaws or superseded technology in the predicate or similar marketed device has been rectified or improved in the new device but evidence would be required to substantiate these claims.

Step 2: Review the intended purpose of the device

The intended purpose of the device relates to the clinical condition being treated, the severity and stage of disease, the patient population and the site of application to/in the body. This should be clearly stated and provided in the manufacturer or sponsor's submission. If the device does not have an intended purpose that is the same as the predicate or similar marketed device it is generally not possible to establish that the devices are substantially equivalent, with the exception of the conditions below.

In some circumstances the clinical evidence for the predicate or similar marketed device may be used to demonstrate the device's compliance with EP 14 even if these other devices do not have the same intended purpose as the device, if the following criteria have been met:

  • clinical evidence that demonstrates safety and performance is available for the predicate or similar marketed device for the intended purpose for which the manufacturer or sponsor of the device is applying, and
  • the other steps of the substantially equivalent process have been met.
Step 3: Compare the technical and biological characteristics

When comparing the technical characteristics of the device and predicate/similar marketed device, a comprehensive assessment of the characteristics should be provided. Technical and biological characteristics include but are not limited to those outlined below by the GHTF:

"Technical characteristics relate to the design, specifications, physicochemical properties including energy intensity, deployment methods, critical performance requirements, principles of operation and conditions of use and Biological characteristics relate to biocompatibility of materials in contact with body fluids/tissues." 66

This comprehensive comparison is best demonstrated in a table which provides a description of the characteristics for the two devices and notes both their similarities and differences. All differences between the devices should be clearly and explicitly stated.

The device in question is considered suitable for the substantial equivalence process if it has substantially similar technical and biological characteristics as the predicate or similar marketed device. A device that does not have substantially similar technical characteristics as the predicate or similar marketed device may only be considered suitable for the substantial equivalence process if the manufacturer can demonstrate that the differences between the devices do not adversely impact on the safety and performance of the device in question (refer to Step 4: Provision of additional data).

The biological characteristics of the device under assessment should be compared with the biological characteristics of the predicate or similar marketed device. If there are differences the manufacturer is required to demonstrate that these would not impact on the safety and performance of the device.

What is considered substantially equivalent?

To be considered substantially equivalent, the differences in the technical and biological characteristics between the predicate/similar marketed device and the device should be minimal. Furthermore, these minimal differences should not be expected to adversely impact safety and performance of the device.

Below are examples of changes to a predicate device. The clinical assessors would be likely to consider the device's technical characteristics to be substantially similar to the predicate in the case of a balloon angioplasty device if the changes consisted of:

  • alterations to the radiopaque markings on an angioplasty balloon
  • ergonomic changes to the handle of a delivery system for an angioplasty balloon or stent
  • colour changes to an angioplasty balloon
Table 2: Example summary table
Evidence presented The Device Predicate or similar marketed device Impact of difference on safety and performance
Clinical characteristics
{e.g. clinical condition treated, intended purpose/ indications, site in body, operational procedures, target population including age, anatomy, physiology}      
Technical characteristics

{e.g. materials, design, function, energy source etc.}

{e.g. deployment methods}

     
Biological characteristics

{e.g. biocompatibility}

     

If the manufacturer considers the technical and biological characteristics of the device and predicate/similar marketed device to be substantially equivalent it is imperative that a justification from the clinical expert is provided which supports this claim.

Step 4: Provision of additional data

A device may be suitable for the substantial equivalence process if the technical characteristics are not substantially similar; however, in this circumstance additional evidence should be supplied that shows that the device is expected to be as safe and perform as well as the predicate/similar marketed device. This additional data may include clinical data and/or pre-clinical data (bench testing or in vivo studies) specifically designed to address the differences between the two devices.

Step 5: Final assessment

The onus is on the manufacturer to ensure that all information relating to the predicate/similar marketed device is provided for clinical assessment, in particular the clinical data which demonstrate safety and performance of the predicate/similar marketed device must be provided. The manufacturer may not have conducted the research, but they should ensure that the research method, clinical data and all information relevant to the assessment are included.

If the manufacturer is unable to demonstrate substantial equivalence, then direct clinical evidence will be required.

Step 6: Review clinical evidence

At this step in the substantial equivalence process, the device is considered substantially equivalent to the predicate/similar marketed device. The manufacturer should ensure there is robust clinical evidence to support equivalence where there are technical and/or biological characteristics of the device that differ materially from the predicate/similar marketed device. This should be summarised in the final component of the assessment.

Clinical evidence for the device must be updated by the manufacturer on a regular basis. If a safety concern is identified by the manufacturer for the predicate or similar marketed device which was used to demonstrate compliance with EP 14 for inclusion on the ARTG, the manufacturer must investigate whether this safety concern also applies to their own device and if it does, report it to the TGA in accordance with the requirements of the Act s.41MP68.

Book pagination