You are here

TGA Internet site archive

The content on this page and other TGA archive pages is provided to assist research and may contain references to activities or policies that have no current application. See the full archive disclaimer.

Presentation: Evaluating risk minimisation effectiveness: Where are we now?

TGA presentation: ARCS Scientific Congress Sydney, 11-12 May 2016

30 May 2016


These presentation papers are provided on the TGA's website solely for the purpose of indicating or suggesting what TGA representatives spoke about to the various conferences and seminars to which it relates. The papers are not legislative in nature and should not be taken to be statements of any law or policy in any way.

The Australian Government Department of Health (of which the TGA is a part) advises that (a) the presentation papers should not be relied upon in any way as representing a comprehensive description of regulatory requirements, and (b) cannot guarantee, and assumes no legal liability or responsibility for, the accuracy, currency or completeness of the information contained in the presentation paper.


  • Presented by: Elspeth Kay, Director, RMP Evaluation Section, Pharmacovigilance and Special Access Branch
  • Presented at: ARCS Scientific Congress Sydney
  • Presentation date: 11-12 May 2016
  • Presentation summary: Challenges of risk minimization activities and strategies for overcoming them with reference to emerging published literature and real-life examples


Evaluating risk minimisation effectiveness

Elspeth Kay
Director, RMP Evaluation Section
Pharmacovigilance and Special Access Branch

ARCS Congress 2016

Slide 1 - Overview

  • Why is risk minimisation evaluation important?
  • What does TGA look for?
  • What has been the experience in Australia?
  • Where would we like to be?

Slide 2 - Why is evaluation important?

Have the risk minimisation activities worked? If not, why not?

Input: sponsor, regulator, experts, HCPs, consumers → Risk minimisation activities → HCP and/or consumer knowledge → Minimal occurrence of ADR

Slide 3 - Risk management cycle

Risk management cycle

  • Data Collection
    • monitor effectiveness and collect new data
  • Identify & analyse
    • risk quantification and benefit assessment
  • Evaluate
    • benefit risk balance and opportunities to increase and/or characterise
  • Select & plan
    • risk characterisation /minimisation and benefit maximisation techniques
  • Implement
    • risk minimisation /characterisation and benefit maximisation

Slide 4 - What does TGA look for?

In the evaluation plan

  • Is the evaluation plan well described in RMP/ASA?
  • Which parts of the program going to be evaluated and why?
  • Are the measures and timeframes clearly defined and appropriate?
  • Is the methodology suitable and feasible?
  • Has consideration been given to how success will be defined?

In the evaluation reports

  • Was the evaluation carried out as intended?
  • Is there a conclusion about whether the risk minimisation activity should change, continue as is, or cease?
  • Is the conclusion well supported by the information presented?

Slide 5 - Resources

  • EMA Guidelines on good pharmacovigilance practices Module XVI
  • CIOMS IX report: practical approaches to risk minimisation for medicinal products

Slide 6 - Risk minimisation evaluation framework


CIOMS IX risk minimisation evaluation framework

  • 1. Programme (go to step 2 or 3)
  • 2. Outcome indicators to evaluate the programme success (safety related outcome of interest).
    • Morbidity/mortality EP
    • Composite EP
    • Surrogate EP
    • Biomarker EP
    • (go to step 4)
  • 3. Single or several interventions (go to step 6)
  • 4. Programme successful?
    • Yes : Continue programme (or provide justification to discontinue)
    • No : Improve interventions or programme design based on process indicator feedback (go to step 5 or 6)
  • 5. Potential moderators
    • Comprehensiveness of strategy description
    • Strategies to facilitate implementation
    • Quality of delivery
    • Participant responsiveness
    • (go to step 6)
  • 6. Process indicators to evaluate implementation fidelity of each intervention
    • Content
    • Coverage
    • Frequency
    • Duration
    • (go to step 2)

Note: EP = endpoint. The 'CIOMS IX risk minimisation evaluation framework' outlines elements to be considered for the evaluation of a risk minimisation programme (modified from Carroll (25).

Slide 7 - RMPs evaluated for Type A applications in 2015

light blue area - No additional risk minimisation (n=24). light green area -  Additional risk min with no effectiveness measure (n=7). dark blue area - Health professional survey (n=6). purple area - Routine pharmacovigilance (n=1). orange area Patient registry (n=1)

Slide 8 - Common challenges

  • Evaluation plans not developed before registration
  • Description of evaluation lacking necessary detail
  • Poor response rates to surveys
  • Uncertainty about how to evaluate consumer-directed activities
  • Use of process indicators without consideration of outcome indicators

Slide 9 - Where would we like to be?

  • Clear and comprehensive description of evaluation plan
  • Consideration of a number of measures, not just knowledge and awareness
  • Use of outcomes data
  • Evaluation of consumer-directed activities
  • Well-justified goal for success
  • Sharing experience through publication/dissemination

Print version

How to access a pdf document