Supporting claims and indications for listed medicines

The TGA has developed learning modules for sponsors of listed medicines that aims to supplement the information in the Evidence Guidelines. The purpose of these modules is to assist sponsors to understand how to use the Evidence Guidelines and to provide additional information regarding common questions and issues encountered by sponsors.

If you have any feedback or questions regarding the content of these modules, please contact complementary.medicines@health.gov.au. 

Listed medicines training

Learning modules for sponsors of listed medicines as a supplement to Evidence Guidelines.

Sponsors legal responsibilities

Listed medicines do not undergo an individual pre-market assessment of safety, quality and efficacy by the TGA and as such, are not ‘approved’ by the TGA. Rather, listed medicines are entered in the ARTG under section 26A of Therapeutic Goods Act 1989 (the Act) following an application and legal certification by the sponsor that their medicine meets all the applicable legislative requirements, including that the medicine:

• only contains pre-approved low-risk ingredients from a list of TGA approved permissible ingredients known as the Permissible Ingredients Determination.
• only makes low-level indications selected from a list of TGA approved permitted indications known as the Permissible Indications Determination (see Indications for more information on indications permitted for use in listed medicines)
• is manufactured in accordance with the principles of good manufacturing practice In relation to the efficacy of listed medicines, sponsors:
• certify [against subparagraph 26A(2)(ja) of the Act] that they hold evidence to support all indications for their medicine and comply with all requirements for those indications in the Permissible Indications Determination
• certify [against subparagraph 26A(2)(j) of the Act] that they hold evidence to support any claims (that are not indications) made for the medicine (refer to 4.1 Differences between claims and indications)
• are required to comply with the condition of listing in subparagraph 28(7) of the Act that they must, at all times while the medicine remains listed, have information/evidence that supports the indications and that such information/evidence will be provided to the TGA upon request

Provisions in s31 of the Act enable the TGA to request the following from a sponsor:

• S 31(2)(fa): information or documents relating to any of the matters covered by the certifications made for the purpose of listing – this would include requesting the information and evidence that the applicant holds to support the claims and indications for the product
• S 31(2)(fb): information or documents relating to whether the goods comply with listing conditions – ss 28(6) and 28(7) require the sponsor to hold information and evidence supporting claims and indications

Reg 16AA(2)(a) of the Regulations expands s 31(2) to permit the Secretary to require information or documents about ‘the efficacy of the goods for the purposes for which the goods are to be used’.

A medicine can be cancelled from the ARTG by the TGA (note this is not an exhaustive list) if:

• the sponsor certifications under 26A(2)(ja) and (j) of the Act (that the sponsor holds evidence for all indications and claims) are incorrect
• the efficacy of the medicine appears to be unacceptable [paragraph 30(2)(a) of the Act]
• the sponsor refuses or fails to comply with a condition of listing, including the condition in subparagraph 28(7) of the Act that the sponsor has, at all times while listed in the ARTG, information/evidence that supports the indications and will provide such information/evidence to the TGA upon request

Note

For the purposes of these Guidelines, efficacy is the capacity of a medicine to produce a therapeutic effect, i.e., that the medicine will do what its indications say it will do.

The expectation that a medicine works as described by its indications applies to all types of indications, irrespective of whether scientific evidence sources or evidence of traditional use are being relied on.

The purpose of a sponsor’s evidence package

Based on the legal responsibilities outlined in Sponsors’ legal responsibilities, the overall purpose of an evidence package is for the sponsor to demonstrate to the TGA that your medicine will do what its indications say it will do (i.e., your medicine is efficacious).

It is up to the sponsor to decide how to demonstrate this. These Guidelines explain the elements that the TGA generally considers when assessing an evidence package for a listed medicine in order to determine whether the medicine’s efficacy is acceptable.

However, there is no ‘one size fits all’ approach and these Guidelines cannot provide for all possible scenarios. Efficacy reviews by the TGA are undertaken on a case-by-case basis and assessed on individual merit. A sponsor may choose to diverge from these guidelines as long as their evidence package demonstrates to the TGA that their medicine’s efficacy is acceptable. As such, it is important that an evidence package includes a critical analysis of the body of evidence that clearly explains the sponsor’s conclusions.

When and how the TGA reviews efficacy

We conduct random and targeted post market compliance reviews of listed medicines on an ongoing basis. At any time, the TGA can request that a sponsor provides the evidence they hold to show that they meet their legal responsibilities as outlined in Sponsors’ legal responsibilities.

As part of a post market compliance review, we may undertake a review to check (note this is not an exhaustive list):

• the accuracy of the sponsor certification that they hold evidence to support all indications and claims made for the medicine
• that the evidence the sponsor holds supports the indications at all times the medicine remains listed
• the efficacy of the medicine is not unacceptable
• whether the presentation of the medicine suggests it has characteristics it does not have
• whether advertising for the medicine includes indications that are not included in the ARTG.

During an efficacy review, the key questions the TGA considers include (but are not limited to):

1. What therapeutic effect is described by the indication?
2. On what basis can it be concluded that the medicine will result in this therapeutic effect? What are the reasons for this conclusion?
3. Is the above conclusion based on data or information that we have confidence in to be true and accurate?
   1. If assumptions have been made, are these valid?
   2. Does the data result from biases that impact on their validity?
   3. Is there competing data? If yes, which dataset best represents what the medicine will do?

Sponsors should consider the answers to the above questions when critically analysing the body of evidence compiled in their evidence package in order to demonstrate to the TGA that their medicine is efficacious (does what its indications say it will do) and meets the legal requirements as outlined in Sponsors’ legal responsibilities.

This section outlines the different types of evidence that can be sourced and provides guidance on how to conduct a literature search. A literature search is usually the first step to identify evidence sources to include in an evidence package for a listed medicine.

Different types of evidence

There are two types of evidence that can be used to demonstrate the efficacy of listed medicines:

• evidence of traditional use
• scientific evidence.

An evidence package can include scientific evidence and/or evidence of traditional use with the appropriate corresponding permitted indications (see Indications for information on indications permitted for listed medicines).

While most listed medicine sponsors rely on existing published literature, some sponsors may choose to generate their own scientific evidence for their medicine e.g., conduct a clinical trial. In such cases, sponsors should follow the appropriate guidelines for clinical trials to ensure the data is robust.

Evidence of traditional use

Traditional medicines are based on an extensive history of use, often measured over thousands of years. This history provides an accumulated repository of systematic observation and underpins the safe use of these medicines in a traditional setting. Usually when a medicine or a relevant ingredient in the medicine has been used over a long period of time, the dosage and formulation have been refined to maximise therapeutic effectiveness and minimise risk.

Many traditional medicines and ingredients with a long and coherent history of use are well documented in pharmacopoeias, monographs, materia medica, other texts and information published by various international regulatory authorities.

Some traditional medicine paradigms have been recorded by people outside the tradition’s indigenous origin and culture. Other traditional medicine paradigms, particularly those that have been developed within smaller and more localised groups, are not well documented; rather they are based on knowledge transmitted orally from generation to generation. Factors that may be relevant to establish a tradition of use include:

• the time over which the medicine or active ingredient has been used
• therapeutic use/s during that time
• continuity of its use
• geographical extent of its use
• use of the medicine is recorded in recognised traditional medicine evidence sources.

A well-established tradition of use is based on:

• evidence of a history of widespread medicinal use of the ingredient/s or medicine that exceeds three generations of use (75 years)
• the traditional use is extensively recorded in internationally recognised evidence sources for traditional medicine use.

Note

Homoeopathic medicine

Homoeopathic medicine is a traditional paradigm where the manufacturing process (of serial dilution and succussion or serial trituration) is a major component of the tradition of use. Provided that a substance is prepared according to principles that are described in a recognised homoeopathic pharmacopoeia and safety requirements are satisfied, indications may be based on traditional use. Evidence of traditional use for homoeopathic medicines can include independent written histories of use in traditional or contemporary homoeopathic literature.

Evidence sources of traditional use

Evidence to demonstrate efficacy of listed medicines with traditional indications can be derived from sources such as:

• materia medica
• official pharmacopoeias
• monographs
• publications from various international regulatory authorities
• texts that are relevant to the traditional paradigm
• well-recognised evidence-based reference texts.

A pharmacopoeia contains a comprehensive list of medicines and describes their properties and how they are prepared.

A materia medica sets out the body of knowledge on the therapeutic properties of medicines. Different materia medica relate to different types of complementary medicines, for example: Traditional Chinese Medicine, homoeopathy.

While the TGA does not have a list of approved sources of information, Appendix 1 provides some examples of internationally recognised resources and texts.

Other sources of evidence for traditional use

Non-reference textbooks

Non-reference textbooks cite, comment on, or build on established sources of evidence and thus do not usually provide sufficient evidence to substantiate traditional indications. If a non- reference textbook is used, the original source of evidence of traditional use should be located from the original documents cited in footnotes. If this is not possible, then the evidence package should include footnotes to clearly indicate that the textbook is based on original historic records or studies.

Where it is not possible to find the original reference that describes the traditional use, efficacy may be supported by more recent references reporting the original traditions of use. However, these references should provide enough information to support that the medicine is consistent, as far as possible, with the one described in the original reference.

Modern textbooks and monographs

Many modern textbooks and monographs include a combination of both traditional and scientific evidence. If using a textbook, monograph, or similar source in your evidence package, you must determine whether the information in the source is traditional or scientific.

In a situation where a traditional indication is used in combination with a scientific statement/claim relating to the mechanism of action of the ingredients, the combined claims must not imply clinical efficacy unless supported by scientific evidence. Indications and claims are discussed in more detail in Differences between claims and indications.

Independent written histories

When supporting evidence includes independent written histories of use in the classical or traditional literature (such as in relation to oral evidence or testimonials), the significance and clarity of references to any health benefit should be assessed by whether the:

• traditional paradigm is defined
• ingredient(s)/medicine is/are characterised
• preparation is described
• dose and dosing details are documented
• route of administration is specified
• target population is defined
• traditional indication is described

Evidence sources in languages other than English

Evidence in a language other than English can be used, if the evidence package includes:

• copy of the relevant pages in the original language
• verified English translation of the relevant pages (a verified translation is one that is accompanied by a signed statement from an accredited translator, fluent in both languages, verifying that the translation is true and complete).

Oral evidence sources

If the traditional indication is from an oral culture, video footage (stored in a digital format, not on film) may be appropriate. To be regarded as high quality, oral evidence must be corroborated from at least two separate sources in different locations.

Scientific evidence

Scientific evidence refers to quantifiable data and usually includes reports of clinical trials in humans; human epidemiological studies; animal studies; and other cellular or pharmacological studies. Due to the quantifiable nature of scientific evidence, scientific indications can imply clinical efficacy for health outcomes where the medicine’s efficacy is supported by such data.

Scientific evidence sources

Evidence types that may be included in the evidence package of listed medicines with scientific indications can be derived from sources such as:

• a systematic review
• a randomised controlled trial (RCT)
• a pseudo-randomised controlled trial (alternate allocation or some other method)
• a comparative study with concurrent controls
• a comparative study without concurrent controls
• Case series with either post-test or pre-test/post-test outcomes
• a review article.

The NHMRC levels of evidence and grades for recommendations for developers of guidelines¹ is a useful resource for definitions and more detailed discussion on types of evidence sources. The NHMRC Guidelines for Guidelines include a section on identifying evidence (in the context of conducting a systematic review) that may also be useful.

Note

Non-clinical studies

Non-clinical studies, such as animal or in vitro studies, on their own are not considered sufficient evidence to demonstrate the efficacy of a listed medicine with a scientific indication. The scientific uncertainties involved in extrapolating human health benefits from non-human data limit their usefulness. However, you may use non-clinical studies to support any discussion on biological plausibility or provide additional weight to a proposed indication to support indications when limited clinical studies are available.

Other sources of scientific evidence

Internationally recognised monographs or pharmacopoeias

High-quality and credible texts such as internationally recognised pharmacopoeias or monographs maintained by other international regulatory bodies or evidence-based reference texts may be appropriate to support the efficacy of listed medicines with non-specific indications (refer to Level of indications for information on how indications are categorised into non- specific and specific).

These texts can also provide additional support for the efficacy of listed medicines with specific indications, but this is only in addition to high level evidence. These texts cannot be used in isolation to support the efficacy a listed medicine with specific indications. Where possible, the relevant studies included in the monograph should be sourced and assessed for quality and relevance to the medicine’s indication.

Abstracts of scientific papers

Abstracts generally do not give sufficient details as to how the research was conducted or the data were analysed to allow objective evaluation of the quality of the research data, or the conclusions drawn by the study authors. Abstracts alone are not sufficient to demonstrate efficacy of a listed medicine with a scientific indication.

Unpublished studies

Unpublished studies or proprietary research can contribute to an evidence package for a scientific indication for a listed medicine if they are relevant and have been reviewed by at least two independent reviewers. To facilitate an accurate interpretation of methodological quality, any original research must be appropriately documented.²

Note

Requirements for independent reviewers to assess unpublished materials

• Independent reviewers should have relevant expertise and experience in the subject area under review.
• If they cannot be objective and judge the study impartially due to direct or perceived conflicts of interest, they should NOT accept the study for review.
• Conflicts of interest may arise if reviewers have professional or personal or financial affiliations with for example, author(s) and academic institution(s).

Literature searches

The following guidance is provided as a tool to assist sponsors find evidence that is high quality and relevant to a medicine. If further assistance is needed, we recommend engaging a specialist librarian to conduct a high-quality search to enable the collation of a strong evidence package.

Identifying literature is the first step in compiling a literature-based evidence package. The document Literature-based submissions for listed medicines and registered complementary medicines provides guidance on the scope of a literature search in the context of listed and registered complementary medicines. This section supplements that guidance document.

A well-constructed literature search identifies the general body of evidence related to a research question. There is no single search strategy that can be applied in all cases. For example, the design of a search strategy will be different if you are answering a research question related to a particular ingredient (e.g., ‘Does oral consumption of garlic reduce symptoms of the common cold?’) versus ingredients for a particular therapeutic use (e.g., ‘What ingredients are associated with reducing symptoms of the common cold?’).

It is good practice for the full details of the search methodology used to obtain evidence sources to be documented in an evidence package.

Note

Search strategies

The information in this section provides guidance on best practice standards for conducting literature searches. However, the TGA does not require sponsors to undertake a particular search strategy for listed medicines.

It is ultimately up to the sponsor to consider what type of literature search is most appropriate for their circumstance.

When a literature search can be conducted

A literature search can be undertaken at any of the stages of a product lifecycle, for example:

• at product development
• when product changes are made, e.g., change in formulation, addition of new indications
• periodically to ensure that the evidence package remains current.

Note that the search parameters used in a literature search (e.g., dose, dosage form) may be affected by whether the product formulation is in development or already established.

Types of literature searches

Literature searches can be systematic or non-systematic. The type of search used will affect the type of evidence retrieved and ultimately, the type of indications that can be used for a medicine. The evidence required to demonstrate the efficacy of listed medicines with different types of indications is described in Cross-paradigm indications.

Systematic literature searches

A systematic literature search is a conceptually logical, explicit, and reproducible approach to identifying and retrieving all potentially relevant published literature (both positive and negative) for a particular topic. Literature-based submissions for listed medicines and registered complementary medicines on the TGA website includes guidance on conducting a systematic literature search to be included as part of a literature based submission. If you require additional resources to help you, the Cochrane Handbook of Systematic Reviews of Interventions provides detailed guidance on best-practice standards for conducting systematic literature searches in the context of performing a systematic review.

Systematic literature searches involve searching a comprehensive electronic bibliographic database such as:

• MEDLINE - a bibliographic database of all health-related publications in journals indexed by the United States National Library of Medicine.
• EMBASE - a biomedical and pharmacological bibliographic database specifically developed to assist finding information to comply with the regulatory requirements of a drug.
• Web of Science - a multi-disciplinary tool which provides website access to multiple proprietary bibliographic databases.
• The Cochrane library - a collection of health-related databases provided by Cochrane and other organisations, with Cochrane Reviews at its core.
• BIOSIS - a bibliographic database of life sciences and biomedical sciences literature, including pre-clinical and experimental research.
• Scopus - a multi-disciplinary bibliographic database covering journals, trade journals and books.
• CABI Databases:
  • Global Health (provides access to all the world’s relevant public health research and practice)
  • CAB Abstracts (a bibliographic database focused on life sciences literature)
• AGRICOLA - a database of agricultural literature indexed by the US National Agricultural Library of the US Department of Agriculture.
• Food Science and Technology Abstracts – a multi-disciplinary database focusing on food and health science.
• AMED - Allied and Complementary Medicine Database run by the British Library with specific information on complementary medicines not found elsewhere.

Access to MEDLINE is through the PubMed search facility and there is a learning tab which provides search instructions, tutorials and FAQs. MEDLINE/PubMed provides abstracts (summaries) and citations for the journal articles listed, and often links to full-text articles online.

Note

• In general, your database search should utilise MEDLINE/PubMed electronic databases and include at least one other relevant database.
• General search engines (for example: Google) are not considered appropriate databases for conducting a literature search.
• Journal abstracts or summaries are not sufficient on their own to establish the efficacy of your medicine. You should access the full text in order to determine if it is high quality and relevant to your medicine.

In addition to the use of bibliographic databases, ‘manual searching’ may be used to identify potential publications. The process of manual searching involves a targeted exploration of specific journals or other sources. This may include searching past issues of a peer-reviewed journal which is particularly topical to the research question but may not be listed on the selected databases. Alternatively, websites which are known to list published research on the chosen topic may be searched to identify any possible ‘missed’ publications.

Determining the review question

Defining a clinical question in terms of the specific therapeutic use for a medicine will help find relevant evidence in the literature.

The ‘PICO Model’ is a format that can be used to help define the review question according to the following critical elements:

• Population: what are the characteristics of the target population, for example: persons with a specific condition or a general population.
• Intervention: what is the intervention under consideration for this target population, for example: a whole medicine or an individual ingredient.
• Comparison (if known): what is the alternative to the intervention, for example: a direct comparator or a placebo or no intervention.
• Outcome: what are the relevant outcomes, for example: specific outcome measurements or general health improvement.

Example

PICO model

• Research Question: ‘Is Echinacea purpurea better than placebo at reducing symptoms of the common cold in adults?’
• Population or Problem: Adults experiencing mild to moderate symptoms of the common cold
• Intervention: Echinacea purpurea
• Comparison (if known): Placebo or no intervention
• Outcomes: Reduction in the severity of common cold symptom.

For qualitative data, you may find it useful to use SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) as an alternative model or in conjunction with PICO.

Planning the search

Once the research question has been defined, the next step is deciding on the search protocol. The search protocol includes:

• the research question
• a list of the databases and other sources used during literature searches
• the search methodology which is the keywords and criteria used to search a database.

Using appropriate search terms is vital to ensure important publications are not missed. Familiarity with the field of study is necessary for this step as search terms should always include related synonyms, e.g., ‘complementary medicine’, ‘complementary therapies’, ‘alternative medicine’, ‘alternative therapies’, ‘integrative medicine’, ‘natural medicine’, ‘herbal medicine’, ‘holistic health’ and ‘holistic medicine’ may all be used for a complementary medicine literature review topic. When conducting a literature search on traditional medicine, you may find it useful to include the following search terms: ethnobotany, ethnomedicine, ethnopharmacology and medical anthropology, etc.

The searches should extend retrospectively for at least 10 years from the present day. Depending on your medicine, you may also consider searching databases that are in languages other than English, for example: the CNKI for traditional Chinese medicines; and multilingual databases such as LILACS and Redalyc. In particular, non-English language literature should be considered where there has been significant scientific work or if the medicine is largely based on a tradition where English is not the primary language of communication. Certified English translations of key references should be obtained, and the help of a specialist librarian may be useful when searching non-English databases.

Inclusion and exclusion criteria

Inclusion and exclusion criteria need to be established in the search protocol to determine which publications to include or exclude. These criteria need to be closely linked to the research question in the search protocol and may be based on:

• study design e.g., randomised or controlled trials only
• participants e.g., individuals who have experienced the symptom for longer than six months
• the intervention e.g., Chinese herbal medicine rather than broader herbal medicine
• language e.g., studies in English
• types of control e.g., intervention group and control group
• types of outcomes.

As a first step, publications with data should be chosen to be included rather than commentary or other reviews. However, the latter may be useful for identifying additional articles for inclusion in the final evidence package.

The systematic and manual search of the literature should be documented such that a reader can understand the logic behind the search methodology. The Literature-based submissions for listed medicines and registered complementary medicines provides some guidance on how this can be achieved. The search terms and databases used, and the numbers of references retrieved should be documented in your evidence package.

How to incorporate critical elements in the search

A literature search should include the following critical elements, where appropriate:

• health benefit
• active ingredient/formulation
• method of preparation of active ingredient
• dose
• dosage form
• route of administration
• frequency and duration of use
• population.

Controlled vocabulary and search strategies

When possible, the terms of the PICO and the critical elements listed above should be translated into the controlled vocabulary of the database (list of standardised subject heading used by catalogues and database indexers to describe what a publication is about). Use subject headings, keywords, free text words, synonyms and truncation (wildcard) searching when necessary.

• Use AND to combine terms and narrow a search e.g., Searching ‘echinacea AND common cold’ will only retrieve results that contain both echinacea and common cold, which limits the results only to those papers -refer to Figure 2 below.
• Use OR to expand a search using synonyms e.g., Searching ‘echinacea OR purple coneflower will include papers that include any of those terms. OR always means MORE - refer to Figure 3 below.
• Apply Limits for publication type, year, age groups or other factors e.g., Limit to age group ‘Aged, 65 and over’; Limit to publication years ‘2010-2020;’ Limit to publication type ‘randomised controlled trial’. Any Limits you use should be justified and reported in your evidence package.

Boolean Search uses a combination of keywords and Boolean operators (AND, OR and NOT) to organise and filter through searches. The ‘NOT’ operator is not generally recommended to be used in systematic searches as it has the potential to inadvertently exclude relevant articles. Figure 2 shows a visual representation of the difference between ‘AND’ and ‘OR’ Boolean operators in terms of the ensuing search results.

Each bibliographic database will also use their own search syntax, which will include various headings and categories, for example: MEDLINE uses Medical Subject Headings (MeSH). The various MeSH can be browsed on the US National Library of Medicine website.

A literature search can (and should) use free-text terminology as well as the relevant search syntax for the database, for example: For the search above, the MeSH for Echinacea pupurea is ‘echinacea’ and the MeSH for common cold is ‘common cold’.

When running a query in PubMed, use the [tiab] field code after each free text term. This will restrict the query to search in the title or abstract of the articles. By using both MeSH and tiab terms, you will increase the likelihood of finding all relevant articles.

It is also recommended to use synonyms to expand search results and ensure that they are comprehensive and relevant – this may include the inclusion of common names or Latin binomials for herbal medicine ingredients, or various synonyms for the condition being searched. This may also include using the term captured by the MeSH in free-text form anywhere in the title or abstract (e.g., [tiab]). The use of wildcards (e.g., * in the example below) can ensure that any term will be included in your search.

An example of a MEDLINE search string for the simple search of echinacea in common cold is provided in the text box below. Note that each PICO category is defined in a search line before these lines are combined.

Example

MEDLINE search string

1. "echinacea" [MeSH] OR “purple coneflower*” OR “coneflower, purple” OR “coneflowers, purple”
2. common cold [MeSH] OR cold* OR coryza OR coryzal OR catarrh*
3. 1 AND 2.

Filtering search results

If a substantial number of results are received (hits) while searching a database, the search can be refined by reducing the date range to the last 5-10 years. The justification for refining the date range should be documented. A search for evidence may retrieve several different types of evidence from a variety of different sources of literature. Review of the literature to produce a body of high-quality evidence that is relevant to the indication or medicine is known as ‘filtering’. However, care should be taken when applying filters as evidence may be inadvertently lost.

Documenting the search methodology and results

The search process for each database should be documented so that anyone reviewing the search process is able to understand whether the search retrieved all potentially relevant published literature. Documenting the following may facilitate this:

• all search terms and variations
• the date searches were carried out
• how many results were retrieved for each search
• the final number of results in the first pass
• all references found, even those that may be discarded later because they are irrelevant or poor quality and/or which do not support a relevant therapeutic use, to show that a balance of evidence approach has been considered
• how many records were duplicates.

The PRISMA guidelines may be a good resource to help you understand what to document in your systematic search. For further information on how to assess the evidence identified in a literature search refer to How to assess evidence. An example of a documented search strategy (including systematic and non-systematic searches) is provided in Appendix 3.

An example of a well-documented search methodology and results is provided below:

Example

Search protocol

Aim: The literature search was structured to identify clinical studies on the efficacy and safety of Echinacea for the common cold.

Database searched: PubMed Date searched: 28 April 2021 

Inclusion criteria:

• study using Echinacea
• clinical trials (any)
• systematic reviews
• safety or tolerability and/or efficacy in the common cold
• human study
• adults 19+ years

Exclusion criteria:

• Duplicate
• Echinacea use in different indications
• detail or design not sufficient to assess outcome/study quality
• conference abstracts
• not a pivotal source of evidence
• animal studies
• age group 0-18 years

Example

Search protocol and results

Search number	Query	Results
#1	"echinacea"[MeSH Terms]	800
#2	echinacea	1,313
#3	"purple coneflower*"	66
#4	"coneflower, purple" OR "coneflowers, purple"	1,313
#5	#1 OR #2 OR #3 OR #4	1,320
#6	common cold [MeSH Terms]	4,271
#7	cold*	177,945
#8	coryza OR coryzal	14,831
#9	catarrh*	7,441
#10	#6 OR #7 OR #8 OR #9	185,519
#11	#5 AND #10	195
#12	#5 AND #10 Filters: Adult: 19+ years	50

Non-systematic literature searches

A non-systematic literature search is an informative search of the literature on a topic and takes an in-depth (but not systematic) approach to a specific research question. Generally, a systematic literature search is considered best practice in the first instance. However, a systematic search of scientific literature may not be the most effective way of retrieving evidence in all circumstances, for example:

• indications based on evidence of traditional use, where a search of traditional literature would be more effective
• some non-specific indications relating to health maintenance, for example: ‘Maintain general health and wellbeing’, for which a systematic literature search may not retrieve many relevant evidence sources.

In these circumstances, a systematic literature search may not be necessary and a non- systematic search can be used. However, be aware that a non-systematic literature search may identify some, but not all, publications that address a particular issue. Shortcomings to conducting a non-systematic literature search include:

• the lack of a structured search strategy which is theoretically not reproducible
• limitations on the number of resources that can be searched
• an increased chance of selection bias.

Evidence for a non-systematic literature search can be sourced from databases and print sources, such as:

• pharmacopoeias
• current and/or classical references in the specific field under review, for example: herbal or homoeopathic pharmacopoeias or materia medica
• standard works on the ethnobotany, or use of medicinal plants in a geographic area
• databases of biological literature, for example: BIOSIS or CAB Abstracts.

It is good practice to document the details of a non-systematic literature search in the evidence package, including:

• where and how the evidence was sourced
• full bibliographic citation/s
• details of where and how terminology was established
• search methodology used
• why a non-systematic literature search was used instead of a systematic search
• why a non-systematic literature search is appropriate for the relevant indications.

Appendix 3 provides an example of a documented search strategy (including systematic and non-systematic searches).

Once evidence sources have been identified from a literature search, each source should be assessed individually to determine its relevance to the medicine and the quality of the evidence source.

The overall contribution an individual evidence source makes to your evidence package as a whole, based on its relevance and quality, should also be considered. This is discussed later in sections How to use evidence and How to document and present evidence.

How to assess if evidence is relevant

There are a range of factors that influence a medicine’s efficacy or effect. It is important that the evidence sourced should be specific to each of these factors or can be reasonably extrapolated to the medicine. Examples of such factors include:

• health benefit/therapeutic use/indication
• active ingredient
• formulation
• method of preparation of the active ingredient
• dose
• dosage form
• route of administration
• frequency and duration of use
• population.

The medicine design should be comparable to the key factors in the evidence source. Where there is a divergence, an appropriate justification should be included in your critical appraisal (see Critical appraisal of the body of evidence) to show why the divergence does not impact the overall conclusions about the medicine’s efficacy.

Most literature-based submissions are based on publicly available literature that does not specifically investigate a unique medicine design. Establishing relevance helps answer the question: ‘On what basis can it be concluded that the medicine will result in this therapeutic effect?'

While compiling your evidence package, you should consider how each of the key factors (such as those listed above, where relevant) in the evidence source compares to the medicine design so that the therapeutic effect described in the source can be extrapolated to the medicine and thus support the efficacy of the medicine.

Note

Ideally, most of these factors in the evidence sources should match those of the medicine. There may be cases where one or more of the factors in the source is not identical to the medicine but may be similar or not specified. These sources may still form part of your evidence package, depending on the other evidence sources included and the accompanying justification of relevance provided. For example, even when certain evidence sources do not have factors that match identically with your medicine, other evidence sources may be able to fill in the gaps. For further information refer to How to collate and present evidence.

Health benefit stated in literature

The health benefit described in the evidence source should be comparable to or be able to be extrapolated, where appropriate, to the therapeutic use described by the permitted indication/s selected for the medicine. When transcribing these permitted indications onto the medicine label and other advertising, the extent, nature, or prominence of the effects observed should not be exaggerated or suggest greater certainty than what was demonstrated in the evidence source. Also note that a medicine cannot advertise indications that are not included in the ARTG for the medicine.

Appendix 2 provides a case study (case study 1) that demonstrates an incorrect extrapolation of the health benefit described in the evidence source to a hypothetical medicine.

Active ingredient

The evidence source should relate to the whole medicine or include the same active ingredient/s and be similar or comparable to the medicine, unless justified, in terms of (where applicable):

• dose
• dosage form
• dosage regimen (including duration and frequency of administration)
• route of administration.

Appendix 2 provides case studies demonstrating:

• where the active ingredient in the evidence source is not equivalent to the medicine in terms of dose (case studies 2 and 3), method of preparation (case studies 4 and 5) and dosage form (case study 6)
• a discrepancy in the active ingredient used in the evidence source but with an appropriate justification for the inclusion of the source in an evidence package (case study 13).

Active ingredients from evidence of traditional use

When the medicine contains a herb or herbal substance, the species (and subspecies where applicable), plant part(s) and route of administration of the medicine should be comparable to that described in the evidence source. The method of preparation and processing, the equivalent dry weight and the dose of active component (where applicable) described in the evidence source should also be comparable to that in the medicine, unless otherwise justified. Traditional methods of preparation include:

• the use of a whole organism or specific parts e.g., leaf, root, fruiting body etc.
• fresh, dried, or preserved with alcohol, honey or sugar
• extracts produced by the application of pressure to the source material
• aqueous extracts such as infusions, decoctions and syrups
• ethanol-based extracts such as tinctures
• glycerine-based extracts
• vinegar-based extracts
• oil, grease or fat-based infusions
• beeswax salves and ointments.

Other methods of preparation may be considered traditional if supported by an appropriate reference describing the use of the method within the traditional medicine paradigm.

Note

Herbal ingredients

For listed medicines with herbal ingredients, additional information can be found in the Guidance on equivalence of herbal extracts in complementary medicine.

Medicinal preparations described in early pharmacopoeias, materia medica and other traditional references may pre-date modern analytical techniques. These are unlikely to provide a comprehensive and satisfactory specification (for the characterisation and establishment of the quality of the ingredient or medicine). In such situations, the active ingredients and method of preparation should be comparable to that described in the traditional literature. Where there are differences in parameters, such as the dosage or method of preparation, justifications (substantiated by evidence) should be included in the evidence package to explain why the discrepancies have no impact on the extrapolation of the traditional evidence to your medicine.

In general, active ingredients may be considered as sufficiently comparable if there are no relevant differences in the method of preparation (such that comparable outputs are yielded or no impact on therapeutic effect or safety can be shown) and if the medicine has the same intended purpose, dosage and the same route of administration. This may include traditional medicines in which the indication, dosage and administration are based on traditional knowledge, the dosage forms have been modified to modern dosage forms (e.g., capsules or tablets) but the outputs have been demonstrated to be comparable.

Modification of traditional formulations

You should consider whether modifications to the traditional method of preparation, formulation and/or dosage in well-established and well-documented traditional medicine systems (such as Traditional Chinese Medicine and Ayurvedic medicine) have impacts on whether the documented therapeutic use can continue to be extrapolated to your medicine.

The overall medicine formulation should reflect the traditional principles of ingredient combinations or substitution of herbal species. If the medicine uses traditional ingredients or formulations which have been significantly altered in their constituent profile from the traditional evidence source, a justification and/or additional information should be provided in the evidence package.

Non-traditional methods of preparation, including the use of non-traditional solvents and/or herbal material extraction ratios that are significantly altered from traditional, can change the chemical profile of the preparation. Such changes may affect the efficacy (and safety) of the medicine. In these circumstances, justifications to substantiate the extrapolation of the therapeutic effects described in the evidence source to the listed medicine should be included in your critical appraisal (see Critical appraisal of the body of evidence).

Active ingredients from scientific evidence

The active ingredient should be well characterised in the evidence source. Preparations used in the source should contain the same ingredient preparation and dosage form as the medicine.

For herbal ingredients, the species (and subspecies if applicable), plant part, method of preparation and processing, the equivalent dry weight and the dose of active component (where applicable) described or used in the evidence source should be considered for comparability to the medicine. If the preparation method used for a particular herbal product is different to that used in the literature, evidence that the constituent profile of the resulting active ingredient/s is not substantially different from the active ingredient in the literature, or a suitable justification should be provided. Unfortunately, many trials inadequately describe or characterise the composition of the herbal treatment. Be mindful that even when the herbal ingredient is standardised to known active therapeutic components or marker compounds, there can be variation in the concentration of other active components in the herbal extract that may result in different pharmacological activity in vivo. This should be documented in the evidence package accompanied with a justification for why the differences have no impact on the extrapolation of the data in the evidence source to your medicine.

Other characteristics of medicines used in clinical trials may also impact on their relevance to a proposed indication e.g., an evidence source with a dosage form designed for slow release of an active ingredient may not be relevant to medicines with indications that imply the health outcomes are achieved rapidly (e.g., ‘for the rapid relief of pain / fast acting formula to relieve pain’).

Frequency and duration of use

The way that an ingredient or formulation is administered to produce the observed health benefit in the evidence source should be comparable to that of the medicine, unless appropriately justified.

In some instances, for a clinical study to be relevant, the study duration needs to be of an appropriate time to validate the health benefit described by the indication. The appropriate study duration depends on the nature of the health benefit. If referring to a short-term benefit such as acute pain relief, a clinical study of only several hours duration may be adequate. On the other hand, where long-term benefits are implied, clinical studies should be of sufficient duration to establish a sustained response that is likely to be meaningful. This is of particular importance for medicines designed for weight loss. This is explained in further detail in Weight loss indications.

Appendix 2 provides a case study example of the study duration not being sufficiently applicable to the indication of a hypothetical medicine (case study 15). Evidence of traditional use may not refer explicitly to a ‘study duration’. However, traditional evidence sources may have a context of use that relates to a particular duration of use for the medicine or ingredient, and this should be comparable to the medicine.

Target Population

The target population for the medicine should be consistent with the population described in the evidence source unless extrapolation can be justified. It is important to note that, in general, listed medicines are intended for use by healthy individuals. The general population is categorised as the following:

• male and female participants
• generally healthy
• aged 18–65 years
• socio-culturally similar to the Australian population.

Where a target population is not specifically mentioned for a medicine (e.g., in the indication, directions for use etc), it is assumed that the target population of the medicine is the above general population.

The relevance of evidence sources that target a population with non-serious disorders or in situations where a continuum of health and disease exists, such as individuals in early disease states, should be considered carefully. In general, evidence sources with populations that have serious diseases, conditions or ailments may not be extrapolated to the general population and may not be considered relevant to a listed medicine indicated for the general population. However, in cases where there is data to suggest that the pathophysiology of the disease does not change the way the active ingredient works in the milder form of disease, compared to the more serious form of disease, the relevance of these evidence sources may be justified. Extrapolation of results obtained from subjects outside the target population of the medicine should be appropriately justified.

When an indication is directed towards a specific subgroup of the population (e.g., children, the elderly, pregnant women) it should be supported by data derived from the same subgroup of the population or otherwise justified. Particular care should be taken (based on appropriate evidence/data) when determining an appropriate dose for such subgroups, especially for young children. Similarly, evidence sources from a specific subgroup cannot be extrapolated to the general population, unless adequately justified e.g., data from clinical studies that only use females in the treatment group may not be generalisable to generally healthy adults (depending on the objectives of the study). Table 1 provides some examples of the characteristics of study populations that are relevant to the target population.

The case studies 8 and 9 in Appendix 2 show examples of evidence sources considered relevant to a medicine. Case studies 10 and 11 are examples of evidence sources not considered relevant to a medicine.

The decision tree in Figure 3 can be used as a tool to help determine if the evidence source is relevant to the medicine and when additional justification of relevance should be included in your evidence package.

How to assess the quality of evidence

Each source of evidence should be assessed to determine if it is of sufficient quality. Quality in this context means how certain or confident we can be that the estimates of the therapeutic effect in an evidence source is correct. Depending on the evidence sources, some of the factors that may increase or decrease our confidence in the information presented include:

• risk of bias
• precision of the effect estimates
• consistency of the individual study results
• how directly the evidence answers the question of interest
• risk of publication or reporting bias.

When we discuss the hierarchy of scientific evidence, we are referring to the type of evidence/study design and how much the reported results may be impacted by bias. Scientific evidence sources can be categorised into a hierarchy based on how much the source is impacted by bias. In contrast, traditional evidence sources do not have a hierarchy based on bias.

Certain sources of scientific evidence provide a lower risk of bias than others due to their design, methodology or level of review. When designed and implemented appropriately, double blinded randomised controlled trials (RCT) and systematic reviews of multiple RCTs are the most likely to achieve low bias and high precision when studying treatment effects (high quality). Conversely, if an RCT is not appropriately designed and implemented, the results generated may not be robust and reliable (poor quality).

Table 2 provides a hierarchy of scientific evidence types. The evidence types under Category A are generally better at controlling bias and are generally attributed the most weight in an assessment of the body of evidence. Category C evidence types can be more affected by bias and therefore are usually attributed the least weight.

When reviewing evidence from any of the categories in Table 2 to include in your evidence package, you should consider the limitations commonly associated with each evidence type. For example, depending on your chosen indication, cohort and case-control studies may not be enough alone to substantiate the indication (i.e., efficacy) due to the higher risk of bias associated with these types of studies. In these situations, the studies may be included in your evidence package to be critically appraised in conjunction with other evidence sources. It is also important to note that this hierarchy is a general guide only; a hierarchy approach should not be used to automatically promote evidence from experimental studies over observational studies – given that the optimal study design will differ depending on the study question being investigated. Refer to the Joanna Briggs Institute³ for more discussion on this topic. The hierarchy provided in Table 2 is intended to provide useful information to assist compiling your evidence package but should not be used in place of conducting your own assessment of the quality and relevance of the evidence sources.

While double blinded randomised controlled trials and systematic reviews of multiple randomised clinical trials are usually associated with low bias and high precision, they are not always available or feasible. Acknowledging this, the TGA allows other study types and a range of other sources of evidence to be submitted as potential support for the claimed efficacy of a medicine. The limitations of these other sources need to be considered e.g., case-control studies and cohort studies may not be a practical means of providing evidence for some indications and are limited in their ability to produce unbiased and unambiguous data regarding the true efficacy of a medicine. They can, however, provide valuable supportive data related to the likely efficacy of a medicine for the general population.

It is ultimately up to you, as the sponsor, to consider while compiling your evidence package, why the evidence types are appropriate for your particular medicine (and indication/s) and why the risks of bias in those evidence types have no impact on the overall conclusion on the medicine’s efficacy.

Appendix 2 includes case study examples (case studies 8 and 9) demonstrating evidence sources that are relevant to a hypothetical medicine. Case study 10 provides an example of evidence sources that are not relevant to a hypothetical medicine and is not of sufficient quality.

Assessing the quality of evidence of traditional use

Each evidence source should be considered on its own merit in relation to the medicine before viewing in context of the overall body of evidence. An evidence source can only be considered legitimate if it establishes a tradition of use, is credible and relevant to the medicine and indication.

Is the evidence of traditional use from a credible source?

Evidence of traditional use for an indication should demonstrate that the medicine or the relevant ingredients in the medicine, have a recognised history of use in the specified tradition for the specified health benefit.

To use a traditional indication, the medicine or ingredient must be an accepted part of a tradition of medicinal use within an established paradigm for over three generations (75 years). This will demonstrate a recognised history of use that underpins the use of the medicine.

Refer to Appendix 2 for case study 11 that shows an example of a traditional evidence source that is not of sufficient quality.

Assessing the quality of scientific evidence

Clinical research should be conducted in a manner that yields meaningful and reproducible results. The design, implementation, results, and reporting of each piece of research are important factors that determine how much confidence we may have in the data generated and the author’s conclusions relied upon. The Grading of Recommendations Assessment, Development and Evaluation approach to assessing the certainty of a body of evidence is considered best practice by many international organisations and may be a useful tool when compiling your evidence package. It provides a structured way to consider the level of confidence that can be placed in an evidence source and may be a useful tool to help assess which sources to include in your evidence package. It may also help determine how much weight to place on certain evidence sources during a critical appraisal of the body of evidence.

The rest of this section outlines important factors for assessing the quality of scientific evidence that was identified in the literature search. As there is no single set protocol for how to conduct research, it is important to critically appraise evidence sources in terms of these factors. If these factors are not adequately described in the evidence source, and therefore you cannot determine whether bias has been adequately controlled, then you should carefully consider whether to include it in your evidence package. If you still decide to include that evidence source in your evidence package, then it should be accompanied by a discussion of the impacts of the potential bias on the data in the evidence source in the context of the body of evidence included in your package.

Evaluating the study design

Evaluation of study design is an essential part of assessing the quality of scientific evidence. The below sections describe important components of study design that, when appropriately considered and applied, contribute to the quality of the evidence source.

Ideally, the evidence source included in your evidence package should be high quality in terms of the factors outlined in the following section. There may be instances where the evidence source does not address all factors. Where information is missing, it is up to the sponsor to rationalise the potential impacts on the conclusions about the efficacy of the medicine.

Note

Not all evidence sources will have applied or discussed all the factors described below. If you choose to include such evidence sources in your evidence package, you should document how the limitations in the evidence sources do not impact on your conclusions about the efficacy of your medicine.

Study design and methods

Clinical trials should have clearly documented aims and methods. Study design (including the presence or absence of randomisation and blinding), measurement tools (and their validation where appropriate) and statistical methods should be clearly outlined. Inclusion and exclusion criteria and the baseline characteristics of study participants should be described. The baseline distribution of potential confounders should be shown, and any potential confounding factors should be considered, and accounted for, during the analysis. In addition, the limitations of the study design and/or the methodology chosen, and their potential implications, should be discussed by the authors.

The randomisation method and other critical study design parameters should be described and meet contemporary standards to allow an assessment of whether any bias was introduced into the study.

Intervention and control groups (study trial arms)

Randomisation of participants to intervention (in the context of these guidelines, the group taking the listed medicine) and control arms of a trial helps reduce innate inter-group differences and potential bias. The method of randomisation should be clearly described. Baseline characteristics of treatment and control groups should always be documented to establish equivalence in key areas such as age, weight, diet, and other factors that may contribute to non-treatment differences in health benefit between groups.

Ideally, trials should be conducted under conditions where the only difference between groups is that one is exposed to the intervention (medicine) and the other is not. This is often achieved in controlled trials but is less likely to occur in cohort studies and case-control studies. In these methodologies, the presence of potential confounders and study biases may impact on study results and should be considered and accounted for in your analysis of the study. When confounders exist within a study, they lessen the study’s quality and the degree of confidence in the reported study outcomes.

Study outcomes: primary and secondary clinical outcomes

A primary outcome is the main health benefit being investigated in a clinical trial. A secondary outcome may be another outcome that is measured during a trial but may not be the main question the study is trying to address. Ideally the medicine’s indication will be included in the study as a primary outcome. However, inclusion of the indication as a secondary outcome may possibly be justified in some cases if, the study design adequately controls for bias and the observed result is shown to be statistically and clinically significant.

Evidence sources that only demonstrate a substance’s mechanism of action are not considered equivalent to the demonstration of a clinical outcome. However, such sources may be useful as part of the body of evidence to justify the biological plausibility of a clinical outcome e.g., a non- clinical study that shows in vitro inhibition of COX enzymes by green lipped mussel should not be used on its own to demonstrate that a medicine can relieve inflammation.

Some evidence sources may refer to a particular measurement method used to assess study outcomes e.g., visual analogue scales to subjectively assess pain, hunger etc. It is important that these methods are validated to ensure that the method is appropriate and accurate for its intended purpose and that the results can be reproduced. If the measurement method has been borrowed from another party, the original authors that developed and validated the method should be cited in the study. The characteristics of the validation methods are often reported in other studies or publications, and it is this original research that validates the scales. It is good practice to check the validation conducted by the original authors. If a measurement method has been appropriated from another party and has been modified, it is also good practice to check for signs that the modified method has been validated.

Where information about method validation is not reported in a study, the impact of this on the conclusions drawn from that study should be accounted for. Whether there is confidence in the measurement method to generate reliable and accurate results should be considered and, where appropriate, documented for each evidence source included in the evidence package.

Appendix 2 provides a case study (case study 14) demonstrating a clinical study where a measurement method was appropriated but not validated.

Statistical analysis

The principles outlined in Note for Guidance on Statistical Principles for Clinical Trials (ICH Topic E9) are a good resource when assessing whether the statistical analysis of a clinical study has been conducted in a robust manner.

Even if a clinical study is well-conducted and sources of bias are limited, there is a possibility that the results arose purely by chance. Studies can use various statistical methods to minimise a ‘Type I error’ (also known as a false positive) where a conclusion has been made that there is a difference between two study interventions when no difference really exists. It is important to bear in mind that statistical significance does not provide information about the degree of health benefit produced or whether it is likely to be clinically meaningful and, as such, clinical significance should be considered in addition to statistical significance. For more about clinical significance, please see Clinical significance.

When compiling your evidence package, you should consider the results reported in each evidence source (taking into consideration factors such as those outlined in Assessing the quality of scientific evidence). In instances where there may be statistical issues (e.g., a study is under powered, confidence interval not reported) a discussion of why the conclusions drawn by the author/s (as a result of the statistical analysis conducted) can be relied upon should be included in your evidence package.

Statistical significance (p-value)

Well-conducted clinical studies will usually report the degree of statistical significance (p-value) associated with the observed difference between treatments. The p-value indicates the probability that an observed effect is due to chance (that is, the probability of making a Type I error). Although there is no definitive p-value threshold, the lower the p-value the greater the likelihood that the effect observed is real. In practice and in general, a p-value of less than 0.05 indicates with acceptable certainty that an observed effect or health benefit is unlikely to be due to chance.

Things to consider relating to statistical significance and the p-value include whether:

• the statistical test used to derive the p-value is appropriate and reliable
• the p-value obtained for the primary outcome is less than 0.05
• all the actual p-values (not just p < 0.05) are reported.

Statistical significance (confidence intervals)

Confidence intervals provide an alternative measure of statistical certainty. The confidence interval (CI) is the range of values within which there is a certain likelihood that the true value can be found. The confidence level is the probability that the CI contains the true difference. Well-conducted studies should usually report the 95% CI. This means that there is a 95% chance that repeated experiments would have outcomes that fall within the specified range.

The precision (or width) of the CI is also an important consideration. A narrow 95% CI is much more desirable than a wide 95% CI. A wide CI indicates a low level of confidence in what the true population effect is.

To demonstrate that there is indeed a difference between the treatment and control groups, the 95% CIs of the treatment and control groups should not overlap.

Factors affecting statistical significance of study outcomes

Attrition rates (dropout rates) are commonly high in clinical studies that evaluate health benefits that are modest and require long-term commitment. High attrition can introduce serious bias (attrition bias) into these studies because the reasons for non-completion vary across initially randomised groups. High attrition rates may also diminish the general applicability of the treatment to any population, as it is unclear which socio-demographic or other factors may be associated with dropout. The resulting data from a high attrition study should be interpreted with caution.

An Intention-To-Treat (ITT) analysis, in which outcomes of the original randomised groups are compared, provides a means of accounting for the effects of dropouts. In an ITT analysis, dropouts from the study are included in the analysis. When an ITT is performed, all efforts should be made to obtain outcome measurements from dropouts at the end of the study. In cases where this is not possible, baseline measurements of study parameters should be carried forward e.g., for a study outcome related to weight loss, body weight recorded at the beginning of treatment would be carried forward to the end of the study for dropouts. In this way, a treatment effect demonstrated in an ITT analysis underestimates the efficacy of the treatment but may be a good reflection of effectiveness under real world conditions4. When dropouts are not accounted for in the analysis of results, attrition bias (exclusion bias) may result.

Number of participants (power calculations): The power of a study is the probability of detecting an effect when the effect is truly there. Power is proportional to sample size i.e., the greater the sample size, the greater the power of the test. However, very little power is gained by increasing the sample size beyond a certain point. A power analysis is used to perform sample size calculations in order to determine how many subjects are required in a clinical study in order to detect a difference between treatment and control groups.

It is important that studies enrol sufficient numbers of participants to detect a significant and reliable treatment effect. The number of participants required to be reasonably certain of a reliable result needs to account for the degree of health benefit (effect size), the variability of individual results and the number of participants dropping out of the study (attrition rate). Therefore, a study may need to include larger numbers of participants to account for a high attrition rate and/or small effect size. Conversely, where an effect size is large, a study may only require a small number of participants to measure the effect reliably and accurately.

A sponsor is not expected to perform power calculations on behalf of study authors, but consideration should be given to: any limitations of the statistical analysis that the study authors have reported, including how the study was powered (and why this is appropriate); why a certain effect size was selected; number of dropouts; and how these factors impact on the reported study outcomes. If you choose to include an underpowered clinical study in your evidence package, you should include a justification for why you think the study outcomes can be relied upon to reflect the efficacy of your medicine in the context of the body of evidence included in your package.

Often high-quality clinical studies have been designed to ensure that the results generated are statistically robust. The authors would have considered the factors that are important to achieve the desired outcome, have designed the study accordingly and determined how they would conduct their statistical analysis prior to running the study (to reduce the risk of data dredging).

Tools for assessing risk of bias

The National Health and Medical Research Council has the following definitions for bias and risk of bias:

• Bias refers to factors that can systematically affect the observations and conclusions of the study, causing them to be different from the truth. Studies affected by bias can be inaccurate. for example: finding false positive or false negative effects or associations by over- or under- estimating the true effect. Often, it is difficult to tell from the study results whether bias is present, instead, we can assess the study design to gauge the risk of bias – see below.
• Risk of bias refers to the likelihood that the features of the study design or conduct of the study will give misleading results.

There are a number of ways to assess risk of bias for different studies (refer to NHMRC assessing risk of bias guideline) with the most commonly employed tool for assessing risk of bias for randomised clinical trials being the Cochrane Risk of Bias tool. Whether you choose to use these tools or not, a discussion of why you believe the evidence sources have adequately controlled for bias should be included in your package.

Note

The following tools may assist you to assess the quality of the evidence source:

• PRISMA / AMSTAR – developed to evaluate systematic reviews of randomised and non-randomised trials
• CASP – series of critical appraisal checklists designed for use with systematic reviews, RCTs, cohort studies, case control studies etc.
• CEBM – has a selection of tools to assist with critically appraising literature
• CONSORT Statement – provides the international standards for reporting randomised trials
• EQUATOR Network – provides reporting guidelines for all main study types
• GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) – provides guidance for assessing how much confidence to place in findings from systematic reviews of qualitative research.

Clinical significance

Not all statistically significant differences are clinically significant. A statistically significant outcome only indicates that the difference observed or measured between two groups (i.e., treatment vs control) is unlikely to have occurred by chance. Whereas clinical significance is the practical importance of the treatment effect – whether it has a real, palpable, noticeable effect on a person’s daily life.

For listed medicines, this may be regarded as ‘a degree of health benefit that is meaningful to the consumer’. The number of participants required to detect a clinically significant difference between treatment and control groups depends on the type and level of health benefit, the standard deviation of the health effect, the significance level (p-value) and statistical power of the study and the type of hypothesis being tested.

Determining the clinical significance of health outcomes associated with listed medicines is particularly difficult for the following reasons:

• Listed medicines are self-selected by consumers from a wide variety of backgrounds, with varied expectations and variable educational and financial resources.
• The health outcomes provided by listed medicines may be modest, not readily apparent, and/or achieved over long periods of time.
• Healthy consumers may be satisfied with smaller gains in health than individuals with a pre- existing condition.

Therefore, it may not always be possible to determine clinical significance of evidence sources, but this concept should be considered when assessing the balance of evidence.

The following section provides information about indications and claims for listed medicines and how evidence can be used to demonstrate efficacy.

Differences between claims and indications

Note

• Health benefit refers to the beneficial effect of a medicine formulation or ingredient on health outcomes as described in an evidence source.
• Indication is defined in the Act as the specific therapeutic use/s of a medicine.
• Claim refers to a statement that does not describe a therapeutic use. Claims which imply therapeutic uses are assessed as indications.

Indications

Indications describe the specific therapeutic use/s of the medicine. Consistent with their low- risk status, listed medicines may only use low level indications included in the Permissible Indications Determination.

• Low level indications may:
• refer to general health maintenance
• refer to health enhancement
• refer to the prevention of a dietary deficiency
• imply a benefit for a non-serious form of a disease or condition.

Low level indications may not:

• refer to, or imply the prevention, cure or alleviation of any disease, ailment, defect or injury.

The Permitted indications for listed medicines guidance provides information about permitted indications for listed medicines including: terminology; structure; use; and how to apply for new indications.

Permitted indication qualifiers

When entering a medicine in the ARTG via the Electronic Listing Facility (ELF), sponsors can select ‘indication qualifiers’ from drop down lists to make a permitted indication more detailed and align with the evidence they hold for the medicine. There are four different types of indication qualifiers available:

• traditional context (for traditional medicines only)
• time of use e.g., ‘after eating’
• population e.g., ‘in men’
• TCM pattern (for TCM medicines only).

For more information on indications qualifiers refer to the Permitted indications for listed medicines guidance.

Claims

In this guidance, a claim refers to a statement that does not describe a therapeutic use. Listed medicines may include claims or statements on their labels or other advertising in addition to permitted indications. Claims are not required to be included on the medicine’s ARTG entry, but a sponsor must still hold evidence to support these ‘claims’ (based on the sponsor certifications made under section 26A of the Act).

However, in some circumstances, a claim or statement may be considered to imply a therapeutic use. A therapeutic use may be implied where the message conveyed to the consumer suggests that the product is intended to influence, inhibit, or modify a physiological process. Where this is the case, a permitted indication of similar intent must be included in the ARTG entry and supported by evidence, for example: a medicine presented explicitly as a calcium supplement with a claim on the label stating, ‘Calcium is an essential component of strong bones’ will require evidence supporting the role of calcium for a related permitted indication such as ‘Maintain/support bone strength’ and the related permitted indication should be included in the medicine’s ARTG entry.

Refer to the Permitted indications for listed medicines guidance for more information on claims.

Types of indications

Listed medicine indications are classified as traditional or scientific indications based on the type of evidence used to demonstrate efficacy. Scientific indications are further categorised according to the level of specificity: non-specific or specific.

Traditional indications

Traditional indications present factual statements of a health benefit relating to a historical record of use within a recognised paradigm outside modern conventional medicine e.g., Traditional Chinese Medicine, Aboriginal and Torres Strait Islander healing practices, Ayurvedic medicine, Western herbal medicine.

Traditional indications must be based on long-term use (more than three generations, that is, 75 years) within a specific paradigm. The traditional use should be extensively documented in internationally recognised evidence sources for traditional medicine use.

In practice, the language used in traditional medicine paradigms can make it difficult to determine if an indication is specific or non-specific. Therefore, the TGA’s focus for traditional indications is whether they are supported by legitimate traditional sources of evidence rather than the specificity of the indication.

A traditional indication must include the traditional context of use (see Permitted Indication qualifiers) to clarify that the efficacy of the medicine carrying the indication is supported by a history of use within a specific traditional paradigm rather than based on quantitative scientific data. A medicine may include multiple traditional paradigms if the ingredient has been used within more than one traditional paradigm for the same indication e.g., ‘Ingredients in this medicine have been traditionally used in Ayurvedic and Chinese medicine for relieving symptoms of the common cold’.

Note

Under the Therapeutic Goods Advertising Code advertisements for listed medicines that include indications (or “one or more claims”, as referred to in that document) based on evidence of a history of traditional use must:

• disclose the reliance on this traditional use
• prominently display or communicate this disclosure in the advertisement.

When using a traditional indication, a sponsor should:

• select the traditional paradigm that supports the formulation of traditional medicine
• ensure the evidence is based on experiences or theories specific to or consistent with the particular tradition, not on scientific clinical evidence
• ensure the indication has the same meaning and intent as specified in the evidence
• ensure the indication uses the same logic and terminology (may be accompanied by English
• terms on the medicine label) as the evidence of use in the specified traditional paradigm.

Traditional indications cannot:

• refer to anatomical, physiological or pharmacological effects that are not envisaged within the specified paradigm e.g., ‘raise haemoglobin levels’
• imply efficacy based on scientific evidence for the medicine e.g., ‘clinically tested’
• use specialist terminology that belongs to a different paradigm e.g., ‘damp heat’ is a term that is specific to the Traditional Chinese Medicine paradigm and would be inappropriate for an Ayurvedic medicine
• include qualifications that require scientific substantiation e.g., ‘assists to increase bone density by 10%’
• refer to conditions that cannot be diagnosed within the specified paradigm e.g., ‘Traditionally used in Chinese medicine to increase bone mineral density’ is inappropriate as increased bone mineral density cannot be monitored or determined without conventional medical intervention
• refer to the modulation of biomarker levels or weight loss.

When choosing a traditional indication, the following questions should be considered:

• Are the terms used to describe the indication the same as those used in the evidence source of traditional use?
• If the terms are different from those in the evidence source of traditional use, can the change be justified?

Evidence sources of traditional use can be based on the medicine formulation or for an individual ingredient. If a traditional indication is linked to an individual ingredient in a medicine formulation, then that ingredient should be clearly linked to that indication on the medicine label.

In some instances, multiple sources of evidence of traditional use may be needed to support the efficacy of a listed medicine with a traditional ingredient or formulation. Together these sources should form a combined collective body of evidence that should be critically appraised in the evidence package.

If there is conflicting evidence between the history of traditional use and contemporary scientific evidence for the medicine, then sponsors should consider including a statement to this effect in any labelling and advertising associated with the medicine, for example: ‘this traditional use is not supported by scientific evidence’. This will ensure that the advertised information relating to your medicine is truthful, valid, and not misleading.

Medicines from more than one traditional medicine paradigm

Many traditional medicine paradigms use similar ingredients. A listed medicine may contain multiple traditional ingredients that are supported by evidence of therapeutic use in different traditional paradigms e.g., a Traditional Chinese Medicine ingredient may be combined with an Ayurvedic ingredient but as the new formulation is neither a Traditional Chinese Medicine nor Ayurvedic medicine, the formulation, as a whole, cannot claim a history of use. Each traditional ingredient should be linked to a separate traditional indication that refers to the specific traditional paradigm of use.

However, when combining ingredients from traditional paradigms, sponsors should ensure that the combination ‘makes sense’ and does not contradict the traditional use of the individual ingredient/s e.g., combining highly dilute homoeopathic preparations with essential oils in a topical cream is not consistent with the homoeopathic paradigm.

Example

Traditional indication for a multi traditional paradigm medicine

A listed medicine includes individual herbal ingredients that are drawn from traditional uses in Traditional Chinese Medicine and Ayurveda. There is no history of use of the particular herbal combination, but there is sufficient evidence of use of the individual herbs for the indication in both Chinese medicine and Ayurveda paradigms.

• Non-allowable indication (refers to whole listed medicine): “Used traditionally in Chinese medicine and Ayurvedic medicine to promote muscle relaxation’.
• Allowable indication (refers to ingredients): ‘These herbs are used traditionally in Chinese medicine and Ayurvedic medicine to promote muscle relaxation’.

Scientific indications

Scientific indications are those that refer to modern conventional medicine paradigms and that are supported by quantitative scientific data derived from human clinical trials, observational studies in humans and/or systematic reviews. When choosing a scientific indication, the sponsor should:

• ensure the body of evidence is based on scientific data obtained in humans
• ensure the indication has the same meaning and intent as the therapeutic benefit specified in the evidence sources.

Scientific indications should not:

• refer to traditional paradigms or be based on evidence of traditional or historical use
• imply a higher level of certainty in the indication than warranted by the body of evidence.

Cross-paradigm indications

A listed medicine can have a combination of traditional and scientific indications where:

• the efficacy of an ingredient in the medicine is supported by both evidence of traditional use and scientific evidence (cross-evidence base ingredient)
• the medicine contains both traditional and non-traditional ingredients with associated traditional and scientific indications (cross-evidence base medicine).

In this case, the traditional and scientific indications should be assessed separately, not as one single mixed-category indication. It is important that the indications for the listed medicine accurately reflect the evidence base for the indication(s). The evidence package should separately demonstrate the efficacy of your medicine for each traditional and scientific indication.

Cross-paradigm indications

Example 1:

If efficacy can be demonstrated, a medicine that contains Echinacea purpurea and ascorbic acid (vitamin C) can have the following indications listed in the ARTG:

• ‘Traditionally used in Western herbal medicine to enhance/improve/promote immune system function’
• ‘Maintain/support immune system health’

These indications can be combined on the medicine’s label as follows:

‘This medicine has been formulated from traditional and modern ingredients for a healthy immune system function. Echinacea purpurea has been traditionally used in Western herbal medicine to promote immune system function. Vitamin C supports immune system health’.

Example 2:

If efficacy can be demonstrated, a medicine that contains evening primrose seed oil (EPO) with a high gamma-linolenic acid content can have the following indications listed in the ARTG:

• ‘Traditionally used in Western herbal medicine to relieve symptoms of mild eczema/dermatitis’ linked symptom indication: ‘soothe/relieve skin inflammation’
• ‘Anti-inflammatory/relieve inflammation’

These indications can be combined on the medicine’s label as follows: ‘Evening primrose seed oil (EPO) has been used traditionally in western herbal medicine to relieve symptoms of mild eczema, such as skin inflammation. Gamma- linolenic acid is a component of EPO that has anti-inflammatory properties.’

Level of indications

Listed medicines may only use low level indications selected from the Permissible Indications Determination. Indications permitted for use in listed medicines may only refer to:

• health enhancement
• health maintenance
• prevention or alleviation of a non-serious vitamin or mineral dietary deficiency; or
• a non-serious form of a disease, ailment, defect or injury.

For the purposes of determining the level of evidence that is expected to support the efficacy of listed medicines, we classify scientific indications into two levels, specific and non-specific Traditional indications are not classified into ‘specific’ and ‘non-specific’ as the evidence expectations for non-specific and specific traditional indications are fundamentally the same. In addition, the terminology used in traditional medicine paradigms does not easily fit in this classification system.

Classifying scientific indications as specific or non-specific is a means to determine when it is acceptable for sponsors to hold a lower level of evidence. This is in recognition that, while all permitted indications are low level (to maintain a low risk medicine), some go beyond general health maintenance claims and refer to enhancement of health or include an effect on a specific target. Accordingly, the efficacy of listed medicines with these types of indications should be supported by a higher level of evidence.

Guidance on the minimum evidence requirements for specific or non-specific indications is included in What types of evidence are expected for each type of indication?

Note

For the purposes of these guidelines, when considering non-specific or specific indications, the following definitions apply:

• Symptoms: Any subjective evidence of disease or of a patient's condition, that is, such evidence as only able to be perceived by the patient, for example: muscle pain.
• Sign: The objective evidence of disease or of a patient’s condition, that is, such evidence that is apparent to a third party, for example: skin rash.
• Condition: Any deviation(s) from the normal structure or function of the body, as manifested by a characteristic set of signs and symptoms.
• For a condition to be ‘named’, it is mentioned/specified/identified in the indication.

Non-specific indications

Non-specific indications for listed medicines refer to general health and wellbeing, such as:

• health maintenance
• relief of general symptoms
• general vitamin, mineral or nutritional supplementation that imply a general health benefit such as the maintenance of good health.

Table 3 in conjunction with Figure 3 explains (with examples) the types of indications that are considered to be non-specific.

Specific indications

A scientific indication for listed medicines that goes beyond maintaining general health and wellbeing is classified as specific. Specific indications refer to:

• health enhancement, with therapeutic actions such as: enhance, reduce, improve, increase, promote, relieve, decrease or those with similar intent
• reduction of occurrence or frequency of a condition or symptoms, a discrete event or a named condition
• management or relief of symptoms linked to a named condition/disease/disorder
• nutritional supplementation that restore, correct or modify a physiological or mental process/function/state.

Indications referring to symptoms can be non-specific or specific depending on the particular target or action. Table 3, in conjunction with Figure 4, explains the types of indications that are considered to be specific or non-specific.

Instructions for using table:

1. The table should be considered in its entirety to best determine which category an indication falls in.
2. Start from the top of the table, and work sequentially through the numbered subcategories, paying close attention to the exclusions.
3. It is highly recommended to double check the classification of indications by using the decision tool at Figure 4.

Note

Maintain/support claims

The presence of the term ‘maintains/supports’ in general, usually implies a ‘health maintenance’ action and therefore, in most cases would be considered a non-specific indication. However, this may not always be the case. When determining whether your indication is about health maintenance or health enhancement, you should consider the indication holistically, considering factors such as if a vulnerable target population is included as a qualifier to the indication or the nature of the condition that is referred to in the indication e.g.,

• 'Maintains/supports healthy bone density’ when in the context of postmenopausal women would be considered ‘health enhancement’, as bone density decreases as a normal process of ageing in this subpopulation. Therefore, referring to maintaining bone density in this subpopulation goes beyond normal functions of the body.

Note

Linking ‘parent’ permitted indications with ‘symptom’ permitted indications in ARTG entries.

When entering a medicine in ELF, indications referring to general symptoms can be linked with an indication referring to 'the symptoms of’ a disease or condition (‘parent indication’). Parent indications are generally considered ‘specific indications’ as they refer to a named disease or condition.

When linking general symptom indications with an indication referring to symptoms of a disease or condition, you are expected to hold evidence for the selected symptoms as it relates to the disease or condition in the parent ‘specific’ indication. This may come from a variety of evidence sources.

If you do not hold that evidence, then you should not link that symptom indication to the parent indication (that is, the indication can be included separately in the ARTG entry and efficacy demonstrated by using separate evidence). You should ensure that the indication on the medicine label is truthful and an accurate representation of the body of evidence.

Selecting indication qualifiers for permitted indications in ARTG entries

When you enter your medicine in ELF, you may choose to select indication qualifiers (such as population qualifiers e.g., ‘in women’ or ‘time of use’ qualifiers e.g., ‘after eating’) to make a permitted indication more targeted and better align with the efficacy evidence you hold for your medicine. Although adding a qualifier does not necessarily make a non-specific indication become specific, please take care to ensure that the selection of a qualifier is relevant to the evidence you hold.

Figure 4 is a guidance tool designed to assist you with the process of classifying indications as specific or non-specific. While the tool is intended to increase the consistency of classification, we acknowledge that there may be some indications that are difficult to categorise for individual medicines. When considering your indication using the decision tool, some factors that should be considered include:

• the target population
• indication qualifiers
• the nature of the condition referred to in the indication
• whether combining indications on the medicine label changes the specificity of the indications.

Further, it is strongly recommended that when determining whether your indication is specific or non-specific, you check to see whether there are similar or related indications in the Permissible Indications Determination. For example, the Determination includes both ‘maintains/supports skin healing/repair/regeneration’ and ‘enhance/improve/promote skin repair/healing’. In this instance, the former indication is more likely to be considered as non- specific, when the target population is generally healthy adults.

We reiterate that when the TGA conducts a compliance review of a listed medicine, the medicine is considered on its individual merits. If you consider there is ambiguity in classifying your medicine’s indication, you may wish to include a justification or rationale in your evidence package for classifying an indication as non-specific or specific to show the TGA how you came to your conclusion.

Explained examples: bolded indications in the decision tool

The indications below (used as examples in the decision tool provided in Figure 5) have been further explained to help you understand the thought processes involved when using the tool.

‘Maintain healthy digestion’

1. Is the indication represented to restore, correct or modify (increase or decrease) a physiological or mental process/function/state?

   No – this indication is represented to maintain healthy digestion in a healthy person, and therefore, does not represent to modify any physiological function. 

   Conclusion: Non-specific

‘Relieve symptoms of dehydration’

2. Is the indication represented to restore, correct or modify (increase or decrease) a physiological or mental process/function/state

   Yes – to relieve symptoms of dehydration, the lost fluids in the body must be restored or corrected.

3. Does the indication explicitly refer to 'symptoms of' or ‘reduce occurrence of’?

   Yes – the indication mentions the words ‘symptoms of’.

   (Note: The definition of a ‘condition’ is ‘any deviation(s) from the normal structure or function of the body, as manifested by a characteristic set of signs and symptoms’. Therefore, it is only possible to have ‘symptoms of’ a condition—not ‘symptoms of’ a symptom.

   Conclusion: Specific

'Relieve skin redness’

1. Is the indication represented to restore, correct or modify (increase or decrease) a physiological or mental process/function/state?

   Yes – to relieve skin redness, physiological processes (such as chemical messengers affecting blood vessel dilation) must be modified in a patient to decrease the level of redness in the skin.

2. Does the indication explicitly refer to 'symptoms of' or ‘reduce occurrence of’?

   No – the indication does not mention the words ‘symptoms of’ or ‘reduce occurrence of’.

3. Does the indication suggest an action of relief?

   Yes – the indication refers to the relief of skin redness.

   (Note: changing the indication to ‘reduce skin redness’ or ‘decrease skin redness’—as allowed in the Permissible Indications Determination—will also answer ‘Yes’ to this question. Reducing or decreasing skin redness has the same meaning and intent as relieving skin redness. However, the words ‘Decrease/reduce’ in indications such as ‘Decrease/reduce time to fall asleep’ does not have the same meaning and intent as relief.)

4. Does the indication refer only to a single symptom/sign?

   Yes – skin redness can be either subjective or objective as this can be perceived or observed by the patient (i.e., the patient can see or feel the skin redness) or to a third party (i.e., a third party can see the skin redness). An average person would be able to conclude that the skin is indeed red and cannot deduce the disease/condition that is causing this sign/symptom.

   Conclusion: Non-specific

'Temporarily relieve mild fluid retention'

1. Is the indication represented to restore, correct or modify (increase or decrease) a physiological or mental process/function/state?

   Yes – relieving mild fluid retention is restoring or correcting the amount of fluid that has been trapped in the tissues of the body.

2. Does the indication explicitly refer to 'symptoms of' or ‘reduce occurrence of’?

   No – the indication does not mention the words ‘symptoms of’ or ‘reduce occurrence of’.

3. Does the indication suggest an action of relief

   Yes – the indication refers to the relief of mild fluid retention.

4. Does the indication refer only to a single symptom/sign?

   No – fluid retention is not a subjective or an objective evidence of a disease/condition. Fluid retention is manifested by a characteristic set of signs and symptoms such as swelling of the extremities and pitting of the skin thus making it a condition that is named in the indication.

   Conclusion: Specific

'Helps maintain weight after weight loss’

1. Is the indication represented to restore, correct or modify (increase or decrease) a physiological or mental process/function/state?

   Yes – the indication is represented to help maintain weight after weight loss by continuously correcting or modifying the physiological process/function, such as increasing one’s metabolism, reducing the absorption of fat or suppressing the appetite.

2. Does the indication explicitly refer to 'symptoms of' or ‘reduce occurrence of’?

   No – the indication does not mention the words ‘symptoms of’ or ‘reduce occurrence of’.

3. Does the indication suggest an action of relief?

4. No – the indication does not refer to the relief of anything.

   Conclusion: Specific

‘Helps maintain healthy blood sugar’

1. Is the indication represented to restore, correct or modify (increase or decrease) a physiological or mental process/function/state?

   No – this indication is represented to maintain existing healthy levels of blood sugar in a generally healthy person, and therefore, is not restoring, correcting or modifying any physiological functions.

   (Note: Indications referring to the management, balance and/or control of biomarker levels, ratios and ranges are not permissible indications and cannot be advertised.)

   Conclusion: Non-specific

‘Maintain/support testosterone level in older males’

1. Is the indication represented to restore, correct or modify (increase or decrease) a physiological or mental process/function/state?

   Yes – the indication is represented to correct the level of testosterone in older males by helping them maintain those levels—whose levels would otherwise naturally decline with older age

   (Note: Without the ‘in older males’ qualifier, the indication would represent to maintain levels in a healthy adult; therefore, would answer ‘No’ to this question.)

2. Does the indication explicitly refer to 'symptoms of' or ‘reduce occurrence of’?

   No – the indication does not mention the words ‘symptoms of’ or ‘reduce occurrence of’.

3. Does the indication suggest an action of relief?

   No – the indication does not refer to the relief of anything.

   Conclusion: Specific

‘Aid/assist digestion of lactose’

1. Is the indication represented to restore, correct or modify (increase or decrease) a physiological or mental process/function/state?

   Yes – the indication is represented to help the digestion of lactose by either directly providing/increasing more lactase enzyme or modifying the physiological function of digesting lactose by other means (e.g., modifying the gut microbiota).

   (Note: Depending on the mechanism of action of the indication, the words ‘aid/assist’— which is synonymous with ‘support/helps’—can either mean to increase/decrease something or to maintain it. Where ambiguity is present, sponsors should justify why they have categorised it one way or another, and how that is suitable based on their medicine’s design. For example, a sponsor may justify that their ‘aid/assist nail growth’ indication is designed and presented for healthy individuals and is categorised as ‘non-specific’ because it is maintaining existing healthy levels of nail growth.)

2. Does the indication explicitly refer to 'symptoms of' or ‘reduce occurrence of’?

   No – the indication does not mention the words ‘symptoms of’ or ‘reduce occurrence of’.

3. Does the indication suggest an action of relief?

   No – the indication itself does not refer to the relief of anything—only refers to the digestion of lactose. Although as a result of enhanced lactose digestion, it may relieve symptoms such as bloating and flatulence, it does not make sense to ‘relieve’ digestion of lactose.

   Conclusion: Specific

What types of evidence are expected for each type of indication?

Minimum evidence requirements

The different types of indications and the corresponding types of evidence that may be included in evidence packages are outlined in Table 4. These are general minimum evidence requirements to be included in an efficacy evidence package, which may not necessarily be sufficient in all cases.

Regardless of what evidence is used, all evidence sources should be assessed for quality and relevance as set out in these guidelines. Just the fact that a study is published should not be relied on to indicate that the study is of sufficient quality. Often clinical studies conclude that an effect might be present, but that further investigation is needed. If there are specific areas of concern for an evidence source, a justification should be included in your evidence package for why these areas of concern have no impact on the validity of your conclusions of your medicine’s efficacy. Sponsors are encouraged to provide additional evidence to strengthen an evidence package, particularly if any of the pivotal studies are limited in some way.

The options presented in Table 4 may not be suitable for every medicine. These requirements represent the lowest threshold below which the efficacy of the medicine cannot be reasonably assessed. This means that a minimum number of evidence sources from the categories of evidence described above should be included in your evidence package to be critically appraised as part of the body of evidence. Please see How to critically appraise the body of evidence.

As with all other aspects of an evidence package, where a sponsor diverges from these guidelines, justifications should be included in the evidence package to explain why the body of evidence compiled is of sufficient quality and relevance to conclude that your medicine’s efficacy is acceptable.

Using ‘clinically proven’ in your indications

Use of the claim ‘clinically proven’ in a scientific indication implies a higher level of certainty in the effectiveness of the medicine: that is, the medicine has been clinically trialled and proven to be efficacious. Such a claim should be supported unequivocally by robustly designed, published peer-reviewed clinical trial/s conducted on the actual medicine (i.e., not based on trials of individual ingredients found in the medicine) being advertised, or an identical formulation and dose (as a minimum).

Similarly, using terms such as ‘clinical’, ‘clinically’, ‘scientifically’ coupled with ‘trialled’, ‘researched’ or ‘tested’ implies a higher level of certainty of the medicine’s effectiveness. Unless these claims are supported by well-designed clinical studies on the actual medicine in question, these terms should not be used as they may mislead consumers about the effectiveness of the medicine.

Due to the additional evidence required to support the use of these terms, indications that include these types of claims are classified as specific scientific indications. Listed medicines with traditional indications should not use these terms unless supported by relevant scientific evidence, as this is misleading to consumers.

Vitamin/ mineral/ nutrient supplementation

Supplementation claims

For the purposes of these guidelines, a supplementation claim for a listed medicine is an explicit or implicit statement that a medicine provides a vitamin/mineral/nutrient for oral ingestion, which is ordinarily consumed from the diet (i.e., from food). A supplementation claim simply conveys that by consuming the medicine, the consumer will ingest additional vitamin/mineral/nutrient than otherwise ingested from dietary sources alone.

Examples of supplementation claims in relation to the mineral magnesium are:

• ‘magnesium supplement’
• ‘dietary magnesium supplement’
• ‘source of magnesium’.

A supplementation claim is not considered a therapeutic indication and, in general, such claims are not included in the Permissible Indications Determination.

While a supplementation claim can be included on a medicine label (providing the statement is truthful), the medicine must also have at least one indication from the Permissible Indications Determination (a Permitted Indication) in the medicine’s ARTG entry and on the medicine’s label.

Evidence required to support supplementation claims

A listed medicine can make a named vitamin/mineral/nutrient ‘supplementation claim’ on the medicine label, if the medicine:

• provides at least 25% of the Recommended Dietary Intake (RDI), Adequate Intake (AI) or Nutrient Reference Value (NRV); and
• the nutrient is in a form able to be absorbed by the body.

Recommended Dietary Intake (RDI)⁵ – the average daily intake level that is sufficient to meet the nutrient requirements of nearly all (97-98%) healthy individuals in a particular life stage and gender group

Adequate Intake (AI) – the average daily intake level based on observed or experimentally determined approximations or estimates of nutrient intake by a group (or groups) of apparently healthy people that are assumed to be adequate.

Nutrient Reference Value (NRV)⁵ – the daily amount of nutrients required for good health, as well as an upper safe level of nutrient intake.

Evidence required to support supplementation content claims

Listed medicines should not refer to the presence of vitamins or minerals unless they are present in the recommended daily dose of the product to at least the level of 10% of the RDI, or there is evidence to support a therapeutic effect at/below this level.

Indications with special evidence requirements

All scientific indications that refer to the effect of a stated vitamin/mineral/nutrient are categorised as specific or non-specific using the exact same principles as any other scientific indication (refer to Level of indications on the categorisation of scientific indications and the Minimum evidence requirements).

However, as specific exceptions, the TGA will accept the following two indications to be supported by the same evidence requirements as Supplementation claims:

1. ‘Maintain/support (state vitamin/mineral/nutrient) levels in the body'
2.  ‘Maintain/support (state vitamin/mineral) within normal range’.

A requirement included in the Permissible Indications Determination associated with the use of the above two indications is that the medicine label must include one of the following label statements:

• (Vitamins/minerals/nutrients/dietary supplements) can only be of assistance if dietary intake is inadequate

  OR

• (Vitamins/minerals/nutrients/dietary supplements) should not replace a balanced diet (or words to that effect).

The rationale for the TGA accepting special evidence requirements for the above two indications is that these indications (with their required label statements) only imply that additional oral ingestion (to that ingested from dietary sources) of the stated vitamin/mineral/nutrient will help the consumer reach the recommended consumption of the nutrient. That is, a consumer would have similar expectations for these indications as they would for similar supplementation claims. Accordingly, these indications can be supported by providing at least 25% of RDI, AI or NRV. The requirements of supplementation claims and the above two indications have been summarised in Table 5.

If you choose to use supplementation indications other than the two in the table above, your evidence package should include evidence source(s) that support the dosage of your medicine. If the dosage in the evidence source is different from your medicine, you should include a justification in the critical appraisal in your evidence package that explains the rationale for why your dosage will produce the therapeutic effect described in the evidence source. 

Additionally, if you believe that, based on your critical analysis of the evidence landscape, 25% RDI (or less) is appropriate for your circumstance irrespective of whether your indication is non-specific or specific, then your rationale should be explained in your critical appraisal. Refer to Justifications in your critical appraisal for more guidance on when justifications should be provided.

Biomarker indications

Biomarkers are molecules in the body that can be measured to monitor a physiological process or monitor a condition or disease. Blood glucose is an example of a biomarker for diabetes, where high levels may indicate the onset or progress of the disease.

The Therapeutic Goods (Permissible Indications) Determination only includes low level biomarker indications relating to general health. Examples include:

• ‘Helps maintain/support healthy blood sugar/glucose’
•  ‘Helps maintain/support healthy cholesterol’.

In addition, indications for listed medicines should only target healthy individuals with biomarker levels that lie within the normal healthy range. Sponsors are encouraged to consider their biomarker indications using the decision tool provided in Figure 4 to determine the appropriate level of evidence required to support their medicine’s efficacy. An example of a biomarker indication is included in the tool to provide clarification. 

Sources of evidence used to support the efficacy of listed medicines with biomarker permitted indications should show a therapeutic effect in the population that is representative of, or can reasonably be extrapolated to, the general healthy Australian demographic.

If an evidence source uses study populations with baseline biomarker levels outside normal healthy levels, the data should not be automatically extrapolated to a medicine with a biomarker permitted indication without further justification (for why it can be extrapolated) in the evidence package. Evidence sources using diseased populations may not be relevant as the pathophysiology of the disease may alter the way a substance works in the body.

The assessed listed [AUSTL (A)] pathway may provide an option for listed medicines to make biomarker indications beyond those included in the permitted indications list when the efficacy of the medicine has been assessed by the TGA. Refer to Assessed listed medicines evidence guidelines for more information.

Weight loss indications

It has been suggested that a mean reduction in Body Mass Index (BMI) of approximately 1 kg/m2 (that is, one BMI unit) across a population could have a significant impact on the prevalence of obesity and overweight individuals within the population13. There are clinical studies^6,7 that support the notion that a 5%–10% weight loss impacts positively in reducing the risk of type 2 diabetes, cardiovascular disease, hypertension, and psychological functioning (such as mood, body image and binge eating).

Indications relating to weight loss or weight management require scientific evidence that demonstrates the weight loss is clinically significant, unlikely to be attributed to diet and exercise alone and can be maintained after the initial weight loss period.

Amount of observed weight loss between obese and overweight individuals

Amount of observed weight loss between obese and overweight individuals In Australia, registered medicines targeting obese individuals [those with a Body Mass Index (BMI) over 30 kg/m2] are required to demonstrate an absolute reduction of the initial body weight of at least 10% over a one year period⁸. However, this amount of weight loss may not be appropriate for individuals who are moderately overweight (those with a BMI between 25 to 30 kg/m2), but otherwise healthy.

Obese individuals expend more energy for a given activity because of their larger body mass. Therefore, in a weight loss context, for the same level of dietary intake and physical activity, the amount of weight lost will be different for obese (BMI >30 kg/m2) and overweight individuals (BMI 25-30 kg/m2). This difference is likely to become increasingly significant as the mean BMI of study participants increases.

As such, studies that include obese participants with a BMI >30 kg/m2 cannot be automatically generalised to overweight, but otherwise healthy, individuals (BMI 25 – 30 kg/m2). If a sponsor chooses to include such studies in their evidence of efficacy package, a justification (substantiated by evidence) for the scientific rationale behind such a generalisation should be included.

Evidence of clinically significant weight loss

In weight loss clinical trials, the control group commonly achieves some degree of weight loss due to changes in lifestyle, such as dietary intake and exercise. It is possible for lifestyle modifications alone to give small weight loss results ^9,10,11,12. Evidence supporting the efficacy of listed medicines with weight loss indications should demonstrate that the degree of weight loss is clinically significant and is the result of the actions of the medicine (that is, unlikely to be attributed to diet and exercise alone).

What is clinically significant weight loss? 

In relation to weight loss indications, clinical significance means a significant weight reduction that is maintained over time (please see caveat below for temporary weight loss indications).

For listed medicines, given that they can only make low level indications, the TGA accepts a minimum level of weight loss of 5% reduction of the initial body weight over six months, as it likely represents a clinically significant level of weight loss. Weight loss of less than 5% of the initial body weight is unlikely to be clinically significant and therefore inadequate to support the efficacy of listed medicines with weight loss indications. Noting that these Guidelines are not mandatory requirements but rather show sponsors what a TGA delegate considers in a compliance review and what will more likely result in a successful compliance review outcome.

Therefore, in weight loss studies in which the participants are overweight but otherwise healthy, we expect the study to demonstrate:

• an overall weight loss of at least 5% of the initial body weight in the treatment group
  • the overall weight loss should be at least 3 % greater in the treatment group than in the placebo/control group in randomised clinical trials; and
  • at least 5 % difference in the treatment group than in the placebo/control group in non- randomised clinical trials, as this demonstrates that the weight loss is likely due to the treatment and not the consequence of lifestyle changes or placebo effects
• that the overall weight loss and the difference between groups is statistically significant
• at least 50% of the participants in the treatment group should achieve at least a 5% reduction of the initial body weight, making it ‘more likely than not’ that consumers will achieve a clinically significant benefit from the appropriate use of the medicine. This demonstrates that the effect of an ingredient or medicine is consistent across the whole target population.

Study duration for weight loss

Several international and local guidance documents have adopted a minimum study duration of at least 12 months for weight loss studies, including:

• The European Medicines Agency ‘Guideline on clinical evaluation of medicinal products used in weight management’¹³. which provides guidelines for clinical studies examining the efficacy of weight loss medications.
• The Food and Drug Administration draft guidance document titled ‘Guidance for industry developing products for weight management’¹⁴, which outlines the conditions under which a product can be considered effective for weight management.
• The 2013 AHA/ACC/TOS Guideline for the management of overweight and obesity in adults¹⁵, which recommends long term comprehensive weight loss programs.
• The National Health and Medical Research Council (NHMRC) ‘Clinical practice guidelines for the management of overweight and obesity in adults, adolescents and children in Australia’¹⁶ which includes statements regarding the duration of weight loss interventions.
• Initial weight loss is commonly followed by a gradual weight regain. This phenomenon has been observed in studies where the treatment intervention occurred for the entirety of the study, as well as in studies where the treatment intervention occurred for a specified length of time and the participants were monitored for the remainder of the study period. There have been several studies in which overweight/obesity interventions (including diet/exercise alone) were observed to cause weight loss in a relatively short time frame, frequently reaching a peak within 6 months of the start of treatment, after which study participants gradually began to regain weight^{17, 18}.
• For the purposes of listed medicines, a reasonable timeframe to achieve a significant degree of weight loss (at least 5% reduction of initial body weight) and to demonstrate clinical significance is considered to be six months. Short study periods fail to demonstrate the purported benefit of a treatment in full, including the ability to sustain weight loss for longer periods of time. Therefore, study lengths will generally need to be at least six months in duration to support the efficacy of listed medicines with weight loss indications. For information about study duration for temporary weight loss indications, see below.
• However, it may be possible to demonstrate clinically significant weight loss through a rigorously designed study of less than six months duration when there is sufficient follow up after the treatment intervention. Adequate follow up is critical because weight regain is a common occurrence following a weight loss intervention, and in the absence of follow up data demonstrating otherwise, it is reasonable to expect that weight regain will occur.

We therefore expect that if the treatment intervention period in a given study is less than six months, there should be follow up data demonstrating at least 5% reduction of the initial body weight at the six-month point (if not later).

Study duration for temporary weight loss indications

For indications specifically referring to temporary weight loss, which must carry the mandatory label statement advising consumers that the ‘Weight loss may not be maintained for longer than 3 months’ (which clarifies for consumers that the weight loss they achieve may only be for a short term), may be appropriate to rely on clinical studies that show at least a 5% reduction of the initial body weight over a minimum duration of three months.

Use of terminology that implies weight loss

Terminology that implies the potential for weight loss (e.g., through increased fat metabolism) should be used with careful consideration of the overall presentation of the medicine as this may imply a weight loss effect. Table 6 provides examples of indications that are at risk of implying weight loss. If you use such indications together with the overall presentation of your medicine to imply weight loss, then you should include an appropriate weight loss indication (e.g., ‘helps enhance/promote/increase weight loss’) in the ARTG entry for your medicine and hold evidence that demonstrates clinically significant weight loss as described above.

Scientific evidence to support weight loss

For listed medicines with indications relating to weight loss in overweight individuals (BMI 25- 30 kg/m2), supporting scientific evidence should demonstrate:

• A mean overall loss of at least 5% initial body weight in the treatment group:
  • the difference between the treatment and placebo group must be statistically significant (p<0.05)
  • the difference between the treatment group and the control group should be a mean overall loss of 5% for non-randomised clinical trials and 3% for randomised clinical trials
• the treatment group should show 5% greater weight loss than placebo group for example: if placebo group showed 2% weight loss, the treatment group should show at least a mean weight loss of 7% of initial body weight; and\at least 50% of participants in the treatment group must have achieved a loss of at least 5% of initial body weight and
• the above must be demonstrated at the 6-month point from the onset of the intervention period or
• for ‘temporary’ weight loss indications, the study duration should be at least 3 months.

When the collected evidence has been assessed for quality and relevance to the medicine, an evidence package should be collated to provide to the TGA if requested to do so [under subsection 31(2) of the Act].

Evidence packages provided to the TGA as part of an evidence review are not required to comply with a specified format. Evidence packages can be submitted using the Evidence package checklists available on the TGA website or you can use another format, as long as it can show the TGA that your medicine meets the requirements as set out in Sponsors’ legal responsibilities.

The purpose of an evidence package is to provide evidence to the TGA to demonstrate that your medicine will do what it says it will do (i.e., it will produce the therapeutic effect(s) described by its indications) and thus meets the evidence requirements set out in Sponsors’ legal responsibilities. The evidence that may be necessary to demonstrate the efficacy of listed medicines with scientific or traditional indications will vary from medicine to medicine.

Therefore, an evidence package should:

• Show that the data or information collected is a true and accurate representation of the evidence landscape by:
  • conducting an objective, comprehensive and transparent review of the literature relating to the indications for the medicine
  • selecting to include literature that is of high quality and relevant to the medicine
  • demonstrating that assumptions made are valid, biases do not impact the data validity and evaluating the impacts of competing data on the conclusions
• Provide reasons for why, based on the body of evidence collected, the medicine is likely to produce the therapeutic effects described by the indications.

It is suggested an evidence package include (where appropriate, noting that this is not a mandatory list of required information):

• Medicine identification details:
  • ARTG identification number
  • Medicine name
  • Sponsor name
  • Sponsor contact details
• General medicine details:
  • Dosage form
  • Route of administration
  • Intended population (e.g., age, gender, ethnicity, health conditions)
  • Recommended dose
  • Maximum recommended daily dose (MRDD)
  • Directions for use
  • Duration of use (where relevant)
• Active ingredient/s
• Method of preparation for active ingredient/s
• Indication/s
• Details of the search strategy used
• A critical appraisal of the body of evidence generated from the literature searches, including justifications for any data gaps or discrepancies
• A comprehensive list of references.

Critical appraisal of the body of evidence

Critical appraisal

A TGA delegate of the Secretary must make statutory decisions about whether a medicine’s efficacy is acceptable based on relevant evidence and data. As such, it is reasonable to expect sponsors to provide evidence and data that clearly explains why their listed medicine will work as described by its indications.

An evidence package should include a critical appraisal of the body of evidence to achieve this. The purpose of a critical appraisal is to demonstrate that the sponsor’s conclusion that the medicine will likely result in the therapeutic effect described by the indication is logical, reasonable and based on data that we have confidence in to be true and accurate.

A critical appraisal is especially important when a sponsor’s evidence package relies on sources of evidence that:

• may be missing critical pieces of information
• may not perfectly match the medicine
• may be subject to significant bias or confounding
• are from an evidence landscape mixed with positive and negative findings.

If a sponsor chooses to include sources of evidence such as those listed above, the critical appraisal should acknowledge these issues and provide an assessment of how and why these issues do not impact on your conclusion that the medicine is efficacious. In this situation, holding several of the above sources that meet the relevance and quality criteria when combined, may help strengthen arguments to support your conclusion.

A critical appraisal should:

• be focused on evidence sources that are most relevant to the research question and are of high quality
• avoid relying on irrelevant or weak evidence sources
• distinguish evidence from opinion, assumptions, misreporting and belief
• assess the validity of the results and conclusions in the relevant literature
• assess the impact of competing datasets
• assess the impact of any potential for bias
• assess the clinical significance of the findings in the literature.

If you are unable to provide a strong case to conclude that the medicine will likely result in the therapeutic effect described by the indication, then we recommend that you reconsider the design of your medicine, for example: indication, formulation, dose, dosage form or target population.

Balanced view of the evidence landscape

Before a medicine is listed in the ARTG, a sponsor should carefully consider the available body of evidence and be satisfied that the overall balance of evidence supports the efficacy of your medicine (that is, by considering both evidence for and against the therapeutic effect of interest). A well-constructed literature search will help to identify the overall evidence landscape related to a particular indication.

The evidence you collate into your evidence package should consist of sources that are largely consistent with the general evidence landscape. In other words, a reasonable person making an objective assessment of all relevant evidence about the medicine would conclude that there is good evidence to support your medicine’s efficacy rather than against it. The medicine’s indications should not, indirectly or by implication, lead consumers to believe that the medicine will assist in a health benefit that is not supported by the balance of evidence.

For scientific indications, a sponsor should regularly check for new relevant data to ensure that the balance of evidence continues to support the efficacy of their medicine, for the time the medicine is listed on the ARTG. This is best achieved by building and maintaining an up-to-date body of evidence for the medicine. Over time, the evidence landscape supporting a particular therapeutic use may change as research and technology advance. Newer clinical studies may augment the strength of evidence related to an indication or they may be inconsistent with previous findings. Maintaining a comprehensive and up-to-date body of evidence for your medicine will help ensure that your indications remain valid, not misleading and consistent with the scientific evidence for the life of your medicine.

Justifications in your critical appraisal

Typically, the evidence that is relied on to demonstrate the efficacy of listed medicines is not derived from clinical studies conducted on the medicine itself. Instead, most of the evidence comes from the available literature. Consequently, there can often be information gaps or discrepancies between sources of evidence. Where a sponsor elects to include evidence sources that have gaps or discrepancies in their evidence package (such as those described under Critical appraisal), justifications should be included as part of the sponsor’s critical appraisal of the evidence. The purpose of these justifications is to demonstrate why it can still be concluded that the medicine will likely result in the therapeutic effect described by the indications, despite the gaps and discrepancies in the body of evidence.

A justification can be in the form of a written explanation or the provision of additional supportive evidence (or both).

When a justification should be provided

If an evidence package includes sources that do not provide certain details (e.g., frequency of dose, dosage form) that are pertinent to establishing relevance to your medicine, a justification should be provided for how the purported effect in those sources can be extrapolated to your medicine.

Where a source includes population groups (e.g., adults with a serious disease) that differ from the target population group for the indication (e.g., generally healthy adults), an evidence-based justification should be provided for how the results observed in the studied population can be extrapolated to your medicine and its target population. In some circumstances, a combination of in vitro, non-clinical and clinical data may show that the mechanism of action of an active ingredient works independently of the disease state and any pathophysiological differences of the study participants. In this instance, data from a diseased population may be justified to be generalisable to a healthy population.

Where there are competing datasets, it is important to explain why a particular dataset best represents what your medicine will do, for example: in some instances, the differences in results may be due to differences in dosage, the population tested or other aspects of study methodology that do not align with your medicine design.

Where there is data to suggest that certain aspects of a population demographic (e.g., diet, gender, age) have an impact on the therapeutic effect that is being examined (e.g., enhances intestinal beneficial bacteria growth), justifications should be provided to demonstrate how the results from a specific demographic (e.g., adults with a Japanese diet) can be extrapolated to the medicine’s target population (e.g., adults with an Australian diet).

Examples of when a justification should be provided in your evidence package may include (but are not limited to):

Where an evidence source:

• is not relevant to your medicine’s design (e.g., different ingredients, method of preparation, dosage, target population)
• is not high quality
• has a moderate to high risk of bias
• has a poor study design, low statistical power or low clinical significance
• is not reflective of the evidence landscape
• includes study endpoints that are not relevant to the indication
• does not include details to establish relevance to your medicine’s design and/or determine risk of bias and/or determine data validity.

Figure 5 summarises the overall process for determining the evidence to include in an evidence package. 

The TGA does not routinely review the information included in internationally recognised resources and texts (for example: pharmacopoeias or monographs). It is a sponsor’s responsibility to ensure that the sources included in an evidence package are relevant to the medicine and of sufficient quality.

The following list includes examples of internationally recognised texts that may be used to support efficacy (note this is not an exhaustive list).

Monographs

• Blumenthal M et al. (eds) (2000) Herbal Medicine – Expanded Commission E monographs, American Botanical Council, Austin, Texas.
• European Scientific Co-operative on Phytotherapy (ESCOP) series (1996) Monographs on the Medicinal Uses of Plant Drugs, ESCOP, Exeter.
• World Health Organization (WHO) (1999) Monographs on Selected Medicinal Plants, Vol 1, WHO, Geneva.
• European Medicines Agency (EMA) Herbal Monographs.

Pharmacopoeias

• British Herbal Pharmacopoeia, British Herbal Medicines Association, West Yorkshire.
• European Pharmacopoeia, Council of Europe, Strasbourg.
• Martindale: the Extra Pharmacopoeia, Pharmaceutical Press, London.
• The British Pharmaceutical Codex, Pharmaceutical Press, London.
• The British Pharmacopoeia, Her Majesty’s Stationary Office, London.
• The United States Pharmacopoeia and National Formulary, USP Convention Inc, Rockville, Maryland.
• Pharmacopoeia of the People’s Republic of China VOL 1.

Note

TGA disclaimer

The following case studies are hypothetical examples only. The indications do not relate to actual medicines and the evidence packages described do not relate to actual evidence.

These case studies are only a snapshot of a portion of hypothetical evidence packages. As such, the TGA assessment in the following case studies is limited to whether an evidence source is shown to be applicable/relevant to the hypothetical medicine – not whether the efficacy of the medicine has been demonstrated. 

The TGA has not assessed whether, and is not implying, there is evidence to support or not support the efficacy of medicines with these the indications. They are provided here as a tool to accompany the information included in this guidance document and provide additional clarity to assist sponsors.

Case study 1: Health benefit does not match

The health benefit investigated in the evidence source does not match the medicine indication.

Case study 2: Active ingredient dose does not match

The active ingredient dose in the evidence source does not match the medicine.

Case study 3: Active ingredient dose does not match

The active ingredient dose in the evidence source does not match the medicine.

Case study 4: Preparation method does not match

The method of preparation (solvent type/ratio) of the active ingredient in the evidence source does not match the medicine.

Case study 5: Preparation method (raw) does not match

The method of preparation (raw vs extract) of the active ingredient in the evidence source does
not match the medicine.

Case study 6: Dosage form does not match

The dosage form used in the evidence source does not match the medicine.

Case study 7: Relevant evidence source

The sponsor’s evidence source is relevant to the medicine and indication and is of sufficient quality.

Case study 8: Relevant evidence sources

The sponsor’s sources of evidence are appropriately relevant to the medicine and are of sufficient quality.

Case study 9: Relevant evidence sources

The sponsor’s sources of evidence are appropriately relevant to the medicine and are of sufficient quality.

Case study 10: Evidence not relevant and/or insufficient quality

The evidence sources in the evidence package are not relevant to the medicine and not of sufficient quality.

Case study 11: Evidence not relevant and/or insufficient quality

The evidence sources in the evidence package not relevant to the medicine or of sufficient quality.

Case study 12: Evidence sufficient

The clinical study included an appropriate analysis of sample size to detect statistical significance.

Case study 13: Appropriate justification

The evidence package included an appropriate justification to support a discrepancy in the active ingredients between the evidence source and the medicine.

Case study 14: Non-validated measurement method

The clinical study used a measurement method that was taken from another study and not validated.

Case study 15: Study duration inconsistent

The study duration of the clinical study is inconsistent with the context of use of the medicine and thus does not support the indication.

Research question: Does supplementation with vitamin B12 prevent vitamin B12 deficiency?

Search strategy

View the TGA recorded information session on the revised Evidence Guidelines for listed medicines.

Footnotes

1. National Health Medical Research Council. NHMRC Guidelines for Guidelines 2009.
2. Schulz et al. (2010). CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomized Trials. Annals of Internal Medicine 152.
3. The Joanna Briggs Institute Levels of Evidence and Grades of Recommendation Working Party. Supporting Document for the Joanna Briggs Institute Levels of Evidence and Grades of Recommendation.
4. Koepsell, T & Weiss, N (2003). Epidemiologic Methods: Studying the occurrence of illness.Oxford University Press, New York
5. https://www.nrv.gov.au/
6. Knowler WC, Barrett-Connor E, Fowler SE, et al. (2002) Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 346(6):393–403.
7. Wing RR, Lang, W, Wadden TA, et al. (2011) Benefits of modest weight loss in improving cardiovascular risk factors in overweight and obese individuals with type 2 diabetes. Diabetes Care. 34:1481-1486.
8. European Medicines Agency (2007). Guideline on clinical evaluation of medicinal products used in weight control (CPMP/EWP/281/96). London
9. Franz M, et al. (2007). Weight-loss outcomes: A systematic review and meta-analysis of weight-loss clinical trials with a minimum of 1-year follow-up. Journal of the American Dietetic Association 107: 1755- 1767.
10. Wu T, et al. (2009). Long-term effectiveness of diet-plus-exercise interventions vs. diet only interventions for weight loss: a meta-analysis. Obesity Reviews 10: 313-323.
11. Sacks F, et al. (2009). Comparison of weight-loss diets with different compositions of fat, protein, and carbohydrates. New England Journal of Medicine 360(9): 859-873.
12. Rose G and Day, S (1990). The population mean predicts the number of deviant individuals.British Medical Journal 301: 1031-1034.
13. European Medicine Agency (2017) Guideline on clinical evaluation of medicinal products used in weight management. https://www.ema.europa.eu/en/clinical-evaluation-medicinal-products-use…- control [Accessed 4 July 2019]
14. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (2007). Guidance for industry developing products for weight management. https://www.fda.gov/regulatory-information/search-fda-guidance-document…- weight-management-revision-1 [Accessed 5 July 2019].
15. Jensen MD, Ryan DH, Apovian CM, et al. (2014) 2013 AHA/ACC/TOS guideline for the management of overweight and obesity in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and The Obesity Society. Circulation. 129(25 Suppl 2):S102– S138.
16. National Health and Medical Research Council (2013) Clinical practice guidelines for the management of overweight and obesity in adults, adolescents and children in Australia.
    https://www.nhmrc.gov.au/about-us/publications/clinical-practice-guidel…- and-obesity [Accessed 5 July 2019]
17. Franz MJ, VanWormer JJ, Crain AL, et al. (2007) Weight-loss outcomes: a systematic review and meta- analysis of weight-loss clinical trials with a minimum 1-year follow-up. J Am Diet Assoc. 107(10):1755-67.
18. 18 Sacks FM, Bray GA, Carey VJ, et al. (2009) Comparison of weight-loss diets with different compositions of fat, protein and carbohydrates. N Engl. J Med. 360(9):859-873.