Evidence guidelines

Guidelines on the evidence required to support indications for listed complementary medicines

Book pagination

24 July 2014

Scientific indications: What evidence do you need to support your scientific indication?

Scientific evidence refers to quantifiable data and includes: clinical trials in humans; epidemiological evidence; animal studies; and other evidence of biological activity. Due to the quantifiable nature of scientific evidence, scientific indications can imply efficacy to health outcomes.

Example of evidence for a scientific indication

  • A medicine containing turmeric (Curcumae longae rhizoma) has undergone randomised placebo-controlled clinical trials in humans and has been shown to provide a statistical significant decrease in gastrointestinal effects (abdominal pain, discomfort and wind) in this well designed study. An appropriate indication for this medicine may be:
  • 'Turmeric has been shown to relieve symptoms associated with mild digestive disturbances (abdominal pain and discomfort)'.

Scientific indications are usually supported with data from relevant controlled human clinical trials, or studies. These studies may be supplemented by other sources of evidence. You may also choose to conduct clinical trials on your medicine. For more information on conducting your own clinical trials refer to the National Health and Medical Research Council (NHMRC) website.

What are the sources of scientific evidence?

High quality sources of scientific evidence include:

  • Peer-reviewed6 original clinical research in well cited journals, for example: New England Journal of Medicine or The Lancet. Well cited journals are those with high Journal Impact Factors of greater than 5 (refer to Appendix 2: Journal impact factors).
  • Peer-reviewed journals are considered a good place to start when searching for supporting scientific evidence.
  • Systematic reviews of the clinical research relating to particular subject areas - such as those conducted by the Cochrane Collaboration. Systematic reviews are also reported in high-quality journals.
  • Unpublished studies or 'Propriety research' (as long as they fulfil the required criteria).
  • Secondary sources or non-clinical studies. Only human studies are considered appropriate to support indications for listed medicines. The scientific uncertainties involved in extrapolating non-human data from animal and in vitro studies limit their usefulness. However, non-human and in vitro studies may be used to support any discussion on biological plausibility.

What databases can you use to search for scientific evidence?

General web search engines are not considered appropriate databases or sources of primary scientific evidence. Similarly, online databases containing a limited collection of scientific abstracts are not considered primary sources of scientific evidence.

To find scientific journal articles that may be relevant to your indication; you should search a comprehensive electronic database such as MEDLINE (an electronic database produced by the United States National Library of Medicine that indexes millions of articles from 5,000-plus reputable biomedical journals from around the world).

Examples of comprehensive electronic databases include:

  • MEDLINE
  • EMBASE
  • Web of Science
  • the Cochrane library
  • BIOSIS
  • Sciverse Scopus
  • Cab Health
  • AGRICOLA
  • Food Science and Technology Abstracts

Access to MEDLINE is through the PubMed search facility, where you will also find search instructions, tutorials and FAQs. MEDLINE/PubMed provides abstracts (summaries) and citations for the journal articles listed, and often links to the full-text articles online.

Database searches

Your database search should utilise MEDLINE/PubMed electronic databases and include at least one other relevant database.

Web search engines are not considered appropriate databases or sources of primary evidence.

During your database search, if a substantial number of results (hits) are received, you can refine your search by reducing the date range to the last 5-10 years (a justification for refining the date range should be recorded in your evidence package).

Your database search of the literature should be documented to best practice standards. The Australian Regulatory Guidelines for Complementary Medicines (ARGCM) provides guidance on appropriate search strategy standards. The search terms, databases and search interfaces you use and the numbers of references retrieved should be documented in your evidence package (Evidence package checklists).

Using internationally recognised monographs or pharmacopoeias to support your scientific indication

High-quality and credible texts such as internationally recognised pharmacopoeias or monographs maintained by other international regulatory bodies (refer to Appendix 3) or evidence based reference texts may also be appropriate to support non-specific (general) indications, such as: nutrient supplementation indications when multiple monographs from different independent regulatory bodies report the same therapeutic benefit.

Internationally recognised monographs and pharmacopoeias can also provide additional support to specific indications referring to health enhancement claims, but such items will need further evidentiary support from primary research articles and/or systematic reviews. The more specific the indication, the more evidence you need to support your indication.

Using clinical studies to support your scientific indication

When citing a publication, you should not rely simply on the fact that a study is published as being sufficient to support your indications. Sometimes the results of published clinical studies are reviewed by others and these thoughts are captured in letters to the editor published separately in the journal. Often clinical studies may indicate that the results are indicative that an effect might be present, but that further work is needed. Such studies cannot be relied on solely to support specific indications.

Studies that have been through a peer review process are more likely to be methodologically robust and valid. You need to evaluate each piece of evidence to ensure that it is relevant and of high quality to support your indication.

Certain clinical studies offer higher quality evidence than others due to their methodological design (NHMRC, 2009). You should select the highest quality of evidence available to support for your indications.

Suitable evidence to support a scientific indication can be obtained from:

  • high quality, preferably multi-centre, random controlled trials (RCT)
  • well-designed controlled trials with randomisation; or
  • well-designed analytical studies preferably from more than one research group, including cohort and case-control studies.

Clinical trials, particularly randomised, placebo-controlled and blinded trials, provide the most robust information regarding the potential efficacy of a particular intervention. Case-control studies and cohort studies may not be practical means of providing evidence for some indications and are limited in their ability to produce unbiased and unambiguous data regarding the true efficacy of an intervention. They can, however, provide valuable supportive data relating to the likely effectiveness of an intervention within the general population. Case studies and epidemiological surveys do not have sufficient strength in their own right to justify a scientific indication.

If you use a systematic review to support an indication, it is necessary for you to demonstrate that the studies included in that review are relevant and satisfy the requirements outlined in the subsequent sections.

Diagram 3 and Table 4 display the hierarchy of evidence sources.

Diagram 3: Hierarchy of evidence sources

Diagram 3 - Increasing quality of evidence (lowest to highest): Expert opinion, observational and case studies, uncontrolled studies/monograohs, small random controlled trials, large random controlled trials, systematic reviews

The hierarchy of evidence is listed from the lowest to the highest sources of increased quality of clinical evidence.

  • Expert opinion
  • Observational and case studies
  • Uncontrolled studies/monograohs
  • Small random controlled trials
  • Large random controlled trials
  • Systematic reviews
Table 4: Levels of evidence associated with clinical studies (adapted from NHMRC)
Level Intervention Definition
I A systematic review of level II studies. Cochrane Reviews are examples of such systematic reviews.
II A randomised controlled trial. An experiment in which investigators randomly allocate eligible people into intervention groups to receive or not to receive one or more interventions that are being compared. The results are assessed by comparing outcomes in the treatment and control groups.
III
  • A pseudo-randomised controlled trial (that is: alternate allocation); or
  • A comparative study with concurrent controls:(Non-randomised); or
  • A comparative study without concurrent controls.
  • A pseudo-randomised controlled trial is a study with an independent, blinded comparison with a valid reference standard between participants with a defined clinical presentation.
  • Comparative studies with concurrent controls include: non-randomised experimental trial; cohort studies; case-control studies; or interrupted time series with a control group.
  • Comparative studies without concurrent controls include historical control studies, two or more single arm studies, cohort studies, case-control studies, and interrupted time series without a parallel control group.
  • Refer to the National Health and Medical Research Council for definitions.
IV Case series with either post-test or pre-test/post-test outcomes. In depth description of the factors related to a disease, disorder or condition in a specific individual or group of individuals.

Can you use unpublished studies to support your scientific indication?

Unpublished studies can contribute to your evidence base for a scientific indication if they are relevant and have been reviewed by at least two independent reviewers. To facilitate an accurate interpretation of methodological quality, any original research must be appropriately documented [Schulz et.al. (2010)].

Can you use abstracts to support your scientific indication?

Abstracts are not primary sources of scientific evidence and cannot stand alone to support scientific indications. These documents usually do not give sufficient details as to how the research was conducted or the data were analysed, thereby not allowing an objective evaluation of the quality of the research data and the conclusions drawn by the study authors.

Can you use non-clinical studies to support your scientific indication?

Animal or in vitro studies cannot stand alone to support scientific indications. The scientific uncertainties involved in extrapolating human health benefits from animal and in vitro studies limit their usefulness. Non-clinical studies are considered secondary sources of scientific evidence, and may provide additional weight to a proposed health benefit when limited clinical studies are available. Only clinical (human) studies are considered primary sources of scientific evidence and sufficient to support indications for listed medicines.

Can you extrapolate target population groups from clinical study groups?

Indications may refer to health benefits for the general population or for specific sub-populations (for example: children or elderly). The study should be carried out in a study group representative of the population group for which the indication is intended to be made. Extrapolation of results obtained from subjects outside the target population group is normally not accepted unless it can be appropriately justified. Given the variability of the indications and evidence such justifications can only be made and assessed on a case by case basis.

Using systematic reviews and review articles to support your scientific indication

Systematic reviews are reports of the outcome of analysis of a large number of clinical trials (sometimes known as a 'meta-analysis') aimed at looking for an overall pattern in the trial results. In a systematic analysis only those trials that meet a number of pre-set conditions in relation to research design (for example: sample size, randomisation) are included in the final meta-analysis. Cochrane Reviews are examples of such systematic reviews.

Review articles report the study outcomes of a collection of study articles with a common theme or on a particular topic. The review author/s will assess the published research, conduct research regarding the major research advances or report on any significant data gaps then present findings in a coherent view. Review articles provide broad information about a given topic, but are limited in their ability to provide support to specific indications. These limitations are primarily driven by limited scientific assessment or scrutiny of the studies reported in the review article, as review author/s often report the study outcomes as reported by study authors without further assessment of the study methodology.

Using Cochrane Reviews to support your scientific indication

Often a good source of high-quality evidence for scientific indications will be the Cochrane Library, a searchable database of the systematic reviews done by the Cochrane Collaboration. The Cochrane Collaboration is highly regarded, internationally recognised source of independent research. Cochrane reviews are systematic surveys of primary research (that is: experimental research) on healthcare topics.

Each Cochrane review starts with a clear question, for example: 'Does Echinacea reduce the occurrence of the common cold?' It then searches for and collates all the existing primary research from around the world that meets certain quality standards. Carefully following protocols that are designed to minimise bias, it then assesses that evidence against strict guidelines to establish whether there are conclusive findings that answer the question. The reviews are updated frequently.

Cochrane reviews concentrate on research conducted using randomised controlled trials (RCT). A randomised controlled trial is a study in which people are randomly allocated to one of two groups. One of the groups receives the treatment being tested (for example. a pill containing Echinacea purpurea). The other, known as the control group, receives a placebo treatment (for example: a sugar pill). No-one in either group knows which treatment they received. The study is controlled so that all participants have similar care in all ways other than the treatment being tested. Ideally the study participants, the scientists running the trial and those assessing the outcomes are also unaware of which group participants are in - this is called 'double-blinding'.

Cochrane reviews are written by and for scientists but each has a plain language summary (under 400 words) explaining the background, methods, results and weaknesses of the review. Even if you are not a scientist, these summaries can help you assess whether the evidence is relevant to your medicine's indication, but it would be beneficial for a person with expertise in critical appraisal to review the full study to determine if it actually supports your claim.

Systematic reviews and RCT are often associated with higher quality evidence and are appropriate to support scientific indications for listed medicines. However, various levels of clinical trials may be of high quality, but offer lower level of clinical significance due to the design of the study. For instance, random placebo-controlled trials are often seen as higher level or quality of evidence compared to cohort studies (refer to glossary for definition of terms) due to a lower level of potential confounders or bias in RCT studies.

Your evidence package should predominately consist of primary sources of evidence that are relevant to your medicine and of high quality. Multiple low-quality sources of evidence are unlikely to be adequate to support indications for listed medicines.

When using any evidence source, including systematic reviews or publications in a peer-reviewed journal, you must ensure that the evidence is relevant to your ingredient/medicine and relevant to your indication (see Using clinical studies to support your scientific indication).

Assessing your evidence

Relevance of the evidence to your indication

During your search for evidence you may come across many different types of evidence from a variety of different sources of literature. Review of the literature to produce an existing body of high-quality evidence that is relevant to the indication or medicine is known as 'filtering' (Evidence package checklists provides a checklist to assist you with this process).

Establishing the relevant evidence base for your proposed indication is a critical step in the review of evidence. This requires an assessment of the relevance of every item retrieved from the literature review of your proposed medicine, dose and indications. The relevant evidence base for your proposed indication includes all studies that are relevant in terms (amongst others) of ingredient, health benefit, population and context of use.

You must determine whether the evidence you have obtained is relevant to support your proposed scientific indications for your medicine.

Below are the concepts to allow you to assess whether the item of evidence is relevant to your medicine/indication.

Health benefit (indication)

You should ensure that the research is relevant to your proposed specific indication for your medicine. In drafting indications, you should take care to make sure that they match the underlying evidence you hold. Indications that do not match the science, no matter how sound that science is, are unlikely to be supported. Indications should not exaggerate the extent, nature, or prominence of the effects achieved in a study (the study outcomes), and should not suggest greater scientific certainty than that which actually exists.

All indications based on scientific evidence must be supported by primary evidence, such as clinical studies. The scientific uncertainties involved in extrapolating human data from animal and in vitro studies limit their usefulness as an evidence base to support your indication. Non-human and in vitro studies may, however, be used to support any discussion on biological plausibility of a potential mode of action in humans.

Participants enrolled in studies used to justify indications for your listed medicine should fit the following eligibility criteria, unless your medicine is directed to a specific population sub-group:

  • male and female participants
  • generally healthy
  • aged 18-65 years; and
  • socioculturally similar to the Australian population.

Secondary evidence or non-clinical studies (such as mechanistic studies) are normally insufficient to support indications implying efficacy. However, secondary evidence may be used in conjunction with primary evidence to strengthen the wording of an indication.

Dosage

For scientific indications, the recommended dosage and duration or frequency of administration of the medicine must be consistent with the evidence supporting the indication.

Active ingredient and route of administration

The evidence must relate to the whole medicine, the same active constituent(s) with a similar dosage regimen, dose form and route of administration to the medicine for which a claim is being made. For specific scientific indications for example, the different formulation between your medicine and that of the formulation reported in the scientific evidence is very important (for example: the use of a novel tablet matrix). If there are differences noted, then further justifications will be required to address the data gaps identified using the checklists provided.

Assessing the quality of the evidence

In addition to determining the relevance of the item of evidence, you need to determine whether the evidence is of high quality. Importantly, to facilitate the assessment of balance view of evidence, both relevant non-supporting and supporting items of evidence will need to be assessed for quality.

Methodology, blinding and randomisation

The clinical research being used to support your scientific indication should be conducted in a reliable manner to yield meaningful and reproducible results. The design, implementation, and results of each piece of research are important in assessing the adequacy of the substantiation of the health benefit or study outcome.

There are some principles generally accepted in the scientific community to enhance the validity of test results. However, there is no single set protocol for how to conduct research. For example, a study that is carefully controlled, with blinding of subjects and researchers, is likely to yield more reliable results. A study of longer duration can provide better evidence that the claimed effect will persist and better evidence to identify potential safety concerns.

You should critically appraise scientific studies in terms of methodological quality and the possibility of bias and/or confounding. Studies that have been peer-reviewed may be more likely to be methodologically robust. You should assess the results of scientific studies for statistical significance and meaningfulness (clinical significance) of the reported therapeutic benefit. Your evidence should demonstrate an overall improvement in the expected health benefit that is statistically and clinically significant. Additionally, studies that incorporate randomisation process of assigning trial subjects to treatment or control groups are often considered of greater quality due to the reduction of potential for bias. The randomisation method should be described in the study report and meet contemporary standards (such as using post-study questionnaires' of study participants to confirm that they remained blinded). Similarly, the incorporation of good blinding methods in the study design tends to result in studies that are methodologically robust.

Statistical analysis

A study that fails to show a statistically significant difference between test and control group may indicate that the measured effects are merely the result of a placebo effect or chance. The results should translate into a meaningful health benefit for consumers. Some results that are statistically significant may still be so small that they may not provide a positive effect to consumer health.

Filtering relevant evidence to those of high quality will involve, as a minimum, an assessment of the following:

  • characterisation of the ingredient/s
  • study design/methods
  • participant eligibility (inclusion/exclusion criteria)
  • adequacy of randomisation and blinding of participants (for example Randomised Controlled Trials (RCT))
  • sample size justification
  • controlling for potential confounders
  • study attrition (for RCT and cohort studies); and
  • statistical analyses undertaken.

For each study, the meaningfulness of the observed effect/s to consumers at an individual and/or population level (clinical significance) must be assessed.

Balanced view of your scientific evidence

Studies cannot be evaluated in isolation of the surrounding context. The context of the scientific evidence is just as important as the internal validity of individual studies. You need to consider all relevant research relating to the claimed benefit of your medicine and should not focus only on research that supports the effect, while discounting research that does not. A well-constructed literature search should normally be undertaken to help ensure that the general body of evidence related to a specific indication is identified.

Before you list a medicine in the ARTG you must be satisfied that the balance of evidence supports your indication. In other words, a reasonable person making an objective assessment of all the relevant, high-quality evidence about your medicine would conclude that the weight of good evidence is in favour of your indication rather than against it. Your indications must not, indirectly, or by implication, lead consumers to believe that your medicine will assist in a health benefit that is not explicitly supported by the balance of evidence.

The evidence you hold to support your indication should consist of studies that are largely consistent with the surrounding body of evidence. Where there are inconsistencies in the evidence, it is important that you examine whether there is a plausible explanation for those inconsistencies, for example: in some instances the differences in results will be attributable to differences in dosage, the form of administration, the population tested, or other aspects of study methodology. You should assess how relevant each piece of research is to your medicine and the specific indication you wish to make, and also consider the relative strengths and weaknesses of each. If a number of studies of different quality have been conducted on a specific topic, you should look first to the results of the studies with more reliable methodologies (that is, RCTs or systematic reviews).

You should also ensure that the evidence supporting your indications remains valid for the life of the medicine, and this is best achieved using a body of evidence approach. As research advances, the body of scientific evidence supporting a particular health benefit may change. Newer clinical studies may enhance the strength of the evidence supporting your claim, or it may be inconsistent with the strength of previous research. Having a body of supporting evidence will allow you to ensure that the indications claimed for your medicine remain true, valid, are not misleading and consistent with scientific evidence for the life your medicine.

What indication sub-type does your scientific evidence support?

Depending on the indication sub-type, the required evidence to support the indication will be different. Non-specific indications such as 'may assist with general health and well-being' can be supported by scientific evidence from monographs or pharmacopoeias. Whereas specific indications should be supported by specific scientific evidence such as results of clinical studies, or systematic reviews (refer to table below).

Table 5: Levels of evidence generally required to support sub-types of scientific indications
Scientific indication Evidence required to support indication
Non-specific indication Descriptive studies, case series or reports of relevant expert committees. Reference texts, such as pharmacopoeias or monographs, or other evidence-based reference texts, may be provided to support non-specific indications.
Specific indication

You must hold scientific evidence to support the specific indication. Such as

  • Evidence obtained from well-designed controlled trials with randomisation; OR
  • Evidence obtained from well-designed analytical studies preferably from more than one centre or research group, including epidemiological cohort and case-control studies; OR
  • Evidence obtained from multiple time series with or without intervention, including within country and between country population studies.

All indications / claims relating to symptoms must be accompanied by the advice "If symptoms persist consult your healthcare practitioner" or words to that effect.

Vitamin or mineral supplementation claims are only permitted where the recommended daily dose of the medicine provides at least 25% of the Recommended Dietary Intake (RDI) for that vitamin or mineral. The RDI in this context refers to the Australian RDI. If there is no Australian RDI for a vitamin or mineral, an RDI from another country may be used. Where vitamins or minerals are the subject of other kinds of claims, the dose must be consistent with the evidence to support the claim being made. Indications / claims should not refer to the presence of vitamins or minerals unless they are present in the recommended daily dose of the product to at least the level of 10% of the RDI, unless there is evidence to support a therapeutic effect below this level.

Choosing or deciding upon your scientific indication

Scientific indications are usually supported with clinical data from relevant human studies that have likely undergone some scientific assessment or scrutiny. Due to this quantifiable scientific research undertaken into their mode of action and/or health benefit, this evidence type is appropriate to support efficacy for your listed medicine.

Scientific indications must not specify or refer to traditional paradigms.

The use of the term 'clinically proven' in scientific indications infers a level of certainty in the implied health benefit associated with the listed medicine in that it has been clinically trialled and proven to be effective. These terms are not acceptable unless supported unequivocally by robustly designed, published peer-reviewed clinical trial(s) conducted on the actual medicine being advertised, or an identical formulation and dose (as a minimum). The use of the terms 'clinical', 'clinically', 'scientifically' coupled with 'trialled' or 'tested' implies a higher level of certainty associated with the health benefit of your medicine and unless matched by well-designed clinical studies on your specific medicine, may mislead consumers about the effectiveness of your medicine.

You must compare your indication with the quoted health benefit in your evidence identified from scientific sources. Your indication will refer to the same clinically significant study outcomes as that reported in the clinical study. In developing your scientific indication you should:

  • make sure that the indication is suitable for use in listed medicines
  • base any statement about the medicine's therapeutic benefit on the demonstrated clinical study outcomes; and
  • imply only the same level of certainty in clinical effectiveness as that reported in clinical studies, for example 'clinically proven to...' compared to 'may assist to...'.

How to compile a summary of the evidence to support your scientific indication

You should compile an evidence package demonstrating that you have conducted an objective, comprehensive and transparent review of the literature relating to your indication/s. The resulting evidence you hold should be of high quality and relevant to your medicine and adequately demonstrate that all indications you make for your medicine are true, valid and not misleading.

Evidence package checklists provides checklists to assist you to collate your evidence summary and filter evidence items to those that are credible and relevant to your medicine. You are required to hold all the information contained within the relevant forms for your medicine before listing it. Refer to the Appendix 1: How to use evidence package checklists for assistance with the process. While presentation in this manner is not compulsory, it will expedite the compliance review process, should your medicine be selected for an evidence compliance review.


Footnote

  1. 'Peer review' is the evaluation of research by other people in the same field to maintain the quality of work in that field.

Book pagination