Building the Menu

Rigorous technical specifications guide the Menu’s review process

The Menu uses a systematic review methodology to identify and assess published evidence to be considered for inclusion.

The methodology was developed by the Centre for Evidence and Implementation and the University of Melbourne and reviewed by the University of Adelaide. It has also been informed by input received through sector consultation across four workshops in 2019.

The methodology responds to concerns with other existing evidence menus by including clear and concise information related to:

  • overview of the program target population, specific outcomes and delivery approach
  • strength of the evidence supporting the program or practice
  • if a program has been evaluated with Aboriginal children or families
  • cost information where available
  • implementation considerations

The methodology and guidance have been prepared for expert review teams to build the Menu and will be published at a later date.

The first stage of identifying the Menu content was to use the methodology across two focus areas of child maltreatment and family preservation and reunification. The University of Adelaide was engaged to undertake this review.

Searching for studies: A search strategy specifies the target populations, topics, study designs and practices that are within the Menu’s scope. The strategy includes six electronic databases that include grey and published literature.

Including relevant evidence: Identified studies are screened against scope, design and quality criteria.

  • Scope screens the relevance to the target population, topics and issues.
  • Design screens the methodology to understand effectiveness such as quasi-experimental, randomised controlled trials or systematic review.
  • Quality includes assessment of the study against the risk of bias tools (ROBINS-I, RoB2, AMSTAR 2).

Identifying the items: Potential Menu items are identified, and key information is extracted from supporting studies.

An additional search is undertaken using the program name to identify any published literature supporting the program. Any identified studies are assessed against the criteria in step 2.

Identify and assess outcomes: Relevant outcomes are identified, and the direction of effect is assessed. Assessing the direction for the Menu considers any potential bias (study design and quality), along with effect size and consideration of the real-world context of what constitutes a positive or adverse outcome.

Assigning the evidence rating: An overall evidence rating is assigned to the item on the Menu, and a table of effects by outcome and domain is prepared.

Summarising evidence: An easy-to-read evidence summary is prepared for the web portal.