Each program is assessed and assigned a rating through a rigorous review of the program’s evidence, taking into account research design, any potential bias, along with the effects of outcomes and consideration of the real-world context of what constitutes a positive or adverse outcome.
An evidence rating is not a rating of impact. A higher rating does not necessarily mean a program is more effective. Similarly, a lower rating does not mean a program is less effective.
The higher the rating, the more confidence we can have in attributing the measurable change to the program. A program may have a lower rating because it has not shown replicable results in another study with a different population or setting.
The Menu assesses evidence generated from comparison trials in published studies that compares outcomes between people who received an intervention and those who did not, including systematic reviews, randomised controlled trials and quasi-experiments.
The evidence rating of a program is the assessment of evidence base on the specific target cohort examined in the included study or studies. This information should be taken into consideration as it may not be the same for your target cohort or the cohort as defined by the program’s eligibility criteria.
This review process is described in more detail on the Building the Menu page.
The 5 ratings
The Menu’s evidence rating distinguishes a program’s impact on a scale of increasing evidence strength.
Evidence from an independent source that shows positive effects from a rigorous systematic review with meta-analysis of studies with similar results.
Currently there are no very well-supported programs on the Menu.
Evidence of positive effects from at least two randomised controlled trials or quasi-experimental studies on at least one child or family outcome.
Evidence of positive effects from at least one randomised controlled trial or quasi-experimental study on at least one child or family outcome.
A program or practice has a planned or underway evaluation using a randomised controlled trial or quasi-experimental design.
A program or practice has a logic model that explains why and how the approach should have positive effects on child and family outcomes.
The plus rating
Programs with demonstrated sustained impact can provide insights into children’s and families’ long-term outcomes.
Programs with outcomes sustained 12 months and beyond, are indicated with a plus rating. For example, Supported+, Well-supported + or Very well-supported+.