The program evidence summary pages provide concise and high-quality information about what is likely to be beneficial based on the best available evidence. An evidence summary provides key information across six sections.
Building the Menu involves screening and excluding studies that do not fall within the scope, design, and quality guidelines.
Currently, the program evidence summaries contain information on the program’s effectiveness in the focus areas of child maltreatment and family preservation and reunification. Evidence for outcomes outside these focus areas is not included.
Overview
- Evidence rating – overall assessment of the evidence
- Risk of bias – overall assessment of the risk of bias for all supporting studies
- Pathway – alignment to the ‘pathways to support’ model of the Victorian child and family service system – early help, targeted and specialist, continuing care
- Cost – if cost information is included
- Australian study – if a local study is included
- Target age group – program age range
About
Program description and aims
This is a summary of the program, including the client needs that are addressed, and its intended system, family and child outcomes.
Impact
Program’s strength of evidence, effectiveness and relevance of the evidence to context
This is a summary of the evidence for the included studies.
It includes a direction of effect table for outcomes relevant to the Victorian child and family system under 6 domains: system, child, family, parent/caregiver, implementation, and cost.
There are 3 directions of effect:
- Positive effect: The estimated effect is positive and measurable change on an outcome can be attributed to the program.
- No effect: No observed effect or change on an outcome can be attributed to the program.
- Negative effect: The estimated effect is negative, and there is an adverse effect on the outcome that can be attributed to the program.
The impact of measured outcomes from the included study or studies shows if measurable change can be attributed to a program.
When interpreting direction of effect, a 'positive effect' should be taken to mean a benefit or improved outcome. For example, a positive effect for 'child protection contact' is assessed as having decreased contact with child protection services.
Outcomes can have more than one direction of effect. For example, both positive and negative effects can be reported.
This is because an outcome can be made up of several measures. For example, Out-of-Home care outcome may be made up of number of days spent in out-of-home placement and number of placement changes. These measures may report a different direction of effect based on single or multiple studies. In such cases, the evidence is indicated as mixed in the written description.
This summarises the strength of the evidence described in the included study or studies.
Each program is assessed and assigned an evidence rating. The evidence rating considers the study design and the number of research studies and provides a consistent way to assess the evidence.
More information about the ratings and their descriptors is available on the Evidence Rating page.
This information is described alongside the risk of bias, which tells us the quality of the evidence or the likelihood of misleading results. Studies assessed with very high risk of bias have been excluded from the Menu and is identified and reflected on the evidence summaries. The description is based on a risk of bias assessment using the ROBINS-I, RoB2 or AMSTAR 2.
This summarises whether the included study or studies included and reported on outcomes for Aboriginal children, young people or families.
Aboriginal knowledge and evidence are critical to recognise and share practices that lead to improved outcomes for Aboriginal children and families. The department will continue to consult with the Aboriginal community and stakeholders on how Aboriginal knowledge and evidence are defined and included on the Menu.
This provides information about the location, demographics and sample of participants who engaged in the included study or studies.
Implementation
Key characteristics of a program including delivery, implementation features and workforce requirements
This summarises information about the program’s delivery model, mode and setting, target cohort, program delivery, duration and intensity, and the availability of manuals and guides.
Where available, a link to the program’s website is also provided.
This information was populated from online research and reflects how the program is designed and delivered as intended rather than how it was delivered in the included study or studies.
This summarises the workforce and training requirements to implement the program.
This summarises if the program or practice is currently available or delivered in Australia and specifies the states or territories where it is available.
Cost
Cost information related to implementing the program
This reports cost information, where available, related to implementing the program.
References
Studies that were included or excluded on the Menu
List of research studies from which the evidence for each summary was drawn from to assess the program's effectiveness. These studies met the inclusion criteria and quality assessment of the Menu.
List of the research studies that met the inclusion criteria for the Menu but were excluded due to study quality, in other words, they were assessed to have high risk of bias in the quality assessment.