Evaluating Regional Implementation of The Roadmap

Evaluation Outcomes

Outcomes to date:

Four evaluation reports are now available for the ‘evaluating regional implementation of the Roadmap to Close the Gap for Vision’ project.

The project commenced in 2019 and these reports were prepared using data collected through: a national survey of stakeholders working in the sector; focus groups and individual interviews in regional case study areas; and a scoping review of publicly available literature. The reports can be accessed here:

Summary Report:  This brief summary report covers key themes and findings from the evaluation. The summary report was prepared by IEHU with artwork by Casey Atkinson and Matty Atkinson and design by Little Rocket.

Next Steps:

The Indigenous Eye Health Unit sees great value in the evaluation. A written IEH response to the findings and recommendations will be shared with the sector over coming months, with input and guidance from The Aboriginal and Torres Strait Islander Reference Group.

We would like to thank all who participated in the evaluation, through the survey, focus groups, interviews and the co-design process.

We would also like to thank ARTD Consultants and Clear Horizon consultants and our project partners for their work and contributions to the project.

If you have any questions about the evaluation reports or next steps contact Associate Professor Mitchell Anjou manjou@unimelb.edu.au

About the Artwork

In December 2021, Indigenous creative agency, Little Rocket, was engaged to assist us in sharing the outcomes of the evaluation project. Bangerang/Wiradjuri artists, Casey Atkinson and Matty Atkinson, were commissioned to develop an artwork, and Little Rocket have based their graphic design around the key elements of Casey and Matty's wonderful piece of art, which presents the key themes that came out of the evaluation:

About the artists: Casey Atkinson (Bangerang/Wiradjuri) and Matty Atkinson (Bangerang/Wiradjuri) are brothers from Shepparton. They take great pride in the works they create and are dedicated to helping achieve positive outcomes for their people and the community. Casey and Matty’s artistic talents come from a line of amazing artists on both sides of their Atkinson and Bamblett families.

Their styles incorporate both traditional and contemporary Aboriginal art and is quite intricate and detailed. They enjoy working across various mediums including acrylic paint on canvas, spray paint, epoxy resin, woodwork, wood burning and also digital media. This collaboration for Indigenous Eye Health was the first time that they have worked together in what was an incredibly rewarding and invigorating opportunity.

Casey and Matty describe the artwork they created

The eye symbolises not only the Indigenous eye health organisation along with its goals and objectives, but also the perspectives of individuals within the wide array of Aboriginal and Torres Strait Islander communities across our beautiful nation. The patterns bordering the eyes show the existing connections between communities and Indigenous eye health. With all stakeholders meeting in the middle (the pupil) to achieve positive outcomes. 

The outer borders represent the broader community and society as a whole. The symbols in each community circle represent the areas in eye

health that we are aiming to strengthen across all of our communities. The Goanna footprints are representative of Indigenous eye health’s on-going journey to help heal our communities, specifically in relation to Trachoma.

The artwork by Casey and Matty is used in the Evaluation Summary Report

November 2021 Update

A fourth co-design workshop was held on 28^th – 29^th September 2021, at which participants considered outcomes and responses to the evaluation findings and the development of a knowledge exchange plan. The outcomes of the co-design workshop are informing the IEH response to the evaluation and IEH is working with the Aboriginal and Torres Strait Islander Reference Group (A&TSIRG) through this process.

A report from the co-design workshop will be available soon.

Next Steps:

The Indigenous Eye Health Unit sees great value in the evaluation and we are planning to make the findings of the evaluation readily available to and shared with all interested parties and stakeholders. An IEH response to the findings and recommendations will also be shared with the sector over coming months, with input and guidance from the A&TSIRG.

We would like to thank all who participated in the evaluation, through the survey, focus groups, interviews and the co-design process.

We would also like to thank ARTD Consultants and Clear Horizon consultants and our project partners for their work and contributions to the project.

If you have any questions about the evaluation reports or next steps contact Associate Professor Mitchell Anjou manjou@unimelb.edu.au

National Survey

EOI for Case Study Regions

Share YOUR Regional Eye Health Experiences and Learning

As part of the evaluation, we are looking for regional groups or networks of stakeholders, who have been working together to identify and address the eye care needs of Aboriginal and Torres Strait Islander people within their local or regional area, to participate as ‘case studies’ for the evaluation. This will involve a focus group and/or interviews with people involved in the regional network.

The aim of the case studies is to explore regional approaches in more detail, to better understand and describe the activities that have been undertaken, the changes that have been seen, the key things that have worked and the challenges involved in improving eye care services and outcomes for Aboriginal and Torres Strait Islander people at a local or regional level. This is also an opportunity for regional groups to share their experiences and learning.

We are seeking representation of groups and networks from across urban, regional and remote geographical areas and from different states and territories. We are interested to hear from both newer and more established regional stakeholder groups and networks.

Two regions are already participating, and a second round of EOIs is now open.

EOIs for round two are due by Friday 5th February 2021.

For EOI forms and more information contact Tessa.Saunders@unimelb.edu.au

BRIEF INFO SHEET

DETAILED INFO SHEET & APPLICATION FORM

December 2020 Update

Click the image below to view a one-page overview of the evaluation methods and timelines.

Ethics approvals have now been received from 7 HRECs, including: The University of Melbourne, Australian Institute of Aboriginal and Torres Strait Islander Studies, Aboriginal Health and Medical Research Council of NSW, Aboriginal Health Research Ethics Committee of South Australia, Human Research Ethics Committee of the Northern Territory, Central Australian Human Research Ethics Committee and the Western Australian Aboriginal Health Ethics Committee. Applications are pending with other entities and appropriate local approvals will be sought as required.

Data collection has started in jurisdictions where ethics approvals are finalised and this has included focus groups and interviews in two regional case-study sites. Interviews with external stakeholders on the role of IEHU in implementation of the Roadmap will start in December.

The second round of EOIs for case-study sites is now open for focus-groups and interviews to take place in February, March and April 2021 (see EOI for Case Study Regions tab)

The national survey will be released in early 2021, once all ethics approvals have been provided. The survey provides an opportunity for those working in Aboriginal and Torres Strait Islander eye care to share their views and experiences and the information gathered will help with planning future activities and allow learning to be shared about what works to improve eye care for Aboriginal and Torres Strait Islander people.

Survey participants will also go into the draw to win one of 20 ‘Check Today, See Tomorrow’ t-shirts or a pair of Deadly Eyewear sunnies!

For more information, see the December 2020 evaluation update provided at the 2^nd codesign workshop here

June 2020 Update

Due to the implications of COVID-19, we have revised the methods and timelines for the evaluation. We are now working through multi-site ethics approvals with relevant ethics bodies and the evaluation tools are being finalized ready for data collection by the independent evaluators – ARTD Consultants and Clear Horizon consultants. The methods for collecting data for the evaluation will include a short online survey, key informant interviews and focus groups. To maintain social distance the key informant interviews and focus groups will need to be conducted through video and telephone. For more information about the evaluation see the evaluation project summary here

The Aboriginal and Torres Strait Islander Reference Group has been providing valuable guidance to the evaluation project. The group was established following the first co-design workshop to prioritise the voices of Aboriginal and Torres Strait islander people, and includes members with a wide range of experience including health care delivery,  community engagement, health promotion, program management, policy development, evaluation, communications and advocacy roles. Current members include: Anne-Marie Banfield and Shaun Tatipata (co-chairs), Nick Wilson (secretariat), Tania McLeod, Tony Coburn, Nicole Turner, Colin Moore, Emma Robertson and Nicole Tujague.

An Expression of Interest (EOI) will be released shortly, inviting regional networks to indicate their interest in being ‘case study’ sites for the evaluation. As agreed with the evaluation co-design group, we will aim to have representation from across different jurisdictions and remoteness areas. The case studies will provide the opportunity for a deeper analysis of the different regional approaches to addressing Aboriginal and Torres Strait Islander eye care needs. Focus Groups and interviews for the case studies will be conducted by the Aboriginal Partnerships and Projects team at ARTD Consultants.

For further information about the evaluation contact Tessa.Saunders@unimelb.edu.au

Aboriginal and Torres Strait Islander Reference Group

The Aboriginal and Torres Strait Islander Reference Group provides valuable guidance to the evaluation project. The group was established following the first co-design workshop to prioritise the voices of Aboriginal and Torres Strait islander people and includes members with a wide range of experience, including health care delivery, community engagement, health promotion, program management, policy development, evaluation, communications and advocacy roles.

Current members include:

Anne-Marie Banfield and Shaun Tatipata (co-chairs), Nick Wilson, Makkaillah Ridgeway (secretariat), Tania McLeod, Tanya Morris, Nicole Turner, Emma Robertson and Nicole Tujague.

Evaluation Co-design

To make sure that we were asking the right questions and using the best methods, the evaluation was co-designed with stakeholders from across Australia and the Aboriginal and Torres Strait Islander eye care pathway.

Participants in the co-design group were selected through an Expression of Interest process and represented: Aboriginal Community Controlled Organisations (health services); ACCHO Jurisdictional Peaks; National Indigenous Health Peak; Jurisdictional Fundholders; Not for Profit Eyecare Organisations; Optometry, Hospitals; and Universities. Over the course of the co-design process, there was representation from all States and Territories except the ACT.

Four facilitated co-design workshops were held over the course of the evaluation:

• Workshop 1: design the evaluation questions and methods: The first workshop, with 38 people from across the country and the pathway of care, was held in October 2019 (see Workshop 1)

• Workshop 2: provide input into interpretation of initial evaluation findings: The second workshop, with 32 people from across the country and the pathway of care was held on 8th December 2020.

• Workshop 3: contribute to shaping the findings and evaluation reports: The third workshop, with 28 people from across the country and the pathway of care was held on 25 May 2021.

• Workshop 4: developing a response to the evaluation findings and a knowledge exchange plan: The fourth workshop was held 28 September 2021, with 27 people from across the country and pathway of care.

Co-design group members also provided input via email to evaluation tools, communications and draft reports.

In addition, the Aboriginal and Torres Strait Islander Reference Group, formed in December 2019, provided ongoing guidance through meetings and out-of-session comment.

Participation in the evaluation

Opportunities were provided for stakeholders to contribute to the evaluation through the national survey and/or through interviews or through case study sites.

If you have any questions about the process or what was involved email Tessa.Saunders@unimelb.edu.au or call (03) 9035 3972

Workshop 1 October 2019

On Tuesday 29 October 2019, the first (of three) codesign workshops for the evaluation of the regional implementation of The Roadmap to Close the Gap for Vision was held at The University of Melbourne. The workshop was organised by Indigenous Eye Heath and facilitated by an external Aboriginal facilitator, Kate Kelleher, with the support of the evaluation team from ARTD Consultants.

The objectives of the workshop were to ensure participants understood their role in co-designing the evaluation and how they can be involved in the evaluation and to reach broad agreement on:

• what ‘success’ of regional implementation looks like from various perspectives across the eye care pathway
• what important elements/issues can be identified to support the development of questions and methods for the evaluation
• the process/principles for identifying case study sites
• the best way to share the evaluation findings with those who will benefit from and use the findings.

Figure 1: Map of where stakeholders were from

Figure 2: Stakeholder roles across the eye care pathway

The workshop was well attended, with 38 people from across the country and the Aboriginal and Torres Strait Islander eye care pathway participating in great discussion throughout the day.

The input provided by workshop participants is now being used to finalise the evaluation questions and methodology. The codesign participants will continue to provide input over the course of the evaluation with the next workshop to focus on initial data once that has been collected.

Further detail about the evaluation will be shared once relevant ethics processes are completed.

Background

In 2019, Indigenous Eye Health (IEH) at The University of Melbourne commenced a formal evaluation of regional implementation of The Roadmap to Close the Gap for Vision (the Roadmap). Independent evaluators ARTD Consultants and Clear Horizon Consulting have been engaged to carry-out elements of this work with an experienced facilitator, Kate Kelleher, supporting the co-design process.

The aim of the evaluation is to assess current progress and effectiveness of regional implementation of the Roadmap. This will help to inform future activities, enable sharing of learnings across regions about sustainable and transferable approaches to improving eye care for Aboriginal and Torres Strait Islander Australians, and to identify any important lessons that may be transferrable beyond eye health.