Evaluation of Trachoma Health Promotion
The aim of this evaluation is to establish recommendations for future health promotion approaches in trachoma beyond June 2024 including improvements to current approaches with consideration for efficiency, effectiveness and appropriateness.
Evaluation of Trachoma Health Promotion
Background and Project Plan
The Minum Barreng: Indigenous Eye Health Unit (Minum Barreng) at the University of Melbourne is currently funded by the Australian Government Department of Health and Aged Care (DoHAC) to provide health promotion activities supporting the sustainable elimination of trachoma as a public health problem in Australia. The current funding agreement ends 30 June 2024.
An evaluation of this work is required as part of the current DoHAC funding agreement and activity work plan for 2023-2024. The evaluation is described as:
An evaluation of the impact of health promotion activities on facial cleanliness in selected communities for the period 1 July 2021 – 31 June 2023. This will include recommendation/s for future health promotion approaches in trachoma and any changes for improvement to current activities, with consideration to efficiency, effectiveness, and appropriateness of the project.
Minum Barreng has been conducting trachoma health promotion since 2009 and has been funded by the DoH to provide health promotion resources and run activities since 2015.
Australia’s efforts to eliminate trachoma are currently at an important stage. The national rate of trachoma has halved since 2011 and is 3.3% in 5 – 9 year old children in 2021 (NTSRU 2021). Recently released data for 2022 shows a further reduction in national prevalence to 2.2%. A dossier seeking certification for the elimination of trachoma in Australia by WHO is expected in the next two to three years.
Aim of the Evaluation
The aim of this evaluation is to establish recommendations for future health promotion approaches in trachoma beyond June 2024 including improvements to current approaches with consideration for efficiency, effectiveness and appropriateness. The evaluation will also consider the impact of health promotion activities in 2021-2023 on facial cleanliness in selected communities.
Method
A desktop review of data and activities in trachoma hotspot communities will be undertaken to explore facial cleanliness in hotspot communities over the calendar years 2020, 2021, 2022. Minum Barreng will undertake a critical internal report and review of activities in the same communities and time frame.
A Delphi method survey will be undertaken to gain consensus advice on effective trachoma health promotion approaches used in trachoma elimination health promotion 2010 to 2022 and advice on recommended future health trachoma elimination health promotion activities for 2024-2027. The Delphi expert consensus method allows for the gathering of practice-based evidence from experts and collaboration amongst key stakeholders to determine best practice (Jorm 2015).
Minum Barreng will establish a small evaluation steering group to guide the evaluation and a small evaluation operations team to manage the evaluation. The steering group will include members providing evaluation expertise and those who are external to Minum Barreng. External consultants may also be engaged to provide support and additional input to the project.
An Aboriginal and Torres Strait Islander reference group will also be established to inform the evaluation and to ensure that the knowledge, systems and frameworks of Aboriginal and Torres Strait Islander people working in the trachoma elimination eco-system are privileged within the evaluation design, implementation and outcomes.
The evaluation steering, operations and Aboriginal and Torres Strait Islander reference groups will be involved in all stages of the evaluation including planning, questionnaire development, ethics (likely needing University of Melbourne and NT Human Research Ethics approvals), Delphi panel recruitment and formation, Delphi consensus survey rounds, data analysis, results and write up.
A list of 50/60 experts working throughout the system will be invited to participate in the survey. This will allow for input from a wide range of stakeholders involved in different parts of the system. It will also allow analysis of First Nations voice and feedback. Feedback will be provided to the panellists after each round.
We anticipate three rounds of Delphi surveying
Round one: to explore the effectiveness of previous health promotion activities and nominate and identify those activities that are considered effective in future trachoma health promotion
Round two: to identify the priority activities for future trachoma health promotion
Round three (possibly by workshop): to finalise the priority future trachoma health promotion activities.
A written report will be provided to the DoHAC at completion of the evaluation and an interim report of the evaluation will be prepared for DoHAC by 30 March 2024.
Aboriginal and Torres Strait Islander Reference Group
The Aboriginal and Torres Strait Islander Reference Group provides valuable guidance to the evaluation project and we acknowledge and appreciate their knowledge, experience and skills. The group was established to ensure that the knowledge, systems and frameworks of Aboriginal and Torres Strait Islander people working in the trachoma elimination eco-system are privileged within the evaluation design and implementation. Members provide a wide range of experience, including community engagement, health promotion, program management, policy development, evaluation, communications and advocacy roles.
Current members include:
Nick Wilson (chair), Lesley Martin, Walter Bathern, Tammy Abbott, Tania McLeod, Angee Ross and Nama Jalu Consulting.
External consultants – Nama Jalu
Nama Jalu Consulting have been engaged by the Minum Barreng: Indigenous Eye Health Unit to support the evaluation of the Trachoma Health Promotion Project.
Nama Jalu is a majority Aboriginal owned and led Aboriginal Social Enterprise consultancy. The Nama Jalu team have many years of experience working with Aboriginal communities, organisations and government agencies on projects, programs and services for community across the government, community and private sectors. Nama Jalu’s work is focused on achieving Aboriginal community outcomes and directed to achieve sustainable and real change for Aboriginal people and communities.
More information about Nama Jalu Consulting can be found here.

Delphi Survey Questions and 19 Initiatives
The Delphi method used two rounds of surveys to rate effectiveness and rank priorities. There was also a workshop in person and online to discuss the findings.
As there have been a range of health promotion initiatives conducted as part of the Trachoma Health promotion Project since 2009, it was decided to group the many resources and activities into groups that could be presented to experts for assessment and comparison through the Delphi surveys. Minum Barreng developed initial categories which were then confirmed after discussion and revision with the Nama Jalu team. Below are the so called 19 initiatives included in the surveys.
Survey #1 included two activities – rate how effective the 19 initiatives were in raising awareness about facial cleanliness on a scale. Effectiveness was explained as success in producing a desired result, in this case raising awareness about facial cleanliness in the elimination of trachoma.
The second part of survey #1 was to rate how high of a priority they think the 19 initiatives should be for trachoma health promotion in the next three years.
Survey#2 provided an opportunity to re-rate the priority of any non-consensus initiatives.
Printed 'Clean Faces, Strong Eyes' trachoma resources

These refer to the posters, cards, and other printouts.
For example, The SAFE Strategy to Eliminate Trachoma card, Milpa Wash Your Face poster, The Trachoma Story Kit and Correcting 10 Myths about Eliminating Trachoma document.
'Milpa's Six Steps to Stop Germs' trachoma resources (broader health and hygiene messages and actions).

These refer to the posters, cards, stickers, and colouring in booklet, that detailed Milpa’s Six Steps to Stop Germs.
Promotional materials, clothing and merchandise

These refer to the physical items to give away for people to keep as reminders.
For example, t-shirts, stickers, temporary tattoos and wristbands.
Branding (Milpa logo, Trachoma branding, Key messages)

These include the use of 'Milpa Trachoma Goanna' logo, 'Clean Faces, Strong Eyes' and 'Trachoma' branding on printed resources and vehicles/water trailers.
Music Videos

Music videos that were made with original songs to convey messages around trachoma promotion, such as Through My Eyes (Docker River, NT), Kulilla Kulilla Pay Attention (Warburton, WA) and Fregon APY Lands Trachoma Elimination Song.
TV and Radio Community Service Announcements (CSAs)

Different CSAs produced to convey messages around trachoma health promotion, such as Milpa's Six Steps to Stop Germs with Drum Attweme and Melbourne Football Club (Deakyn Smith & Kozzy Pickett). The CSAs have been broadcast on several channels over the years including Imparja and Go TV, ICTV, CAAMA Radio, NG Media, PY Media and in many languages.
Melbourne Football Club (MFC) support for the elimination of trachoma

This refers to the players and staff from the Melbourne Football Club (MFC) who were involved in promoting the key messages of the campaign and delivering football clinics, for example 2022 IEHU and MFC trip.
School-based Educational Resources

These refer to all the different teacher books, student workbooks, lesson plans, and other interactive educational material as part of the promotion. Including poster competitions, classroom presentations and support from the Department of Education and schools and early learning centres. The teacher books and lesson plans have been developed in partnership with the departments of education in the NT and WA and include links to the relevant curriculum for various ages including early years and primary.
SAFE Bathroom Resources

These refer to the various resources developed around the SAFE Bathroom campaign, including the printed checklist, animated video. These resources were developed with input from NT, WA and SA to encourage discussion about reporting any maintenance issues in houses so they can be fixed.
Resource Adaptations

These refer to adaptations of resources external organisations have made using the branding and logos. Some examples include 'trachoma rates in communities', 'social media posts', 'drink bottles', 'bags' and 'soft footballs'.
Milpa the Trachoma Goanna Character Costume

Milpa the Trachoma Goanna life size costume appearing at events, schools and community visits.
Online Training

Remote Area Health Corps modules are also available on the AHCWA portal, and the CERA grading module is available on the Mium Barreng website.
Online Marketing and Communications

This refers to the online activities and platforms set up for the promotion, such as the website, e-Bulletin / newsletter and social media pages (Facebook and Instagram).
Community Engagement

This refers to all face-to-face community activities such as yarning with community members about trachoma, trachoma presentations and activities in early learning and primary school settings, and running stalls at community events.
Hygiene Bags

This refers to bags with different hygiene products, given out at events and community days. Hygiene bags can include items such as soap and towels, as well as other trachoma resources.
Community Art Murals

These refer to the painted murals that feature community stories and trachoma messaging, painted by a commissioned artist and local artists from each community. These community murals then act like a billboard. See some of the stories behind the art mural projects here Art Mural and Hip Hop Project, Willowra Art Mural Project, Areyonga Art Mural Project, Utopia Homelands Art Mural Project.
Partnerships and Collaborations (connecting the sector)

Minum Barreng could not undertake our work in health promotion and community engagement without partnerships and collaborations. These occur regularly out in community and in meetings and projects. Some examples include bringing groups together to discuss sector wide issues, for example facilitating the Central Australian Environmental Health Working Group, facilitating the NT Health and Education group, or healthy homes workshops with partners such as Katherine West Health Board (KWHB) and Environmental Health Project WA and healthy housing officer roles such as with Central Australian Aboriginal Congress and partners Fred Hollows Foundation and Nourish Our Communities Ltd.
Augmented Reality App

This refers to the 3D Augmented Reality App, developed to make learning about Milpa’s Six Steps to Stop Germs more interactive and fun for kids and adults.
Clinical Resources

This refers to clinical resources such as the WHO Trachoma Grading Card, Trachoma Resource Book and clinical information posters and cards developed to assist clinical staff in the diagnosis and treatment of trachoma.
Evaluation Findings Summary
Minum Barreng at the University of Melbourne has been conducting trachoma elimination health promotion activities since 2009 and has been funded by the Australian Government Department of Health and Aged Care (DoHAC) to provide health promotion resources and run activities since 2015. An evaluation of this work was included in the DoHAC funding agreement and workplan for 2023–2024. The aim of the evaluation was to establish recommendations for future trachoma elimination health promotion approaches with consideration for efficiency, effectiveness and appropriateness.
A Delphi method survey was undertaken to establish consensus on the effective trachoma health promotion activities undertaken from 2010 to 2022 and recommendations on future trachoma health promotion activities for 2024-2027. Nama Jalu Consulting were engaged to support the evaluation. Nama Jalu is a majority Aboriginal owned and led social enterprise consultancy.
An Aboriginal and Torres Strait Islander reference group was established to guide and inform the evaluation and to ensure that the knowledge, systems and frameworks of Aboriginal and Torres Strait Islander people working in the trachoma elimination eco-system were privileged within the evaluation design, implementation and outcomes. Additional external evaluation expertise was obtained through the project.
Fifty experts working throughout the trachoma elimination system were invited to participate in the two online Delphi surveys. Minum Barreng grouped the previously used trachoma elimination health promotion activities into 19 initiatives. The experts were asked to rate the 19 initiatives using a five-point scale for their effectiveness and also to nominate the priority initiatives for the next three years. A workshop with the trachoma experts was held in Alice Springs to discuss the survey results and allow for further discussion of the findings. A report was prepared and circulated late in 2024.
Some key statistics from the evaluation report (attached) include high response rates - 78% from the first survey and 70% from the second survey. 20% of experts in the first survey and 28% of experts in second survey identified as Aboriginal and/or Torres Strait Islander. One third of the participants at the workshop were Aboriginal and/or Torres Strait Islander.
The five initiatives that rated as highly or very highly effective by the expert panel were:
- Milpa the trachoma goanna character costume
- Branding
- Community Engagement
- Partnerships and collaboration
- Milpa’s Six Steps to Stop germs trachoma resources
Most of the 19 initiatives (14/19) rated 60% and above for effectiveness. The initiatives involving community engagement via team members, in person, working in communities and on the ground with organisations and rated the highest for both effectiveness and future priority. Milpa and the Clean Faces, Strong Eyes branding and messaging were highly recognised and considered effective. There were notable geographic/jurisdictional differences reflecting that some of the activities had occurred in the Northern Territory, for instance, and not in other states. In the key learnings section of the report – the experts considered it critical that trachoma elimination health promotion activities continue – there was strong consensus amongst the panellists on this point. Although the surveys enabled some ranking of activities, discussion at the workshop and within the Aboriginal Reference Group suggested that all of the initiatives were considered valuable. One of the greatest strengths of the trachoma health promotion was the community engagement activities involving local Aboriginal people. The evaluation report includes all the findings from the surveys, workshop discussions and some key learning and suggestions.