Evaluation of Impact

Sponsors should monitor and evaluate the impact of CE to demonstrate its value to research.

There is limited evidence regarding the impact of CE, as it is a relatively recent part of TB clinical research. This makes it difficult to convince donors and other stakeholders that meaningful CE is worth the time and investment. In addition, there is no widely accepted best practice on how to assess the desired impact of CE activities, which is a prerequisite for any impact evaluation.

The Sponsor began collecting STREAM CE monitoring data in 2016 and sharing a monitoring dashboard with CABs in 2018. The dashboard tracked activities by site and over time, as well as highlighting CE successes and challenges. In 2019 and 2020, a theory of change – which identified and incorporated the desired impact of STREAM CE – was developed in a participatory process involving all STREAM CABs. A logic model provided a useful framework for identifying evaluation questions; however, it would have been more useful to develop the CE logic model at the start of the trial to guide monitoring and evaluation throughout the trial.

Donor support for CE in STREAM was substantial, in part because we were able to provide information on its impact from regularly collected monitoring data.

STREAM Sponsor
Infographic summarising community engagement activities for the whole trial, including: 653 small meetings, 83 trial team meetings, 172 large meetings, 154 trainings, 423 outreach events, 139 policy meetings and 318 patient support activities.

Recommended best practices

Build in monitoring and evaluation of CE impact through the following measures:

  • Donors and sponsors should commit appropriate resources to monitoring and evaluating CE

  • Early in a trial, a logic model should be developed through a participatory process involving CAB members

  • The sponsor should develop and implement a monitoring and evaluation plan for CE, based on the agreed logic model

  • Where possible, the sponsor should evaluate the impact of CE using an experimental or quasi experimental design (and, where useful, qualitative methods)