Accurate diagnosis, appropriate treatment and referral, timely reporting and tracking of malaria cases are the cornerstones of malaria service delivery. As the flagship global malaria service delivery project of the U.S. President’s Malaria Initiative (PMI), PMI Impact Malaria supports National Malaria Programs (NMPs) in their efforts against malaria. One way we do this is by supporting countries to improve the quality of health services for those most at risk, especially children and pregnant women. In 12 partner countries, PMI Impact Malaria supports the implementation of a quality improvement approach called Outreach, Training, and Supportive Supervision Plus (OTSS+).
Outreach, Training and Supportive Supervision (OTSS) was first developed in 2007 under PMI’s Improving Diagnostics project (2007 – 2011) and carried through to the PMI MalariaCare project (2012 – 2017), and PMI Impact Malaria from 2018 to the present. This facility-level approach is intended to improve health facility readiness and providers’ skills and knowledge on the management of uncomplicated and severe malaria, prevention and treatment of malaria in pregnancy, and malaria diagnosis leading to increased adherence to national guidelines and improved patient outcomes. Central to its success is that supportive supervision and problem-solving take place within the health facility.
When conducting OTSS+ visits, trained supervisors like Sam Enow Agbor in Cameroon observe providers in the outpatient and antenatal care clinics, review records, and assess health facility readiness using a standardized set of competency-based checklists that have been adapted to fit the needs of the specific country and NMP guidance. These checklists are uploaded into electronic tools facilitating the collection of data and have built-in analytic capacity that provides real-time scoring during the OTSS+ visit. When these scores identify areas of weakness, OTSS supervisors use targeted on-the-job training and coaching to address areas of poor performance, troubleshoot gaps in service readiness, and work with facility staff to develop action plans for service quality improvement. Through PMI Impact Malaria, countries have supplemented OTSS+ with other targeted quality improvement approaches, including mentorship, peer-to-peer learning, and targeted classroom trainings that target low-performing providers and facilities identified through OTSS+.
(Source: Quality Improvement Technical Brief, PMI Impact Malaria, pg. 3)
After 15 years of implementation and refinement under three PMI-sponsored projects, including an expansion of services under PMI Impact Malaria, an independent evaluation was commissioned by PMI and conducted by Tropical Health LLP. This evaluation considered the effectiveness of the OTSS approach, lessons learned and best practices, and identified areas for improvement and the potential streamlining of this approach.
Read the full independent evaluation here.
Watch our webinar co-hosted with the RBM Partnership to End Malaria Case Management Working Group and Tropical Health LLP sharing the results (English / French).
In their report the evaluators stated, “Overall, the evaluation found that OTSS improves malaria quality of care through improving health workers performance and health facility readiness. This was consistent with the overwhelming positive perceptions from stakeholders about OTSS being relevant and effective in bringing about positive changes in malaria service delivery.”
“The success of OTSS is attributed by stakeholders to three main factors: sense of ownership at country level, the delivery approach, and that OTSS creates a useful mutual accountability framework.”
Key findings from the report include:
-
Respondents stated that the whole OTSS package of quality improvement interventions was beneficial; the relevance of the framework and delivery of the package of components, combined with generation of real-time data and the shared accountability were the main perceived success factors.
-
The approach was perceived to stand out from other supervisory approaches by the effective use of ‘soft skills’ (communication, coaching, and problem-solving approach). The digitization of the checklists and the development of action plans at health facility and district levels that are followed-up with post supervisory visits also distinguished OTSS from other supervision approaches.
-
OTSS supervisors and supervising health authorities overwhelmingly prefer the digital tool over paper-based systems, including the availability of facility scoring during the OTSS visit, allowing targeted on-the-job training, coaching, troubleshooting, and data-driven action planning.
-
OTSS has been adopted by the national malaria programs in all targeted countries. Country stakeholders share a sense of ownership over OTSS from the significant contributions they have made to the tool’s revisions based on their experience and needs. However, with some exceptions, OTSS is still perceived by national stakeholders as an ‘add-on’ donor-driven and resourced process. PMI Impact Malaria has begun transitioning data hosting and management to national programs to increase their sense of ownership and to sustain the program beyond the project lifespan.
Evaluation Recommendations
The report further states that evaluation respondents did not feel it was necessary to streamline the OTSS approach in terms of malaria services covered, the package of quality improvement activities, or capacity strengthening approaches used. Rather, operational and strategic considerations need to be reflected upon in each country’s context to make OTSS implementation more efficient and effective:
- Operational considerations:
- Refine the checklists and explore the potential of digitization to further customize checklists to each context,
-
Accompany each country to find the best strategy for OTSS visit targeting and implementation approaches,
-
Supervise the supervisor and develop/expand and maintain a pool of skilled supervisors, and
-
Improve the approach to the follow-up of action plans and their monitoring as a potential game changer in closing the quality improvement loop.
- Strategic considerations:
- Articulate overarching and country specific OTSS theories of change,
-
-
Make better use of the triangulation of data derived from the OTSS process to drive systemic changes,
-
Promote inclusion of OTSS in country strategic plans, guidelines and budgets,
-
Support countries in finding the right balance with regards to integration with other diseases or health service levels, based on their respective contexts, and
-
In the next iteration of PMI support, focus on institutionalization of OTSS in terms of data management and increased accountability and capabilities to manage at the lower levels of the system.
We are pleased that this evaluation was able to independently validate the effectiveness and acceptance of the OTSS quality improvement approach and hope these results will be used to inform decision-making by ministries of health, funders, and implementers that are planning to introduce or, where already implemented, consolidate and expand the use of the OTSS approach to achieve universal access to high-quality malaria service delivery.
Based on insights from this evaluation, we will continue to support NMPs to adapt OTSS+ to their distinct contexts and consistently improve the quality of their malaria services to achieve their strategic goals.
I would like to thank Tropical Health LLP for their independent evaluation of the OTSS quality improvement approach and PMI for their continued support for implementation of this approach through the PMI Impact Malaria project. We also would like to thank the many health workers, supervisors, NMPs, and PMI Impact Malaria project staff who work every day to improve the quality of malaria services in their countries, with the vision of A World Without Malaria.
Technical Director, PMI Impact Malaria
PMI Impact Malaria is funded and technically assisted by the U.S. President's Malaria Initiative (PMI) and is led by Population Services International (PSI) in partnership with Jhpiego, MCD Global Health, and the Malaria Elimination Initiative (MEI) at the University of California, San Francisco.