Systematic review on reducing missed opportunities for vaccinations in Latin America

ABSTRACT Objectives. To estimate the prevalence of missed opportunities for vaccination (MOV) in Latin America and the effect of interventions targeting health systems, health workers, patients, and communities on MOV. Methods. Searches were conducted in MEDLINE, EMBASE, CINAHL, and LILACS electronic databases and relevant organizations were contacted, including the Pan American Health Organization (PAHO), to identify studies meeting eligibility criteria. A pair of reviewers identified 27 randomized and non-randomized studies quantifying the effectiveness of any intervention for reducing MOV and 5 studies assessing the rate of MOV in Latin America. Results are reported narratively when criteria to pool results were not met, and the certainty of this evidence was assessed using the GRADE approach. Results. Evidence suggests the rate of MOV in Latin America ranged from 5% to 37% with a pooled estimate of 17% (95% CI [9, 32]) (low certainty) and that monetary incentives to healthcare teams, training for healthcare teams on how to communicate with patients, and educational interventions for caregivers probably reduce MOV (moderate to very low certainty). Conclusions. There is insufficient evidence supporting the implementation of any intervention as policy based only on the potential reduction of MOV without considering several factors, including costs, feasibility, acceptability, and equity.

(2015), and measles (2016) through immunization (5,6). Immunization can also contribute to considerable cost-savings for health systems (4). Despite this, many children and adults are still affected by vaccine-preventable diseases. Estimates from the World Health Organization (WHO) suggest that approximately 6.9 million children died from vaccine-preventable diseases in 2011, while approximately 22.6 million infants were not provided essential vaccines (2).
Immunization schedules and vaccine coverage vary nationally and internationally. For example, coverage in Latin America and the Caribbean has considerably decreased since 2013, less than half of countries in the Region reached the WHO 90% coverage target in 2017, and coverage continues to vary dramatically between countries in the Region (3). Due to the COVID-19 pandemic, recent declines in vaccine coverage have the potential to erode historic and hard-fought gains in the Region. A comparison of the number of children receiving diphtheria-tetanus-pertussis (DTP3) and measles, mumps, and rubella (MMR) vaccinations from January to June of 2019 and 2020, respectively, indicates that vaccination coverage decreased by 33% for DPT3 and by 24% for MMR in the Region (7). A number of challenges impede vaccination coverage in the Region, including hard-to-reach communities, political instability, unfavorable socioeconomic factors, religious/cultural beliefs, vaccine hesitancy, limited resources, poor management of health systems, and inadequate monitoring and supervision (2,(8)(9)(10).
WHO recommends that immunization should be offered to all eligible individuals at every contact point with health services (3,11). When it is not offered, this is referred to as a "missed opportunity for vaccination" (MOV). An earlier review identified four main reasons for missed opportunities: 1) failure to simultaneously administer all vaccines for an eligible child; 2) false contraindications to immunization; 3) health worker practices; and 4) logistical problems (11). Though patients may seek immunization services, healthcare systems may not accommodate these requests due to lack of an appointment, immunization not being the primary reason for the visit, and despite there being no health-related contraindications. Additionally, patients may seek care in health facilities where immunization is simply not offered (12).
Research has identified several potential approaches (e.g., one-on-one coaching with caregivers) to reduce MOVs (13). Interventions aimed at reducing MOVs are likely to increase vaccination coverage and therefore magnify all beneficial downstream consequences associated with immunization.
Preexisting reviews have identified the main reasons for low vaccine coverage and MOVs (11,14,15). Additionally, an earlier systematic review included evidence from randomized controlled trials (RCTs) to measure the effect of interventions on MOVs (13). Although RCTs are the ideal study design for comparing effects of interventions, utilizing evidence from non-randomized studies can provide a comprehensive summary of all evidence on benefits and harms (16). Additionally, previously published reviews have not adequately assessed the certainty of, or confidence in, included evidence.
Regional decisionmakers have called for a comparison of interventions for reducing MOVs (Objective 1) and a summary of evidence (published by national and international organizations) on the rate, or prevalence, of MOVs in Latin America (Objective 2) to comprehensively inform regional decisionmakers on the magnitude of MOVs and potential strategies for reducing these.

MATERIALS AND METHODS
This report is primarily an update of another review-an internal, unpublished systematic review for the Pan American Health Organization (PAHO)-developed in response to a call for evidence on interventions to reduce MOVs. In this update, we included randomized and non-randomized studies on interventions for reducing MOVs (Objective 1). For this current review, we also summarized results from studies measuring the rate of MOVs in Latin America (Objective 2). Reporting of this systematic review was informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement and conducted according to a protocol approved by PAHO (available upon request) (17).

Study eligibility
Interventions for reducing MOVs (Objective 1). We included studies on interventions for reducing MOVs for immunizations recommended by WHO with a particular focus on vaccination of the following populations: children under 5 years of age, women of childbearing age, and older adults. We did not include immunizations for human papillomavirus (HPV) due to inconsistent inclusion of HPV vaccination in immunization schedules within the Region (18).
We included RCTs, quasi-randomized controlled trials (CCTs), controlled before-and-after studies (CBAs), and interrupted time series (ITS) quantifying the effect of any intervention (of any duration and in any setting) used for reducing MOVs. We included studies in any language published after 1990. We categorized the interventions as follows: • Interventions targeting the health system including changes in organization of care within a particular healthcare setting; • Interventions targeting health workers including hospital administrators, clinicians, nurses, and any other individuals involved in the immunization process; • Interventions targeting patients or community members including children, adults, caregivers or parents, women of childbearing age, and older adults; • Mixed interventions targeting one or more of the above categories.
We included studies that compared interventions to another intervention or no intervention, or where the same intervention was monitored over time.

Outcomes of interest.
We summarized the following outcomes from included studies: MOVs defined as any healthcare visit that did not result in participants of any age receiving the needed vaccine (11). MOVs were further classified as: a. Attributable to the healthcare system; b. Attributable to the knowledge, attitudes, and/or decisions of individual patients, caregivers, etc.; c. Attributable to the knowledge and/or attitudes of healthcare personnel.
a meta-analysis), we reported results narratively. We also accounted for differences in study design and synthesized data separately for RCTs and non-randomized studies. For each comparison, we reported any pooled estimates at an outcome level as risk ratios (RR) or odds ratios (OR) in Grading of Recommendations Assessment, Development and Evaluation (GRADE) Summary of Findings Tables (21). We used the event rate of the control groups to estimate the baseline risk of associated outcomes and obtain absolute estimates of effect.
Assessment of certainty of the evidence. The certainty of the evidence can be rated as high, moderate, low, or very low and is influenced by study design and other domains that can decrease (issues of risk of bias, inconsistency, imprecision, indirectness, and publication bias) or increase the certainty (large magnitude of effect, dose-response gradient, and residual confounding acting opposite to what was observed). We assessed the certainty of the evidence at an outcome level using the using the GRADE approach (22).
Handling of missing data and other analyses. We planned to contact study authors to obtain missing outcome data if such missing data were judged to affect the certainty of the evidence. We did not conduct any subgroup or sensitivity analyses.

Data collection and analysis (Objective 2)
Data abstraction and management. We abstracted data from selected studies using a pretested data abstraction form. We collected information on the country, target population, data collection period, number of included participants and healthcare centers, number of participants eligible for vaccination, number of MOVs, and reasons for MOVs. For studies including child populations, we collected information about caregivers and any other person(s) responsible for making vaccinationrelated decisions.
Data analysis. When criteria for pooling were not met, we summarized data narratively. When studies included/used similar populations and methodology, we calculated an overall rate of MOVs by pooling the proportion of MOVs across studies using a generalized linear mixed model, implemented using the metaprop package in the software R (23,24). We assessed the certainty of the evidence using the same methodology detailed above for Objective 1.

How to interpret the results
The GRADE certainty of the evidence reflects how certain we are that the true effect lies within a particular range of results. For example, low certainty evidence implies that the results may be importantly different to what was observed (21). In addition to the certainty of the evidence, interpreting results requires, for example, making a judgment of the importance of the magnitude of reduction in MOVs observed. For example, an absolute risk reduction of 3% of MOVs when providing training to the healthcare team on how to communicate with patients (compared with standard of care) may be too small to be considered important and worth the efforts necessary to Rate of MOVs in Latin America (Objective 2). We included any report or study on the rate, or prevalence, of MOVs in Latin America that followed PAHO methodology (19).

Search strategy
Interventions for reducing MOVs (Objective 1). We searched the MEDLINE, EMBASE, CINAHL, and LILACS electronic databases. The original systematic review conducted a search between 1990 and April 2014; we updated the original search strategy, with minor modifications, and ran the search from April 2014 to November 2019. A combination of MeSH and keyword terms on the topics of immunization and missed opportunities for vaccination were used (full search strategy available upon request).

Rate of MOVs in Latin America (Objective 2).
We included all studies developed by the Comprehensive Family Immunization Unit of the Family, Health Promotion and Life Course Department of PAHO, which were retrieved by one author (MV), an advisor in the department. Reference lists. We reviewed reference lists of relevant citations to retrieve any additional study meeting our eligibility criteria for Objectives 1 and 2.

Data collection and analysis (Objective 1)
Selection of studies. We used a pretested form to select studies meeting eligibility criteria. After removing duplicate results, we screened titles/abstracts and full-text articles (authors MT and RB) in succession. We consulted a third author (ACL) to resolve disagreements.
Data abstraction and management. Data abstraction was conducted independently and in duplicate by authors (MT, RB, ACL) using a pretested form. We abstracted study characteristics (author, country, journal of publication, year of publication) in addition to data on included populations, interventions, comparisons, and outcomes of interest. We contacted corresponding authors for clarification when needed.
Risk of bias assessment. We (MT, RB) independently and in duplicate assessed risk of bias for which we used the Cochrane Risk of Bias tool for randomized clinical trials, and suggested risk of bias criteria for non-randomized studies by the Cochrane Effective Practice and Organization of Care group (20).

Data synthesis.
When criteria for pooling results were met (the population, intervention, and comparator were judged to be similar enough across studies), we conducted meta-analysis in Review Manager 5.3 to calculate summary estimates. We used a frequentist approach and random-effects models to account for potential, unexplained heterogeneity between studies. We used the participants or clinic visits as the unit of analysis. When it was not possible to conduct meta-analysis (e.g., important clinical heterogeneity precluded us from conducting 36,37,40,43,47,49,50), 8 in a hospital setting (27,30,38,39,42,(44)(45)(46), 1 in a private clinic (32), and the 3 in a community setting (35,41,51). One study was based in both a primary care setting and hospital setting (48).

Results (Objective 1)
The original electronic search retrieved 469 citations, and the contacted organizations suggested 30 papers. We screened all associated titles and abstracts, and 49 articles in full text. The updated search identified 350 new citations, of which 42 were ultimately full-text screened. A total of 27 studies from both searches met eligibility criteria, 6 of which are from the updated search . The results of the search and screening process are detailed in a PRISMA flow diagram (Figure 1).
We summarized the effect of interventions for reducing MOVs and the certainty of this evidence in Table 1. Most studies compared interventions to standard of care (or to no intervention). by clinicians (32), quality improvement programs (25), and pharmacist presence (50).
There were significant variations in how the primary outcome of interest, reduction in MOVs, was measured. Some studies reported MOVs per patient per year (48), while others reported MOVs for specific vaccines (41), the number of visits at which MOVs occurred (45), mean number of MOVs per infant (47), captured opportunities (i.e., opposite of MOV) (31), and/ or opportunistic immunizations (i.e., opportunities that would have been missed) (46).

Risk of bias assessments
Major issues of risk of bias were observed in 15 RCTs (risk of bias assessments available upon request); these included issues

Results of studies developed by PAHO (Objective 2)
We included five studies conducted between 2012 and 2018 in Honduras, Ecuador, Peru, Panama, and Bogotá (Colombia) (52)(53)(54)(55)(56). Studies included 368 to 2 495 participants. Three of the studies focused on reducing MOVs in children under 5 years of age (52,53,55), one on children under 3 (54), and another on children under 2 years (56). All studies used a cross-sectional design, and one added a qualitative design component (52). Table 2 summarizes the main characteristics of these studies.
Studies reported a total of 579 MOVs for 2 030 children eligible for vaccination (a rate of 5%-37% MOVs per country) ( Table 2). The pooled rate of MOVs, estimated from data from five countries in the Region, was 17% (95% CI [9,32]). Statistical tests suggest important heterogeneity among included studies (I 2 97%). All studies reported that parents/caregivers, primarily women, made vaccination-related decisions for their children, and MOVs did not seem to be associated with the caregiver's level of education (32%-72% of caregivers completed high school) ( Table 2). The reasons for MOVs varied across countries, and the largest proportion of MOVs were attributable to healthcare personnel knowledge, attitudes, and practices ( Figure 2).

Certainty of the evidence for Objectives 1 and 2
The certainty of the evidence on interventions for reducing MOVs ranged from very low to moderate certainty, due to serious issues of risk of bias and imprecision, and studies measuring the prevalence of MOVs in Latin America were of low certainty. The main limitation of this second body of evidence was indirectness (applicability concerns) and imprecision, with the extremes of the confidence interval of the pooled estimate suggesting a small prevalence of MOVs at one extreme and a large prevalence of MOVs at the other.

DISCUSSION
This review summarizes the proportion of MOVs in the Region and the best available evidence on interventions (targeting healthcare teams, patients or caregivers, and healthcare systems) for reducing MOVs. Results from studies from the Region suggest that 17% of recommended vaccines are missed (low certainty evidence). Results also suggest that monetary The study characteristics table (available upon request) details all comparisons.

Interventions targeting healthcare team
Eight studies identified interventions targeting the healthcare team. There was moderate certainty evidence that monetary incentives (7.5% reduction in MOVs) (29) (27,44,46). One study on web-based training (49) and another using provision of expert recommendations (32) did not report reductions in MOVs.  (41). The certainty of evidence was very low for reminders and alerts, either mailed or by telephone (26,35,36), and SMS (short message service) (40). One study assessed patient tracking and outreach for reducing MOVs, but no estimates could be abstracted (43).

Interventions targeting the healthcare system
Five studies identified interventions targeting healthcare systems. There was low certainty evidence suggesting that presence of a pharmacist (OR 0.56; 95% CI [0.35, 0.90]) (50), offering free vaccination programs (RR 1.00; 95% CI [0.95, 1.05]) (29), and exempting patients from caregiver authorization (researchers reported no statistical differences) may reduce MOVs (48). There was very low certainty evidence suggesting that quality improvement programming may result in little to no reduction of MOVs (no reduction reported) (25). One study measuring outcomes of improving accessibility to vaccination sites did not report reduction of MOVs (38).

Implications for research
We suggest the development of higher-quality randomized and non-randomized studies on interventions for reducing MOVs, especially for adult and special populations that require vaccination (e.g., women of childbearing age and seniors). We also suggest that this research be conducted in low-or middleincome countries (or those with lower vaccination coverage) and in the context of issues associated with access to care, vaccine hesitancy, and the COVID-19 pandemic. We encourage testing of cross-cutting health system changes, interventions targeting vaccine hesitancy, and community-level interventions. Furthermore, we encourage reporting of outcomes such as costs, vaccine wastage rate, satisfaction with care, occurrence of vaccine-preventable diseases, and healthcare personnel's attitudes. In addition, uniform definitions of MOVs should be adopted to promote comparability of studies.

Implications for practice and policy
The results of this review suggest an important rate of MOVs in the Region. This is in line with evidence on decreased vaccine coverage, weakening of epidemiological surveillance, emerging outbreaks of vaccine-preventable diseases, inadequate long-term sustainable financing, the challenge of scientific misinformation, and increasing vaccine hesitancy. Every effort must be made to consider interventions for reducing MOVs while maintaining the trust of populations in the Region in vaccination services, establish and follow national guidelines for immunization in the context of COVID-19 transmission, prioritize newborn vaccinations, and maintain periodic and systematic registries of the population pending vaccination (7,57). Now more than ever, ensuring that every child who accesses a health center has their vaccination status reviewed and their vaccination schedule completed will be an opportunity to improve vaccination coverage and reduce incidence of vaccine-preventable diseases (1).

Strengths and limitations of this review
The strengths of this review include not limiting evidence to studies in English, assessment of gray literature, and conducting screening and data abstraction independently and in duplicate. This review is limited by reporting issues in and methodological quality of the included evidence, which largely prevented us from conducting meta-analyses or from presenting the results in a consistent manner across interventions/ outcomes. Furthermore, a paucity (and poor reporting) of study data resulted in our analysis not comprehensively accounting for instances where unit of randomization differed from unit of analysis. This may have resulted in confidence intervals that are more precise than they should be. Because the certainty of many of our estimates was rated down for imprecision, we believe that unit of analysis issues did not have an important impact on our conclusions.

Conclusion
MOVs can increase the incidence of vaccine-preventable diseases in the Region. Clinicians and other decisionmakers within the Region should consider this review's limitations, paucity of incentives to healthcare teams, training healthcare teams on patient communication, and educational interventions for caregivers probably reduce MOVs (moderate to very low certainty across interventions and outcomes).

Overall completeness and applicability of evidence
For our first research objective, reports meeting eligibility criteria studied interventions for reducing MOVs for children and were conducted in Australia and the United States of America-countries with high vaccine coverage. This may limit the applicability of the findings to adults or settings in low-and middle-income countries, including several countries within the Region. Although there was a variety of included interventions, which readers might interpret as increased applicability, each of the interventions seemed to have its own unique effect. Because of this, the effect of a specific intervention should not be generalized to other interventions aimed at the same recipient. Cross-sectional studies on the prevalence of MOVs in Latin America were conducted in five countries. The heterogeneity observed among these studies decreases the applicability of the findings to settings outside of the Region.

Agreement and disagreement with other studies or reviews
In a recently published systematic review, authors addressed our first research objective (13). In comparison, we did not limit eligibility criteria to RCTs (resulting in 26 additional studies added to this review), performed more comprehensive searches, and evaluated certainty of the evidence using the GRADE approach; this allowed us to more accurately present limitations that may influence results and conclusions. Finally, authors from the previous review suggested that patient education/tracking, outreach sessions, and provider prompts reduced MOVs. Our review suggests that, in addition to the interventions suggested by the previous review, monetary incentives for healthcare teams probably reduce MOVs.  Revisión sistemática sobre la reducción de las oportunidades perdidas de vacunación en América Latina RESUMEN Objetivos. Estimar la prevalencia de las oportunidades perdidas de vacunación en América Latina y el efecto de las intervenciones dirigidas a los sistemas de salud, los trabajadores de salud, los pacientes y las comunidades.