Evaluating Quality of Contraceptive Counseling: An Analysis of the Method Information Index

Abstract

The Method Information Index (MII) is calculated from contraceptive users’ responses to questions regarding counseling content—whether they were informed about methods other than the one they received, told about method‐specific side effects, and advised what to do if they experienced side effects. The MII is increasingly reported in national surveys and used to track program performance, but little is known about its properties. Using additional questions, we assessed the consistency between responses and the method received in a prospective, multicountry study. We employed two definitions of consistency: (1) presence of any concordant response, and (2) absence of discordant responses. Consistency was high when asking whether users were informed about other methods and what to do about side effects. Responses were least consistent when asking whether side effects were mentioned. Adjusting for inconsistency, scores were up to 50 percent and 30 percent lower in Pakistan and Uganda, respectively, compared to unadjusted MII scores. Additional questions facilitated better understanding of counseling quality.

The renewed focus on quality of care in the era of sustainable development goals (SDGs) has accelerated efforts to define and develop measures of service quality (Leisher et al. 2015; Kruk, Pate, and Mullan 2017). The Method Information Index (MII) is a relatively new entrant into the suite of family planning (FP) quality indicators. It can be thought of as one way to assess the “information given to clients” element of the well‐established quality of care framework put forth by Judith Bruce (1990). The purpose of making sure that the client receives complete information about her method is both to ensure informed choice as well as contraceptive continuation. It is also a way to assess that women's care is consistent with their reproductive rights, specifically the right to sexual and reproductive health (SRH) services, information, and education as described by Erdman and Cook (2008). Counseling women appropriately with instructions that they understand preserves their rights to information as well as reproductive self‐determination (Hardee et al. 2014). Assessments of counseling have traditionally been done either through direct observation, exit interview, or retrospective report by the FP user.

In the absence of direct observation of the client‐provider interaction, asking women about the information they received is used as a proxy indicator of the quality of the services provided (Bessinger and Bertrand 2001; Chin‐Quee, Janowitz, and Otterness 2007). When used in this way, the MII can be reported at the provider, clinic, or program level as an indicator of program performance. When used as a component of a population‐based survey, as done in the Demographic and Health Surveys (DHS) Women's Questionnaire or the Performance Monitoring and Accountability 2020 (PMA2020) surveys, the MII can be reported at a national or international level and allows for comparison across countries and over time, making it an extremely versatile indicator. The MII has also been adopted as one of the core indicators for the Family Planning 2020 (FP2020) initiative. It is the only indicator that addresses the concepts of counseling, informed choice, and process quality. The use of client perspectives has been emphasized as an important aspect of evaluating service quality (Donabedian 1988; Calnan 1998; Williams 1998). Although clients’ views are subjective, prone to measurement error, and may differ from providers’ perspective of quality (Petersen 1988), they are insightful for service providers and programmers to understand clients’ perception of service quality (Andaleeb 2001).

In nationally representative surveys, the MII is calculated from current contraceptive users’ responses to three input questions about the information providers gave when they received their current method: whether they were informed about other methods aside from their current method, told about possible side effects from their current method, and advised what to do if they experienced side effects (ICF International 2015). For each of these questions, responses are coded 1 if the respondent answered “yes” or 0 otherwise, and the reported index score is the percent of women who responded “yes” to all three questions. Comprised of only three binary questions, the MII is simple and relatively easy to collect. However, how well it reflects client understanding of counseling information has not yet been demonstrated. With its increased use, comparing the MII with other data of client understanding may help to build greater confidence in this indicator.

Both a 2016 United Nations Population Fund (UNFPA) study of informed choice in 24 countries and a 2016 comparison of two rounds of DHS datasets in 25 countries identified a great deal of variability in the MII and informed choice between countries and over time (Jain 2016; Loaiza, Liang, and Snow 2016). Jain (2016) noted that MII scores on average increased between two time points (34–39 percent), suggesting counseling quality had improved over time. Jain (2016) also described high variation in the proportion of women receiving all three pieces of information, both across countries and within a given country by method, household wealth, and respondent education. This suggests good quality counseling is not consistently delivered across settings and populations. Furthermore, little is known about the specific content of information exchanged between a provider and a client (Jain 2016). The MII alone does not demonstrate whether a client leaves her visit with a complete understanding of what to expect with her method. The successful transfer of counseling information may be important for a client's willingness to accept any side effects and her continued contraceptive use.

As part of a broader longitudinal study in Uganda and Pakistan to assess correlations between measures of FP service quality at the facility level, client‐reported MII at baseline exit interviews, and 12‐month all‐method contraceptive discontinuation, we performed an analysis of baseline client‐level data to assess the consistency between responses to additional questions about specific information exchanged and the FP method received.

METHODS

Study Setting

Pakistan

With 208 million inhabitants and a growth rate of 2.3 percent, Pakistan is the sixth‐most populous country in the world (Population Reference Bureau 2017). Large segments of the population reside in rural areas where there is little access to quality health services and skilled providers (UNDESA Population Division 2008; NIPS Pakistan and ICF International 2013). This has resulted in Pakistan having the second‐highest risk of maternal deaths in South Asia (NIPS Pakistan and ICF International 2013), and the highest newborn mortality rate in the world (UNICEF 2018).

Pakistan's total fertility rate (TFR) remains high at 3.8 with at least one in four births resulting from an unintended pregnancy (NIPS Pakistan and ICF International 2013). Use of modern contraception is low at 26 percent; female sterilization (8.7 percent) and condoms (8.8 percent) are the predominant modern methods, while long‐acting reversible contraceptives (LARCs) (<3 percent) are uncommon (NIPS Pakistan and ICF International 2013). Despite concerted efforts to increase access to modern contraception in the last decade, unmet need remains high at 20 percent (NIPS Pakistan and ICF International 2013). As a consequence, Pakistan was among the few middle‐income countries that failed by 2015 to achieve Millennium Development Goal (MDG) 5 that aimed to reduce the maternal mortality ratio by 75 percent and to achieve universal access to reproductive health (Countdown to 2015 2015). Information given to clients about benefits and potential side effects of modern methods of contraception may not be sufficient, potentially leading to low contraceptive uptake and discontinued use (Khan and Shaikh 2013; NIPS Pakistan and ICF International 2013). The most recent DHS survey indicates that two‐thirds of current contraceptive users report not being informed about potential side effects, whereas 70 percent said they were not informed about the wide range of contraceptive methods, and even more (72 percent) said they were not informed about what to do in case they experienced side effects (NIPS Pakistan and ICF International 2013).

Uganda

Uganda is an East African country with 35 million inhabitants and a total fertility rate of 5.4, resulting in the fifth fastest growing population worldwide (Hussain 2013; UBOS and ICF 2018). Unintended pregnancies make up close to half of all pregnancies in Uganda (Hussain 2013; UBOS and ICF 2018) and only 35 percent of married women of reproductive age and 47 percent of sexually active unmarried women use a modern method of contraception (UBOS and ICF 2018).

Knowledge of family planning is high with 99 percent of all women of reproductive age reporting having heard of at least one contraceptive method (UBOS and ICF 2018). Uganda's method mix is predominantly short‐term methods, with injectables accounting for about half of the method mix; 19 percent of women use LARCs, including the intrauterine device (IUD) and implant. Method discontinuation is an issue in Uganda: 45 percent of contraceptive users discontinue within 12 months. Discontinuation rates are highest for users of pills (67 percent) and injectables (52 percent). The most common reason for discontinuation was side effects or health concerns, with more than one in five users of pills or injectables discontinuing for this reason. Of those who discontinued any method and stated wanting another method, just 5 percent switched to another method within two months (UBOS and ICF 2018). In terms of informed choice, 67 percent of all women using a modern method were informed about side effects or other problems associated with the method they used, and 57 percent were informed about what to do if they experienced side effects (UBOS and ICF 2018). A higher proportion of women (74 percent) were informed about other available methods (UBOS and ICF 2018).

Study Design

As part of a prospective cohort study conducted between December 2016 and April 2018 in Pakistan and Uganda, women of reproductive age from two social franchise networks were enrolled to assess the relationship between quality of FP services and contraceptive discontinuation. In Pakistan, married women of reproductive age, 15–49 years old, who were receiving clinical services at 75 Marie Stopes Society (MSS) Suraj social franchise clinics, were eligible for inclusion in the study. In Uganda, both married and unmarried women of reproductive age, 15–49 years old, were recruited from 30 high‐volume clinics in the ProFam network located in the Central, Southwest, and Eastern regions, supported by PSI's network partner in Uganda, Programme for Accessible Health, Communication and Education (PACE). Clients were eligible to participate if they received a modern FP method (male/female condom, pill, injectable, implant, IUD, or emergency contraceptive) and were first‐time users (reported using contraception for the first time in their life), switching to a different modern method, or lapsed users returning to use (reported not using any FP method in the three months prior to the baseline interview). Women who obtained a resupply of an existing method, received sterilization, or were using nonmodern methods (e.g., withdrawal) were not eligible.

For women who provided written informed consent, client exit interviews were conducted in a private setting to ask about their visit and demographics immediately after they adopted a modern method at a social franchise site. Using the EquityTool, clients were asked a shortened form of questions about their household assets to assess relative wealth benchmarked to the wealth index from the most recent DHS surveys in Pakistan and Uganda (Chakraborty et al. 2016). (These questionnaires can be found online at www.equitytool.org.) To be eligible in Uganda, women must also have provided at least one phone number at baseline where they could be reached for follow‐up interviews. Enumerators were trained to obtain written informed consent, to ensure privacy during interviews, to administer the questionnaire, and to build rapport with enrolled clients. Quality of FP services at each participating social franchise site was measured using facility audit scores through direct observation collected within a year of the period of enrollment. Clients were followed up over a 12‐month period to assess contraceptive discontinuation. In Uganda, women received a small incentive of 5,000 Ugandan shillings (∼US$1.30) to complete the baseline survey.

MSS Suraj Network

At the time of enrollment, the Suraj Network in Pakistan consisted of over 500 clinics franchised by MSS for provision of FP services. Primarily offering maternal and child health services in underserved rural areas, these clinics are usually comprised of two to three rooms and are privately owned and operated by mid‐level service providers who have two‐year midwifery diplomas. Under the franchise agreement, service providers receive training, commodities, demand‐generation support, and quality audits for provision of FP services including condoms, pills, injectables, and IUDs.

PACE/Uganda ProFam Network

Since 2008 PACE/Uganda has worked with more than 150 ProFam clinics and more than 300 affiliated providers. PACE/Uganda has supported these providers across a broad range of health interventions, including family planning, HIV, malaria, cervical cancer screening and treatment, and maternal health. Under the franchise agreement, PACE supports the providers with training, commodities, demand generation, quality audits, and supportive supervision, as well as improved clinic/facility management.

Ethical Approval

Approval for the study arm in Pakistan was obtained from Ethical Review Committee (ERC) Marie Stopes International (MSI), UK (022‐16), and the National Bioethics Committee (NBC) at Pakistan Medical Research Council (PMRC), Islamabad (4‐87/17/NBC‐227/RDC/ 2308). Approval for the study arm in Uganda was obtained from both the Makerere University School of Public Health Research and Ethics Committee (451) and the Uganda National Council of Science and Technology (UNCST), Kampala (SS4215).

Analysis

We enrolled 813 clients across 75 Suraj facilities in Pakistan and 1,185 clients from 30 ProFam facilities in Uganda. In an analysis of baseline data, we constructed measures of the MII, disaggregated by client‐reported FP method. We present an index similar to the MII of the DHS and PMA2020, where the index score indicates the percent of women who answered “yes” to all three questions.

A few differences exist between our approach and the household survey index scores. Our analysis included condom and emergency contraceptive users while national surveys exclude these users. Only first‐time users, lapsed users, and those who switched methods were recruited in the longitudinal study since we expected these clients to receive full counseling as a best practice. In contrast, less than complete counseling for women seeking resupply of a method may be reasonable, which is why these clients were excluded from the study. In Pakistan, almost all clients reported using a single method and a significant proportion reported using condoms as their main method. This suggests the use of condoms was for FP rather than solely for sexually transmitted infections (STI) prevention. Additionally, the exclusion of condoms is not supported by strong rationale other than the limitations of national survey data. In surveys, it may be difficult to conclude whether condoms were used as a dual method and if they were used primarily for STI prevention. In this study, clients were enrolled immediately following their visits at a clinic and were asked about their reproductive intentions.

We also allowed for a response from all women to the question asking whether they were told what to do if they experienced side effects, while national surveys administer this question only to those who reported being informed about side effects. In doing so, we anticipated that some women might be told what to do about side effects without being informed about the side effects. Taking these differences into consideration, we refer to the index constructed from our approach as the MII for discussion purposes in this article.

In addition to the input questions used to calculate the MII, we also asked respondents to describe specifically which other FP methods they were told about, what method‐specific side effects they were told about, and what they were specifically advised to do if they experience side effects (Appendix Table A1, ¹). These additional open‐ended questions were only asked of clients who answered “yes” to the input questions of the MII. We then assessed the consistency between these additional responses and the method the respondents received—that is whether these specific responses were appropriate given their contraceptive method. For the question regarding other FP methods, a single definition of consistency was applied: a woman's response was deemed consistent if she could name at least one additional method aside from the one she received. For the questions that asked clients to list method‐specific side effects and actions to take if they experienced side effects, we applied two definitions of consistency. In our strict formulation, consistency was defined as the absence of any responses that were discordant with the FP method the client received. For example, if a woman received the pill and reported she was told spotting was a potential side effect, this was deemed a concordant response. However, if the same woman reported swelling of the arm as a potential side effect, this was deemed a discordant response. As multiple responses were allowed for each question, using this strict formulation, her response overall was consistent if a woman reported only concordant responses and none that were discordant. Alternatively, we also developed a less stringent definition of consistency, where the presence of any concordant response was deemed consistent.

In the above example, again had the woman reported being told that both spotting and swelling of the arm were potential side effects of the pill, her response was deemed consistent with her method under this less stringent formulation. For both formulations, no adjustments were made for the number of responses given, for example if a woman had listed five types of side effects, rather than just one or two. For IUD users, since both provider networks offered only copper IUDs during the time of the baseline visit, we assume that clients who reported receiving an IUD had received the copper IUD and considered only those side effects associated with the copper IUD as consistent. For all assessments, we used Contraceptive Technology to identify side effects that were concordant with the method received (Hatcher 2011).

Using both the less stringent and strict definitions of consistency, we constructed adjusted measures of the index. If a woman's response overall was not consistent with her method for a given question, then her score was adjusted down by one point. We compared unadjusted and adjusted MII index scores using a McNemar's paired test to assess differences in proportions, accounting for clustering at the health‐facility level.

To make our estimates more comparable to those of the DHS and PMA2020 surveys, we also recalculated the MII excluding condom and emergency contraceptive users. Additionally, we restricted the question that asked clients whether they were informed of what to do about side effects to only those who said they had been informed about side effects. Statistical analysis was conducted using Stata Version 14.0 (StataCorp).

RESULTS

The mix of FP methods varied across the two countries. In Pakistan, the majority of enrolled clients reported using short‐term methods (57 percent) as their primary method, whereas in Uganda most clients were using LARCs at baseline (60 percent). In Uganda, only a small number of users (n = 21) reported using male/female condoms or emergency contraception, so these were grouped together under other short‐term methods. We observed differences in client characteristics between the two countries. In Pakistan, the majority of the clients were 25 years or older, had higher parity, were uneducated, and belonged to the lower‐middle wealth quintiles. In contrast, clients in Uganda were younger, with lower parity, more educated, and more likely to belong to the wealthiest quintile of the population (Table 1).

Table 1.

Characteristics of clients by country

	Pakistan (N = 813)		Uganda (N = 1,185)
	n	%	n	%
Method
IUD	350	43.1	276	23.3
Implant	—	—	431	36.4
Injectable	199	24.5	335	28.3
Pill	149	18.3	122	10.3
Other ST methoda	115	14.1	21	1.8
User type
First‐time user	294	36.2	314	26.5
Lapsed adopter	42	5.2	177	14.9
Method‐switcher	477	58.7	694	58.6
Age (years)
15–24	125	15.4	475	40.1
25–34	443	54.5	531	44.8
35–49	245	30.1	179	15.1
Highest education
None (never went to school)	468	57.6	29	2.4
Primary	145	17.8	401	33.8
Secondary	151	18.6	595	50.2
Beyond secondary	49	6.0	160	13.5
Parity
Nulliparous	1	0.1	112	9.5
Primiparous	97	11.9	255	21.5
2–3 live births	317	39.0	448	37.8
4–5 live births	235	28.9	239	20.2
6 or more live births	163	20.0	131	11.1
Wealth quintile
1 (poorest)	54	6.6	21	1.8
2	126	15.5	46	3.9
3	210	25.8	43	3.6
4	228	28.0	180	15.2
5 (wealthiest)	195	24.0	895	75.5

NOTE: Descriptive statistics do not account for clustering at the facility level.

^aOther short‐term (ST) methods include male condom in Pakistan and male condom, female condom, and emergency contraception in Uganda.

The white bars in Figure 1 display the percent of clients who reported having been given information for each input question. Of the three questions, women most reported that they had been informed of other FP methods (83.8 percent in Pakistan and 90.6 percent in Uganda) while fewer clients, 76.1 percent and 79.8 percent, respectively, said they were told about side effects. In total, 76.9 percent of women in Pakistan and 88.1 percent of women in Uganda reported they were told what to do if they experienced side effects. The results using different adjustment formulations are also presented in Figure 1. For the question asking if clients were told about other FP methods, only one adjustment criterion was used, where a response was consistent if a client could name at least one additional method aside from the one she received. These adjusted results in both countries were equal to or close to the unadjusted results. When clients were asked to specifically describe what they were told to do if they experienced a side effect, the adjusted results were only slightly lower than the unadjusted results in both countries.

Figure 1.

Percent of clients who reported being given information and percent of specific responses consistent with their method

NOTE: Vertical line represents 95 percent confidence interval for the estimate.

In contrast, consistency between specific side effects and the method received resulted in greater variability. When applying the less stringent criteria of any concordant answer, the percent dropped to 67.3 percent in Pakistan and 71.0 percent in Uganda (Appendix Table A2). Our strict formulation of consistency produced a 38.3 percentage point drop in Pakistan and a 22.7 percentage point drop in Uganda. When the differences were disaggregated by method, in both countries, IUD users were the least likely to mention side effects that made sense for their method.

Looking across the two countries, the side effects most frequently reported by clients that were consistent with the method they received were heavy bleeding and pain at the site (copper IUD and implant), heavy bleeding (injectable), and headache (pill) (Appendix Table A3). Side effects reported that were inconsistent with the method received included infrequent bleeding/no bleeding (copper IUD), breast tenderness (injectable), heavy bleeding (pill), and weight gain (copper IUD, condom).

The overall percent of women answering “yes” to all three MII questions was 64.6 percent in Pakistan and 72.7 percent in Uganda (Figure 2). Our less stringent formulation produced adjusted MII index scores that were 12 percent lower than unadjusted scores in both countries. By comparison, strict criteria consistency estimates were reduced by 53 percent in Pakistan and 31 percent in Uganda, respectively, compared to unadjusted results.

Figure 2.

Unadjusted and adjusted Method Information Index scores

NOTE: Vertical line represents 95 percent confidence interval for the estimate.

By method, IUD users had the highest unadjusted MII score (73.4 percent in Pakistan; 79.4 percent in Uganda); however, they also had large reductions in adjusted MII scores (by 10–74 percent in Pakistan and by 26–63 percent in Uganda) (Table 2). The MII was most stable for injectable users in Pakistan, starting with an unadjusted score of 72.4 percent and dropping to 60.8 percent when applying our strict definition of consistency, and for implant users in Uganda, decreasing only slightly from 69.4 percent to 65.2 percent employing the strict criteria. Using our less stringent definition of consistency, the overall adjusted index score significantly differed from the unadjusted score in both Pakistan (difference = 8.1; 95% CI: 5.3–10.9; p‐value < 0.001) and Uganda (difference = 8.5; 95% CI: 5.7–11.4; p‐value < 0.001). This difference was even more appreciable when applying our strict criteria of consistency: Pakistan (difference = 34.4; 95% CI: 28.4–40.5; p‐value < 0.001) and Uganda (difference = 22.3; 95% CI: 15.2–29.1; p‐value < 0.001).

Table 2.

Method Information Index Scores: Unadjusted and adjusted for consistency of responses by family planning method

	Unadjusteda % (95%CI)	Adjusted for the presence of any concordant responseb % (95%CI)	Adjusted for the absence of discordant responsesc % (95%CI)
Pakistan d
IUD	73.4	65.7	18.9
(n = 350)	(65.2–80.3)	(57.3–73.2)	(13.7–25.4)
injectable	72.4	68.8	60.8
(n = 199)	(61.3–81.2)	(57.2–78.5)	(49.8–70.8)
Pill	59.7	57.1	36.2
(n = 149)	(46.2–71.9)	(43.8–69.4)	(25.8–48.2)
Male condom	30.4	6.1	3.5
(n = 115)	(18.4–45.9)	(2.9–12.5)	(1.3–9.0)
TOTAL	64.6	56.5	30.1
(N = 813)	(56.4–72.0)	(49.3–63.4)	(24.9–36.0)
Uganda e
IUD	79.4	58.7	29.7
(n = 284)	(67.7–87.6)	(45.8–70.5)	(20.0–41.7)
Implant	69.4	66.4	65.2
(n = 424)	(58.6–78.4)	(56.1–75.3)	(54.9–74.2)
Injectable	73.1	71.0	56.1
(n = 337)	(61.3–82.4)	(59.4–80.5)	(43.2–68.3)
Pill	71.3	61.5	38.5
(n = 120)	(56.5–82.7)	(47.3–73.9)	(23.4–56.2)
Other ST methodf	57.1	0.0	0.0
(n = 22)	(37.5–74.8)
TOTAL	72.7	64.2	50.5
(N = 1,185)	(63.5–80.4)	(55.1–72.4)	(43.2–57.7)

NOTE: Estimates account for clustering at the facility level.

^aCalculated from raw reported responses.

^bCalculated from responses adjusted for consistency (presence of any concordant response) between specific responses and FP method received.

^cCalculated from responses adjusted for consistency (absence of discordant responses) between specific responses and FP method received.

^dPakistan: McNemar test comparing raw and adjusted (presence of any concordant response) proportions scoring 3: difference = 8.1 (95% CI: 5.3‐10.9), p‐value < 0.001. McNemar test comparing raw and adjusted (absence of discordant responses) proportions scoring 3: difference = 34.4 (95% CI: 28.4‐40.5), p‐value < 0.001.

^eUganda: McNemar test comparing raw and adjusted (presence of any concordant response) proportions scoring 3: difference = 8.5 (95% CI: 5.7‐11.4), p‐value < 0.001. McNemar test comparing raw and adjusted (absence of discordant responses) proportions scoring 3: difference = 22.3 (95% CI: 15.3‐29.3), p‐value < 0.001.

^fOther short‐term (ST) methods include male/female condom and emergency contraception.

In a sensitivity analysis, we compared our MII estimates to those of the DHS and PMA2020 surveys. We excluded condom and emergency contraceptive users and only asked those who said they were told about side effects if they were also informed of what to do about side effects. Excluding condom and emergency contraceptive users increased estimates in both countries in our study since scores were lower for these types of users. Employing the same skip pattern as that used in national surveys did not change our estimates due to the binary nature of the MII: answering “yes” to all three questions versus less than three questions. In this case, a client's response to the third question was dropped if she had answered “no” to the side‐effects question. Mathematically, if a client answered “no” to the side‐effects question, she already belongs to the group answering “yes” to less than three questions. Asking or not asking the question about what to do about side effects does not affect the binary categorization of her responses. Therefore, no differences exist in the calculation of the MII when administering this question to all women as done in this study versus asking only a subset of women who said they were told about side effects as administered in surveys.

In comparison to national surveys in Pakistan, our unadjusted and both adjusted MII scores excluding condom and emergency contraceptive users (unadjusted = 70.2 percent, less stringent adjustment = 64.8 percent, strict adjustment = 34.5 percent) were consistently higher than the estimates calculated from 2012–13 DHS data (13.2 percent) (NIPS Pakistan and ICF International 2013; Jain 2016). In Uganda, our unadjusted and less stringent adjustment scores (73.0 percent and 65.4 percent, respectively) were higher than that of the 2016 DHS estimate (52.9 percent) (UBOS and ICF 2018), but our score applying the strict definition was comparable (51.2 percent). All scores from our study were higher than the estimate reported in the 2016 PMA2020 Uganda data (44.6 percent) (PMA2020 2016).

DISCUSSION

Employing a new approach to investigate properties of the MII, our study utilized additional questions to probe for specific responses related to the method a client received to supplement and verify responses to input questions of the MII. We calculated the range of possible adjustment to MII scores using two criteria—that is, the presence of any response concordant with the method received (less stringent) versus the absence of discordant responses (strict). We observed a significant decrease in the MII scores in both the countries when using both adjustment criteria. The finding suggests that the MII calculated from input questions alone may overestimate counseling quality.

Consistency was higher for all three questions among clients in Uganda compared to clients in Pakistan. This may be partly attributed to socio‐economic differences between the client populations in the two samples. Clients with a higher level of education and those belonging to higher wealth quintiles may be more likely to better understand counseling information they received from their providers (Assaf, Wang, and Mallick 2016). Differences in the profile of the providers in the two franchises may also explain the higher consistency of client reports in Uganda.

Taking client perspectives into consideration has been highlighted as a critical component of evaluating service quality (Donabedian 1988; Calnan 1998; Williams 1998). Although clients’ views are subjective, the perception of poor service quality may deter them from utilizing health services and compel them to seek alternatives (Petersen 1988; Andaleeb 2001). Therefore, improving quality of services from a client's perspective is essential to increasing uptake and retaining clients within the health system (Petersen 1988; Andaleeb 2001). Informed choice is a required component in most family planning programs, but good counseling requires not only providing correct information but communicating that information effectively. From a rights‐based perspective, counseling a client appropriately with information she understands is the responsibility of the provider (Hardee et al. 2014). If a client comes away from her visit with incomplete information, the way the information is communicated can be adjusted to facilitate better understanding. Prior work has shown that more client‐centered counseling strategies can help to engage clients more in the decision‐making process and provide them with better‐tailored information (Kim, Kols, and Mucheke 1998; Johnson, Kim, and Church 2010).

It may also prove useful to assess the agreement between more objective measures of service quality and client perception. Earlier studies have found varying levels of inconsistency between observed and client‐reported indicators of counseling. These inconsistences are reflective of recall bias, information being transmitted in a way that is not comprehensible to the client, or clients reporting positively when the behavior did not occur, which could signify a social desirability bias (Bessinger and Bertrand 2001; Tumlinson et al. 2014; Assaf, Wang, and Mallick 2016; Choi 2018). Although these studies have not shown high degrees of consistency between observations and client reports, observations of the counseling interaction should not be considered more important than client reports. In fact, what the patient retains or understands may matter more for her continued FP use. For this reason, the use of follow‐up questions in client reports may help to better understand what the MII is capturing when it comes to quality of counseling. Our study indicates that seemingly simple “yes/no” questions about the counseling that the woman received during her visit have many layers. Given the discrepancy between the unadjusted and adjusted MII scores, this study suggests that follow‐up questions to the three input questions used to calculate the MII may help to provide context for improved understanding of the process of information exchange. There is increasing interest in the potential application of the MII to programs delivered at lower levels of the health system. Here, results of FP counseling can be used to adjust training curricula or to provide results‐based payments, where financing is linked to agreed‐upon targets and payment is only made once targets are achieved (Blacklock et al. 2016). In these situations, verification of the MII findings, even in a subsample of clients, is advised.

Reporting of the MII in national surveys makes it possible to routinely measure and monitor important elements of information women receive (ICF International 2015). To allow for comparisons to DHS and PMA2020 surveys, we recalculated our MII estimates excluding condom and emergency contraceptive users. Our MII estimates were generally higher than those reported in surveys in both countries. The reasons for the difference may be attributable to bias and limited generalizability of our study population. When used in retrospective surveys such as the DHS or PMA2020, the MII is subject to recall bias. Women may forget what happened during a clinical visit, especially if asked about particular aspects of a conversation up to five years after it occurred. By asking women about their visit directly after it took place, this study reduced recall bias. However, our use of client‐exit interviews probably introduced or enhanced the effect of any courtesy or acquiescence bias, where clients may be reluctant to share what may be perceived as undesirable responses about the care they received when interviewed at the health facility (Hameed et al. 2018).

It is important to note that the difficulty in reporting method‐specific information that is consistent with the method received is not equal across the three questions. For example, naming one other FP method and mentioning that she was told to return to the provider if any side effects were experienced are more likely to be consistent with any given method than reporting an appropriate method‐specific side effect. For this reason, low consistency was relatively confined to the side‐effects question. This suggests that targeting improvements to this one component of informed choice, namely clients’ understanding of method‐specific side effects, could increase adjusted scores dramatically. The successful transfer of counseling information from provider to client may be important for a client's understanding of what to expect from her new method. Prior studies have found that clients who were not adequately counseled about side effects were more likely to stop using contraception while those who received more extensive counseling on side effects were more likely to continue using their method or switch to another method (Cotten et al. 1992). Knowing that specific side effects are common for a method may lead a woman to be more likely to accept the side effects if she experiences them herself and may mean she is less likely to discontinue. Low consistency in the side‐effects question could be indicative of poor provider knowledge or the reluctance of providers to discuss contraceptive side effects with clients out of concern that clients may worry unnecessarily or opt against using the method (Kim, Kols, and Mucheke 1998; Murphy et al. 1999). In both countries, a slightly higher percent of women reported having been told about what to do if they had side effects than being told about side effects. This analysis suggests that in some cases providers may have expected that clients would not adopt a method upon hearing about side effects and, instead, informed clients to return to the clinic if they have any problems, without mentioning specific side effects.

Notably, the index scores differed substantially (56.5 percent to 30.1 percent in Pakistan; 64.2 percent to 50.5 percent in Uganda) depending on the adjustment criteria applied. An alternative explanation may be that a substantial amount of information was communicated by the provider. The majority of women said they had been informed about other methods. Service providers may have discussed possible advantages and disadvantages of each method, including potential side effects, with the clients. Both the MSS Suraj and PSI/PACE ProFam networks train providers on counseling strategies that first help clients identify a short list of methods that satisfy their needs and preferences before detailing specific information about potential side effects, in an effort to avoid overwhelming clients. However, clients may still have found it difficult to retain side‐effects information correctly for each method. When asked in the interview, they may have cited multiple side effects that may or may not be related to their adopted methods. Previous studies have also suggested that information retention is reduced as more medical and technical information is provided by the provider (Ley 1979; Selic et al. 2011). Nonetheless, provision of information about potential side effects has its own importance as it not only helps clients make an informed decision, but also may help to increase the likelihood of sustained contraceptive use (Davie et al. 1996; National Collaborating Centre for Women's and Children's Health (UK) 2005; Dehlendorf, Krajewski, and Borrero 2014).

There are certain underlying caveats that should be considered while drawing inferences about adjusted MII scores in our study. One possible explanation for the decrease in the MII scores in both the countries when adjusting for specific responses could be clients’ low retention of technical information provided to them (Ley 1979; Selic et al. 2011). However, it is important to note that in our study, interaction with the service provider took place a few minutes before the interview. Therefore, the influence of recall bias should have been minimal. Another possible reason could be the close‐ended dichotomous nature of the input questions, which could have inflated unadjusted scores due to courtesy bias, as discussed earlier.

In a recent comparison of client exit interviews and direct observation of side‐effects counseling collected in Service Provision Assessments (SPA) datasets from five countries, systematic overestimation of client‐reported estimates versus observation‐based estimates were attributed to courtesy bias (Choi 2018). Last, the responses to the input questions may have also been affected by acquiescence bias. Clients may have been inclined to agree with the three questions as they are unipolar and are usually asked together sequentially (Hinz et al. 2007). We had attempted to minimize both courtesy and acquiescence bias by conducting comprehensive training of enumerators with special focus on rapport building, questionnaire administration, and ensuring privacy. Given the study methodology, it is not possible to determine the extent of any courtesy, acquiescence, or recall bias, nor whether clients were only reporting on the interaction completed immediately prior to the baseline interview. To know what the providers actually communicated to clients, this study would have had to compare client reports with direct observation. Nevertheless, this study does assess information exchanged, from the perspective of the client. It assumes that additional questions about the content of information exchange actually occurred at the clinic visit. However, clients may have combined prior knowledge with the information transmitted by the provider.

Additional limitations of our study include the fact that our sample population was not nationally representative, even of users of FP. Our study has limited generalizability on two levels. First, the population of clients are those who attend social franchises, which are primarily urban, and entirely private. In Uganda, 75 percent of clients belonged to the highest national wealth quintile, while in Pakistan, only 22 percent of clients came from the poorest 40 percent of the population. Second, the method mix of these women does not reflect the national method mix. In Pakistan, approximately 6 percent of contraceptive users in the 2012–13 DHS survey used the IUD, compared to 43 percent of clients in our sample. LARCS are more popular in Uganda, with 23.4 percent of married contraceptive users reporting these methods in the latest PMA 2020 survey. Still, nearly 60 percent of clients in our sample used LARCS. This discrepancy in the method‐mix—especially the higher adoption of LARCs could be attributed to the nature of demand‐generation activities carried out in each of the social franchise programs (Viswanathan, Schatzkin, and Sprockett 2014; Gul et al. 2015; Boddam‐Whetham et al. 2016).

CONCLUSION

To our knowledge, this is one of the first studies to further explore the MII by investigating not only whether women were given information during a counseling session but also what specifically they could recall being told and whether this information was consistent with the method they received. This study finds that the MII may be overestimating the information successfully exchanged during counseling from provider to client and retained by the client, primarily due to low consistency in the side‐effects question. From a rights‐based perspective, evaluating client understanding and successful transfer of information to ensure informed choice during her counseling visit may be the primary goal, and an analysis of consistency can aid in identifying areas for improvement. However, it is unknown whether these unadjusted and/or adjusted scores are related to contraceptive discontinuation or other measures of service quality. A client's ability to accurately describe specific information about her method may mean she is better prepared for any side effects she may experience and is more likely to continue using family planning.

Further investigation is needed to assess whether a client's perception of the information‐exchange interaction versus her correct recollection of information is more important for her continued use of contraception. With increasing interest in using the MII to monitor FP programming and performance, recognizing how this measure may differ from its use in national surveys is necessary. In client exit interviews, a woman may be primed to think about the discussion with her provider that occurred during her FP visit immediately prior to the interview. In household surveys, which may take place months or years after she first obtained her current method, she likely has had some experience using her method and interim visits to seek resupply, all of which may influence her response to the input questions of the MII. More work is needed to better understand potential differences in what the MII may be capturing in the context of programmatic applications as compared to national measures.

Supporting information

Appendix Tables A1–A3