Validating the self-reported annual monitoring grid for the 2007 Caribbean Community Declaration of Port-of-Spain on noncommunicable diseases

ABSTRACT

Objectives

In 2007, the Caribbean Community (CARICOM) convened the world's first-ever heads of government summit on noncommunicable diseases (NCDs) and issued the landmark Declaration of Port-of-Spain: Uniting to Stop the Epidemic of Chronic NCDs. Since then, ministry of health (MoH) focal points in each country have self-reported annually on their NCD efforts, using a 26-indicator grid created to assess implementation of the Declaration. Our objective was to assess the validity of those grid responses, as compared to information from in-depth interviews and document reviews.

Methods

Seven national case studies on policy responses to the Declaration were undertaken in 2015. In-depth, semistructured interviews were conducted with stakeholders from multiple sectors, including the MoH. Policy documents were also identified and reviewed. The results from the 2015 case studies were compared to the 2014 MoH focal point grid responses. Kappa statistics evaluated chance agreement.

Results

The information from the grid and from the case studies agreed closely. Out of a total of 182 indicators (26 each for seven countries), there was a lack of agreement on just 9 (4.9%). All the differences were between policy statements and implementation. Except for physical activity, kappa statistics indicated that agreement was good to excellent for all the clusters of the grid and for the grid as a whole, but with wide confidence intervals.

Conclusions

In general, the monitoring grid accurately assessed the national situation, but with a possible tendency to overstate performance in some areas. These findings contributed to the design of a new, 50-indicator monitoring grid in 2016. Alongside these improvements, CARICOM countries face a substantial burden from having to complete many other required NCD reports, mainly for the Pan American Health Organization and the World Health Organization.

Governments have a primary role in creating health-promoting environments as an essential input for sustainable development (1, 2). In 2007, the Caribbean Community (CARICOM) convened the world's first-ever heads of government summit on noncommunicable diseases (NCDs) and issued the landmark Declaration of Port-of-Spain: Uniting to Stop the Epidemic of Chronic NCDs (3).

In 2014, CARICOM and the Pan American Health Organization/World Health Organization (PAHO/WHO), on behalf of the CARICOM member countries, commissioned a formal evaluation of country compliance with the mandates from the 2007 NCD Declaration. This evaluation was funded by the Canadian International Development Research Centre (4).

The overall objective was to evaluate, seven years on, the implementation of the 2007 Port-of-Spain Declaration, in order to learn lessons to accelerate its further implementation and to inform the attainment of the WHO's Global Monitoring Framework for NCDs. Adopted in 2013, the WHO Framework stemmed from the 2011 United Nations Political Declaration of the High-level Meeting of the General Assembly on the Prevention and Control of Non-communicable Diseases. Similarly, the 2030 Sustainable Development Goal #3 aims to reduce the burden of NCDs through improvements in coverage and reduction of risk factors (5).

As described elsewhere (6), this formal evaluation had six research objectives. Objective 1 was to add to knowledge on the implementation of the Port-of-Spain Declaration by determining the extent to which the mandates in the Declaration were reported to have been implemented within the 20 CARICOM countries and territories. These 20 consist of 15 full members plus 5 associate members that are United Kingdom Overseas Territories.

The work to meet this evaluation's Objective 1 built on the existing system of annual monitoring of 26 progress indicators utilizing a grid developed in 2008 and first revised in 2010 (7). In each CARICOM member state, the NCD focal point in the ministry of health (MoH) has reported to the University of the West Indies (UWI) annually on whether they had met, not met, or made progress towards meeting each of the Port-of-Spain Declaration mandates. This information has then been summarized by UWI in a color-coded grid. The completed grid has been presented each year to the annual meeting of the CARICOM ministers of health, and to the CARICOM Heads of Government.

The objective of this paper was to assess the validity of self-reported responses to the grid, as compared to in-depth interview data and document reviews. We report on the process and outcome of a second revision of the grid in 2016, and also on the number of NCD reports required from countries annually or every two, three, or five years by the World Health Organization, the Pan American Health Organization, the Caribbean Pubic Health Agency, and the UWI.

METHODS

Validation of the grid used in 2008–2015

Case studies on policy responses to the Declaration were undertaken between April 2015 and July 2015 in seven CARICOM member states: Antigua and Barbuda, Belize, British Virgin Islands, Grenada, Jamaica, Saint Kitts and Nevis, and Trinidad and Tobago. These seven member states were purposely selected to reflect large, medium, and small population sizes; mainland and island member states; full members and associate members; and countries with high or low levels of achievement of the Port-of-Spain Declaration mandates. A total of 76 in-depth, semistructured interviews were conducted with 80 stakeholders, from the ministry of health, other government ministries and agencies, civil society organizations, and the private sector. Interviews lasted 30 to 60 minutes, and the number of interviews in each country ranged from 7 to 12 (Table 1).

TABLE 1. Number of key informant interviews done in 2015, by country/territory and by sector, for national case studies on policy responses to the 2007 Caribbean Community Declaration of Port-of-Spain on noncommunicable diseases.

Country/Territory	Ministry of health	Other government	NGOa	Private	Academia	Total
Antigua and Barbuda	3	3	5	-	-	11
Belize	4	3	4	1	-	12
British Virgin Islands	5	3	3	1	-	12
Grenada	3	1	4	-	1	9
Jamaica	5	1	4	2	1	13
Saint Kitts and Nevis	3	3	-	1	-	7
Trinidad and Tobago	4	2	3	3	-	12
Total	27	16	23	8	2	76

Source: M. M. Murphy et al., based on the study results in article titled “Evaluating policy responses to promote diet and physical activity in seven Caribbean countries,” in this supplement of the Revista Panamericana de Salud Pública/Pan American Journal of Public Health.

^aNGO = nongovernmental organization.

The interview schedule was structured around the major policy recommendations contained in the 2007 Port-of-Spain Declaration, and thus also covered the policy areas contained in the monitoring grid. In addition, the interviews covered policy on alcohol-related harm (not covered in the 2007 Declaration, or in the grid), as well as facilitators and hindrances to policy agenda-setting, development, and implementation.

All the 2015 interviews were recorded and transcribed verbatim. A framework analysis was undertaken in 2015 using Dedoose software (SocioCultural Research Consultants, University of California Los Angeles, Los Angeles, California, United States of America), which included identifying reported policy responses across the different domains found in the annual monitoring grid. A distinction was made between “policy statements” (such as written statements in policy documents, or statements made by policymakers) and “policy implementation,” where key informants indicated that there was clear evidence that policies had been implemented and were active.

Findings from the 2015 interviews and document review were compared to responses that the respective MoH focal points had provided in the NCD grid 2014 reports (Table 2).

TABLE 2. Country implementation of mandates, in seven clusters, from the 2007 Caribbean Community Declaration of Port-of-Spain on noncommunicable diseases (NCDs), comparing ministry of health reports of 2014 vs. validation of response in 2015 from key informant interviews and document review^a.

Statistical analysis

Agreement on the number of grid indicators met was assessed using the kappa statistic. This was done separately for each of the seven clusters (Table 2) of the grid, and for the grid as a whole. For each cluster of the grid, each country was given a score, which simply reflected the number of indicators met. Stata 13 statistical software (StataCorp, College Station, Texas, United States) was used to apply weighted kappa analysis, assessing the agreement between the scores reported by the NCD focal points and those derived from the analyses of the in-depth stakeholder interviews. Kappa values are reported with 95% confidence intervals based on the standard error reported by Stata and with p values testing the null hypothesis of no agreement.

Ethical approval was received from the University of the West Indies/Ministry of Health Institutional Review Board in Barbados and from corresponding institutional review boards and/or ministries of health in the seven case study countries.

RESULTS

Validation of the 2008–2015 grid

The results from the 2015 case studies agreed closely with the responses that the MoH focal points had given in 2014 to the NCD grid. Out of a total of 182 indicators (26 across seven countries), there was lack of agreement on just 9 (4.9%). All the disagreements arose from the difference between policy statements and implementation (Table 2). For 17 of the 26 indicators, the self-reported grid data from 2014 aligned with the evidence collected in the seven countries from key informants and document reviews in 2015. For an additional 5 of the 26 indicators, one out of the seven case study countries had a change by one increment. For example, for one country, “multisectoral NCD commission appointed and functional” was downgraded from “yes” to “partial.” In one case, there was an upgrade from “partial” to “yes.” (In Table 2, differences in indicator status can be seen when comparing results from the middle of the table (“Response to grid”) and the right side of the table (“Modified from KI, doc review”). It is considered a downgrade where the specific indicator changes from “in place” to “in process/partial,” “not in place,” or “missing.”)

Among the four remaining indicators, “mandatory PA [physical activity] in all grades in schools” was the least reliable indicator. While four countries had self-reported “yes,” the review downgraded this to two countries. One country went from “yes” to “no,” and in one country no evidence could be found to support the assertion.

Three other indicators had inadequate alignment. Having an NCD plan had two countries downgraded from “yes” to “partial.” In one country, having a specific NCD budget and “tobacco taxes > 50% sale price” were downgraded from “in place” to “missing,” since there was no data or key informant answers that supported this response.

Validation by country

Two of the seven countries—the highest performer and the lowest performer—had self-report 2014 grid responses that aligned exactly with data from the 2015 case-study key informants. One country was upgraded when we found evidence of partial compliance with “mandatory provision for physical activity in new housing developments.” The Ministry of Health, which completed the grid, had not been aware of that.

One country was quite an outlier, where two indicators were downgraded from “in place” to “partial” and for two other “in place” indicators were downgraded to “missing,” as there was no evidence from any of the 2015 case study stakeholders to support the 2014 self-report responses.

Table 3 shows the agreement (weighted kappa scores) between the indicators that the NCD focal points had reported as having been met when they completed the grid in 2014 and the indicators assessed as having been met based on the in-depth stakeholder interviews in 2015. With the exception of physical activity, agreement was good to excellent for all the clusters of the grid and for the grid as a whole. However, the confidence intervals are wide, reflecting the small sample size, and, with the exception of physical activity, are consistent with poor or fair agreement through excellent agreement. For the physical activity cluster of the grid, confidence intervals are consistent with no agreement through to good agreement.

TABLE 3. Agreement for the number of indicators for the 2007 Caribbean Community Declaration of Port-of-Spain on noncommunicable diseases, comparing ministry of health reports in 2014 of indicators having been met vs. validation of those responses from key informant interviews and document review in 2015.

Grid cluster (no. of indicators)		Weighted kappa (95% CI)a	p value
Commitment (4)		0.59 (0.12-1.00)	0.007
Tobacco control (4)		0.81 (0.28-1.00)	0.001
Nutrition (5)		0.81 (0.28-1.00)	0.001
Physical activity (3)		0.30 (-0.11-0.71)	0.073
Education/Promotion (4)		0.87 (0.30-1.00)	0.001
Surveillance (4)		1.00 (0.26-1.00)	0.004
Treatment (2)		1.00 (0.26-1.00)	0.004
	All indicators (26)	0.63 (0.18-1.00)	0.003

Source: Author NU, based on statistical analysis.

^aThe maximum value of the upper confidence interval (CI) is limited to 1.00

DISCUSSION

The annual monitoring grid for 2008–2015 had relied on self-reported data from the ministries of health, without objective validation of the reports. Self-reports may not align with objective measures (8–10). However, our analysis determined that, overall, the monitoring grid provided an accurate assessment of the national situation, with 95.1% of reports assessed as being accurate. However, due to a small sample size, the kappa statistic had wide confidence intervals.

The inaccuracies were almost exclusively overreporting of achievements. These findings have validated the self-reported annual monitoring grid, and have shown objectively that self-reports on progress on implementation of NCD policies and programs are a valid methodology for assessing country compliance.

Around the world, the importance of monitoring NCD performance is recognized (11, 12). The literature shows NCD policy evaluation in several countries, and other countries rely on policy analysis (13–14).

In West Africa, an assessment by Sodjinou et al. (15) of nutrition programs used a conceptual model that included four levels: 1) tools; 2) skills; 3) staff and infrastructure; and 4) structures, systems, and roles. That research assessed capacity, assets, and gaps at the individual, organizational, and system levels through interviews with key informants from government, development partners, tertiary training institutions, and health professional schools, and then compared the results across countries.

Small island developing states (SIDS), which are primarily located in the Pacific and the Caribbean, share many characteristics, including a high burden of NCDs and their risk factors (16–18).

In the Caribbean and the Pacific, small populations and resultant suboptimal capacity require tailored solutions for monitoring the NCD agenda (19–22).

Like their Caribbean counterparts, the Pacific islands have been monitoring and assessing accountability for their responses to the NCD crisis. The Pacific Monitoring Alliance for NCD Action (MANA) simply monitors and records components of NCD country process and outcome indicators (mortality, morbidity, risk factors, health system responses, environments, and policies) captured in a dashboard to demonstrate national and regional progress towards agreed NCD policies and actions. However, the publications do not indicate that any secondary validation process is employed in the Pacific (23, 24).

This paper goes beyond the initial monitoring and evaluation by validating the Port-of-Spain Declaration grid. It is important to use a method that ideally avoids the same types of potential errors that the completion of the grid has. Similar approaches, such as using another checklist-type questionnaire, would be a poor method of validation. Using in-depth stakeholder interviews, from a range of stakeholders in each country, is a very different methodological approach.

In the peer-reviewed literature, we were not able to identify NCD programs with post-hoc validation of results. Other generic approaches might have been taken to the “validation” of the grid responses.

Baisch et al. (2016) used a qualitative study to compare the “state of the art” and the “state of the practice” and identify the gap between the two (25). Our validation compared the mandates with the practice.

Zhao et al. (26) used experts to conduct a two-round Delphi study and developed a quality assessment index framework for public health services, which was unified and hierarchical. That model is being further tested. While this method focuses on quality of services, it could probably be adapted to evaluate implementation of quality services.

Limitations

Our in-depth interviews were not specifically designed to validate the grid, but we believe them to potentially give a fuller and more complete picture than one individual (even a well-informed individual) completing the grid.

The kappa statistic takes into account the possibility of similar answers based on chance agreement, and the values have this range: ≤ 0 (no agreement), 0.01–0.20 (none to slight), 0.21–0.40 (fair), 0.41–0.60 (moderate), 0.61–0.80 (substantial), and 0.81–1.00 (almost perfect). One criticism is that in this ranking, a kappa score of 0.61 is deemed to be substantial agreement, despite there being little agreement among raters. Health-related studies may need to use a higher threshold for acceptable agreement between raters. McHugh suggested that both kappa and percent agreement should be reported (26).

The kappa results are often used to reduce the estimate of agreement, but because the method of utilizing marginal sums may not be reliable, the kappa may not be representative of the amount of rater agreement caused by chance. This is because the assumptions underlying the calculations have not been met, the raters guessed the answer to each item similar to the marginal proportions, or the raters were entirely independent (26).

A further limitation is that kappa is dependent on prevalence and is unable to determine the type or source of disagreement (27). Our small sample size resulted in large confidence intervals, known to be a characteristic of small studies. It is recommended that a Kappa study should have at least 30 comparisons (26).

2016 revision of the grid

The overarching goal of surveillance of country implementation of the Port-of-Spain NCD summit mandates is to provide a “snapshot” of government responses to NCDs. This is a prerequisite to identifying performance by country and across the region to monitor progress, and to influence policy, given that policy reforms will be necessary to achieve the NCD agenda (28).

While the evaluation grid has proved very useful, and indeed is proving of international interest, it also has limitations that need correcting. In addition, some indicators are no longer needed and others need to be added. The original format of the grid was refined and used for the eight years of 2008 through 2015. Table 2 displays the 26 indicators. The work reported here, on the validity of the original format, was used to help update the grid in 2016, when a revised version of the grid with 50 indicators was piloted and introduced.

The first draft of the revised grid came out of discussions at a surveillance workshop in October 2015. A draft was prepared by author TAS and circulated for comments. Then the amended draft was circulated to countries for their comments. Finally, two conference calls were held with countries, at which time they made further edits and they were assisted with filling out the new grid for the first time. Seventeen of 20 countries submitted in 2016, and 19 of 20 did so by 2017, with Haiti being the only country with no response and all indicators missing (Table 4).

TABLE 4. Ministry of health reports on compliance with 2016 revision of monitoring grid of mandates of the 2007 Caribbean Community Declaration of Port-of-Spain on noncommunicable diseases (NCDs), plus additional NCD indicators, according to 2016 reports and 2017 updates^a.

Cluster/Source of indicator/NCD progress indicator			Country
			A	A	B	B	B	B	B	C	D	G	G	H	J	M	S	S	S	S	T	T
			N	N	A	A	E	E	V	A	O	R	U	A	A	O	K	T	V	U	R	C
			G	T	H	R	L	R	I	Y	M	E	Y	I	M	N	N	L	G	R	T	I
Commitment
	POS; CC	National NCD policy, strategy, action plan integrates NCDs and risk factors	+	+	+	+	+	+	+	+	+	+	+	…	+	+/-	+	+	+	+	+	+/-
	POS	NCD plan/programs explicit re gender	+	+	-	-	+	+	+	-	+	+	+	…	+	+	+	+	+	+/-	+	+
	POS; CC	Funding for NCD and risk factor programs	+	+	-	+	+/-	+	+	+	+	+	+	…	+	+	+	+/-	+	+	+	+
	POS	Earmarked taxes/revenue for NCDs	-	+	-	-	-	-	-	-	-	-	-	…	+	-	-	-	-	-	+	-
	Hds16	National tax on sugar-sweetened beverages	+/-	-	-	+	+/-	-	-	-	+	…	-	…	-	…	+	…	-	-	-	-
	Hds16	National tax on high fat foods	-	-	-	-	-	-	-	-	-	…	-	…	-	…	-	…	+	-	-	…
	Hds16	National tax on high salty foods	-	-	-	-	-	-	-	-	-	…	-	…	-	…	-	…	+	-	-	…
	CC	Price subsidies for healthy foods	-	+	-	+	-	-	-	-	-	…	+/-	…	-	…	-	…	-	-	…	…
	POS; CC	NCD unit or equivalent in ministry of health (MoH)	+	+	+	+	-	+	+	-	+	+	+	…	+	+	+	+	+	+	+/-	+
	POS	Dedicated NCD focal point > 50% time on NCDs	+	+	+	+	-	+	+	+/-	-	+	+	…	+	-	+	+/-	+	+	+	+
	POS; CC	Multisector national NCD commission (NNCDC) or equivalent	+	+/-	+	+	+	+	+	-	+	+	+/-	…	+/-	-	+/-	+	+/-	+/-	-	+
	POS	NNCDC cabinet submission in last 24 months	+	-	+	+	-	+	-	-	-	+	+/-	…	-	-	-	-	-	-	-	-
	POS	Inter-Minister Task Force (IMTF) /equiv: “all of govt”	-	-	+/-	+	-	-	+	-	+	+/-	-	…	+/-	-	+/-	-	+	-	+	-
	POS	IMTF cabinet submission in last 24 months	-	-	-	+	-	-	-	-	-	+/-	-	…	-	-	-	-	-	-	-	…
Tobacco
	POS; FCTC	National anti-tobacco TV/radio campaign in last 24 months	-	+	-	+	-	+	+	+	+	-	+/-	…	-	-	-	-	+	-	-	+
	POS; FCTC	Total taxes in the retail price of most widely sold brand of cigarettes >50% sale price	+	-	-	+	-	-	-	+	+	…	+/-	…	+	-	-	+	-	+	…	…
	FCTC	All public spaces 100% smoke free by law	-	-	-	+	-	+	+	+/-	+/-	+	-	…	+	-	-	+/-	-	+	+	+
	FCTC	Graphic warnings on tobacco packages	-	-	+/-	+/-	-	-	-	-	-	-	-	…	+	-	-	+	-	+	+/-	…
	FCTC	Advertising, promotion & sponsorship bans	-	-	-	-	-	+	+/-	+	+	-	+/-	…	+/-	-	-	-	-	+	+	+
Alcohol
	CC	Written national alcohol policy/action plan	-	-	-	-	+/-	+	-	-	+/-	…	+/-	…	+/-	…	-	+/-	-	-	+	…
	CC	Legal blood alcohol conc (BAC) for driving	-	-	+	-	-	+	-	-	-	…	+	…	+	…	+	-	-	+	+	…
	CC	Sobriety checkpoints/random breath testing	-	-	-	-	-	+	-	+/-	-	…	+	…	+	…	-	-	-	+	+	…
	CC	National anti-alcohol TV/radio campaign in last 12 months	+	-	-	+	-	+	-	+/-	+/-	…	+/-	…	+	…	-	-	-	-	+	…
	CC	Restrictions on sales (hours, days)	+/-	+	+	-	-	+	-	+	+/-	…	+/-	…	+	…	+	+	-	-	+/-	…
Nutrition
	POS; CC	Implement policies to reduce salt	-	-	-	+	-	-	-	-	+/-	+/-	-	…	+	-	-	-	+	+	+/-	-
	POS; CC	Implement national policies to limit saturated fats & virtually eliminate trans fats	-	-	-	-	-	+/-	-	-	+/-	+	-	…	+	-	-	-	+/-	-	-	-
		Implement policies to increase fruits & vegs	+/-	-	-	+	-	+/-	-	-	-	+	+/-	…	+	+/-	-	+/-	+	-	-	+/-
	POS; PAHO	Implement front-of-package labeling for easy ID of unhealthy foods	-	-	+/-	-	-	-	-	-	+/-	-	+/-	…	-	-	-	-	+/-	-	+/-	-
	POS; Hds16	MoH sits in trade negotiations re food security & health goals	+	-	-	-	+/-	-	-	+/-	…	+	+/-	…	-	-	-	-	+	-	…	…
CC; Hds16		Implement WHO recommendations on marketing of foods & beverages to children	-	-	-	+/-	-	-	-	-	+/-	-	-	…	-	-	-	-	+/-	-	+/-	+
POS; PAHO		School feeding programs in line with national food-based dietary guidelines	+	+	+/-	+	+	+	-	+	+/-	+	+/-	…	+	+	+	+	+	-	+/-	+/-
POS; CC		Full implementation of International Code of Marketing of Breast-milk Substitutes	+/-	-	-	+	+/-	-	-	-	-	+/-	+/-	…	+/-	…	…	+/-	-	-	+	+/-
Physical activity
	POS	In the past year, Govt provided additional support for physical activity in schools	+	+/-	-	-	+	+	+	-	+	+	+/-	…	+	+/-	+	+/-	+	+	-	-
	POS	MoEd implement policy requiring 150 minutes of PA/wk for all students	+/-	+	-	+	-	+	-	-	-	…	-	…	+	…	+/-	…	-	-	+/-	…
	POS	Mandatory provision for physical activity spaces in new housing developments	-	+	-	+	-	-	-	-	-	+	+/-	…	+/-	-	-	-	-	-	-	+/-
	POS	Government promotes or incentivizes workplace wellness programs	+/-	-	+/-	-	+/-	+/-	+	+	-	+	+	…	+/-	+	-	+/-	-	-	+	+/-
	POS; PAHO	Population access to “open streets” programs (closed to vehicular traffic)	+	-	-	-	-	+	-	+	+/-	+	+/-	…	+/-	-	-	+/-	-	-	+	+
Education/Promotion
	POS	National wellness TV/radio/social media campaign in last 12 months	+	+	+	+	+/-	+	+	+	+	+/-	+	…	+	+	+	+	+	-	-	+
	POS	CWD multi-sectoral, multi-focal celebrations	+	+	+	+	+/-	+	+	+	+	+	+	…	+	+/-	+	+	+	+	+	+
	FPs	Local NCD research in peer reviewed journal in past 12 months	-	-	-	+	-	-	-	-	…	…	+/-	…	+	…	-	+	-	-	-	+
Surveillance
	POS	STEPS/equiv NCD risk survey in past 7 yrs	+	-	+	+	-	+	+/-	+	+	+	-	…	+	-	-	+	+	+	+	+/-
	POS	Minimum data set reporting in past 2 yrs	…	+	+/-	+	+/-	+	-	+	+	+	+/-	…	+	+/-	+/-	+/-	+	-	+	…
	POS	Global Youth Tobacco Survey in last 5 yrs	-	+	+	+	-	+	-	+	-	+	+	…	+	-	-	+	+	+	+	…
	POS	Global School Health Survey in last 5 yrs	+	+	+	+	-	-	+/-	+	-	+	+	…	+	…	-	+	-	+	+	…
	POS	WHO Country capacity survey in last 5 yrs	+	…	+	+	-	+	+	+	-	+	+	…	+	…	+	+	+	+	+	…
Treatment
	POS; CC	Availability of evidenced-based national guidelines for management of major NCDs	+	+	+	+	+/-	+/-	+	+	+/-	+	+	…	+	+	+	+	+/-	+	+	+/-
	POS; PAHO	National Drug Formulary revised in last 3 yrs to include high quality generics	+	+	-	+	+	-	+/-	+	-	+	-	…	+	+	+	+	-	-	+	+
	CC	Utilize PAHO Strategic Fund for NCD medicines & technologies	+	-	+/-	-	+/-	+	-	+/-	-	…	+	…	+	…	-	…	+/-	-	+	…
	FPs	Universal Health Coverage indicator, no user fee in PHC	-	+	+/-	+	+/-	-	+	+/-	+	-	+	…	+	+/-	+/-	+/-	-	+	+	+
	FPs	Electronic medical records in primary care	-	-	+/-	+	+	-	+	+	…	+	-	…	-	-	-	+	-	-	-	+/-

Source: George Alleyne Chronic Disease Research Centre, University of the West Indies.

^aNotes on the table: 1) The symbols and shading used in the table cells are: = in place; = in process/partial; = not in place; … = no information; the 2017 updates are indicated by the yellow highlighting of the respective country abbreviations. 2) The countries are: ANG = Anguilla; ANT = Antigua and Barbuda; BAH = Bahamas; BAR = Barbados; BEL = Belize; BER = Bermuda; BVI = British Virgin Islands; CAY = Cayman Islands; DOM = Dominica; GRE = Grenada; GUY = Guyana; HAI = Haiti; JAM = Jamaica; MON = Montserrat; SKN = Saint Kitts and Nevis; STL = Saint Lucia; SVG = Saint Vincent and Grenadines; SUR = Suriname; TRT = Trinidad and Tobago; TCI = Turks and Caicos Islands. 3) The other abbreviations used are; CC = WHO country capacity survey; CWD = Caribbean Wellness Day; FCTC = Framework Convention on Tobacco Control; FPs = added by consensus of CARICOM NCD focal points in ministries of health; Hds16 = communique from CARICOM Heads of Government July 2016; MoEd = ministry of education; PA = physical activity; PAHO = PAHO Regional Plan; PHC = primary health care; POS = Port-of-Spain NCD summit Declaration.

The features of the revised grid include 50 indicators (up from the previous 26 indicators). These include: 1) indicators from the 2016 Heads of Government communiqué on NCDs; 2) standardized questions to identify performance; 3) questions from the WHO country capacity survey; 4) indicators from the Framework Convention on Tobacco Control (FCTC); and 5) PAHO or WHO process indicators. For the countries, use of these standardized indicators reduces duplication of reporting efforts. Indicators on alcohol-related harm were added, although this is not part of the Port-of-Spain Declaration.

The indicator on ratification of the FCTC in the version of the grid used in 2008–2015 was dropped in the 2016 revision, since all the independent countries, with the exception of Haiti, have ratified the FCTC. Instead, the four specific FCTC commitment indicators were included, utilizing FCTC language and definitions.

Burden of NCD reporting

As part of the evaluation of the Port-of-Spain NCD Declaration that was carried out by a consortium of researchers, led by the University of the West Indies, in 2014–2017, a 2016 audit was done by author TAS of routine, periodic NCD reporting that regional and international health organizations require from all countries in the Caribbean Community. Those requirements included: a) 3 reports to be made annually (including reporting on the annual evaluation grid); b) 12 reports every two or three years; and c) 2 reports and 5 population-based surveys every five years. That is a total of 22 reports (Table 5).

TABLE 5. Reporting on noncommunicable diseases (NCDs) required of Caribbean Community countries by the Pan American Health Organization (PAHO), World Health Organization (WHO), Caribbean Public Health Agency (CARPHA), and University of the West Indies (UWI) annually or every two, three, or five years, as identified in 2015.

Frequency of reporting	Report	Survey with sampling frame
Annually	1.UWI Port-of-Spain grid
	2.CARPHA to PAHO mortality data
	3.CARPHA CCH 4 (Caribbean Cooperation in Health)
Every 2 years	4.WHO Global Report on the Tobacco Epidemic/monitor implementation of six tobacco control policies/interventions
	5.WHO FCTCa Parties Progress Report
	6.WHO Global Information System on Alcohol and Health (GISAH)
	7.WHO Mental Health Atlas
	8.PAHO Report on Plan of Action for the Prevention of Obesity in Children
Every 3 years	9. WHO country capacity survey	13. PAHO Nutrition Survey
	10. PAHO risk factor regulatory capacity monitoring tool	14. PAHO NCD report (add chronic kidney disease treatment, treated end stage renal disease, social protection, strategic fund for drugs, seatbelts)
	11. WHO Global Status Report on Road Safety - multi-sectoral
	12. PAHO/WHO Global Monitoring Framework (GMF)
		15. PAHO Health Analysis Unit - services & coverage
Every 5 years	16. WHO Global Information System on Resources for the Prevention and Treatment of Substance Use Disorders, every 4 years	18. PAHO STEPS NCD risk survey or equivalent
		19. PAHO Global Youth Tobacco Survey (GYTS)
	17. PAHO coverage of micronutrient supplementation programs	20. PAHO Global School Health Survey (GSHS}
		21. PAHO Global Adult Tobacco Survey (GATS) (or include core GATS policy questions to STEPS surveys)
		22. PAHO nutritional indicators (anthropometric, anemia, vitamin A, breast-feeding)

Source: Study research by author TAS.

^aFCTC = Framework Convention on Tobacco Control.

In addition, there were 3 more reports being developed or being pilot-tested as of 2015: 1) a disability survey to estimate the prevalence of disability within the population; 2) a risk factor (RF) regulatory-capacity monitoring tool to assess country capacity to regulate risk factors; and 3) the Mapping Instrument for NCDs/RFs Surveillance (MAPI), which is a PAHO tool to assess a country's capacity to conduct mortality and risk factor surveillance.

PAHO or WHO require 19 of these 22 reports, which have been developed in response to global and/or regional commitments that PAHO Member States have made. However, due to the small size of many Caribbean countries and territories, and therefore, the small size of their ministries of health, this extensive NCD reporting is a burden. The reporting demands on small island developing states with limited numbers of staff for their small populations is the same as the reporting requirements for large countries. A review of these requirements (especially by PAHO and WHO) might identify a core set of reports, and within them, a core set of questions that could form a minimum data set for reporting from countries with limited human resources..

Conclusions

Despite a heavy burden of required NCD reports, CARICOM countries have invested in and supported annual reporting of the indicators of the Port-of-Spain NCD Declaration. During the 2016 revision, country NCD focal points were actively involved in determining which indicators they wished to report, and in insisting that alcohol-harm indicators be added, although this was not addressed in the 2007 Declaration nor in the 2008–2015 version of the grid. The development, validation, revision, and uptake of the new, 2016 version of the grid indicate country support for this process.

This article validates country self-reports of the Port-of-Spain NCD Declaration progress indicators by using the seven country case studies, which collected objective data to compare with the self-reports. This piece also reports on the process and outcome of the 2016 revision of the grid. This article therefore is likely to contribute to the discourse around methodology and validity of middle-income, small island states with limited human resources. The piece also highlights the burden of reporting for small island developing states, including reporting on the WHO NCD Global Monitoring Framework.