Skip to main content
. 2019 Sep 29;15(3):e1045. doi: 10.1002/cl2.1045
Description Question Coding
Report identification Unique study identification # For example, PES001
First author—impact evaluation Surname Surname
Other papers used for coding First author surname and type of paper of any qualitative, descriptive quantitative, process evaluations or project documents used for coding
General comments (1) General comments: any general comments on study not coded elsewhere (2) Issues of comparability: please report any potential issues of comparability between different documents (e.g., different documents assess a programme/intervention at different scales [geographic/time scale]). If the issue of comparability related only to a certain secion of a document (e.g., cost data), please put in brackets in relevant cell Open answer
Publication date Year (letter) XXXX (a)
Publication type What is the impact evaluation publication type?

1 = Peer‐reviewed journal

2 = Book chapter/book

3 = Conference paper

4 = Organisation report

5 = Working paper

6 = Implementation document

7 = Other grey

8 = PhD thesis/dissertation

Funding agency Who is funding the evaluation/study?

1 = Public institution (e.g., govt, NGO, university, research institute)

2 = Private institution (e.g., private company)

3 = Multilateral Organisation (World Bank, UN)

4 = Foundations

8 = Not clear

9 = Not applicable (Non‐funded)

Name of funding agency Please add name of the agency funding the evaluation Open answer
Independence of evaluation What level of independence is there between the impleenting agency and study team?

1 = Funding and author team independent of implementers/funders of programme

2 = Funding independent of implementers/funders of programme, but includes authors from funder/implementer

3 = Evaluation funded and undetaken by funders/implementers

8 = Unclear

Independent data collection Has the data been collected by an independent party? 1 = Yes, 2 = No, 8 = Not clear
Conflict of interest Is there a potential conflict of interest associated with study which could influence results collected/reported? (e.g., is there a declaration of conflict of interest? Is any of the authors related in any way to the funding or implenting institution?) 1 = Yes, 2 = No, 8 = Not clear
Comments on conflict of interest Please add reason for your answer to whether there is a conflict of interest Open answer
Language of publication Language of publication of the impact evaluation, for example, Spanish, English and for forth Open answer
Other methods If the impact evaluation addresses other questions than effectiveness note questions and methods used here Open answer (this will include, e.g., mixed‐methods to assess implementation, adherence, participant views etc.)
Intervention descriptives Programme or project name State the programme or project name. If no name, then list the location (e.g., Town, village etc.) Open answer
Intervention type Indicate type of intervention

1 = PES alone

2 = PES + other intervention

Type of ecosystem targeted Indicate the type of ecosystem targeted

1 = Forests

2 = Farmland

3 = Grassland

4 = Mangroves

5 = Wetlands

Intervention description Provide descriptive details about the intervention. Include detail on any other intervention provided alongside the PES, including alternative livelihoods strategies, awareness raising activities, increased forest monitoring etc. Open answer
Objectives of intervention Type of objective(s) of intervention

1 = Conservation only

2 = Restoration

3 = Environmentally beneficial/preferable to BAU land‐use

4 = Socioeconomic (livelihoods, poverty reduction etc.)

5 = Other (add description in comments)

Objectives of intervention State any objectives stated in study or project document, including whether the study targets both environmental and poverty objectives
Size of payment Indicate the size of the regular payment Open answer, $
Frequency of payment Indicate how frequently the payment is made (annual, monthly, etc.) Open answer
Method of payment Indicate how payment made to participants Open answer
Conditionality Indicate the stated conditions of the PES programme Open answer
Intervention scale What is the scale of the intervention?

1 = Local

2 = Regional

2 = National

Intervention implementing agency Who is implementing the intervention? State the name (and department) of the implementing agency Open answer
Intervention funding agency Type of funder

1 = Government

2 = User financed (companies using env service)

3 = NGO

4 = Multilateral/bilater organisation

5 = Carbon offset mechanism

6 = Other

Intervention funding agency Name of intervention funding agency Open answer
Intervention target group What were the characteristics of beneficiaries used to target the intervention? Open answer
Targeting methods How were beneficiaries targeted for the programme (e.g., how was the targeting implemented)? Open answer
Intervention start Start date (if not stated, state study date) of intervention XX/XXXX
Intervention end State end date (if ongoing state ongoing) XX/XXXX
Follow up How long after the last payment was outcome data collected? indicate number of months (numerical only). If not clear state so
Program theory Do the authors make explicit reference to program theory, theory of change or similar? 1 = Yes, 2 = No, 8 = Not clear
Program theory Report any description/statement of program theory as stated by author(s). Open answer
Context Country List countries the study was conducted in Country 1, Country 2, and so forth.
Detailed location If provided, give detailed information on where the study took place within a country, for example regions/districts covered Open answer
World Bank Region Select region(s) the study was conducted in according to World Bank. For more info on region classification see http://data.worldbank.org/country

1 = East Asia & Pacific

2 = Europe & Central Asia

3 = Latin America & Caribbean

4 = Middle East & North Africa

5 = South Asia

6 = Sub‐Saharan Africa

WB Income category Select the World Bank income classification of the country at the time of the study

1 = Low income country

2 = Lower‐middle income country

3 = Upper‐middle income country

REDD+ status Is the country where the evaluation took place a REDD+ country? 1 = Yes, 2 = No, 3 = Unclear
Environmental performance index How does the country rank on the Environmental Perfomance Index: http://epi.yale.edu/? Open answer—to be filled in after coding complete
Baseline deforestation rates Report any data/description on deforestation rates in programme/comparison area Open answer
Baseline socioeconomic status of participants Report any data/description on baseline socio‐economic status of participants Open answer
Property right regime Report any description in the primary evaluation or qualitative documents of the existing property rights regime Open answer
Process and implementation Information about program take‐up/adherence (among beneficiaries)

Is there any information about program take‐up/adherence (among beneficiaries)?

Commentary by authors should be used when information on program adherence and so forth. is not backed up by some sort of research/when the authors do not report that/how they collected data to assess these areas

1 = Yes, commentary from author; 2 = No; 4 = Yes, formally assessed
Methods of assessing take‐up/adherence Which methods are used to assess program take‐up/adherence?

1 = Observation by intervention staff

2 = Reporting by participants

3 = Other

4 = Commentary from author

9 = Not measured

Results of the assessment of take‐up/adherence What is the result/information provided of the assessment of program take‐up/adherence? Open answer
Information about implementation fidelity/service delivery quality

Is there any information on implementation fidelity/service delivery quality?

Commentary by authors should be used when information on program adherence and so forth. is not backed up by some sort of research/when the authors do not report that/how they collected data to assess these areas

1 = Yes, commentary from author; 2 = No; 4 = Yes, formally assessed
Methods of assessing intervention fidelity Which methods are used to assess implementation fidelity/service delivery quality

1 = Observation by intervention staff

2 = Reporting by participants

3 = Other

4 = Commentary from author

9 = Not measured

Results of the assessment of intervention fidelity What is the result/information provided of the assessment of implementation fidelity/service delivery quality Open answer
Other description of process factors Any other description of process factors not covered above Open answer
Barriers and facilitators Do the study identify any barriers and facilitators not included above? Open answer
Cost Cost Are any unit cost data/cost‐effectiveness estimates provided? 1 = Yes. 2 = No
Cost details If yes, report any details of unit cost and/or total cost. Please also report year and currency Open answer
External Validity Length of study Length of study in months (where study length not reported, code as length of intervention, noting that in brackets) # months, if not reported N/A
Efficacy or effectiveness trial Was the intervention implemented under "real world" conditions? By real world we mean a programme implemented independently of the evaluation, either by government, NGO or international agency. For example, the programme is not designed and implemented for the purpose of research 1 = Yes, 2 = No, 9 = N/A
Personell implementing the programme Who was in charge of implementing the program? 1 = PI/researchers (study authors); 2 = implementing agency staff, 3 = external agency (eg: survey firm); 4 = others; 8 = not clear
Sampling frame for the study State the sampling frame (list of all those within a population who can be sampled, i.e., households, communities) for selection of study participants (i.e., census, etc.). Open answer
Author discussion of external validity Do the authors discuss or explicitly address generalisability/applicability? Open answer
Theory Is there any reference to theory of change underlying intervention? 1 = Yes, 2 = No, 9 = N/A
Theory based evaluation Is the study using theory to inform the evaluation design and analysis? Open answer—describe if and how the authors use theory in the evaluation. Do they for example use it to inform data collection? Do they do any causal chain analysis?
Equity Consideration of equity Does the study consider equity? 1 = Yes, 2 = No
Equity methods How does the study consider equity?

1 = intervention target a disadvantaged group

2 = study measures inequality

3 = subgroup analysis by dimension of inequity

Equity dimension What dimension(s) of equity does the study consider?

1 = gender

2 = socioeconomic status

3 = place of recidence

4 = land ownership

5 = landsize