Table 2.
Application of the Open Quality tool in an a household survey (data collection step)
| Failure modes | Data quality attributes | Standards/criteria | Preventive strategies | Verification activities | Corrective actions |
|---|---|---|---|---|---|
| 1. What can go wrong? | 2. Which data quality attribute will be affected if this goes wrong?a | 3. How can we determine (measure) if we have fulfilled the attribute? | 4. How can we prevent things from going wrong and ensure we fulfill the attribute? | 5. How can we check whether we are on track to fulfill the attribute?b | 6. What should we do to correct things if we are not on track? How can we prevent it from happening again?c |
| Data fabrication | Accuracy (and credibility) | All fields in the questionnaire should be filled in with information genuinely observed or provided by the respondents | Use tool for electronic data collection (EDC) with tablet, program start and end time of the interview and collect GPS location of households visited to be reviewed regularly by survey team |
100% daily review of questionnaires by supervisors Random spot-checks by supervisor 10% household revisited/call back survey independently by a team of independent monitors |
Replacing teams that are not functioning well (with 'reserve' interviewers not part of the initial team) |
| Interviewers do not fill in questionnaire completely | Completeness | Only completed questionnaires shall be uploaded |
Built-in EDC functionality whereby data cannot be sent if questionnaire is incomplete Built in EDC functionality whereby cannot proceed to next questions if all previous not completed |
||
| Interviewers do not visit all households in the sample (only those that are easier to access) | Completeness | All sampled households should be visited except if they are in a cluster excluded from the sample for security reasons | Daily plans for each supervisor, submitted to field managers | 100% review of incoming data on weekly basis compared to targets | Immediate contact with survey manager in case targets are not being met |
| Responses to various questions are not coherent | Coherence | Answers to related questions should be coherent [select related questions] | Built in EDC functionality with consistency checks between responses to selected questions, prompting interviewer to double-check responses and ask for clarifications to the respondent | ||
| Data cannot be uploaded to the server (no internet connection) | Accessibility (and timeliness) | Data should be uploaded on the same day or next day at the latest | Each tablet has a sim-card with data bundle to send data in case wireless connection cannot be obtained | 100% review of incoming data on weekly basis compared to targets | Immediate contact with survey manager in case targets are not being met |
aHere we refer to the OECD data quality dimensions [17] but other data quality frameworks can be used
bThis field may not always be applicable
bOnly applicable if a control practice is defined