CLEAR | COMP. | VALID | FEASIBLE | SAMPLE STATEMENTS FROM EXPERT | |
---|---|---|---|---|---|
Framework | Y | Y | Typically, a researcher will evaluate data quality/availability and develop a research design appropriate for the data. Recording a diagnosis code for "diabetes" in an EHR during a visit in which the clinician orders a test for diabetes is not necessarily an error, it might be a local policy that all diabetes tests ordered get coded with that diagnosis. I want to know whether my cohort has the right data available for a study, during the study period. It is one question for me, not three. These recommendations…are where this comes to life. |
||
Constructs | 0/3 | 0/3 | |||
Cells | 4/9 | 1/3 | 3/9 | ||
Recs. | 8/9 | 8/9 | |||
Framework | N | N | I think missing from the framework is actually the frame—when in the research process are we supposed to use this? It seems to be aimed at the analysis of a data set—after the data-collection process has been specified. [T]he concept missing for me is my data-quality workflow, as a quality assessor or researcher. [Regarding] the "pathway" of data from physical event to recording in the dataset. These three operationalizations don't cover all of them, so I presume you are making a choice based on some sort of tradeoff, having to do with ease of checking. |
||
Constructs | 0/3 | 0/3 | |||
Cells | 1/9 | 0/3 | 6/9 | ||
Recs. | 5/9 | 6/9 | |||
Framework | Y | N | Again it is contextual—fitness for purpose definition. But overall the logic of self-assessment and self-determination of what "sufficient" is makes sense. Progression on data over time reflects clinical course and will vary depending on a number of diagnostic, management and prognostic factors. So need constraints in framing the research question(s). I think the realist approach should be emphasized, i.e. the importance of context. The issue of "actors" is another important scope question as EHR-based research can be used for research about the care provider and interventions as well as impact on patients. |
||
Constructs | 3/3 | 2/3 | |||
Cells | 6/9 | 0/3 | 6/9 | ||
Recs. | 7/9 | 9/9 | |||
Framework | Y | N | I don't see anything to address the quality issue of, "Are the right patients included in the data?" Perhaps this is more of a research question…but it seems to cross into the data quality boundary when someone attempts to use the data for something that's not fit for purpose. [I]t feels as if [completeness] depends on the 'task at hand.' If the goal is to estimate an effect, then completeness requires that the estimate can be generated without bias due to confounding. [Using an] external reference is a good idea, but practically is quite difficult, both in terms of logistics and methodologically ensuring that the external reference should be comparable to the source population. |
||
Constructs | 0/3 | 2/3 | |||
Cells | 9/9 | 3/3 | 9/9 | ||
Recs. | 4/9 | 3/9 | |||
Framework | N | Y | Each construct seems like it should be followed by the term "for the task at hand." [The completeness across patients recommendation] may be difficult for larger datasets, composite variables, and deciding when to do this… where does this get represented? [For the current across patients recommendation,] this is clear—I'm not sure how feasible it is. [For the current across time recommendation,] without metadata for recording data I'm not sure how feasible this is. |
||
Constructs | 2/3 | 3/3 | |||
Cells | 7/9 | 3/3 | 8/9 | ||
Recs. | 8/9 | 7/9 | |||
Framework | Y | Y | I found the questions to be a nice way to frame the framework! I think the questions will help users understand the framework and the subsequently presented Recommendations. [Y]our framework fits into the larger picture of data quality, and into the frameworks created by others. These Guidelines beg the question of how these fit together. What about single-site versus multiple-site EHRs? What is the bigger picture? Does presentation of the final version of your Guidelines call for a short description of context? |
||
Constructs | 3/3 | 3/3 | |||
Cells | 9/9 | 3/3 | 9/9 | ||
Recs. | 9/9 | 9/9 | |||