Table 5.

Catalogue of encountered problems

Problem	Explanation
Data not available in	Data items required to compute many of the indicators, such as those contained in the pathology reports, were only
structured format	available in non-structured free text, and therefore not directly (re)usable. Also structured data to exclude patients based on
	the exclusion criteria recurrent carcinoma and TEM-resection as well as ‘resection’ via colonoscopy was not available in our EMR
	nor in the DSCA dataset. Non-recorded exclusion criteria can lead to lower indicator results, wrongly underestimating the
	quality of care for indicators whose percentages are to be maximised [16,17].
Incorrect data items	The double data entry in our case study helped us to discover incorrect data items. Furthermore, we identified imprecise
	and/or incorrect diagnosis codes in our EMR.
Incomplete view of	Hospitals throughout the country refer patients to our hospital, which specialises in gastro-intestinal oncology. Some of
patient history	these patients are only treated for a short time, and then referred back. Likewise, our hospital maintains an alliance with a
	nearby hospital. Referral letters are typically posted as physical letters, making a complete, consistent view on a patient’s
	history difficult to obtain. For example, it is hard to retrace whether preoperative imaging of the colon has taken place in
	another hospital.
Lack of relations	Our EMR does not store any relations between diagnoses and procedures, making it impossible to select the diagnosis that
between data items	was the underlying reason for a procedure. For example, the lymph node indicator should only select lymph node
	examinations that have been carried out in the context of a primary colonic carcinoma, and not, for example, a previous
	mamma carcinoma. As a partial solution, we imposed the constraint that the diagnosis should have been established before
	the related operation was carried out, which resulted in some missed patients.
Lack of detail	None of the diagnoses in the EMR was detailed enough to meet the information required by the indicators, which include
	patients with primary colonic and rectum carcinomas. The only relevant diagnoses in the EMR were malignant neoplasm
	of colon, rectum and rectosigmoid junction. Therefore, the concepts employed in the queries to compute the indicators
	had to be generalised. Furthermore, only the type of endoscopies is registered, such as colonoscopy, but not whether the
	complete colon is affected.
Lack of standardisation	For example, the urgency of an operation is defined in the EMR according to 8 categories, but the DSCA dataset only
	differentiates urgencies according to 4 categories. It was not clear how these categories should be mapped, as their
	meaning was not unambiguously described (for example, one of the categories was called “extra”).