Monitoring health information services: a new tool for information quality review

Alan Sweeting (Project Manager) and Tom Hain (Manager) 
The Centre for Health Information Quality (CHIQ), The Help for Health Trust Highcroft, Romsey Road, Winchester, SO22 5DH, UK. E‐mail: thain@chiq.org

Background

The Commission for Health Improvement (CHI) has a statutory obligation to review progress made in implementing the coronary heart disease (CHD) National Service Framework (NSF). In order to fulfil this obligation CHI, in partnership with the Audit Commission (AC), is planning to undertake reviews of CHD communities within England and Wales. CHD communities comprise a group of interconnected National Health Service (NHS) trusts and primary care organizations.

One of the aims of this process will be to review systematically the quality of information available to patients and the public associated with the CHD NSF, and in order to undertake this review an information quality review (IQR) tool was required for this purpose.

The Centre for Health Information Quality (CHIQ) was established by the Department of Health in 1997, and has a lead role in accrediting patient and public information. ¹ CHIQ is part of The Help for Health Trust, a registered charity.

CHIQ has managed a two‐stage project on behalf of CHI/AC to develop a tool to enable the information quality review to be undertaken, and to be used to undertake pilot evaluation in selected communities in Spring 2003.

Coronary heart disease: information for patients

The CHD NSF was published in March 2000. It sets 12 standards for prevention, diagnosis and treatment of CHD. The standards are to be implemented over a 10‐year period. They are underpinned by a number of ‘fundamental values and guiding principles’ relating to access, efficiency, equity and quality. The NSF also fixed immediate priorities and milestones against which progress would be measured. In July 2000, the NHS Plan identified extra investment for CHD services and set out a number of targets additional to those in the NSF. ²

The patient information needs associated with the CHD NSF will be delivered through the local implementation strategy (LIS) that all NHS trusts should have developed as a response to the NHS information strategy ‘Information for Health’. ¹

Issues to be addressed by the CHI/AC review include how far implementation of the NSF reflects the published ‘guiding values and principles’ and whether these are reflected in the experience of patients and carers. In the case of information, the method identified for achieving this is by systematically reviewing the quality of patient information, as part of the local review process.

The IQR tool has been developed for use during local reviews and will enable an assessment to be made of the quality of information available to patients and carers in reviewed communities.

Guiding values and principles that underpin the CHD NSF are intended to provide the foundation for the development of national policies. Their inclusion helps underpin the drive to raise the quality of clinical care, and reduce variations in access to, and the quality of services.

Guiding values and principles include the following:

• • Information.
• • Quality.
• • Health impact.
• • Public health programmes.
• • Evidence base.

Aims of the IQR tool

The CHD NSF identifies the following goals for information, and it is these goals that define the tests for information quality in the review:

• • consistent, accurate and clear information should be readily available for members of the public about how they can play their part in preventing and managing CHD;
• • health professionals and others should be actively involved in educating members of the public, and should be supported in communicating information about risks and how to reduce them;
• • patients should expect, at every stage of their care, to be treated with respect and sensitivity;
• • they should be provided with accurate, relevant and clear information, so that they and their families can understand the illness and its treatment, and be as involved as they wish in the planning of their care.

Developing the IQR tool

The IQR tool was required for piloting purposes by late November 2002. CHIQ was engaged by CHI/AC to develop the tool in mid‐September. A development programme was proposed and approved, involving a range of stakeholders in the initial scoping phase to ensure patients, carers, health professionals, strategists and communication experts were able to influence the development of the tool.

A two‐stage development plan was approved, with the following components:

Stage 1:

Development:

Scoping, background work, desktop study

Facilitated Consultative Group

Development of model

First draft IQR tool developed

Tool circulated to Consultative Group and CHI, for comment

Stage 2:

Refinement:

Detailed tests developed

Assessment and scoring criteria developed

Second draft IQR tool developed

Tool circulated to Consultative Group, CHI, for comment

Pilot IQR tool available for use.

A health care framework

The CHD NSF introduces the concept of the health care framework (Fig. 1) as a tool to support performance assessment at a local level. This tool combines two groupings together to build a model of conditions and interventions that reflects the interactions between the population as a whole, and the health service.

Figure 1.

A health care framework.

The health care framework provides the model to test the information available to patients, carers, and members of the public across the range of interactions that they may have with the health service related to CHD.

The IQR tool provides the means for CHI/AC reviewers, in consultation with CHD information leads, to map their information inventory against the services a Primary Care Trust (PCT), Primary Care Group (PCG), or health care community as a whole, provides in order to assess the scope of information available.

The benefits of this approach are threefold:

• • A consistent approach to information provision is adopted.
• • A comprehensive appraisal for information associated with services can be undertaken.
• • The scope of information provision against service provision can be assessed.

Information quality review

This review is concerned with evaluating the quality of available information in CHD communities.

The information available within a CHD community should include both nationally produced information and locally produced information.

National and local information

The CHD NSF indicates information that is available nationally to the public should include, ² but not be limited to, the following resources in order to ensure that complete and accurate information on prevention and care is available.

• • NHS Direct telephone service.
• • NHS Direct Online.
• • The National Electronic Library for Health (NeLH).

This is further supported by the CHD NSF Information Strategy (CHDIS). ³

Nationally produced information should be tested for its availability.

The CHDIS develops the actions and time scales necessary for the implementation of the information requirements of the CHD NSF. The responsibility for delivery lies with local information strategy (LIS) teams, who have the following goal:

Local information provision should ensure that patient information covers every stage of the patient pathway, using electronic and paper media. National resources should be tailored to local circumstances if necessary.

Locally produced information relating to CHD should be tested for its information quality.

Using the IQR tool

The four categories of health care framework intervention should be used to assign CHD information available within an NHS trust and ensure that information is tested across the whole range of patient engagement that the CHD community is involved in.

As part of the review information exchange should be identified in the following health care framework interventions:

• • prevention and health promotion;
• • investigation and diagnosis;
• • clinical management;
• • continuing care.

The information that is identified across the health care framework is more likely to range from public health promotion, condition specific information, treatment options and rehabilitation.

Sampling information

CHD information available in a health care community should be mapped against the interventions of the health care framework in order to identify the information associated with each specific intervention.

One information resource should be selected randomly from each intervention for review, although the reviewer should ensure that all types of media identified in an individual trust are reviewed.

This may entail, for example, reviewing a different media type for each intervention.

Using the questionnaire

Questionnaires have been developed that address the key information values of the CHD NSF. Each questionnaire contains specific questions that aim to test the quality of the information addressing a specific issue, and contains both evaluation criteria and scoring, and grading guidance. The questionnaires are:

• • understanding;
• • patient involvement;
• • respect and sensitivity;
• • information quality.

The aim of these questionnaires is to assess the quality of the information exchange, and enable comparative assessment between CHD communities in the England and Wales.

Following discussion with CHD leads the relevant health care framework interventions will have been agreed for the CHD community. The questionnaire matrix indicates which questionnaires should be used to test information relevant to a particular intervention.

The basis of the assessment is that each intervention will be tested with the relevant questionnaires in order to evaluate information quality across the health care framework.

Only one information resource should be assessed, and this should be selected at random. The key points are:

• • some or all of the interventions may be relevant to the CHD community;
• • each intervention may have two or more relevant questionnaires;
• • each intervention is tested individually, rather than an aggregate approach being adopted.

The questionnaire for information quality is a generic questionnaire that can be used to appraise information in any media type.

Scoring

Each questionnaire has both mandatory and additional criteria to be assessed.

All mandatory criteria must be met before a particular information exchange can be graded ‘Good’. Failure to meet all mandatory criteria will result in a lower grade. If all mandatory criteria and a range of additional criteria are met, then a higher grade may be awarded.

The scoring system is summarized in Table 1.

Table 1.

Scoring criteria

Assessment category	Criteria
Very poor	Fifty per cent or less of mandatory  criteria fulfilled
Poor	More than 50%– but less than all  of mandatory criteria fulfilled
Good	All mandatory criteria fulfilled
Very good	All mandatory and 50% of additional  criteria fulfilled
Excellent	All mandatory and between 50 and  100% of additional criteria fulfilled

Questionnaires

Each of the four questionnaires has its own range of scores for the review, dependent on the number of mandatory and additional questions within the questionnaire. Each questionnaire includes specific scores and guidance on which overall category should be given Finally, the scores, both for mandatory and additional questions, and overall grade, should be recorded on each questionnaire.

Conclusion

Health services are increasingly expected to provide the highest quality information, delivered appropriately to the consumer. The IQR tool equips reviewers of information services to assess information services in systematic fashion.

The IQR tool will be subject to on‐going re‐development. As the tool is CHD‐specific, it will require re‐designing to support implementation of other NSFs. The most recent version of the IQR tool can be found at the CHIQ website (http://www.hfht.org/chiq).