Feasibility, understandability, and usefulness of the STEP self-rating questionnaire: results of a cross-sectional study

Abstract

The study was designed to evaluate the acceptance of the self-rated version of the Standardized Assessment of Elderly People in primary care in Europe (STEP) by patients and general practitioners, as well as the feasibility, comprehensibility, and usefulness in gaining new information. In all, 1007 of 1540 patients aged 65 and above, from 28 different Saxon general practices took part. We recognized that 96% of the patients were able to fill in the questionnaire by themselves. It took them an average of approximately 20 minutes to do so. Further analysis of 257 randomly selected patients identified 281 previously unknown problems (1.1 per patient). In the practitioners’ opinion, 16% of these problems, particularly physiological and mental ones, could lead to immediate consequences. Remarkably, newly identified psychosocial problems were not followed by any consequences. Fourteen of the 75 questionnaire items were not answered by more than 9% of the participants. Eight of the 14 frequently unanswered items were marked as difficult to understand by the patients. Altogether the self-rating version of the STEP was found to be feasible and useful. It was well accepted among patients; however, some questions need further review to improve their comprehensibility. Furthermore, it should be investigated why some identified problems do not have consequences and whether there is a need to record these issues at all.

Introduction

In general practice, geriatric screening can detect unidentified problems. It has been seen that once detected and dealt with, a high proportion of previously unknown problems show improvement.¹ A meta-analysis has shown that geriatric assessment exerts certain benefits toward the survival and function of older adults.² Thus, geriatric assessment should be a tool of every general practitioner. Despite this and the fact that geriatric screening and assessments have been described for a long time,^2–4 in Germany, only a minority of general practitioners regularly uses geriatric assessment tools.⁵

However, geriatric assessment instruments and guidelines are heterogeneous with respect to content and quality.⁶ In general, geriatric assessment should be effective, feasible, and acceptable throughout general practice. To achieve this, a consensus-based guideline and systematic review of the evidence were prepared by an expert panel of general practitioners (the core group) and epidemiologists/geriatricians. The Standardized Assessment of Elderly People in primary care in Europe (STEP) group identified eight health domains to be considered in a preventive care assessment: client’s perspective and attitudes, physical state, functional state, significant symptoms, mental function, social circumstances, medication, and primary preventive care issues.^7–9 The complete STEP contains a self-rating questionnaire for the patient as well as a structured examination carried out by the practice nurse and general practitioner. The general practitioner briefly evaluates the overall health, discusses preliminary results, and initiates interventions, if necessary.¹⁰ Earlier studies,^1,10 using an earlier version of the questionnaire (see Methods), found that the STEP revealed low to moderate rates of previously unknown problems and impairments.

Our investigation set out to determine whether the self-rating version of the STEP is accepted by patients and general practitioners, whether it is feasible to use, understandable, and useful for gaining new information. Since this recent investigation was focused on the self-rating questionnaire, a physical examination of the participating patients was not performed.

Methods

Participating general practitioners

We contacted 51 general practitioners in a rural (southwest Saxony) and urban (Leipzig) area of Saxony, by personal contact or mail. In total, 28 general practitioners agreed to participate in the study, after one reminder. The general practice nurses were informed about the process of the study and taught how to use the STEP questionnaire.

Participating patients

All patients older than 65 years were eligible for inclusion in the study. The patients had to be able to present to the practice for an encounter. There were no exclusion criteria except for being obviously unable to complete the self-rating questionnaire (eg, because of physical or cognitive handicaps, such as severe visual or cognitive impairment, terminal illness, obvious writing impairment). The patients completed the self-rating questionnaire either immediately in the general practice office or at home and returned it at their next visit. For each general practice office, the number of patients who refused to participate was recorded. The sex and age of the patients who explicitly declined participation was also recorded.

Characterization of the patients

We asked 1540 patients to participate and received 1007 completed self-rating questionnaires (response rate 65.4%); 133 patients explicitly declined participation, and 400 self-rating questionnaires were not returned. Twenty self-rating questionnaires lacked information about age and ten about sex. The sociodemographic characterization of the participants is given in Table 1. Of the 133 patients who explicitly refused to participate, the nurses documented the age of 57 and the sex of 68 patients. The sex and age distribution of the patients who declined participation was not statistically significantly different from those who participated (P = 0.864 and P = 0.156 for sex and age, respectively).

Table 1.

Sociodemographic description of the participating patients

Sociodemographic variable	Males	Females
Age [years]
Mean ± SD	72.61 ± 5.34	73.24 ± 5.85
Range	65 to 90	65 to 94
Age distribution
65 to 69	141 (35.6)	198 (33.5)
70 to 74	116 (29.3)	150 (25.4)
75 to 79	87 (22.0)	154 (26.1)
80 to 84	47 (11.9)	70 (11.8)
≥85	5 (1.3)	19 (3.2)
Marital status
Married	344 (87.3)	311 (53.7)
Widowed	37 (9.4)	195 (33.7)
Single	13 (3.3)	73 (12.6)
Children
0	42 (10.9)	82 (14.1)
1	123 (31.9)	198 (34.0)
≥2	221 (57.3)	302 (51.9)
Living conditions
Own flat	367 (93.4)	554 (95.2)
With family	26 (6.6)	28 (4.8)
nursing home	0 (0.0)	0 (0.0)

Notes: n = 987; females: n = 591 (59.9%). If not stated otherwise, all values are absolute numbers, and the corresponding frequency is given as a percentage.

Design of the self-rating questionnaire

The content of the original STEP questionnaire was published by Junius et al¹⁰ and Sandholzer et al.⁹ In our study, the layout was modified and an introductory page was added that contained information about the aims of our investigation. Ten fields of interest (personal data, diseases and medications, physical fitness and activities of daily life, social environment, general complaints, physical symptoms, mood, memory, health behavior, and hospital treatment) were highlighted, with a total of 75 questions organized into 39 complexes. These ten fields of interest covered the eight health domains that should, according to the STEP group, be considered in a preventive care assessment. Fifty-three of the questions were dichotomous, eight had to be answered in one’s own words, ten could be rated on a three- to six-point scale, and four of the questions could be answered with a catalogue of three to eleven keywords. Some of the question complexes started with a filter question. On the last page, patients were asked to provide additional information with a questionnaire containing seven questions relating to: length of the self-rating questionnaire (too short, adequate, too extensive), the need for help in completing the self-rating questionnaire (yes, no), and the understandability of the self-rating questionnaire (good, number of poorly understandable questions), as well as relating to the need to ask about living arrangements (necessary, superfluous), perception that the general practitioner knows the health problem (good, fair, not), the level of discussion of the identified health problems with the general practitioner (completely, partial, none), and the time needed to complete the self-rating questionnaire (estimated in minutes).

Assessing feasibility

The patients were asked how they were able to handle the STEP self-rating version and how long it took them to complete the STEP self-rating questionnaire. Additionally, we created a questionnaire for completion by the nurses that consisted of three questions, with a four-point scale, concerning the disruption of daily routine caused by the STEP self-rating questionnaire, additional workload due to the distribution of the questionnaire, and the effort needed to motivate patients to participate.

Assessing understandability

We used three criteria to identify questions of poor understandability. First, we assessed the frequency with which each question was left unanswered by the patients. Second, concordant questions that were obviously incorrect or contradictorily answered by the patients were suspected of being poorly understandable. The last criterion for poor understandability was the report by patients that the question was poorly understandable.

Assessing usefulness

Twenty-six of the 28 participating general practitioners agreed to perform a case discussion. About ten cases per general practitioner (in total 259 cases) were selected randomly. Two cases dropped out because of transfer to another general practitioner. Conclusively, 257 interviews with the general practitioners were conducted and documented verbatim on a standardized form, to compare the questionnaire data to the previous knowledge of the general practitioner. New information or health problems were documented. It was also recorded whether these problems had consequences or not.

Assessing acceptance

One observer talked to patients, nurses, and physicians about their acceptance of the STEP self-rating version.

Statistical analysis and graphical presentation of the data

Statistical analysis was performed with SPSS® Statistics 18.0 Software for Windows (IBM®, Armonk, NY, USA). We determined absolute and relative frequencies. Comparisons for the categorical variables were performed by using the χ²-test and the exact test with the Monte Carlo method when needed. Differences were considered to be statistically significant for P < 0.05.

Results

Assessing the feasibility of the STEP self-rating questionnaire

The vast majority (95.8%) of the participating patients were able to complete the questionnaire on their own. However, the frequency of patients who could not complete the questionnaire on their own increased statistically significantly with age (age 65 to 69: 1.2%; age 70 to 75: 4.3%; age 75 to 79: 3.9%; age 80 to 84: 10.1%; and 85 years and older: 22.7%) (P < 0.001). The main difficulties experienced in completion arose from problems with vision (12/24), understanding (8/24), writing (3/24), and re cognition (1/24). The mean time for completing the questionnaire increased statistically significantly with age (age 65 to 69: 17 minutes; age 70 to 74: 19 minutes; age 75 to 79: 22 minutes; age 80 to 84: 25 minutes; and 85 years and older: 24 minutes) (P < 0.001). The use of the STEP self-rating questionnaire caused no burden in 46.2% of the general practice offices and little burden in 53.8%, due to the need to explain the study to the patients. We found that 81.5% of the nurses were able to integrate the STEP self-rating questionnaire without problems during consultation hours, while 18.5% felt somewhat bothered.

Assessing the understandability of the STEP self-rating questionnaire

Fourteen of the 75 questions were unanswered by at least 9% of the patients (Table 2). The questions that were marked by the patients as poorly understandable were the same as those with a high percentage of answers omitted. Questions containing filter questions, comprising subquestions, or requiring scaled answers were most likely to be unanswered or were frequently (>5%) contradictorily or apparently incorrectly answered (eg, “highest physical effort you could stand for 2 minutes during the last 2 weeks:” 5%; “use of hearing aids” and “satisfying hearing with hearing aids:” 27%; and “feeling that memory gets worse” and “being concerned about worsening memory:” 12%). At least 10% of the patients did not complete the address, the diseases, the blood pressure, and the pulse. Also, the questions related to over-the-counter drugs, social problems, and the consumption of alcohol were often unanswered (>9%). In the 5-year age groups from 65 to 84, 6.5%–11.1% of the participating patients reported a general poor understandability of the STEP self-rating questionnaire. Four of 19 patients older than 84 years (21.1%) reported a poor understandability.

Table 2.

Absolute numbers and percentage of items that remained unanswered and of items that were marked as difficult to understand by participating patients (n = 1007)

Subject area and item number	Item
	Not answered		Marked as difficult
	n	%	n	%
Personal data
1. Address	114	11.3	0	0
Diseases and medication
2. What of your diseases does your general practitioner treat?	130	12.9	0	0
3. Over-the-counter drugs	261	25.9	6	7.0
Number of drugs taken	103	10.2	0	0
4. Difficulty in doing usual daily activities	<91	<9.0	2	2.3
Physical fitness
5. Highest physical effort for 2 minutes during last 2 weeks?	221	21.9	33	38.4
6. Problems with IADL/ADL	<91	<9.0	3	3.5
Social environment
7. Do you care for someone who depends on your assistance?	183	18.2	0	0
Does someone care for you in case of illness or emergencies?	108	10.7	0	0
Is there a person you confide in?	98	9.7	0	0
9. List of problems concerning living conditions	<91	<9.0	5	5.8
Physical symptoms
11. Situations with dyspnea	<91	<9.0	22	25.6
Walking	118	11.7	ne	ne
Rest periods	350	34.8	ne	ne
At night in bed	325	34.8	ne	ne
18. What was the last blood pressure value?	177	17.6	0	0
19. Was the pulse arrhythmic?	142	14.1	0	0
29. Are there difficulties with hearing (even with hearing aid)?	490	48.7	0	0
Health behavior
36. Did you ever feel you drink too much alcohol?	128	12.7	17	19.8
Were you ever criticized for your drinking behavior?	129	12.8	0	0
38. Hospital treatment in the last year: inpatient, outpatient, ED?	<91	<9.0	3	3.5

Notes: Items remained unanswered by more than nine percent of participants. Items were marked as difficult to understand by 86 participants.

Abbreviations: ADL, activities of daily living; ED, emergency department; IADL, instrumental activities of daily living; ne, not estimated.

Assessing the usefulness of the STEP self-rating questionnaire

The general practitioners thought that the health state of 64.5% of the older adults was not or was only slightly impaired. A further 29.1% were thought to be chronically ill. During the discussion of the 257 cases, 2170 health-related problems or issues were detected. Of these, 281 (13.2%) were not previously identified by the general practitioners. A mean 1.1 health problems per discussed case were new for the physicians. The general practitioners stated that 41 (16.4%) of the 281 new health problems had a consequence. These findings are presented in detail in Table 3.

Table 3.

The frequency of newly identified health-related problems (n = 281) and the percentage of problems that could have consequences in a subsample of n = 257 randomly selected patients

Health-related problems	Total	New		Consequences
	n	n	%	n	%
Drugs
Number of drugs used	235	9	3.8	0	0
Over-the-counter medication	63	16	25.8	0	0
Functional state
Problems with IADL/ADL	67	10	15.2	0	0
Difficulty in doing usual daily activities	117	6	5.3	0	0
Decreased physical fitness	94	3	1.3	0	0
Social environment
Financial difficulties	21	9	42.9	0	0
Social isolation	117	33	28.7	0	0
Problems concerning living conditions	63	14	25.0	0	0
Medical/social care
Use of services	79	6	7.9	0	0
Flu vaccination	49	6	14.0	0	0
Hospitalization	82	2	2.5	0	0
Physical and mental/emotional complaints
Teeth or mouth complaints	68	33	49.3	12	37.5
Falls	52	13	25.0	7	53.8
Urinary incontinence	81	27	34.2	9	36
Memory impairment	81	21	25.9	1	5
Digestive problems	33	4	12.1	1	33.3
Visual problems	30	5	17.2	0	0
Sleep disorder	107	10	9.4	3	33.3
Weight loss	20	2	10.5	0	0
Hearing problems	19	1	5.3	0	0
Depressive mood	58	5	8.8	0	0
Dyspnea	84	8	9.5	1	16.2
Elevated blood pressure	6	0	0	0	0
Pain in the chest	89	11	12.4	2	28.6
Pain in the legs while walking	90	5	5.6	1	50
Joint problems	157	5	3.2	3	60
Pain	144	6	4.2	1	33.3
Health behavior
Physical inactivity/unhealthy nutrition	40	11	30.6	0	0
Tobacco or alcohol abuse	24	0	0	0	0

Abbreviations: ADL, activities of daily living; IADL, instrumental activities of daily living.

Assessing the acceptance of the STEP self-rating questionnaire

The self-rating version of the STEP questionnaire was reported to be easily integrated in the consultation hour. The content and the length of the STEP self-rating questionnaire were acceptable for 89% of the participating patients. We found no relevant age-dependent differences. However, only 6.8% of the patients reported thinking that their general practitioner did not or did not completely know their health problems. A further 16.2% of the patients reported that they had not at all or only incompletely told their general practitioner about the identified problems. In general, one-third of the general practitioners reported they would use the STEP self-rating questionnaire and 15.4% would only use it for selected patients. The most frequent reasons for not using the STEP self-rating questionnaire were the loss of time (92.9%) and the small amount of new information provided by the questionnaire (23.1%).

Discussion

To the best of our knowledge, we performed the first cross-sectional investigation of the STEP self-rating questionnaire with a sufficient sample size in a German general practice setting. However, physician and staff acceptance of the approach of patient self-evaluation of geriatric problems has not yet been explored.

The only comparable studies were performed by Junius et al¹⁰ and Piccoliori et al¹ who used STEP in German and South Tyrolean general practices.^1,10 The study by Junius et al was carried out in a very small group of patients (n = 62), from ten general practitioners.¹⁰ Later Junius et al undertook another study with a larger group of patients (n = 466), from 67 German general practitioners (Piccoliori et al).¹ On average, they found four new problems per patient, compared with 1.7 new problems reported by Piccoliori et al and 1.1 identified in our study. However, the small sample size (n = 62) of the first study limits the comparability of the results with other investigations. Our findings and those of Piccoliori et al support the fact that using a geriatric assessment in primary care discloses relevant health problems and treatment needs that general practitioners otherwise may overlook.^1,11 The earlier investigations suggest that the number of unknown problems decreases with the time the patient is known to the general practitioner.¹ It seems to be conclusive that the self-rating STEP questionnaire is particularly useful with previously unknown patients, patients known for only a short time, or patients irregularly encountered. However, it should be kept in mind that there is a low agreement on health and treatment priorities between patients and physicians that demands better communication between the two parties, to strengthen mutual understanding.¹² This was also shown by Piccoliori et al.¹ They found that 17% of all diagnosed problems were new ones, whereas, in our investigation these amounted to 13%. Furthermore, they stated that for half of the newly identified problems, general practitioners and patients agreed in their view about the relevance of the problem (24% of the newly identified problems were relevant for both, and 27% were unimportant for both). Less than a fifth (18%) of all newly identified problems were relevant for the general practitioners, while a third of all newly identified problems were relevant only for the patient (31%). The upper findings of 42% of newly identified problems being relevant for the general practitioners are not in accordance with our findings that about 16% of the newly identified problems were stated to be relevant by the general practitioners. Piccoliori et al¹ found that a high proportion of the previously unknown problems showed improvement. However, the earlier reported results^1,10,13 as well as our own recent findings also suggest that STEP does not only identify problems that may be successfully treated: there was a high percentage of new socioeconomic problems identified as well as problems of health behavior (see Table 3 for details), although these might not have any consequences. It is possible that the general practitioners may have perceived that they would not be able to intervene in such problems.

The amount of time to complete the STEP self-rating questionnaire was concordant between our and other studies. However, other groups were not able to tell what percentage of patients was able to complete the questionnaires by themselves, for eg, Piccoliori et al¹⁴ asked patients to complete any unanswered questions, to enforce the completion of questionnaires. In contrast to our study, earlier studies did not provide any information regarding patient acceptance of the STEP self-rating questionnaire. In addition, we reported, for the first time, facts about the understandability of the STEP self-rating questionnaire. Our study points out that the self-rating STEP questionnaire was generally feasible and accepted by a majority of older patients, in a routine general practice setting. We provided detailed information on single items of the self-rating STEP questionnaire that were based on a sufficient sample size. Furthermore, our study provides suggestions for further research for improving the self-rating STEP questionnaire.

Comparable with other investigations,¹⁵ the cooperating general practitioners cannot be assumed to have been a representative sample of the total Saxon general practitioners. The study design intended to provide a representative sample of older patients, including all patients older than 65 years, who consulted the general practice office within a period of 3 months. Therefore, the sample cannot be assumed to be representative for the whole population of older adults that the general practitioner cares for. Due to the high amount of work, most general practices participated for only a few weeks. Therefore, the investigation ended after receiving about 1000 completed questionnaires. Detailed information about the nonresponders was unavailable for reasons of data security, and the overly large effort expended by the staff. But when comparing the sex and age distribution of the participating patients with an earlier study, namely, the Sächsische Epidemiologische Studie in der Allgemeinmedizin (SESAM 2) Saxon epidemiologic study in general practice¹⁶ that was based on randomly selected patients in a general practice setting, no relevant differences occurred.

Implications for future research

Some items of the STEP self-rating questionnaire should be modified to reduce the frequency of badly understandable and unanswered questions. Also, further efforts should aim to shorten the questionnaire. This might enhance acceptability and feasibility. Another area of interest would be the complex questions that bear new information and that do not have any consequences. It should be elucidated whether the new information and problems obtained with the questionnaire reflect facts and conditions that might be influenced by the general practitioner and whether this may benefit the patient. If both questions are answered with a “no,” there would be no sense in assessing these. Further efforts should be made to determine which subpopulations of patients should be targeted (eg, patients previously unknown to the general practitioner, patients discharged from hospital), to define the groups of patients who could especially profit from a geriatric self-assessment.

Conclusion

We conclude that the STEP self-rating questionnaire is feasible and useful. It is widely accepted by patients. The self-rating questionnaire should be modified to improve understandability and to minimize the rate of unanswered items. The STEP self-rating questionnaire is not feasible for some disabled people, for example, the visually impaired. In such cases, it may be alternatively used as an interview guide.