Developing a reliable and valid instrument to assess health-affecting aspects of neighborhoods in Tehran

Abstract

Background:

Residence characteristics can affect health of residents. This paper reports the development of an instrument assessing these aspects of neighborhoods.

Materials and Methods:

Literature search and focus group discussions with residents were carried out and relevant items were extracted. Five experts reviewed and commented on the items. An observation instrument with 54 items was composed and completed by two independent observers in 20 randomly selected locations. Due to lack of acceptable reliability in some items, the checklist was revised. The new 22-items checklist in four categories (general characteristics, public green area characteristics, access to services and undesirable features) was completed by two independent trained observers in 28 randomly selected locations.

Results:

The items in the final checklist had kappa statistics ranging from 0.63 to 1, with an exception of the item assessing “presence of beggars, homeless or working/street children”, with kappa as low as 0.27 due to variability of their presence in different times. Average Kappa statistics was 0.78 for general characteristics, 0.79 for public green area characteristics, 0.84 for access to services, and 0.54 for undesirable features.

Conclusion:

Neighborhood and health observation instrument seems to have good reliability in city of Tehran. It can probably be used in other large cities of Iran and similar cities elsewhere.

BACKGROUND

Residence characteristics can have direct and indirect effect on the health status of the residents.[1] Many studies have tried to highlight neighborhood characteristics that affect general health of the residents[1–3] or have certain links with some health conditions[2,4,5] or risk factors.[6] Some researchers have provided reliable tools for this purpose.[7] Many of these tools use secondary data (census, municipality reports, etc.)[8] and are generally based on aggregated individual data, which are considered imperfect in assessing health-affecting aspects of neighborhood by some researchers.[9] Some others use questionnaires to interview the residents and determine the levels of health-affecting characteristics.[4]

The characteristics that are assessed in these tools vary based on the culture and other specifications of the area, such as the prevalence of health determinants in the target society and their degree of impact.[10,11] Instruments have been developed to measure different aspects of neighborhood that can affect health,[12–15] but they cannot be used in Tehran in their original form due to cultural and contextual differences.

Tehran is among the 10 largest cities in the world, a metropolis of complex cultural and social characteristics, with some 12 million inhabitants (about 16% of total population of Iran) in 707 km². About 30% of Tehran's population live in suburb. There is an unbalanced distribution of population: average density of the city is 92 persons per hectare (pph), ranging from 44 to 412 in different districts. About 60% of the inhabitants are born outside Tehran and there are diverse ethnic and linguistic groups from all over the country. Although overall quality of housing is acceptable, there is an unequal development and luxury between northern part and other parts of the city, specially suburb and recently added districts.[16] High air pollution,[17] low levels of physical activity in residents,[18] high incidence of road traffic injuries[19] are some of the characteristics of this city which may affect health of residents.

We need reliable and valid tools to evaluate features of neighborhood that affect health of the residents of Tehran, before any intervention is designed for improving the situation. Efforts have been made to evaluate quality of neighborhood in Iran and other countries of the region but to our best knowledge, there is no reliable and valid tool to assess health-affecting aspects of neighborhood in Tehran. To design such tools, we need to know attitudes and beliefs of the people about health and social life. In this study, we tried to develop an observation instrument to assess the aspects of residence in Tehran that affect health status of the residents directly or indirectly, and are not targeted in questionnaires regarding social capital.

MATERIALS AND METHODS

The first step in developing such instrument was to identify relevant concepts from literature. We reviewed literature thoroughly for relevant scales and extracted items from articles and instruments regarding neighborhood characteristics. We searched Scopus, Google scholar and Proquest using neighborhood, residence, urban health, and direct observation, as keywords. In addition, we reviewed the articles cited by the relevant articles.

Next step in our study was to identify the viewpoints of residents of Tehran about health-affecting aspects of neighborhood. We performed 4 focus group discussions (FGD) in four different locations in Tehran. Participants were asked to identify the features of a neighborhood they consider important for their health. First FGD was performed with health volunteers and patients attending a public health facility in south of Tehran, as a low socioeconomic area. The participants of the second interview were health volunteers and patients attending a health facility in the suburb, with mediocre socioeconomic status. Third FGD was performed in a park in east of Tehran, where elderly men and women gathered for daily activities. People living in this part of city are known to be in a good socioeconomic status. Last FGD took place in a small park in a luxurious residence in north of Tehran. In this location, young parents who were accompanying their children and a few elderly people participated. We recorded the conversations with permission of the participants. Recorded files were transcribed and analyzed using thematic analysis approach.

The results of the literature reviews and analysis of FGDs were summarized in a list and sent to five experts who had prior research in social determinants of health, environment and social health in Iran. Semi-structured interviews were performed with these experts once they had reviewed the list and commented. Finally, we composed an observational checklist with 54 items and asked the expert for their final approval.

We trained two observers to complete the checklist. The training sessions consisted of two parts: first part was a detailed explanation of the study and in-depth discussion about each item in the checklist. The second part was completing the checklist in two locations (one for each trainee) along with the researchers. These observers completed the final checklist in 20 randomly selected locations of Tehran independently. Time spent to complete the checklist was recorded. Kappa statistics were calculated to evaluate the reliability of the checklist.

Due to lack of reliability of some of the items, the checklist was revised. The revision was based on the conclusions of three research team sessions, where results from the statistical analysis and feedbacks of the observers were considered. Similar items were merged into a single one and items that could be determined more accurately from other sources were omitted from the list. Items with low reliability were modified due to experts’ and observers’ opinion and items with low variance between locations were eliminated. Another modification was reducing the answering options from three or four options into two (appropriate, inappropriate). Some changes were made according to the feedbacks from the observers. The expert team were requested to comment on the final version. After their approval, two independent trained observers completed the new list of 22 items in 28 randomly selected locations in Tehran. Time spent to complete the checklist was recorded. Kappa statistics where calculated using Stata/version 8 software.

RESULTS

After reviewing the literature, we categorized and summarized the mentioned features into five domains:

• Activity friendliness

• Physical and social disorder

• Access to health services

• Access to social services

• Undesirable amenities

Each of these domains consisted of a variety of items based on the cultural and social characteristics of the place where the research was conducted.

We extracted 27 items from the FGDs through thematic analysis [Table 1].

Table 1.

List of items extracted from FGDs

By merging scales derived from these two sources (literature review and FGDs) and consulting with the experts, we developed the primary checklist containing 54 items. Kappa statistics for this checklist ranged from 0.15 to 0.89 (median 0.53).

After revising the checklist, the final version contained 22 items. The items in the final checklist had kappa statistics ranging from 0.63 to one (median = 0.78), with an exception of the item assessing presence of homeless or working/street children, with kappa as low as 0.27. List of the items and their kappa statistics are shown in Table 2. Average Kappa statistics was 0.78 for general characteristics, 0.79 for public green area characteristics, 0.84 for access to services and 0.54 for undesirable features. The average time spent for completing the checklist was 17.5 (SD = 6.3) for the primary checklist and 13.3 (SD = 6.1) for final checklist.

Table 2.

Kappa statistics of the final observation instrument

The final checklist was sent to the members of the expert panel and they approved the content and structure.

The Pearson coefficient was less than 0.9 for all pairs of items which excluded colinearity. Bartlett test was significant (P < 0.000), indicating the possibility of performing Factor analysis. Principal component analysis was applied on the results of final instrument and revealed eight empirical factors. Two factors perfectly corresponded to “access to services” and “public green area characteristics”. The items of general characteristics scale were aggregated in two factors and the three questions regarding the undesirable features were placed in three different factors [Table 3].

Table 3.

Factor analysis of the instrument of health-affecting aspects of neighborhoods

DISCUSSION

The results show that this observation instrument can be considered a reliable and valid tool to assess neighborhood characteristics. The content validity of the checklist was confirmed by experts. The inter-rater reliability of the checklist is good (median of kappa statistics = 0.79). The time needed to complete the checklist is less than 15 min, and it can be completed by observation and without further need to ask questions from residents or local officials.

The primary checklist consisted of numerous items and 3--4 answering options. Raters frequently failed to grasp the main idea of the checklist, which resulted in low reliability in certain items. The final checklist did not possess as much details and rating categories. Other researchers have used similar strategies to improve reliability of their checklist.[7,12]

We took measures to improve inter-rater reliability. Training the observers, practicing in real situation and discussing the answers and editing the checklist according to their feedbacks are all part of these efforts. Although they improved the reliability of the checklist in many aspects, the reliability of “Presence of beggars, working/street children and the homeless” was still low, which is probably due to the nature of these disorders (presence of these people in a neighborhood may vary from day to day and even in different hours of the day). We believe that this item should not be eliminated because it reflects an important aspect of neighborhood characteristics. In our setting, the reliability of general sanitary condition was good, unlike the instrument developed by Brian et al., where it was unreliable.[12]

Although many researchers use aggregate measures and census data as proxies for the residence characteristics,[4] the main idea of this study was to develop an instrument which would not use such measures for the neighborhood variables, and thus, would not endure their limitations, as indicated by Diez-Roux.[9]

Many studies focus on a particular aspect of neighbourhood, such as quality of parks and green area,[14] playgrounds,[12] walkable neighbourhood,[13] socioeconomic characteristics,[4,8] etc. To have a more general overview of residence, one may need to use multiple instruments. Although more precise, doing so would be difficult and time-consuming. Park features,[20,21] attractiveness of public open spaces and their distance,[22,23] activity friendliness of the neighborhood,[24] visual aesthetics,[25,26] transportation,[25,26] physical or social disorders,[27–29] and access to services[30] are some of the important aspects of neighborhood that have been investigated in resident health studies.

We eliminated some of the items from the primary checklist (e.g., safety from crimes and accidents) as they could be determined more precisely from other sources than observation. These sources have been used by other researchers to adjust the effect of walkable streets and park access.[13]

The results show that our neighborhood observation instrument provides a simple and quick method to assess neighborhood characteristics. It seems to have good validity and reliability in Tehran. Researchers can use this instrument in Tehran and other large cities of Iran and other cities in the region, which have similar context. This instrument can be best used in combination with data gathered from other resources on security and socioeconomic level of the area; social capital and other important characteristics of the neighborhood.