New York City Neighborhood Wellness Survey: an expansive population-level behavioral health assessment of New York City residents, 2023

Abstract

Objective

To identify and contextualize behavioral health needs among New York City residents ages 18 or older.

Methods

The New York City Neighborhood Wellness survey was a population-based, cross-sectional survey conducted from May to September 2023. An address-based sampling approach was used to select households for participation, and one adult per household was asked to complete the survey. The survey primarily focused on mental health and substance use but also included questions regarding participants’ care for children and youth, interactions with the criminal legal system, social support, and general stressors.

Results

From the 210,426 households randomly selected and sent a survey, 43,606 surveys were returned and included for analysis, a yield rate of 20.7%, and 2023 American Association of Public Opinion Research (2023 AAPOR RR3) response rate of 21.3%.

Conclusions

The New York City Neighborhood Wellness Survey provides population estimates of mental health and substance use across diverse communities to inform tailored local interventions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-026-26457-7.

Background

Mental health and substance use problems have worsened in New York City (NYC) as a result of the COVID-19 pandemic [1–3] with disproportionate effects on Black, Latino, Indigenous, and other Persons of Color (BIPOC), [4] and communities of high poverty [5]. In NYC, higher proportions of BIPOC people reside in neighborhoods of high poverty [6]. The NYC Department of Health and Mental Hygiene (NYC Health Department) is invested in addressing the behavioral health of New Yorkers through the prioritization of populations and communities at most risk [7]. In support of this goal, NYC Health Department utilizes population-based health surveys to assess current social and programmatic needs, geographical areas of focus, and to help guide local public health planning [8, 9].

While several NYC-based health surveys have measured the prevalence of mental health and substance use, [10–13] data gaps (e.g., opioid use) exist and there is currently a paucity of information regarding service and treatment needs, access, and use. Further, surveys with limited sample sizes have restricted the ability to contextualize behavioral health needs across intersectional identities, communities, and smaller geographic areas.

To fill these gaps, the NYC Health Department in collaboration with the City University of New York’s Graduate School of Public Health (CUNY SPH) conducted a needs assessment to inform planning, policy, and programming efforts to meet the behavioral health needs of New Yorkers. Launched in May 2023 and continuing through September 2023, the NYC Neighborhood Wellness survey (NWS) is the largest population-based behavioral needs assessment conducted in the city to date. A cross-sectional survey, NWS was designed to improve our understanding of mental health and substance use issues, as well as stressors and support experienced by NYC residents at the neighborhood level. This paper describes the rationale, design, and methods used in the survey.

Methods

The NYC Health Department and CUNY SPH contracted with Ipsos Public Affairs LLC for study design consultation, data collection, and processing. This cross-sectional survey was selected to gain a robust and representative sample of NYC communities, with a larger sample size compared with other local surveys [13]. We utilized a population-based approach using address-based sampling (ABS) methods to align with previous local survey methods [14]. The study was approved by the City University of New York, University Integrated Institutional Review Board.

Eligibility and sampling

The target population for this study was non-institutionalized adults ages 18 years or older residing in the five boroughs of NYC, including non-English speakers, pregnant people, and mentally or developmentally disabled individuals. Adults living in group quarters, such as college dormitories and military or other non-institutional group quarters were excluded. Adults who were unhoused, or in transitional settings, including carceral or institutional settings, were also excluded.

With ABS methods utilizing mailed surveys with the option to complete via web, the overall expected survey response rate was determined to be 22.5%, or 47,500 completed surveys among approximately 210,000 mailed surveys. We intentionally focused on ZIP code sampling to enable neighborhood estimations and identified 183 ZIP codes based on population estimates. We determined that a minimum of 350 participants per ZIP code would be required to detect 50% prevalence with 95% confidence interval and 5% margin of error. In order to ensure representation within ZIP codes, the next step was to calculate expected response rates within ZIP codes to calculate number of households needed per ZIP code. We reviewed NYC Community Health Survey response rates [15] and NYC 2020 Census response rates for ZIP codes with low response rates [16]. This expected response rate was applied to the population of each ZIP code to determine the total number of households to be sampled. ZIP codes with historically low survey participation rates were identified and sent a relatively higher proportion of surveys to help mitigate this issue [16].

A sampling frame of all residential mailing addresses within the five boroughs of NYC was procured from the United States Postal Service (USPS) Computerized Delivery Sequence (CDS) file for March 2023. Addresses in the CDS file are delivery point verified. For NYC, single USPS mail delivery addresses may be associated with multiple units and are called drop point addresses. Information about the units is not known, therefore drop point addresses were provided pseudo unit numbers to aid in the enumeration of households within a zip code. Addresses in each ZIP code group were randomly selected to receive an invitation to participate in the survey. To ensure only one survey was completed per household, the person in the household with the next upcoming birthday was designated eligible to take the survey.

Survey development

We developed the domains for the survey with the goal of describing community behavioral health at the neighborhood-level [17] and across intersectional identities [18]. Survey constructs and items were determined through an iterative process with the NYC Health Department and CUNY SPH subject matter experts. Community feedback on priority populations and topics was solicited via an anonymous and voluntary survey to NYC mental health and substance use service organizations. Survey items were classified into six domains: adult mental health; substance use; caregiver; criminal legal system; support and stressors; and demographics and were from previous NYC surveys or validated scales [19–30]. Caregiver items were selected because the NYC Health Department has a bureau focused on children, youth, and family health, and work with community providers focused on these populations. The NYC Health Department also has a bureau focused on the health of justice impacted populations, and these items were included to identify the needs of communities who intersect with the carceral system. Cognitive and readability testing were conducted to inform survey language, layout, length, and skip patterns. The ordering of the paper and online surveys was identical (Additional File 1).

Marketing

A website was created specifically to provide information about the survey and indicate legitimacy. The website included information about the survey, responses to anticipated frequently asked questions, links to the study’s social media accounts (Facebook, Instagram) and contact information for the help desk. Web-based surveys were not accessible via the website.

General information about the survey was published in small local newspapers and shared on social media via Facebook, Instagram, and X. These posts encouraged New Yorkers to look for the survey in the mail.

Mailings

The survey materials were disseminated in five separate mailings across a three-month period and included: (1) an initial invitation letter, (2) a reminder postcard, (3) a second invitation letter, (4) a second reminder postcard, and (5) a third reminder postcard. The initial survey packet included an invitation letter in English and Spanish, a paper survey in English and Spanish, a pre-paid envelope addressed to the survey processing center, and a $5 cash incentive. English and Spanish were selected for paper survey packet because they are the predominate languages spoken in NYC. Participants were instructed to fill out the paper survey or to access the web-based survey, which was available in English, Spanish, Traditional Chinese, Simplified Chinese, Haitian Creole, and Russian. For the web-based survey option, the participant was instructed to enter a unique passcode provided on their survey invitation or reminder postcard. Participants who completed the survey were mailed a thank you letter containing an additional $20 cash incentive, for a total of $25. A detailed description of the mailings, their contents, and the timeline for distribution can be found in the survey methodological report.

Informed consent

Information about the purpose and nature of the study was included in all invitation letters. In the invitation letters, participants were informed that the survey was voluntary and should only be completed by one household member, an adult age 18 years or older with the “next birthday”. Participants were notified that their responses were confidential and would be used for research purposes only, that their identifying information would never be used in any analysis or reports, and that they may be contacted to participate in future research opportunities. In the invitation letter, participants were directed to the NYC NWS informational website for the full informed consent document. The full informed consent document was translated into all survey languages. A waiver of signed informed consent was applied as a signed consent document from participants would be the only record linking the participant and the research and could lead to potential harm from a breach of confidentiality. A waiver of signed informed consent was approved by the City University of New York, University Integrated Institutional Review Board. Participants indicated their consent to participate by returning the completed survey by mail or successfully completing and submitting the web-based survey.

Data processing and cleaning

All paper surveys were processed in a secure space and included scanning the surveys, reviewing written text generated by open-ended questions, double data entry of written responses, and mailing address corrections. The web-based surveys were reviewed and cleaned. Data files by survey type (paper and web-based) were combined and further reviewed for completeness. Surveys that had less than 80% completion were excluded. Duplicate surveys, surveys completed by a person younger than age 18, surveys where ZIP code did not match ZIP code stratum mailed, or surveys by a non-NYC resident were also excluded.

Weighting

Survey weights were created and applied to adjust for design, non-response, and population estimates. The design weight accounted for the number of households with adults age 18+ within ZIP codes and boroughs/counties. An adjustment was applied to the initial design weight to account for the selection of one adult age 18+ per household, as the adult with the “next birthday” was eligible. The weighting variables included age, sex assigned at birth, education, race, Hispanic or Latino/a identity, household income, and homeownership. Missingness was assessed for all weighting variables. Missing data for sex at birth was imputed using first name and cell-based imputation [31]. Missing data for the other weighting variables were imputed using hot-deck imputation. The weights aligned the weighted sample on the selected demographic characteristics at the borough and citywide levels. ZIP code population totals within boroughs were also reviewed such that the weights proportionally represent ZIP code groups within boroughs, and boroughs within the citywide sample. To adjust for non-response bias, multivariate adjustments to weights, or ‘raking’, were applied using demographic distributions from the 2021 American Community Survey [17] (ACS) at the ZIP code, borough, and citywide levels. The 2021 ACS was restricted to population aged 18 years or older, living in non-institutionalized settings and non-group quarters. These adjustments were applied at the county level and trimmed at the 1st and 99th percentiles to adjust for extreme estimates.

Data analysis

We described the sample demographic characteristics, and selected response indicators for mental health, substance use, caregiver, and criminal legal system involvement by calculating unweighted totals, age-adjusted weighted prevalence estimates, and 95% confidence intervals for the age-adjusted weighted prevalence estimates. We conducted analysis using SAS 9.4 (SAS Institute Inc., Cary, NC).

Results

Of the 210,426 selected households, 52,802 surveys were submitted, over 80% of which were paper surveys (Fig. 1). A total of 2,379 surveys had less than 80% completion and were excluded (Fig. 1). The survey was primarily completed in English (93%) with fewer than 1% of participants opting to take the survey in Traditional Chinese, Simplified Chinese, Haitian Creole, or Russian combined. After deduplication and removal of ineligible surveys, a total of 43,606 surveys remained equating to a yield rate of 20.7%, and 2023 American Association of Public Opinion Research (2023 AAPOR RR3) response rate of 21.3% [32] (Fig. 1).

Fig. 1.

Flowchart of New York City Neighborhood Wellness Survey implementation, New York City, 2023. Total surveys mailed to households in New York City counties, total survey responses received, exclusion criteria totals, and final analytic cohort

Demographic characteristics of participants are shown in Table 1. Over 20% of NYC adults had a mental health diagnosis in their lifetime (23.9%; 95% CI:23.4–24.5), with anxiety/generalized anxiety disorder (18.4%; 95% CI: 17.9–18.9) and depression (10.8%; 95% CI: 10.4–11.2) being the most common diagnoses (Table 2). Prevalence of serious psychological distress (SPD) was 8.2% (95% CI: 7.9–8.6) as measured by the Kessler K6 scale [21]. Approximately 14.0% (95% CI: 13.6–14.5) of adults reported a perceived need for mental health treatment in the past 12 months. Among adults who were caregivers, 25.2% (95% CI: 23.6–26.8) reported that their child had needed mental, behavioral, or emotional support in the past 12 months. Three out of ten caregivers of children said that their child could not receive support every time they needed. Lastly, the estimated prevalence of lifetime alcohol use disorder and substance use disorder diagnoses were 1.8% (95% CI: 1.7-2.0) and 1.3% (95% CI: 1.2–1.5), respectively, while the use of other drugs, including cocaine, crack, heroin, fentanyl, and methamphetamines in the past 12 months was 3.2% (95% CI: 2.9–3.4).

Table 1.

Descriptive summary of New York City Neighborhood Wellness Survey participants, New York City, 2023

Characteristic	Unweighted N	Weighteda Percent (95% Confidence Interval)
Borough of residence
The Bronx	6,907	15.8 (15.4–16.1)
Brooklyn	12,469	30.0 (29.6–30.4)
Manhattan	8,465	19.9 (19.6–20.2)
Queens	13,251	28.9 (28.5–29.2)
Staten Island	2,514	5.5 (5.3–5.7)
Age Group
18–24	2,028	10.9 (10.4–11.4)
25–29	2,958	10.2 (9.8–10.6)
30–44	10,641	29.1 (28.5–29.7)
45–64	14,438	30.9 (30.4–31.5)
65–74	7,945	11.2 (10.9–11.5)
75+	5,596	7.7 (7.4–7.9)
Race/Ethnicity
American Indian, Native, First Nations, Indigenous Peoples of the Americas, or Alaska Native	172	0.2 (0.2–0.3)
Asian	6,837	15.8 (15.3–16.2)
Black or African American	7,985	21.1 (20.6–21.6)
Hispanic or Latino/Latinab	10,579	27.4 (26.8–28.0)
Middle Eastern or North African	518	1.4 (1.3–1.6)
Native Hawaiian or Other Pacific Islander	63	0.1 (0.0–0.1)
White	15,860	32.2 (31.7–32.7)
Multiple races	1,001	1.2 (1.1–1.3)
Another race	591	0.6 (0.5–0.7)
Gender
Cisgender man	16,565	45.2 (44.6–45.9)
Cisgender woman	26,452	53.1 (52.4–53.7)
Transgender man	99	0.3 (0.2–0.3)
Transgender woman	105	0.3 (0.2–0.4)
Non-binary	300	0.9 (0.8–1.1)
Another gender identityc	85	0.2 (0.1–0.3)
Sexual orientation
Gay or lesbian	2,182	5.3 (5.0–5.6)
Straight or heterosexual	36,405	86.0 (85.6–86.5)
Bisexual	1,526	4.1 (3.9–4.4)
Not sure/Sexual orientation not listed	1,573	4.6 (4.3–4.9)
Birthplace
United States and U.S. territories	26,301	60.2 (59.5–60.8)
Outside of the United States	17,022	39.8 (39.2–40.5)
Number of years in the United Statesd
Less than 5 years residency	864	8.6 (7.9–9.4)
5 to 10 years residency	1,676	14.8 (13.9–15.7)
More than 10 years or since birth	14,379	76.6 (75.5–77.6)
Education
Did not graduate from high school and did not get a GED	4,340	11.8 (11.4–12.2)
High school graduate (diploma, GED)	7,458	25.6 (25.0–26.2)
Some college, technical school, or associate’s degree	9,327	22.1 (21.5–22.6)
Bachelor’s degree	11,499	22.2 (21.7–22.7)
Graduate degree or professional degree	10,982	18.4 (18.0–18.8)
Insurance status
Insured	41,296	95.7 (95.4–96.0)
Not insured	1,340	4.3 (4.0–4.6)
Housing
Owned by me or someone in my family	17,586	36.6 (36.0–37.2)
Rented	25,179	62.2 (61.6–62.8)
Occupied without payment of rent	430	1.2 (1.0–1.4)
Public housing
A public housing (NYCHA) resident	3,017	7.3 (7.0–7.7)
Part of a household that receives rental assistance	3,259	6.7 (6.4–7.0)
Part of a household living in a rent-controlled or rent-stabilized home	7,265	18.4 (17.9–18.9)
None of these	28,927	67.6 (67.0–68.2)
Employment
Salaried	20,977	54.0 (53.4–54.7)
Self-employed	4,364	10.1 (9.7–10.5)
Unemployed, more than 1 year	2,473	6.7 (6.3–7.0)
Unemployed, less than 1 year	1,614	4.7 (4.4–5.0)
Homemaker	2,151	5.0 (4.8–5.3)
Student	1,988	9.9 (9.5–10.3)
Retired	11,363	15.6 (15.4–15.9)
Disability	3,551	6.8 (6.5–7.1)
Household povertye
< 100 - High poverty
100–199	7,580	16.8 (16.3–17.3)
200–399	6,646	16.6 (16.1–17.1)
400–599	9,640	23.5 (23.0–24.1)
≥ 600 - Low poverty	6,495	15.1 (14.6–15.5)

^a Weighted estimates were age-adjusted using Census 2000

^b Latino includes people of Hispanic or Latino origin, as identified by the survey question “Are you Hispanic or Latino?” and regardless of reported race. All other race categories exclude Latino ethnicity

^c Another gender identity includes genderqueer, or a gender identity not mentioned

^d Limited to respondents who were born outside of the United States

^e Household poverty using 2023 Poverty Guidelines Computations. US Department of Health and Human Services Office of the Assistant Secretary for Planning and Evaluation

Table 2.

Prevalence of mental health, substance use, criminal legal system interactions, and caregiver experiences among New York City Neighborhood Wellness Survey participants, New York City, 2023

Characteristic	Unweighted N	Weighteda Percent (95%Confidence Interval)
Mental Health
Lifetime Mental Health Diagnosis	10,649	23.9 (23.4–24.5)
Depression	4,728	10.8 (10.4–11.2)
Post-Traumatic Stress Disorder	2,467	5.3 (5.1–5.6)
Anxiety or Generalized Anxiety Disorder	8,061	18.4 (17.9–18.9)
Schizophrenia, Schizoaffective Disorder, or Psychosis	417	1.1 (0.9–1.2)
Bipolar Disorder, Mania, or Manic Depression	1,179	2.6 (2.4–2.9)
Non-specific psychological distress, past 30 days	3,049	8.2 (7.9–8.6)
Received mental health treatment or counseling in past 12 months	11,505	25.2 (24.7–25.7)
Has unmet mental health treatment or counseling needs in the past 12 months	5,728	14.0 (13.6–14.5)
Alcohol and Substance Use
Lifetime diagnosis, Alcohol Use Disorder	840	1.8 (1.7-2.0)
Binge drinking, past 12 months	12,673	32.4 (31.8–33.0)
Received alcohol treatment or counseling, past 12 months	773	1.8 (1.6-2.0)
Lifetime diagnosis, Substance Use Disorder	605	1.3 (1.2–1.5)
Any drug use, past 12 monthsb	9,877	24.4 (23.9–25.0)
Cannabis use, past 12 months	9,063	22.8 (22.2–23.3)
Any other drug use past 12 monthsc	1,255	3.2 (2.9–3.4)
Misuse of prescription drugs, past 12 monthsd	1,214	2.9 (2.6–3.1)
Needed substance use treatment in the past 12 months	494	1.3 (1.2–1.5)
Received treatment (medication and/or counseling) for substance use, past 12 months	977	3.2 (3.0-3.5)
Criminal Legal System
Stopped, searched, or questioned by police	7,001	17.9 (17.4–18.4)
Any time spent in a juvenile or adult correctional facility or other detention center	2,231	5.3 (5.0-5.6)
Has a close family member who has spent time in a juvenile or adult correctional facility or other detention center	5,547	13.0 (12.6–13.5)
Has a close family member currently in a juvenile or adult correctional facility or other detention center	697	14.3 (13.0-15.8)
Caregiver Experiences
Parent or caregiver to a child under 18	7,163	20.2 (19.7–20.7)
Child under 18 needed mental, behavioral, or emotional support, in the past 12 monthse	2,063	25.2 (23.6–26.8)
Child under 18 was able to receive mental, behavioral, or emotional support every time they needed it, in the past 12 monthse	1,427	31.1 (27.3–35.2)

^a Weighted estimates were age-adjusted using Census 2000

^b Use of any drugs, including cannabis, cocaine, crack, heroin, fentanyl, methamphetamines, and prescription drug misuse

^c Use of other drugs, including cocaine, crack, heroin, fentanyl and methamphetamines

^d Misuse of a prescription refers to use of a prescription medication without a prescription or in a manner other than prescribed

^e Limited to caregivers of a child under the age of 18

Discussion

NYC Health Department and CUNY School of Public Health are committed to improving the health of New Yorkers and this study was strengthened by substantial institutional support and an interdisciplinary approach to survey development and design. The NWS is the largest population-based behavioral health survey conducted in New York City to date. The survey was conducted immediately after the federal government removed the US national emergency status for the COVID-19 pandemic and provides a snapshot of New Yorkers’ well-being in the last year of the pandemic. The NWS included questions that fill current gaps in population surveys, including mental health and/or substance use diagnosis and treatment needs. It gave insight that NYC adults have ongoing mental health needs, including having recent psychological distress (8% of adults) or having an unmet mental health treatment need (14% of adults). The caregiver domains give insights into a population that could use additional support. The finding that one in four caregivers of children younger than age 18 say their child experienced an unmet mental or behavioral health need or emotional support need. Further analysis of mental health and/or substance use needs by race/ethnicity and age could provide key insights in creating culturally tailored education on mental health wellness for aging populations. Indicators such as social networks and social support in a neighborhood could also inform service planning for community activities and events. Similarly, neighborhood level estimates for these behavioral health indicators are helpful for the NYC Health Department and other community partners in advocating for funding or program expansion in their supporting communities. The robust survey sampling afforded the opportunity to understand mental health and substance use issues by multiple categories for race/ethnicity, sexual orientation, and gender identity. Overall, there are many ways that the NWS findings will be used to inform data-driven interventions that improve behavioral health of NYC communities.

The NWS still had some limitations. First, as a cross-sectional survey, it provides a snapshot of the respondent’s experience. While questions were specific to time periods (current, past 30 days, past 12 months, lifetime), temporality and causality cannot be determined. Furthermore, asking respondents to think back in time is subject to recall bias. Another limitation was the lack of diverse languages via paper. It possible that the lack of paper surveys in other languages limited participation for non-English or non-Spanish speaking adults because of preference to complete the paper survey than to complete the online survey. Additionally, the online survey platform limited survey participation to NYC adults who had access to the internet and the tools to complete the online survey. The survey also potentially left out various topics due to limitations on survey length. At the same time, the survey length was a deterrent for some respondents with over 2,000 respondents excluded because of less than 80% completion. It is possible that respondents who did not respond fully had more behavioral health needs, which are not included in the findings. Similarly, another missed population with behavioral health needs that will not be captured are NYC adults who were unhoused, or resided in institutions such as shelters, residential treatment, or carceral settings, which were not included in the residential address sampling frame.

Additionally, survey dissemination timing could have resulted in lower response rates. The first survey mailings were distributed on May 11, 2023, on the same day the federal public health emergency for COVID-19 officially expired. Given the numerous public health surveys that were fielded during the pandemic [33, 34], responses may have been limited due to survey fatigue. Difficulties were also encountered during the mailing process. USPS experienced delays in processing first class mail which negatively impacted survey processing and participant incentive mailings. Despite adjustments made to improve response, the final survey response yield of 21.3% was lower than the anticipated 22.5%, which impacts the stability of some estimates made at the ZIP code level.

Conclusion

NWS provides evidence that implementation of a large population-based survey is feasible in a large metropolitan area. The expansive sample size provides the ability to look at smaller communities, demographics, and geographies, in order to understand the behavioral health needs of these diverse communities. In addition to contributing invaluable information for NYC Health Department’s resource allocation and targeted funding initiatives, a survey of this magnitude also allows for smaller organizations to use community-level data to inform their programming and grant writing activities. Public health researchers should invest in population-based surveys that can estimate needs across their communities.

Authors’ contributions

ET, MM, AH, CN, and SF contributed to the conception or design of the work. ET, MM, AH, CN, and SF contributed to the acquisition of data. ET, MM, and SB, contributed to the analysis of data. ET, MM, SB, AH, CN, and SF contributed to the drafting the work or reviewing it critically for important intellectual content. ET, MM, SB, AH, CN, and SF contributed to the review of manuscript final approval of the version to be published.

Funding

The NYC Neighborhood Wellness Survey was funded by the NYC Health Department.

Data availability

The datasets generated and/or analyzed during the current study are not publicly available due to embargo but are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The study was approved by the City University of New York, University Integrated Institutional Review Board and conducted in accordance with the Declaration of Helsinki. All participants were informed about the survey purpose and methods. Participants indicated their consent to participate by returning the completed survey by mail or successfully completing and submitting the web-based survey.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.