Abstract
Background:
The Supplemental Nutrition Assistance Program (SNAP) is a federal public benefit providing food assistance to millions of Americans. However, it is typically administered by states, creating potential variation in accessibility and transparency of information about enrollment for people with disabilities.
Objective:
To develop and demonstrate the use of a method to assess the accessibility and transparency of information about the disability-inclusive process and practices of SNAP enrollment.
Methods:
Cross-sectional data was collected from SNAP landing and enrollment webpages from all 50 U.S. states, the District of Columbia, and New York City from June–August 2021. Based on principles of universal design and accessibility, scores were determined for each SNAP program across three areas: flexibility in the enrollment process (6 points), efficiency of finding information about enrollment on SNAP websites (6 points), and the accessibility of SNAP webpages (6 points). Total scores were the sum of these sub-categories (18 points maximum).
Results:
Of the 52 SNAP programs assessed, mean scores were 10.66 (SD = 2.51) for the total score, 2.67 (SD = 0.91) for flexibility in the enrollment process, 3.32 (SD = 1.19) for efficiency of finding information about enrollment on SNAP websites, and 4.67 (SD = 1.72) for the accessibility of SNAP webpages. No programs received the maximum flexibility score (6 points) on flexibility, 2 programs received the maximum on efficiency, and 31 programs the maximum on accessibility.
Conclusions:
We found differences in the accessibility, flexibility, and efficiency of SNAP program enrollment information available on SNAP websites and outline room for improvement across all three of these areas.
Keywords: Disability, Food assistance, Food insecurity, Socioeconomic factors
Adults with disabilities are more likely to be food insecure than adults without disabilities but may experience more barriers to accessing food assistance programs, such as the Supplemental Nutrition Assistance Program (SNAP). Approximately 33% of households that include an adult with a disability are food insecure, compared with only 8% of households comprised of adults without disabilities.1 Enrollment in SNAP can reduce food insecurity2,3 and is associated with better health outcomes including less health care utilization,4-6 less cost-related medication non-adherence7 and better glucose control among diabetics.8
Individuals with disabilities likely face disproportionate challenges to SNAP enrollment. Despite being more likely to be food insecure, people with disabilities tend to be under-represented among SNAP participants; in 2016 about 9% of young and middle-aged SNAP participants had a disability,9 but at least 20% and 45% of income-eligible young and middle-aged adults, respectively, were estimated to have a disability.10 The SNAP enrollment process can be cumbersome, which may discourage enrollment.11-13 Yet there has been little attention to identifying and addressing these SNAP enrollment challenges for people with disabilities.
Ensuring SNAP programs are accessible and inclusive must be a key component in efforts to address food insecurity in the U.S. Although SNAP is a federally funded public benefit, it is typically administered by states, creating potential variation in SNAP program practices. Applicants typically rely on state agency websites as a first step in the SNAP enrollment process. The accessibility and transparency of information on these websites is a critical component of removing barriers to SNAP enrollment for people with disabilities. Prospective applicants must be able to easily find information about SNAP, determine eligibility, understand the application process, and find information about accommodations.
Established principles of universal design should be drawn upon to ensure equitable access to SNAP for all who are eligible, including people with disabilities.14 Key principles of universal design include accessibility, flexibility, and efficiency. Accessibility captures whether information is communicated effectively regardless of an individual's disability status. Flexibility allows a wide range of formats to meet individual preferences and needs. Efficiency assesses whether processes within systems are simple, intuitive, and easy to navigate.14 Together, these principles can be used to evaluate whether systems and processes are inclusive of and provide equitable access to people with disabilities15,16 and have been applied in other studies to evaluate disability inclusion in U.S. higher education and COVID-19 vaccine accessibility.17,18 However, there is a dearth of research using these principles to evaluate programs and processes, such as SNAP enrollment. This information is needed to identify areas for improvement, track change, and develop strategies to make SNAP enrollment processes more equitable.
This project's aim was to develop and pragmatically validate a novel method to evaluate and compare the accessibility, flexibility, and efficiency of SNAP enrollment information available on SNAP websites based on principles of universal design and build an open source dashboard to share this information.19
Methods
Because the goal of this study was to measure and describe a process (e.g. SNAP enrollment) rather than measure and describe a latent variable, the study sought to develop an index, rather than a scale,20,21 and used mostly inductive and pragmatic approaches to validation to ensure construct validity and relied less on deductive psychometric validation methods. Institutional Review Board approval was not required as data are publicly available and did not involve human subjects. From June–August 2021, five trained student researchers conducted qualitative coding regarding the landing and enrollment pages of SNAP websites in all 50 U.S. states, the Washington District of Columbia, and New York City. The study team included people with disabilities. The SNAP landing page was defined as the introductory SNAP webpage for a given state, as linked on the United States Department of Agriculture SNAP State Directory of Resources webpage. Landing pages generally summarize eligibility criteria and benefit utilization information and have links to SNAP enrollment applications. In four cases where the website led to an error message, the landing page was identified by inputting “[state name] SNAP website” into a search engine and finding the page that best matched the landing page. The enrollment pages requested an applicant's name, household, income, and other information for determination of SNAP benefits.
Data were collected in a coding instrument in Microsoft excel based on prior similar tools17,18 that were adapted to include SNAP-program specific data items. As previously done, the custom coding instrument allowed coders to input the description of a program's indicator-specific performance, and subsequently calculated the corresponding numerical point allotment (total score). This data collection form was pilot tested in an iterative process prior to data collection first for a randomly selected set of 12 SNAP programs and then for all 52. Researchers asynchronously recorded information in unlinked documents prior to three consensus meetings. During consensus meetings, the researchers provided feedback and the team discussed improvements to the form. The final form collected data for 13 indicators related to flexibility and efficiency and identified the URL to be used for estimating accessibility for SNAP program websites (Supplemental Table 1). After data was extracted by one reviewer, quality reviews were conducted by one researcher for the 12 indicators of flexibility and efficiency among a randomly selected group of 10 SNAP programs. Discrepancies were adjudicated by the first and senior author in a review meeting. This adjudication process resulted in changes for 14 values, or 12% of the 120 indicators reviewed across the 10 programs.
Flexibility indicates the number of modalities for SNAP enrollment to support access to a wide range of individuals. Based on principles of universal design, flexibility in SNAP enrollment was operationalized as having multiple enrollment modalities. Since a validated tool was not available and to support construct validity,20,21 an index was developed using all modalities supported for enrollment as described on SNAP websites, including in-person, online, email, mail-in, telephone and TTY. Programs were given 1.0 or 0.5 points, where 1.0 indicates greater flexibility of options based on whether the modality was offered state-wide or by jurisdiction (Supplemental Table 1).
Efficiency scores determine how readily available SNAP enrollment information was on program websites. A previously-used website efficiency tool22 was modified to evaluate the following criteria relevant to the SNAP enrollment process: if (1) the SNAP program landing page had eligibility information or a direct link to an eligibility determination page, (2) the program landing page had information on how to get accommodations for SNAP enrollment, (3) a telephone help line was provided on the program website, (4) there was a TTY help line listed on the program website, (5) the program landing page had links to an online enrollment form and/ or a PDF enrollment form, and (6) the program landing page had links to a large print PDF version of the SNAP enrollment form.
Accessibility scores, as in prior work,17 were calculated using an average of automated accessibility test data across a broad sample of SNAP site pages and high-level manual accessibility testing on both the SNAP information and registration pages, when available. The WAVE accessibility tool23 was used to collect automatic accessibility data. Each site was assigned an automated accessibility score based on the number of detected errors, the density of errors (number of errors by number of web page elements), and number of likely or potential accessibility errors. To standardize scores in comparison to pages across the web, automated scores were aligned to tertiles based on the WebAIM Million sample of one million homepages.24 As previously done,18 WebAIM testers with experience assessing web accessibility then manually evaluated the landing and enrollment pages for 10 aspects of the Web Content Accessibility Guidelines,25 assigning scores based on potential impact of accessibility issues on individuals with disabilities. Final accessibility scores were the average of the automated scores and manual scores (0 points indicated accessibility scores < 4.3, 3 points indicated scores 4.3 < 7.0, and 6 points indicated scores ≥ 7.0).
Total SNAP scores were calculated as the sum of the flexibility, efficiency, and accessibility scores for each program. A maximum of 18 total points were possible.
Statistical analysis
Summary statistics were provided for the total and each sub-category (flexibility, efficiency, and accessibility) score. Total and sub-category scores were each divided into tertiles: lowest, middle, and highest scores. Using this data, heat maps of the U.S. were created to represent the geographical distribution of these score tertiles (Fig. 1). Spearman correlations across domain scores were calculated. Statistical analyses were completed using R statistical software (version 4.1.1).
Fig. 1.
Supplemental Nutrition Assistance Program (SNAP) Enrollment Accessibility and Disability Inclusion Scores across the United States, 2021, including (a) total scores (Tertile cut-offs are 9 and 12.5), (b) flexibility (tertile cut-offs are 2.5 and 3), (c) efficiency (tertile cut-offs are 2.5 and 4), and (d) accessibility (tertile cut-offs are 2.5 and 4).
Results
Of the 52 SNAP programs assessed, the mean total score was 10.66 (SD = 2.51) out of 18 possible total points. The mean scores were 2.67 (SD = 0.91) for flexibility, 3.32 (SD = 1.19) for efficiency, and 4.67 (SD = 1.72) for accessibility of SNAP program enrollment (Table 1). Correlations between flexibility and efficiency (Spearman ρ = 0.23), flexibility and accessibility (ρ = 0.06) and efficiency and accessibility (ρ = 0.16) suggest that the three domains are related, but distinct.
Table 1.
Supplemental Nutrition Assistance Program (SNAP) for program characteristics in all 50 U.S. states, Washington D.C. and New York City, 2021.
| SNAP program characteristics | |||
|---|---|---|---|
| Flexibility of SNAP enrollment | |||
| Mean enrollment score of SNAP websites (SD) | 2.67 (0.91) | ||
| Enrollment options described on website (%) | No, n (%) | Yes, only by jurisdiction, n (%) | Yes, for the state, n (%) |
| In-persona | 8 (15%) | 0 (0%) | 44 (85%) |
| Online | 4 (8%) | 0 (0%) | 48 (92%) |
| 48 (92%) | 0 (0%) | 4 (8%) | |
| Mail in | 13 (25%) | 26 (50%) | 13 (25%) |
| Telephone | 39 (75%) | 2 (4%) | 11 (21%) |
| TTY | 47 (90%) | 0 (0%) | 5 (10%) |
| Efficiency of SNAP enrollment | |||
| Mean Efficiency score of SNAP websites (SD) | 3.32 (1.19) | ||
| Efficiency of SNAP websites (%) | No, n (%) | Yes, but not clearly indicated, n (%) | Yes, and clearly indicated, n (%) |
| Has eligibility information | 8 (15%) | N/A | 44 (85%) |
| Has enrollment accommodation information | 36 (69%) | 0 (0%) | 16 (31%) |
| Has a telephone help line | 8 (15%) | 6 (12%) | 38 (73%) |
| Has a TTY help line | 28 (54%) | 0 (0%) | 24 (46%) |
| Has online enrollment form linked to state | 3 (6%) | 15 (29%) | 34 (65%) |
| Has large-print PDF enrollment form | 46 (88%) | 0 (0%) | 6 (12%) |
| Accessibility of SNAP Enrollment | |||
| Mean accessibility score of SNAP websites (SD) | 4.67 (1.72) | ||
For most enrollment options, state-wide services were the preferred option, except that by jurisdiction was preferred for in-person enrollment.
In the flexibility sub-category, the majority of program websites had described in-person (85%), online (92%), and mail-in (75%) options, but not email (8%), telephone (25%), and TTY (10%) options. In the efficiency sub-category, the majority of programs listed eligibility information (85%), a telephone help line (85%), and an online enrollment form linked to state (94%), but not enrollment accommodation information (31%), TTY help line (46%), or a large-print PDF enrollment form (12%). Scores ranged across SNAP programs from 1 to 5 for the flexibility sub-category, 1.5 to 6 for efficiency, and 0 to 6 for accessibility (Supplemental Table 2). The number of SNAP programs that received the maximum score (6 points) within each sub-category differed: 0 SNAP programs for flexibility, 2 programs for efficiency, and 31 programs for accessibility. There were differences in these scores when examined by tertile ranking across the U.S. (Fig. 1).
Discussion
This project developed, provided preliminary validation, and demonstrated the use of a tool to describe the accessibility, flexibility and efficiency of SNAP program enrollment information available on SNAP websites, and developed an open-source dashboard to share these results. Results using the tool found differences across SNAP programs and outline room for improvement across all three of these areas. These results are noteworthy because all income-eligible households are legally entitled to receive SNAP benefits, which can reduce food insecurity, but this study documents variability across locations in how programs are enacting principles of universal design. Together with evidence showing disproportionate food insecurity burden among people with disabilities,1 these results suggest that people with disabilities are doubly disadvantaged with regard to food access. They are not only more likely to be food insecure than people without disabilities, but also may face more barriers in the information gathering and enrollment process for the SNAP program.
This study contributes to the literature by providing initial evidence of variability in the accessibility and universal design approaches of government programs. These results are among the first to quantify these aspects for the SNAP enrollment process. Our findings identify important barriers to preparing a SNAP application for people with disabilities, but this is only the first step of the SNAP enrollment process. Multiple additional steps were not examined in this study, including submitting applications with complete documentation, completing an interview, utilizing benefits, updating household information as needed, and recertification. All these critical steps should also be examined regarding inclusiveness for people with disabilities. Additional work is needed to further validate this tool and apply this general approach to other websites and government programs and policies, including housing programs, unemployment benefits, health insurance, tax credits, and income supplements.
These results have important implications for policies, processes, and practices for public benefits such as SNAP by encouraging 1) a system of accountability for inclusion, 2) state partnerships with disability communities to understand and address barriers from the user perspective, and 3) federal funding to support these efforts. First, there is a need for a better system of accountability to ensure inclusion of and equitable access for people with disabilities in entitlement programs like SNAP. Although Section 504 of the Rehabilitation Act of 1973 and Title II of the Americans with Disabilities Act (ADA) prohibit government programs from excluding or discriminating based on disability, the current paradigm of these laws puts the onus of compliance on the individual, as it usually requires the person with a disability who has experienced accessibility issues to file a complaint. This tool intends to begin to build a data infrastructure to change that paradigm. By collecting and sharing data on SNAP enrollment information accessibility, flexibility, and efficiency, we aim to support the advancement of disability inclusion in SNAP. Second, state-administered programs should include people with disabilities as advisors, leaders, and staff to improve practices, processes, and policies. Further, there must be clear lines of communication between these state agencies and disability communities, including transparent processes for sharing grievances or concerns with the current systems. Input from disability communities is needed on an ongoing basis as technology evolves and programs change. Third, federal funding is needed to assist states in becoming more accessible and inclusive of people with disabilities. Funding should be commensurate upon ensuring processes and practices stay up to date with best practices and new technology, and address issues that are raised by those in the disability community.
Importantly, states should take action to make the SNAP enrollment process more efficient. This includes identifying and eliminating unnecessary steps of the enrollment and recertification process for people with disabilities. For example, the Combined Application Project26 and the Elderly Simplified Application Project reduce or eliminates application requirements for people who are already known to be low-income because they receive Supplemental Security Income (SSI) and Social Security income, respectively. Because 86% of SSI recipients also have a disability, this can both improve inclusion for beneficiaries and reduce administrative burden for SNAP programs. Finally, allowing broad-based categorical eligibility can simplify the SNAP enrollment process by raising the income eligibility limit and/or eliminating the asset test for people with disabilities. Since individuals with disabilities need 28% more income to maintain the same standard of living as a household of the same size where no one has a disability,27 the use of broad-based categorical eligibility may level the playing field to ensure their food needs are met.
In addition to eliminating burdensome steps in the enrollment process, SNAP programs could improve disability inclusion by providing more flexibility, assistance and accommodations with the enrollment process. Targeted enrollment interventions have been shown to increase SNAP participation28 and may be particularly important for individuals with disabilities who may face disproportionate challenges navigating the complex enrollment processes. This study builds on prior work by providing a tool and open-source dashboard that SNAP program administrators can use to identify gaps, and address gaps in enrollment processes.
Limitations and strengths
Results are limited to information obtained from websites; there may be additional enrollment modalities beyond those described on the website and accommodations may be offered after an individual initiates an online application or downloads a PDF form. However, based on the principle of efficiency, it is important to provide that information before initiating applications so that individuals with disabilities have the opportunity to make an informed enrollment plan based on the modalities and accommodations that would work best for them. Strengths of this study include having a conceptual grounding in key principles of universal design and inclusion of SNAP programs across the U.S.
Conclusions
In addition to demonstrating the use of the dashboard tool to evaluate the inclusiveness of government programs, these findings highlight potential accessibility gaps for individuals with disabilities in the initial online steps of the SNAP enrollment process. There is a need to create best practices that improve accessibility and universal design and increase SNAP enrollment, additional federal support, and improved input from the disability community.
Supplementary Material
Funding
This work was supported by the National Institute on Aging (K01AG054751) awarded to LJS. The funder was not involved in the conduct or content of this study.
Footnotes
The information in this article was disseminated via the Johns Hopkins Disability Health Research Center Supplemental Nutrition Assistance Program (SNAP) Disability Dashboard [https://disabilityhealth.jhu.edu/snapdashboard/].
Preprint
This article has been posted in MedxRiv, a pre-print server, and is available at [https://www.medrxiv.org/content/10.1101/2022.03.09.22271551v1.full].
Appendix A. Supplementary material
Supplementary material to this article can be found online at https://doi.org/10.1016/j.dhjo.2022.101366.
Conflicts of interest
No conflicts of interest to declare for any authors.
References
- 1.Coleman-Jensen AUS. Food insecurity and population trends with a focus on adults with disabilities. Physiol Behav. Jun 1 2020;220, 112865. 10.1016/j.physbeh.2020.112865. [DOI] [PubMed] [Google Scholar]
- 2.Mabli J, Ohls J. Supplemental Nutrition Assistance Program participation is associated with an increase in household food security in a national evaluation. J Nutr. Feb 2015;145(2):344–351. 10.3945/jn.114.198697. [DOI] [PubMed] [Google Scholar]
- 3.Nord M, Golla AM. Does SNAP Decrease Food Insecurity? Untangling the Self-Selection Effect. 2009. [Google Scholar]
- 4.Berkowitz SA, Seligman HK, Rigdon J, Meigs JB, Basu S. Supplemental Nutrition Assistance Program (SNAP) participation and health care expenditures among low-income adults. JAMA Intern Med. Nov 1 2017;177(11):1642–1649. 10.1001/jamainternmed.2017.4841. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Samuel LJ, Szanton SL, Cahill R, et al. Does the Supplemental Nutrition Assistance Program affect hospital utilization among older adults? The case of Maryland. Popul Health Manag. Apr 2018;21(2):88–95. 10.1089/pop.2017.0055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Szanton SL, Samuel LJ, Cahill R, et al. Food assistance is associated with decreased nursing home admissions for Maryland's dually eligible older adults. BMC Geriatr. Jul 24 2017;17(1):162. 10.1186/s12877-017-0553-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Pooler JA, Srinivasan M. Association between supplemental Nutrition Assistance Program participation and cost-related medication nonadherence among older adults with diabetes. JAMA Intern Med. Nov 19 2018. 10.1001/jamainternmed.2018.5011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Mayer VL, McDonough K, Seligman H, Mitra N, Long JA. Food insecurity, coping strategies and glucose control in low-income patients with diabetes. Public Health Nutr. Apr 2016;19(6):1103–1111. 10.1017/S1368980015002323. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lauffer S. Characteristics of Supplemental Nutrition Assistance Program Households: Fiscal Year 2016. 2017. [Google Scholar]
- 10.Okoro CA, Hollis ND, Cyrus AC, Griffin-Blake S. Prevalence of disabilities and health care access by disability status and type among adults – United States, 2016. Mmwr – Morbid Mortal Wkly Rep. Aug 17 2018;67(32):882–887. 10.15585/mmwr.mm6732a3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Gaines-Turner T, Simmons JC, Chilton M. Recommendations from SNAP participants to improve wages and end stigma. Am J Public Health. Dec 2019;109(12):1664–1667. 10.2105/Ajph.2019.305362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kaye L, Lee E, Chen YY. Barriers to food stamps in New York state: a perspective from the field. J Poverty. 2013;17(1):13–28. 10.1080/10875549.2012.747995. [DOI] [Google Scholar]
- 13.Pinard CA, Bertmann FMW, Shanks CB, et al. What factors influence SNAP participation? Literature reflecting enrollment in food assistance programs from a social and behavioral science perspective. J Hunger Environ Nut. 2017;12(2):151–168. 10.1080/19320248.2016.1146194. [DOI] [Google Scholar]
- 14.Iwarsson S, Stahl A. Accessibility, usability and universal design—positioning and definition of concepts describing person-environment relationships. Disabil Rehabil. Jan 21 2003;25(2):57–66. 10.1080/dre.25.2.57.66. [DOI] [PubMed] [Google Scholar]
- 15.Thompson S, Johnstone CJ, Thurlow ML. Universal Design Applied to Large Scale Assessments. 2002. [Google Scholar]
- 16.Chan EHW, Lee GKL, Chan ATS. Universal design for people with disabilities: a study of access provisions in public housing estates. Prop Manag. 2009;27(2):138. 10.1108/02637470910946435. [DOI] [Google Scholar]
- 17.Jo G, Habib D, Varadaraj V, et al. COVID-19 vaccine website accessibility dashboard. Disabil Health J. Apr 12 2022, 101325. 10.1016/j.dhjo.2022.101325. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Campanile J, Cerilli C, Varadaraj V, et al. Accessibility and disability inclusion among top-funded U.S. Undergraduate Institutions. medRxiv. 2022. 10.1101/2022.02.17.22271105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Swenor BK, Samuel LJ, Campanile J, et al. The Johns Hopkins Disability Health Research Center SNAP Dashboard. Johns Hopkins Disability Health Research Center. https://disabilityhealth.jhu.edu. [Google Scholar]
- 20.Diamantopoulos A, Winklhofer HM. Index construction with formative indicators: an alternative to scale development. J Mark Res. May 2001;38(2):269–277. 10.1509/jmkr.38.2.269.18845. [DOI] [Google Scholar]
- 21.Bollen K, Lennox R. Conventional wisdom on measurement – a structural equation perspective. Psychol Bull. Sep 1991;110(2):305–314. 10.1037/0033-2909.110.2.305. [DOI] [Google Scholar]
- 22.Huang Z, Benyoucef M. Usability and credibility of e-government websites. Gov Inf Q. Oct 2014;31(4):584–595. 10.1016/j.giq.2014.07.002. [DOI] [Google Scholar]
- 23.WAVE Web Accessibility Evaluation Tool. Version WAVE API https://wave.webaim.org/api/; 2001. [Google Scholar]
- 24.WebAIM (Web Accessibility in Mind). The WebAIM Million. An Annual Accessibility Analysis of the Top 1,000,000 Home Pages. Logan, Utah: Institute for Disability Reesarch, Policy, and Practice; 2021. [Google Scholar]
- 25.Accessibility Guidelines Working Group. Web Content Accessibility Guidelines (WCAG) 2.1, W3C Recommendation. 2018. https://www.w3.org/TR/WCAG21/;05 June 2018. Accessed 21.06.22. [Google Scholar]
- 26.U.S. Department of Agriculture Food and Nutrition Service. Combined Application Projects Guidance for States Developing Projects. 2004. [Google Scholar]
- 27.Goodman N, Morris M, Boston K. Financial Inequality: Disability, Race and Poverty in America. 2019. [Google Scholar]
- 28.Finkelstein A, Notowidigdo MJ. Take-up and targeting: experimental evidence from SNAP. QJ Econ. Aug 2019;134(3):1505–1556. 10.1093/qje/qjz013. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.