ChatGPT: Artificial Intelligence as a Potential Tool for Parents Seeking Information About Autism

Abstract

Autism Spectrum Disorder has seen a drastic increase in prevalence over the past two decades, along with discourse rife with debates and misinformation. This discourse has primarily taken place online, the main source of information for parents seeking information about autism. One potential tool for navigating information is ChatGPT-4, an artificial intelligence question and answer-style communication program. Although ChatGPT shows great promise, no empirical work has evaluated its viability as a tool for providing information about autism to caregivers. The current study evaluated answers provided by ChatGPT, including basic information about autism, myths/misconceptions, and resources. Our results suggested that ChatGPT was largely correct, concise, and clear, but did not provide much actionable advice, which was further limited by inaccurate references and hyperlinks. The authors conclude that ChatGPT-4 is a viable tool for parents seeking accurate information about autism, with opportunities for improvement in actionability and reference accuracy.

Introduction

Autism Spectrum Disorder (hereafter autism) is one of the most common neurodevelopmental disorders, currently diagnosed in one out of every 36 youth.¹ As the rates of autism have rapidly increased,² so has web-based information surrounding it.³ Given that parents/caregivers use the internet as their primary source when seeking information about autism,⁴ it is more important than ever to ensure the information available online is easily accessible and accurate.

With the sheer amount of online information, evidence-based information coexists among misinformation and myths. Misinformation has been particularly salient since autism's conception, with myths (e.g., the role of vaccines, “refrigerator mothers”) permeating common knowledge even after being debunked.⁵ Separating fact from fiction can be overwhelming, especially when the majority of evidence-based information is presented in scientific writing, which is not always easily interpretable.⁵ One potential solution to this conundrum has emerged in the arena of artificial intelligence: ChatGPT.

ChatGPT was launched in November 2022, boasting its approach as a conversational question-and-answer system.⁶ ChatGPT was constructed and trained by OpenAI and is based on a revised Generative Pretrained Transformer framework currently on the fourth edition.⁶ ChatGPT's natural language proficiency is further bolstered by reinforcement learning from human feedback.⁶ It shows great promise: it can pass licensing exams,⁷ generate literature reviews,⁸ debunk myths and misconceptions about COVID-19 vaccines,⁹ and demonstrate proficiency as a source of health information that could, “challenge conspiracy ideas with clear, concise, and nonbiased content.”⁹ In a recent systematic review, benefits were highlighted in 51/60 studies, underscoring its utility for health care research, including improving health literacy.¹⁰ These potential strengths make ChatGPT an excellent candidate for sharing evidence-based information about autism in an easily accessible format.

With the excitement about ChatGPT's abilities comes skepticism from academic communities.^9,11,12 A systematic review reported concerns regarding its potential ethical issues, limited knowledge, inaccurate citations, and misinformation.¹⁰ Thus, it has the potential to be a valuable resource about autism; and also another contributor to misinformation. The current study sought to characterize ChatGPT's responses to common questions about autism, including questions seeking general information, questions surrounding common myths and misconceptions, and questions seeking resources from a parent perspective.

Method

We conducted an IRB-exempt, pre-registered qualitative search using ChatGPT on April 7, 2023. Thirteen open-ended questions were divided into three sections addressing common questions asked by caregivers: (a) basic information about autism (e.g., “What are the first signs of autism?”; n = 7; Table 2), (b) myths and misconceptions about autism (e.g., “Do vaccines cause autism?”; n = 3; Table 3), and (c) resources (e.g., “How long is the waitlist for autism services?”; n = 3; Table 4). ChatGPT's responses were collected and qualitatively analyzed in a new ChatGPT account with no previous activity. Due to the rapid learning environment, ChatGPT was asked to “regenerate” a second response to each question⁹ and thus provided two codable responses for each question: response 1 (R1) and response 2 (R2). ChatGPT was also asked to provide a list of references for each question.

Table 2.

Basic Information About Autism

Question	R1	R2
1. What is ASD? Please provide references.	ASD is a neurodevelopmental disorder characterized by persistent deficits in social communication and social interaction, and restricted, repetitive patterns of behavior, interests, or activities. It is a spectrum disorder, meaning that the severity and symptoms of the disorder can vary widely among individuals. The diagnosis of ASD is based on clinical observation and assessment, and there is no single diagnostic test. The DSM-5 is the current diagnostic manual used by mental health professionals in the United States and provides the criteria for the diagnosis of ASD. The exact cause of ASD is unknown, but it is believed to involve a combination of genetic and environmental factors. There is no cure for ASD, but early intervention and treatment can help improve outcomes and quality of life for individuals with ASD. References: American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi.org/10.1176/appi.books.9780890425596 National Institute of Mental Health. (2021). Autism spectrum disorder. https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml	ASD is a neurodevelopmental disorder that affects social communication and interaction, as well as the presence of restricted and repetitive behaviors and interests. It is a spectrum disorder because symptoms can range from mild to severe, and individuals with ASD can have different strengths and challenges. The exact causes of ASD are not yet fully understood, but it is believed to be a combination of genetic and environmental factors. Here are some references for further information on ASD: Centers for Disease Control and Prevention (CDC) - Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/index.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml Autism Speaks - What is Autism?: https://www.autismspeaks.org/what-autism American Psychiatric Association - Autism Spectrum Disorder: https://www.psychiatry.org/patients-families/autism/what-is-autism-spectrum-disorder World Health Organization (WHO) - Autism Spectrum Disorders: https://www.who.int/news-room/q-a-detail/autism-spectrum-disorders-(asd)
2. What causes autism? Please provide references.	The exact causes of ASD are not yet fully understood, but it is believed to be a combination of genetic and environmental factors. Studies have suggested that there are several genes associated with ASD, and that there may be interactions between these genes and environmental factors, such as prenatal and perinatal conditions, that can increase the risk of developing ASD. Here are some references for further information on the possible causes of ASD: Centers for Disease Control and Prevention (CDC) - Causes and Risk Factors: https://www.cdc.gov/ncbddd/autism/causes.html National Institute of Mental Health (NIMH) - What Causes Autism Spectrum Disorder?: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml#part_145426 Autism Speaks - What Causes Autism?: https://www.autismspeaks.org/what-causes-autism American Academy of Pediatrics - Autism Spectrum Disorder: What's Causing This Epidemic?: https://www.aappublications.org/news/2018/08/27/autism082718 World Health Organization (WHO) - Autism Spectrum Disorders: https://www.who.int/news-room/q-a-detail/autism-spectrum-disorders-(asd)	The exact causes of ASD are not yet fully understood, but research suggests that it is likely a combination of genetic and environmental factors. Some studies suggest that there may be a genetic predisposition to ASD, as it tends to run in families. Environmental factors, such as exposure to certain chemicals or toxins during pregnancy or early childhood, may also play a role in the development of ASD. Here are some references for further information on the causes of ASD: National Institute of Neurological Disorders and Stroke (NINDS) - Autism Spectrum Disorder Fact Sheet: https://www.ninds.nih.gov/Disorders/Patient-Caregiver-Education/Fact-Sheets/Autism-Spectrum-Disorder-Fact-Sheet Centers for Disease Control and Prevention (CDC) - Causes and Risk Factors for Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/causes.html Autism Science Foundation - Causes of Autism: https://autismsciencefoundation.org/what-is-autism/causes/ Autism Speaks - What Causes Autism?: https://www.autismspeaks.org/what-causes-autism American Psychological Association (APA) - Autism Spectrum Disorder: https://www.apa.org/topics/autism-spectrum-disorder
3. What are the first signs of autism? Please provide references.	The signs and symptoms of ASD can vary widely, and may be different in each individual. However, there are some common signs that may indicate the presence of ASD, particularly in young children. These can include: 1. Delayed or no verbal communication, such as not saying “mama” or “dada” by 12 months of age 2. Lack of eye contact or facial expressions 3. Difficulty with social interaction or playing with others 4. Repetitive behaviors or actions, such as flapping hands or spinning objects 5. Delayed or no response to their name 6. Lack of interest in sharing experiences or interests with others 7. Delayed or no response to gestures, such as pointing or waving It is important to note that not all children with ASD will show these signs, and some children who do may not be diagnosed until later in life. Here are some references for further information on the signs and symptoms of ASD: Centers for Disease Control and Prevention (CDC) - Learn the Signs. Act Early.: https://www.cdc.gov/ncbddd/autism/screening.html Autism Speaks - Early Signs of Autism: https://www.autismspeaks.org/signs-autism National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml Mayo Clinic - Autism Spectrum Disorder: Symptoms & Causes: https://www.mayoclinic.org/diseases-conditions/autism-spectrum-disorder/symptoms-causes/syc-20352928	The first signs of ASD may vary from child to child, but there are some common early indicators that parents and health care providers can look out for. These include delayed or absent social and communication skills, as well as repetitive behaviors or interests. Early detection and intervention can improve outcomes for children with ASD. Here are some references for further information on the first signs of ASD: Centers for Disease Control and Prevention (CDC) - Learn the Signs. Act Early: https://www.cdc.gov/ncbddd/autism/screening.html Autism Speaks - Learn the Signs of Autism: https://www.autismspeaks.org/signs-autism National Institute of Child Health and Human Development (NICHD) - Autism Spectrum Disorder: What You Need To Know: https://www.nichd.nih.gov/health/topics/autism/conditioninfo/signs American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Autism/Pages/default.aspx First Signs - Early Signs of Autism: https://www.firstsigns.org/what-is-asd/early-signs/
4. How is autism diagnosed? Please provide references.	ASD is typically diagnosed through a combination of behavioral evaluations, developmental assessments, and medical screenings. The diagnostic process may involve input from health care providers, educators, and other specialists, and may vary depending on the age and developmental level of the individual being evaluated. Here are some references for further information on the diagnosis of ASD: American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: Evaluation and Diagnosis: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Autism/Pages/Evaluation-and-Diagnosis.aspx Centers for Disease Control and Prevention (CDC) - Diagnosis of Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/screening.html National Institute of Child Health and Human Development (NICHD) - Autism Spectrum Disorder: Diagnosis: https://www.nichd.nih.gov/health/topics/autism/conditioninfo/diagnosis Autism Speaks - Diagnosing Autism Spectrum Disorder: https://www.autismspeaks.org/diagnosing-autism-spectrum-disorder American Psychiatric Association - Autism Spectrum Disorder: Diagnostic Criteria: https://www.psychiatry.org/patients-families/autism/what-is-autism-spectrum-disorder/diagnostic-criteria	Diagnosing ASD typically involves a comprehensive evaluation by a health care professional, which may include observations of the individual's behavior and developmental history, as well as standardized assessments and tests. There is no one specific test that can diagnose ASD, and the diagnostic process may vary depending on the individual's age, symptoms, and needs. Here are some references for further information on the diagnosis of ASD: Centers for Disease Control and Prevention (CDC) - Diagnosing Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/screening.html National Institute of Child Health and Human Development (NICHD) - Autism Spectrum Disorder: Diagnosis: https://www.nichd.nih.gov/health/topics/autism/conditioninfo/diagnosed American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: Evaluation and Diagnosis: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Autism/Pages/Evaluation-Diagnosis.aspx Autism Speaks - Diagnosing Autism: https://www.autismspeaks.org/diagnosis-autism National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Diagnosis: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml#part_153043
5. What are the interventions for autism? Please provide references.	Interventions for ASD can include a range of approaches that aim to improve communication, social skills, and behavior. The most effective interventions are often individualized to the needs of each person with ASD and may involve a combination of behavioral, educational, and medical therapies. Here are some references for further information on interventions for ASD: Centers for Disease Control and Prevention (CDC) - Treatments and Therapies: https://www.cdc.gov/ncbddd/autism/treatment.html Autism Speaks - Treatment: https://www.autismspeaks.org/treatment National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Treatment: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml#part_153049 American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: Treatment and Management: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Autism/Pages/Treatment-and-Management.aspx National Institute of Child Health and Human Development (NICHD) - Autism Spectrum Disorder: Intervention and Support: https://www.nichd.nih.gov/health/topics/autism/conditioninfo/intervention	There are a variety of interventions available for ASD, and the most effective approach will depend on the individual's specific needs and symptoms. Some common interventions include behavioral therapies, educational interventions, and medication. Early intervention can improve outcomes for individuals with ASD and help them reach their full potential. Here are some references for further information on interventions for ASD: Centers for Disease Control and Prevention (CDC) - Autism Treatment: https://www.cdc.gov/ncbddd/autism/treatment.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Treatment and Support: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml#part_153045 Autism Speaks - Treatment and Intervention: https://www.autismspeaks.org/treatment American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: Treatment Options: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Autism/Pages/Treatment-Options.aspx National Autism Center - Findings and Conclusions: National Standards Project, Phase 2: https://www.nationalautismcenter.org/national-standards-project/findings-conclusions/
6. Are certain demographics more likely to be diagnosed with autism? Please provide references.	Research suggests that ASD affects individuals of all races, ethnicities, and socioeconomic backgrounds, although it is more commonly diagnosed in males than females. Some studies have also found higher rates of ASD among certain racial and ethnic groups, such as non-Hispanic white and Asian children. Here are some references for further information on the demographics of ASD: Centers for Disease Control and Prevention (CDC) - Data & Statistics on Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/data.html Autism Speaks - Prevalence of Autism: https://www.autismspeaks.org/autism-facts-and-figures National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Prevalence and Incidence: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml#part_153042 American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: Demographics: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Autism/Pages/Demographics.aspx Autism Science Foundation - Diversity and Autism: https://autismsciencefoundation.org/what-is-autism/diversity-and-autism/	ASD affects individuals of all races, ethnicities, and socioeconomic backgrounds. However, studies have shown that certain demographics may be more likely to be diagnosed with ASD, including males and individuals with a family history of ASD. It is important to note that these trends may be influenced by factors such as differences in access to health care and diagnostic criteria. Here are some references for further information on the demographics of ASD: Centers for Disease Control and Prevention (CDC) - Data & Statistics on Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/data.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Data and Statistics: https://www.nimh.nih.gov/health/statistics/autism-spectrum-disorder.shtml Autism Speaks - Facts and Statistics: https://www.autismspeaks.org/facts-and-statistics American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: Demographics: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Autism/Pages/Demographics.aspx Journal of Autism and Developmental Disorders - Racial and Ethnic Disparities in Diagnosis and Treatment of Children with Autism Spectrum Disorders: https://link.springer.com/article/10.1007/s10803-009-0810-8
7. Is autism becoming more common? Please provide references.	The prevalence of ASD has increased in recent years, but it is unclear whether this is due to a true increase in the number of cases or to improved awareness and diagnostic criteria. Many studies have found an increase in the prevalence of ASD over time, but it is difficult to determine the exact magnitude of this increase and the reasons behind it. Here are some references for further information on the prevalence of ASD: Centers for Disease Control and Prevention (CDC) - Data & Statistics on Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/data.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Data and Statistics: https://www.nimh.nih.gov/health/statistics/autism-spectrum-disorder.shtml Autism Speaks - Facts and Statistics: https://www.autismspeaks.org/facts-and-statistics Journal of the American Academy of Child and Adolescent Psychiatry - Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years - Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2016: https://www.sciencedirect.com/science/article/pii/S0890856719310103 Pediatrics - Changing Demographics of Autism Spectrum Disorder: https://pediatrics.aappublications.org/content/133/3/472	The prevalence of ASD has increased in recent years, although it is unclear whether this is due to a true increase in the number of individuals with ASD or to improved diagnostic and screening methods. The CDC estimates that ∼1 in 54 children in the United States have ASD, based on data from the agency's ADDM Network. Here are some references for further information on the prevalence of ASD: Centers for Disease Control and Prevention (CDC) - Data & Statistics on Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/data.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Data and Statistics: https://www.nimh.nih.gov/health/statistics/autism-spectrum-disorder.shtml Autism Speaks - Facts and Statistics: https://www.autismspeaks.org/facts-and-statistics Journal of the American Academy of Child and Adolescent Psychiatry - Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years: Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2016: https://www.sciencedirect.com/science/article/pii/S0890856719332614 Pediatrics - Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years: 2014–2016 Autism and Developmental Disabilities Monitoring Network: https://pediatrics.aappublications.org/content/147/4/e20200776

ADDM, Autism and Developmental Disabilities Monitoring; ASD, Autism Spectrum Disorder; CDC, Centers for Disease Control and Prevention; DSM-5, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; R1, Response 1, R2, Response 2.

Table 3.

Myths/Misconceptions About Autism

Question	R1	R2
1. Do vaccines cause autism? Please provide references.	No, vaccines do not cause autism. Numerous studies have been conducted on this topic, and there is no evidence to support a link between vaccines and autism. The original study that suggested a link between the MMR vaccine and autism has been thoroughly discredited and retracted by the journal that published it. Here are some references for further information on the link between vaccines and autism: Centers for Disease Control and Prevention (CDC) - Vaccines Do Not Cause Autism: https://www.cdc.gov/vaccinesafety/concerns/autism.html World Health Organization (WHO) - Vaccines and Autism: What Do We Know?: https://www.who.int/news-room/q-a-detail/vaccines-and-autism-what-do-we-know National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Treatment and Support: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml#part_153045 American Academy of Pediatrics (AAP) - Immunizations and Autism: A Review of the Literature: https://www.aap.org/en-us/Documents/immunization_vaccine_studies.pdf Journal of Autism and Developmental Disorders - Vaccines and Autism: A Tale of Shifting Hypotheses: https://link.springer.com/article/10.1007/s10803-011-1257-0	No, vaccines do not cause autism. The idea that vaccines could cause autism was based on a 1998 study that has since been discredited and retracted by the journal that published it. Numerous studies since then have shown no link between vaccines and autism. Here are some references for further information on the link between vaccines and autism: Centers for Disease Control and Prevention (CDC) - Vaccines Do Not Cause Autism: https://www.cdc.gov/vaccinesafety/concerns/autism.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: Data and Statistics: https://www.nimh.nih.gov/health/statistics/autism-spectrum-disorder.shtml World Health Organization (WHO) - Vaccine Safety: No Link Between Vaccines and Autism: https://www.who.int/vaccine_safety/committee/topics/thiomersal/Jun_2012/en/ American Academy of Pediatrics (AAP) - Vaccine Safety & Autism: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/immunizations/Pages/Vaccine-Safety-Autism.aspx Journal of the American Medical Association - Vaccines and Autism: A Tale of Shifting Hypotheses: https://jamanetwork.com/journals/jama/fullarticle/2275444
2. Does autism mean my child will have savant skills? Please provide references.	No, autism does not mean that your child will have savant skills. While some individuals with autism do possess extraordinary abilities in specific areas, such as music, math, or art, this is not a universal characteristic of autism. Most individuals with autism have a wide range of abilities and challenges, and their skills and strengths vary widely. Here are some references for further information on the relationship between autism and savant skills: National Institute of Neurological Disorders and Stroke (NINDS) - Savant Syndrome: https://www.ninds.nih.gov/Disorders/All-Disorders/Savant-Syndrome-Information-Page Autism Research Institute - Savant Syndrome: https://www.autism.org/savant-syndrome/ Verywell Health - Do All People With Autism Have Savant Skills? https://www.verywellhealth.com/do-all-people-with-autism-have-savant-skills-260205 Scientific American - Why Are Savant Skills Associated with Autism? https://www.scientificamerican.com/article/why-are-savant-skills-associated-with-autism/ Journal of Autism and Developmental Disorders - Savant Skills in Autism Spectrum Disorder: An Overview: https://link.springer.com/article/10.1007/s10803-012-1525-x	No, having ASD does not necessarily mean that an individual will have savant skills. While there is a higher prevalence of savant skills among individuals with ASD compared to the general population, the majority of individuals with ASD do not have savant abilities. Savant syndrome is a rare condition in which individuals with intellectual or developmental disabilities, including ASD, have exceptional abilities in specific areas such as memory, mathematics, music, or art. However, only a small percentage of individuals with ASD have savant skills. Here are some references for further information on the relationship between ASD and savant skills: Autism Research Institute - Savant Skills: https://www.autism.org/savant-skills/ National Center for Biotechnology Information (NCBI) - Savant Syndrome and Autism: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6780733/ Journal of Autism and Developmental Disorders - The Prevalence and Correlates of Savant Syndrome in Autism Spectrum Disorder and Other Neurodevelopmental Disorders: https://link.springer.com/article/10.1007/s10803-015-2564-9 Frontiers in Neuroscience - Savant Syndrome and Its Possible Relationship to Epilepsy: https://www.frontiersin.org/articles/10.3389/fnins.2021.630023/full
3. Will my child outgrow autism? Please provide references.	ASD is a lifelong developmental disorder, and while individuals with ASD may experience improvements in their symptoms over time, they do not typically outgrow their autism. Research has shown that early intervention and targeted therapies can improve the outcomes for individuals with ASD and help them develop the skills needed to lead more independent and fulfilling lives. However, there is no cure for ASD and individuals with this disorder will continue to experience challenges related to social interaction, communication, and behavior throughout their lives. Here are some references for further information on the lifelong nature of ASD: Centers for Disease Control and Prevention (CDC) - Autism Spectrum Disorder (ASD): https://www.cdc.gov/ncbddd/autism/facts.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml Autism Society - About Autism: https://www.autism-society.org/what-is/ American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Pages/Autism-Spectrum-Disorder.aspx Journal of the American Academy of Child and Adolescent Psychiatry - Long-Term Outcomes of Children With Autism Spectrum Disorders: A Review of Research in Population-Based Cohorts: https://www.sciencedirect.com/science/article/pii/S0890856717310345	ASD is a lifelong condition, and while some individuals with ASD may make significant progress in their social, communication, and behavioral skills over time, most will continue to experience challenges associated with the condition throughout their lives. There is currently no known cure for ASD. However, early intervention and support can significantly improve outcomes for individuals with ASD, and many individuals with ASD can lead fulfilling lives with appropriate support and accommodations. Here are some references for further information on the outlook for individuals with ASD: Centers for Disease Control and Prevention (CDC) - Autism Spectrum Disorder: https://www.cdc.gov/ncbddd/autism/facts.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml Autism Speaks - What is Autism?: https://www.autismspeaks.org/what-autism American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: https://www.aap.org/en-us/about-the-aap/Committees-Councils-Sections/Council-on-Children-with-Disabilities/Pages/Autism-Spectrum-Disorder.aspx National Autism Association - Long-Term Outcomes for Individuals with Autism: https://nationalautismassociation.org/resources/autism-fact-sheet/long-term-outcomes-for-individuals-with-autism/

MMR, measles, mumps, and rubella.

Table 4.

Searching for Resources

Question	R1	R2
1. If I think my child has autism, what should I do next? Please provide references.	If you suspect that your child may be showing signs of ASD, it is important to seek an evaluation and diagnosis from a qualified health care professional. The earlier the diagnosis, the earlier intervention and support can be provided to help your child reach their full potential. Here are some steps you can take if you suspect your child may have ASD: 1. Talk to your child's health care provider: Your child's pediatrician or family doctor can help you determine if your child needs further evaluation for ASD and refer you to a specialist. 2. Seek a comprehensive evaluation: A comprehensive evaluation for ASD typically involves a team of health care professionals, including a developmental pediatrician, psychologist, speech-language pathologist, and occupational therapist. The evaluation may include observation of your child's behavior and interactions, developmental assessments, and interviews with you and other caregivers. 3. Connect with support services: If your child is diagnosed with ASD, there are a variety of support services and resources available to help your child and family, including early intervention services, therapy, educational services, and support groups. Here are some references for further information on what to do if you suspect your child may have ASD: Centers for Disease Control and Prevention (CDC) - If You're Concerned: https://www.cdc.gov/ncbddd/autism/screening.html National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml Autism Society - First Steps: https://www.autism-society.org/living-with-autism/first-steps/ American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: https://www.aap.org/en-us/about-the-aap/Committees-Councils-Sections/Council-on-Children-with-Disabilities/Pages/Autism-Spectrum-Disorder.aspx Autism Speaks - How to Get a Diagnosis: https://www.autismspeaks.org/diagnosis/next-steps-after-diagnosis	If you suspect that your child may be showing signs of ASD, it is important to seek professional evaluation and support as soon as possible. Early intervention and support can significantly improve outcomes for individuals with ASD. Here are some steps you can take if you suspect that your child may have ASD: 1. Talk to your child's doctor or pediatrician: Share your concerns with your child's doctor, who can refer you to a specialist or diagnostic center for further evaluation. 2. Seek a comprehensive evaluation: A comprehensive evaluation for ASD typically involves a multidisciplinary team, including a developmental pediatrician, psychologist, speech-language pathologist, and occupational therapist. 3. Connect with support services: If your child is diagnosed with ASD, connect with local support services and resources, such as early intervention programs, support groups, and advocacy organizations. Here are some references for further information on what to do if you suspect that your child may have ASD: Centers for Disease Control and Prevention (CDC) - If You're Concerned: https://www.cdc.gov/ncbddd/autism/concerned.html Autism Speaks - First Concerns to Action: https://www.autismspeaks.org/first-concerns-action National Institute of Mental Health (NIMH) - Autism Spectrum Disorder: How to Recognize the Signs and Symptoms: https://www.nimh.nih.gov/health/publications/autism-spectrum-disorder-how-to-recognize-the-signs-and-symptoms/index.shtml American Academy of Pediatrics (AAP) - Autism Spectrum Disorder: Evaluation and Diagnosis: https://www.aap.org/en-us/about-the-aap/Committees-Councils-Sections/Council-on-Children-with-Disabilities/Pages/Autism-Spectrum-Disorder-Evaluation-and-Diagnosis.aspx
2. Where should I go for more information about autism? Please provide references.	Here are some reliable sources of information and support for ASD: 1. Autism Speaks: A leading autism advocacy organization that provides a wealth of information and resources for individuals with ASD and their families. https://www.autismspeaks.org/ 2. Centers for Disease Control and Prevention (CDC): A government agency that provides comprehensive information on ASD, including facts, diagnosis, treatment, and research. https://www.cdc.gov/ncbddd/autism/index.html 3. National Institute of Mental Health (NIMH): A government agency that conducts research on mental health conditions, including ASD, and provides information on the latest advances in diagnosis and treatment. https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml 4. Autism Society: A national organization that provides information and support for individuals with ASD and their families. https://www.autism-society.org/ 5. American Academy of Pediatrics (AAP): A professional organization of pediatricians that provides guidance on the diagnosis and management of ASD. https://www.aap.org/en-us/about-the-aap/Committees-Councils-Sections/Council-on-Children-with-Disabilities/Pages/Autism-Spectrum-Disorder.aspx 6. National Autism Association: A nonprofit organization that provides support and resources for individuals with ASD and their families. https://nationalautismassociation.org/7. Organization for Autism Research: A nonprofit organization that funds research on ASD and provides educational resources for families and professionals. https://researchautism.org/ These resources can provide further information on ASD, support, and resources for individuals with ASD and their families, and the latest research and treatment options.	There are many reputable sources of information and support for individuals with ASD and their families. Here are some resources that you may find helpful: 1. Autism Speaks: Autism Speaks is the world's leading autism science and advocacy organization, dedicated to promoting solutions across the spectrum and throughout the life span. Their website provides a wide range of information on autism, including diagnosis, treatment, and support services. https://www.autismspeaks.org/ 2. Centers for Disease Control and Prevention (CDC): The CDC provides information on autism, including data and statistics, screening and diagnosis, treatment options, and resources for families. https://www.cdc.gov/ncbddd/autism/index.html 3. National Institute of Mental Health (NIMH): NIMH provides information on autism, including research updates, treatment options, and resources for families. https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml 4. Autism Science Foundation (ASF): The ASF is a nonprofit organization that funds autism research and provides information on the latest findings in the field. Their website also provides resources for families and educators. https://autismsciencefoundation.org/ 5. Autism Society of America (ASA): The ASA is a nonprofit organization that provides advocacy, education, and support for individuals with autism and their families. Their website provides information on autism, resources for families, and local chapters across the United States. https://www.autism-society.org/ 6. American Academy of Pediatrics (AAP): The AAP provides information on autism and other developmental disorders, including diagnosis and treatment options, as well as resources for families. https://www.aap.org/en-us/Pages/Default.aspx These resources are just a few examples of the many organizations that provide information and support for individuals with ASD and their families. It is important to find trusted sources of information and support that meet your specific needs.
3. How long is the waitlist for autism services? Please provide references.	The waitlist for autism services can vary depending on the country, region, and specific services that you are seeking. In some areas, there may be long waitlists for diagnosis, therapy, and support services for individuals with ASD. Here are a few references that provide information on wait times for autism services in different areas: 1. Canada: According to a 2020 report by the Canadian Autism Spectrum Disorders Alliance (CASDA), the wait time for a diagnostic assessment for autism can range from 6 months to 2 years, depending on the province. The report also notes that waitlists for autism services can be up to 4 years long in some areas. (Source: https://www.casda.ca/wp-content/uploads/2020/06/CASDA_Report_on_the_State_of_Autism_in_Canada_2020.pdf) 2. United Kingdom: A 2018 report by the National Autistic Society (NAS) found that the average wait time for an autism diagnosis in the UK was 2.5 years. The report also noted that waitlists for support services can be equally long. (Source: https://www.autism.org.uk/get-involved/media-centre/news/2018-10-11-wait-for-autism-diagnosis.aspx) 3. United States: Wait times for autism services in the US can vary widely depending on the state and specific services needed. According to a 2019 report by the Autism Speaks Autism Treatment Network, waitlists for autism services can range from a few weeks to over a year. (Source: https://www.autismspeaks.org/autism-treatment-network-release-report-wait-times-and-insurance-barriers-autism-services) It is important to note that these wait times are subject to change and may not reflect current wait times in your area. If you are seeking autism services for yourself or a loved one, it is best to contact local providers and organizations for the most up-to-date information on wait times and availability of services.	The waitlist for autism services can vary depending on the location, type of services needed, and the age of the individual. In some areas, waitlists can be long, while in others, services may be more readily available. It is best to contact your local service provider or health authority for the most up-to-date information on wait times in your area. Here are some references with information on waitlists for autism services: Autism Speaks: Autism Speaks provides information on waitlists for autism services and suggests reaching out to local autism organizations and advocacy groups for assistance in navigating the system. https://www.autismspeaks.org/tool-kit-excerpt/waiting-lists National Autism Center: The National Autism Center provides information on access to autism services and notes that waitlists can vary depending on the location and type of services needed. https://www.nationalautismcenter.org/resources/access-to-autism-services/ Centers for Disease Control and Prevention (CDC): The CDC provides information on accessing autism services and notes that waitlists can be a barrier to receiving services. They suggest reaching out to local service providers for more information on wait times. https://www.cdc.gov/ncbddd/autism/treatment.html Autism Society of America (ASA): The ASA provides information on accessing autism services and notes that waitlists can vary depending on the location and the type of services needed. They suggest contacting local service providers for more information on wait times. https://www.autism-society.org/living-with-autism/access-to-services/ It is important to note that waitlists for autism services can be long, and accessing services may require persistence and advocacy. It is best to reach out to local service providers and advocacy organizations for support in navigating the system and accessing the services needed.

Coding

Accuracy

The criteria for evaluating ChatGPT responses were replicated from previous research evaluating ChatGPT's responses to COVID-19 questions⁹ and included the “3Cs”: (a) scientific accuracy of content (Correctness), (b) Clarity of response, and (c) Conciseness (degree to which all the available knowledge is conveyed). Scores for the Correctness, Clarity, and Conciseness are scored on a scale from one to four (4  = Completely correct, clear, or concise, 3 = Almost correct, clear, or concise, 2 = Partially correct, clear, or concise, 1 = Completely incorrect, unclear, or unconcise). Scores on the 3C domains were then averaged to one overall 3C score, replicating previous research.⁹

Language

We assessed language use by ChatGPT to (a) mirror a bias measure used in previous research,⁹ and (b) reflect recent debates about language preferences in the autistic community.¹³ Language was coded according to recent recommendations related to “medical” versus “neurodiversity-affirming” language guidelines for discussing autism^a,¹³ (1 = use of medical language, 2 = use of a combination of medical and neurodiversity-affirming language, 3 = use of neurodiversity-affirming language).

Understandability and actionability

Responses were assessed for Understandability and Actionability using the Patient Education Materials Assessment Tool for Printable materials (PEMAT-P). The PEMAT-P is a validated instrument used to assess how digestible and actionable printed patient education materials are for a lay audience.¹⁴ ChatGPT responses were coded for understandability (n = 10 items) and actionability (n = 6 items). Each PEMAT-P statement is scored as 0 (disagree) or 1 (agree). Understandability and Actionability are reported in percentages, wherein scores are averaged across construct items. As per PEMAT-P guidelines,¹⁴ n = 8 codes were excluded due to the short nature of ChatGPT's responses.

References

References were evaluated for hyperlink accuracy (0 = incorrect Uniform Resource Locator [URL], 1 = correct URL) and date published. As responses often included multiple references, reference scores were coded as percentages of references that met the above criteria for each question (e.g., 80 percent URLs were correct).

Interrater reliability

ChatGPT responses were generated for 13 questions. Each evaluator independently assessed responses, followed by a comparison of the scores to assess the degree of interrater agreement for the seven scores. Interrater reliability was assessed using intraclass correlations^b for continuous measures (References, Understandability, Actionability) and weighted Cohen's Kappa^c for categorical measures (3C's and Language¹⁵). Rater disagreements were resolved by consensus. The coding manual is available on OSF. Interrater reliability ranged from fair to near perfect (Table 1).

Table 1.

Interrater Reliability Metrics

	Weighted Cohen's Kappa				Intraclass correlation
	Correctness	Clarity	Conciseness	Language	Understandability	Actionability	Reference URL
Interrater reliability	0.614	0.455	0.346	0.476	1.000	0.934	0.913

ICCs >0.90 and Kappa >0.80 = near-perfect reliability.

ICCs 0.75–0.9 and Kappa 0.61–0.81 = substantial reliability.

ICCs 0.5–0.75 and Kappa 0.4–0.6 = moderate reliability.

Kappa 0.21–0.41 = fair reliability.

ICCs <0.5 = poor reliability.

ICCs, intraclass correlations; URL, Uniform Resource Locator.

Results

Basic information about autism

Basic information questions and responses are available in Table 2. The average 3C score was 3.67 out of a maximum possible 4.0 (standard deviation [SD] = 0.27) for R1 and 3.76 (SD = 0.42) for R2; there was no significant difference between responses, t(6) = −0.548, p = 0.604. PEMAT-P Understandability was an average of 78 percent for R1 and R2. PEMAT-P Actionability was 0 percent for R1 and R2. Lastly, language was predominantly medical, comprising 85.7 percent (R1) and 71.4 percent (R2) of text. The two responses did not significantly differ in language use, X² = 0.467, p = 0.495.

Myths and misconceptions about autism

Myth-related questions are available in Table 3. The average 3C score was 3.56 (SD = 0.19) for R1 and 3.33 (SD = 0.88) for R2; there was no significant difference between responses, t(2) = 0.378, p = 0.742. PEMAT-P understandability was an average of 78 percent for R1 and 75 percent for R2; there was no significant difference between responses, t(2) = 0.718, p = 0.547. PEMAT-P actionability was 0 percent for both R1 and R2. Lastly, language was evenly split between neurodiversity-affirming (33 percent), medical (33 percent), and a combination (33 percent), and did not significantly differ between responses, X² = 6.00, p = 0.199.

Autism resources

Resource questions are available in Table 4. The average 3C score was 3.44 (SD = 0.38) for R1 and 3.89 (SD = 0.19) for R2; there was no significant difference between responses, t(2) = −4.00, p = 0.057. Understandability from the PEMAT-P was an average of 79 percent for R1 and 78 percent for R2; there was no significant difference between responses, t(2) = 1.00, p = 0.423. PEMAT-P actionability was an average of 40 percent for R1 (SD = 0.20) and 47 percent for R2 (SD = 0.12); there was no significant difference between responses, t(2) = −1.00, p = 0.423. Lastly, language was a combination of medical and neurodiversity-affirming, comprising 67 percent of text from R1 and R2. The two responses did not significantly differ in language use, X² = 0.750, p = 0.386.

Quality of references

Functional hyperlinks were provided for 42 percent of references, with no significant differences between R1 and R2 for accuracy, t(12) = 0.880, p = 0.396 (M_R1 = 45.5 percent, SD = 0.32; M_R2 = 39.1 percent, SD = 0.30). When dates were available for references that existed (n = 57/123 total references, 46 percent), the range of dates was from 2006 to 2023 (Mode = 2023).

Domain similarities and differences

Scores on the three domains of interest (Basic Information, Myths/Misconceptions, Resources) were submitted to a one-way analysis of variance (ANOVA) to evaluate whether ChatGPT was more accurate, understandable, or actionable in one domain of questions over the other. Intuitively, actionability was significantly higher in the Resources domain compared with Basic Information or Myths/Misconceptions domains, R1: F(2,12) = 23.077, p < 0.001; R2: F(2,12) = 94.231, p < 0.001. No other significant differences emerged (ps > 0.401). Table 5 includes average scores for primary outcomes of interest for ChatGPT's overall responses as well as domain-specific responses.

Table 5.

ChatGPT Scores Overall and Within Question Domain

	Average 3C score			Average understandability percentile			Average actionability percentile			Average reference link accuracy
	R1	R2	t	R1 (percent)	R2 (percent)	t	R1 (percent)	R2 (percent)	t	R1 (percent`)	R2 (percent)	t
Basic information	3.67	3.76	−0.548	78	78	0.548	0	0	N/A	57	46	1.082
Myths and misconceptions	3.56	3.33	0.378	78	75	0.547	0	0	N/A	27	27	0.000
Resources	3.44	3.89	−4.00	79	78	1.000	40	47	−1.000	37	36	0.052
Overall	3.59	3.69	−0.634	78	77	1.105	9.2	10.8	−1.000	46	39	0.880

^*** p < 0.001, ^**p < 0.01, ^*p < 0.05.

t, repeated measures t-test result; N/A, not applicable.

Discussion

In the current climate where the internet serves as a primary source of medical information for consumers,⁴ the aims of this investigation were to evaluate whether ChatGPT could be an accurate, reliable, and useful tool for parents/caregivers seeking autism information. Using 13 questions across three domains, we scored ChatGPT's responses on correctness, conciseness, clarity, language use, understandability, actionability, and reference accuracy.

Was ChatGPT correct, concise, and clear?

ChatGPT produced accurate, concise, and clear information as indicated by domain-specific and overall scores on the 3C metric. Responses were, on average, completely clear, concise, and accurate. Of the three, ChatGPT had lower Conciseness scores. Some responses provided too much information, whereas others did not convey the information required to answer the question. The only areas of inaccuracy observed were due to lack of updated information (i.e., outdated statistics), which is a limitation noted by ChatGPT's creators. Importantly, the correctness of information was maintained when asked about common myths and misconceptions, which suggests ChatGPT is a viable tool for combating stigma and misinformation commonly found online.

Does ChatGPT produce actionable and understandable information?

ChatGPT's content was evaluated for actionability and understandability, two crucial elements for widespread use by a lay audience. Whereas understandability was high for ChatGPT's responses, lending support to its role as a lay-audience friendly source of information, actionability was limited with 0 percent for “Basic Information” and “Myths/Misconceptions,” and 44 percent for “Resources.” Although next steps may not be needed for understandability, users would benefit from more actionable recommendations.

Does ChatGPT provide accurate references?

A previously noted area of weakness,¹⁶ references were often incorrect. Fewer than half the hyperlinks worked, let alone took the reader to the correct website. Incorrect references frequently generated a 404 error code and occasionally took viewers to unrelated websites. When links were correct, ChatGPT frequently cited relevant medical and government webpages (i.e., Centers for Disease Control and Prevention [CDC], World Health Organization [WHO]), which provided up-to-date references. The likely cause for this discrepancy is the outdated nature of ChatGPT sources (updated through 2021).¹⁷ The inaccuracy of the references significantly hindered the requester's actionability, especially in the Resources domain, which represents a significant limitation and area of weakness.

What language did ChatGPT predominantly use?

Although ChatGPT used a combination of medical and neurodiversity-affirming language throughout, it primarily used “medical” language observed in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition.¹⁸ This may reflect the language used in the predominantly medical and psychiatric resources where ChatGPT gets its information. As recent debates about language use have emerged in the autism research community,^13,19 this may be an area of growth as ChatGPT adapts to specific user preferences and societal shifts.

Did regeneration improve ChatGPT's responses?

ChatGPT added a “regenerate” response feature to capture the quickly changing learning environment and to enable human feedback to improve responses.⁶ As such, we were interested in whether a regenerated response to the same question would improve ChatGPT's scores. Our results suggested no significant differences on any scores between R1 and R2, suggesting no significant improvement at this time.

Conclusion

ChatGPT is a viable tool for parents/caregivers seeking information about autism. It provides responses that are clear, concise, accurate, and understandable to the public, but is limited by inaccurate references and hyperlinks. However, overall, as a tool to acquire information, learn more about their child's potential presentation, and combat myths and misconceptions, ChatGPT is a valuable instrument for parents and caregivers.

Notes

• a. Examples of medical language include “person with autism spectrum disorder,” “disorder,” “comorbid,” “risk of autism.” Examples of neurodiversity-affirming language include “autistic person,” “autism,” “co-occurring,” and “elevated likelihood for autism” to name a few. For the full list, see Table 1 in Bottema-Beutel et al., 2021.

• b. Intraclass correlations (ICC's) were interpreted according to Koo and Li²⁰: <0.50 = poor reliability, 0.5–0.75 = moderate reliability, 0.75–0.9 = good reliability, and >0.9 = excellent reliability.

• c. Kappa values were interpreted according to McHugh²¹: 0–0.20 = no to slight agreement, 0.21–0.40 = fair, 0.41–0.60 = moderate agreement, 0.61–0.80 = substantial agreement, 0.80–1.00 = almost perfect agreement.

Authors' Contributions

T.C.M.: conceptualization, methodology, software, validation, formal analysis, investigation, data curation, writing––original draft, writing––review and editing, visualization, and project administration. S.B.: software, validation, formal analysis, investigation, data curation, and writing––review and editing. O.P.: conceptualization, methodology, and writing––review and editing. C.H.: conceptualization, methodology, writing––review and editing, resources, and supervision.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This research was supported by a training grant from the U.S. Department of Education (H325D180099; O.P.) and a training fellowship from NICHD (T32 HD040127-21; T.C.M.).