Abstract
A growing body of literature highlights measurable indices of pain through facial expressions, vocalizations, body movements, as well as physiological and neural responses. To enhance the reliability of pain monitoring, automated pain assessment has emerged as a promising approach. Although available multimodal pain datasets remain limited, they are steadily increasing, helping to drive research forward. Despite notable progress, this field is still in its early stages. The 6th edition of the AAP workshop continues to seek to highlight current research and foster interdisciplinary collaborations and discussions to accelerate progress in this important area.
Keywords: Pain, Health Informatics, Human Centered Computing, Clinical Datasets
1. Introduction
Pain is a personal experience influenced by personal, psychological, and social factors [1]. Operationalized verbal reports (e.g., 0–10 numeric rating scale) are the standard for pain assessment and are implemented widely in health-care settings as a fifth vital sign [3]. However, concerns about their potential shortcomings are common. Limitations include idiosyncratic use, inconsistent metric properties across scale dimensions, efforts at impression management or deception, differences between clinicians’ and sufferers’ conceptualizations of pain, as well as important instances in which verbal reports cannot be obtained (e.g., in infants, individuals with advanced cognitive impairment, or non-verbal patients).
The advance in the fields of affective computing, computer vision and machine learning for multimodal recognition of human behavior, raised the prospect that advanced technology might overcome some of the constraints limiting objective, valid, and reliable pain assessment in clinical and research settings [2–4]. A substantial body of research demonstrates that a set of facial movements can encode pain intensity across the lifespan, not only in humans but also in several non-human species [3]. In addition, vocalizations [5], physiological and neural responses [6], and body movements [7, 8] have all been identified as containing indicators of pain. Recent advances in sensing technologies for the automatic detection of pain-related markers [6] are enabling both in-hospital and out-of-hospital personalized pain assessment and management support [9–10]. These findings underscore the multimodal nature of pain communication, revealing that no single behavioral measure can fully capture its complexity.
The AAP workshop aims to support research in multimodal computational assessment of pain. Recent computational models have demonstrated significant progress in analyzing facial expressions [11, 12], vocalizations [13], body movements [14], as well as physiological and neural responses [15–18]. Yet pain is fundamentally multimodal, highly variable, culturally shaped, context-sensitive, and deeply personal. Consequently, automated pain assessment systems must navigate, complexity, ambiguity, adapt to individual baselines, and confront pressing ethical issues related to autonomy, privacy, and informed consent. The increasing, though still limited, availability of multimodal datasets capturing both experimentally induced and clinically observed expressions of pain (e.g., [19–24]) is contributing to foster research in this area as well as facilitating a shift from unimodal to increasingly multimodal approaches in automatic pain recognition.
The sixth edition of the Automatic Assessment of Pain (AAP) Workshop, held in 2026, provides a dedicated forum for presenting new research, identifying open challenges, and fostering interdisciplinary collaborations. It responds to the growing demand for interdisciplinary computational methods capable of integrating and interpreting multimodal signals relevant to the expression and experience of pain. The workshop also addresses the critical need for greater awareness of, and contributions to, multimodal datasets that can drive progress in this emerging field.
2. Workshop Content
AAP’26 brings together researchers from affective computing, health technologies, neuroscience, machine learning, and human-computer interaction to collectively advance the state of the art in pain assessment. To discuss how the technological developments remain grounded in the lived experiences of the individuals they are designed to support, the sixth edition of the AAP workshop invited three keynote speakers:
Dr. Serena Moscato, PhD, Assistant Professor at the University of Bologna, Italy, presented her keynote talk titled “Beyond the Clinic: Wearable Sensors for Real-World Pain Assessment”. Her keynote abstract was as follows: “Pain assessment still relies mainly on subjective self-reports and occasional clinical evaluations, which may miss its dynamic and context-dependent nature. Wearable sensors can move assessment beyond the clinic by continuously monitoring physiological signals in real-world settings. This talk will present examples from clinical and ecological studies using wearable data to characterize pain-related patterns, monitor daily and overnight changes, and support automatic pain-state classification. It will also discuss key pitfalls, including signal non-specificity, contextual and clinical confounders, motion artifacts, missingness, and fragmented recordings, highlighting the need for quality-aware, context-aware, and robust analytical approaches to obtain a reliable automatic real-world pain assessment tool.”
Dr. Francesco Cutugno, PhD, Associate Professor of Computational Linguistics at the University of Naples Federico II, Italy, presented his keynote talk titled “Remembering, Feeling, Naming: Multimodal and Textual Approaches to Pain Representation in Cancer Patient Narratives”. His keynote abstract was as follows: “The central aim of the project I will present is to analyze interviews with patients who are not experiencing pain at the time of recording. The interview protocol is designed to elicit a range of affective and cognitive states in participants through targeted questioning. The resulting utterances span several categories: pain-neutral expressions, retrieved memories of acute or breakthrough pain episodes, accounts of joyful moments unrelated to the illness, and metaphorical expressions capturing the perceived essence of pain. Alongside the interviews, biological data were collected via sensors, and the sessions were video-recorded. The resulting multimodal dataset is being analyzed by an interdisciplinary team comprising bioengineers, psychologists, physicians, and linguists. My talk will present preliminary results addressing two complementary strands of the broader project. The first concerns the multimodal analysis of the video recordings, structured along three dimensions: facial expression, vocal attitude, and textual content. The second offers a more focused examination of the textual data, presenting initial attempts to automatically classify the produced utterances using different models of abstract pain representation.”
Dr. Maria Pia Riccio, MD, PhD, Associate Professor of Child Neuropsychiatry at the University of Naples Federico II, Italy, presented her keynote talk titled “Pain perception and neurodevelopmental disorders: clinical issues and future perspectives”. Her keynote abstract was as follows “Abnormal pain reactivity is often reported by the parents of children with autistic spectrum disorder (ASD), other neurodevelopmental disorders and syndromic conditions. This is related to some neurochemical findings frequently reported in autism and to impairment of modulate genes involved in nociceptive circuitry. Autistic people may react paradoxically to particular sensory stimuli. Sometimes they are oblivious to pain, while patients with cerebral palsy, experience pain more frequently from musculoskeletal, neurological and gastrointestinal sites. Also sleep problems are estimated in 18–37% of children with neurodevelopmental disorders and might correlate with pain. In general, complex bidirectional pain and other clinical symptoms exists and might exacerbate pain by lowering pain thresholds. However, alterations of autonomic responses, irritability, facial expressions were not used to express pain in every conditions, but also other manifestations might be observed (e.g., teeth grinding, aggressiveness, etc.). Such abnormal pain reactivity is often reported by the parents, but it is difficult to measure, especially in subjects presenting language compromission and intellectual disability associated. Finding useful tools to objectify the presence of pain and stabilize its intensity level represents a fundamental objective for identifying therapeutic objectives and treatment tools that can be customized for each patient, improving the quality of life.”
CCS Concepts.
• Applied computing; • Life and medical sciences; Health informatics;; • Human-centered computing; • User interface design; User models;
Acknowledgments
This work was partially supported by the National Institute Of Nursing Research of the National Institutes of Health under Award Number R01NR018451. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Contributor Information
Zakia Hammal, Robotics Institute, Carnegie Mellon University, Pittsburgh, USA.
Nadia Bianchi-Berthouze, Interaction Centre, University College London, London, United Kingdom.
Steffen Walter, University Hospital Ulm, Ulm, Germany.
Francesco Cutugno, The University of Naples Federico II, Naples, Italy.
References
- [1].Raja Srinivasa N, Carr Daniel B, Cohen Milton, Finnerup Nanna B, Flor Herta, Gibson Stephen, Keefe Francis, Mogil Jeffrey S, Ringkamp Matthias, Sluka Kathleen A, Song Xue-Jun, Stevens Bonnie, Sullivan Mark, Tutelman Perri, Ushida Takahiro and Vader Kyle. 2021. The revised Intern. Association for the Study of Pain definition of pain: concepts, challenges, and compromises. Pain, 161(9), 1976–1982. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Hammal Zakia and Cohn Jeffrey F.. 2018. Automatic, Objective, and Efficient Measurement of Pain Using Automated Face Analysis. Handbook of Social and interpersonal processes in pain: We don’t suffer alone, 121–146. Springer. [Google Scholar]
- [3].Prkachin Ken and Hammal Zakia. 2021. Computer mediated automatic detection of pain-related behavior: prospect, progress, perils. Frontiers in Pain Research. 2, 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].Prkachin Ken and Hammal Zakia. 2021. Automated Assessment of Pain: Prospects, Progress, and a Path Forward. In Companion Publication of the 2021,ACM International Conference in Multimodal Interaction Workshops. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Lautenbacher Stefan, Salinas-Ranneberg Melissa, Niebuhr Oliver, and Kunz Miriam. 2017., Phonetic characteristics of vocalizations during pain. Pain Report,2(3):e597. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Vitali Diego, Olugbade Temitayo, Eccleston Christoper, Keogh Edmund, Bianchi-Berthouze Nadia, and de C Williams Amanda C. 2024. Sensing behavior change in chronic pain: a scoping review of sensor technology for use in daily life. Pain, 1,165(6):1348–1360. [DOI] [PubMed] [Google Scholar]
- [7].Strand Liv Inger, Gundrosen Kim Fredrik, Lein Regina Küfner, Laekeman Marjan, Lobbezoo Frank, Defrin Ruth, and Husebo Bettina S. 2019.. Body movements as pain indicators in older people with cognitive impairment: A systematic review. Eur J Pain. [DOI] [PubMed] [Google Scholar]
- [8].Sullivan Michael J L, Thibault Pascal, Savard André, Catchlove Richard, Kozey John, and Stanish William D. 2006. The influence of communication goals and physical demands on different dimensions of pain behavior. Pain, 5, 125(3), 270–277. [DOI] [PubMed] [Google Scholar]
- [9].Fritz Roschelle L, Wilson Marian, Dermody Gordana, Schmitter-Edgecombe Maureen, and Cook Diane J. 2020.. Automated Smart Home Assessment to Support Pain Management: Multiple Methods Analysis. J Med Internet Research. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].Singh Aneesha, Bianchi-Berthouze Nadia, and Williams Amanda CdeC. 2017. Supporting Everyday Function in Chronic Pain Using Wearable Technology. In Proceedings of the 2017CHI Conference on Human Factors in Computing Systems (CHI ‘17). Association for Computing Machinery, New York, NY, USA, 3903–3915. [Google Scholar]
- [11].Szczapa Benjamin, Daoudi Mohamed, Berretti Stefano, Pala Pietro, Bimbo Alberto Del and Hammal Zakia. 2022. Automatic Estimation of Self-Reported Pain by Trajectory Analysis in the Manifold of Fixed Rank Positive Semi-Definite Matrices. IEEE Transactions on Affective Computing, 13, 4, 1813–1826 [DOI] [PMC free article] [PubMed] [Google Scholar]
- [12].Erekat Diyala, Hammal Zakia, Siddiqui Maimoon and Dibeklioğlu Hamdi. 2020. Enforcing Multilabel Consistency for Spatio-Temporal Assessment of Shoulder Pain Intensity. In Companion Publication of the 2020 ACM International Conference in Multimodal Interaction Workshops.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [13].Borna Sahar, Haider Clifton R, Maita Karla C, Torres Ricardo A, Avila Francisco R, Garcia John P, De Sario Velasquez Gioacchino D, McLeod Christopher J, Bruce Charles J, Carter Rickey E and Forte Antonio J. 2023. A Review of Voice-Based Pain Detection in Adults Using Artificial Intelligence. Bioengineering, 10(4):500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Olugbade Temitayo, Williams Amanada C.d.C., Gold Nicolas and Bianchi-Berthouze Nadia 2024. Movement Representation Learning for Pain Level Classification. IEEE Transactions on Affective Computing, 15, 3, 1303–1314. [Google Scholar]
- [15].Philipp Werner, Lopez-Martinez Daniel, Walter Steffen, Al-Hamadi Ayoub, Gruss Sascha and Picard Rosalind. 2019. Automatic Recognition Methods Supporting Pain Assessment: A Survey. IEEE Transactions on Affective Computing. [Google Scholar]
- [16].Vianto Jo, Divakaran Anjitha, Yang Hyungjeong, Yeom Soonja, Kim Seungwon, Kim Soohyung, and Shin Jieun. 2025. Multimodal Model for Automated Pain Assessment: Leveraging Video and fNIRS. Applied Sciences 15, no. 9: 5151. [Google Scholar]
- [17].Hammal Zakia, Walter Steffen, Berthouze Nadia (2025). The Fifth Edition of the Automated Assessment of Pain (AAP 2025). In Proceedings of the 27th International Conference on Multimodal Interaction, Canberra, Australia. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Hammal Z, Walter Steffen, Berthouze Nadia Bianchi, Fernandez-Rojas Raul, Seymour Ben, Goecke Roland (2024). Automated Assessment of Pain (AAP) and Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4Pain). The 12th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), September 2024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- [19].Lucey Patrick, Cohn Jeffrey F., Prkachin Kenneth M., Solomon Patricia E. and Matthews Iain. 2011. Painful data: The UNBC-McMaster shoulder pain expression archive database. Proc. IEEE Int. Conf. Autom. Face Gesture Recog., Workshop, 57–64. [Google Scholar]
- [20].Aung Min S H and al. 2016. The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset. IEEE Trans Affect Comput., 7, 4, 435–451. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [21].Olugbade Temitayo et al. 2024. The EmoPain@Home Dataset: Capturing Pain Level and Activity Recognition for People With Chronic Pain in Their Homes. IEEE Transactions on Affective Computing, 16, 2, 471–484. [Google Scholar]
- [22].Walter S, Gruss S, Ehleiter H, et al. 2013. The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system. IEEE International Conference on Cybernetics (CYBCO), 128–131 [Google Scholar]
- [23].Velana M, Gruss S, Layher G, et al. 2017. The SenseEmotion Database: A Multi-modal Database for the Development and Systematic Validation of an Automatic Pain- and Emotion-Recognition System. Schwenker F, Scherer S. (eds) Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction. MPRSS 2016. Lecture Notes in Computer Science, vol 10183. Springer, Cham. [Google Scholar]
- [24].Gruss S, Geiger M, Werner P, Wilhelm O, Al-Hamadi A, Walter S. 2019. Multi-Modal Signals for Analyzing Pain Responses to Thermal and Electrical Stimuli. J Vis Exp., 5, 146. [DOI] [PubMed] [Google Scholar]
