Abstract
Patient and Public Involvement and Engagement (PPIE) is essential for high-quality healthcare research, yet significant challenges persist in achieving diverse input. Traditional PPIE panels can struggle with recruitment limitations, geographical constraints, and resource intensity, resulting in panels that may not reflect population diversity or lived experiences. In order to address these challenges, we developed Panelyze, an AI-powered co-production system to augment existing PPIE approaches. Panelyze follows a five-step workflow: (1) programmatic generation of synthetic personas and panels based on census data and lived experiences; (2) semantic analysis of research proposals; (3) generation of the synthetic panel's discussions; (4) generation of PPIE co-production artefacts (e.g., Plain English Summaries, Infographics); and (5) generation of the Panelyze Score, a novel algorithmic metric evaluating healthcare proposals against UK Standards for Public Involvement. We validated the system using the CAPRIE-2 cardiovascular research proposal. Results demonstrated the system's capacity to simulate qualitative dialogue, identifying critical socio-economic barriers (e.g., wage loss from clinic visits) and validating terminology choices (e.g., drug naming conventions). Critically, the system functions as an augmentation tool, enabling research leads to stress-test and refine their proposals against a synthetic audience. This process amplifies missing voices rarely heard in physical meetings and generates accessible materials that facilitate easier adoption and higher-quality engagement for subsequent human panels.
Keywords: co-production, digital health, health equity, large language model, natural language processing, patient and public involvement and engagement, synthetic personas, text analytics
1. Introduction
Patient and Public Involvement and Engagement (PPIE) is widely recognised as a prerequisite for ethical and effective healthcare research (1). However, despite widespread recognition of its importance, the operational reality often falls short. Panels are frequently dominated by “frequent flyers” (2)—available, articulate individuals from majority demographics—while marginalised voices remain absent (3). This representational failure undermines the legitimacy and effectiveness of healthcare research designed to serve diverse populations. Furthermore, the materials presented to these panels—often dense, academic proposals—create cognitive barriers that prevent lay contributors from exercising genuine influence (4).
This paper introduces Panelyze™, a generative AI framework designed to bridge the gap between healthcare research design and public understanding. Unlike general-purpose Large Language Model (LLM) applications (5, 6), Panelyze employs a structured, multi-agent architecture to simulate diverse personas and generate the materials necessary for effective human engagement. We present the system's five-step workflow and introduce the Panelyze Score, a novel algorithmic metric derived from the UK Standards for Public Involvement (7). While the Score provides a quantifiable baseline, the system's primary innovation lies in its ability to surface qualitative “lived experience” insights in silico, allowing researchers to iteratively identify and resolve structural barriers in addition to engaging real patients.
Beyond its immediate utility in the co-production of PPIE panels, this framework addresses a critical dimension of technology adoption in the social sciences: AI for Inclusivity. While AI is often critiqued for perpetuating bias, this research explores its potential as a tool for public policy equity. By deliberately engineering synthetic agents to embody under-represented demographics, Panelyze leverages the capabilities of LLMs to shift the window of engagement, ensuring that digital health interventions are not designed solely by, and for, majority populations.
2. Materials and methods
2.1. System architecture
The Panelyze platform is built on a React and Node.js stack and is deployed as a web application. It features a bespoke prompt engineering framework developed specifically for PPIE use cases, enabling experimentation with a range of State-of-the-art (SOTA) Large Language Models (LLMs). The core simulation engine architecture is designed to ground LLM outputs in specific technical contexts, utilising text analytics to bridge the gap between unstructured clinical research protocols and structured, programmatically defined persona prompts.
Central to this architecture is the AIServiceFactory, a task-oriented orchestration layer that programmatically manages model selection and deployment. The AIServiceFactory acts as a traffic controller, routing specific linguistic and analytical tasks to a picklist of frontier models from providers including Anthropic, Google, and OpenAI. By abstracting the underlying API logic, the factory ensures the framework remains model-agnostic while ensuring that each stage of the research workflow is handled by the most capable agent.
In the default configuration, model assignment is optimised for distinct workflow requirements: Claude 4.5 is used for its high-context narrative capabilities to generate rich synthetic persona backstories and consistent panel transcripts; GPT-5 serves as the primary “Assessor Agent” to evaluate proposals against established NIHR standards (the Panelyze Score); and Gemini 2.5 Flash is employed for the generation of visual co-production artefacts and infographics. This orchestration facilitates a transparent, human-AI interaction loop, as detailed in Figure 1, ensuring that AI-generated insights are always subject to researcher oversight before implementation. In the current implementation, cross-validation of model outputs is not performed programmatically; rather, output fidelity is assured through the mandatory Human-in-the-Loop review phase described in Section 2.2. The introduction of an independent “Judge Model”—an AI agent tasked with auditing the fidelity and utility of generated personas and panel discussions—is identified as a near-term development priority and forms part of the platform's ongoing roadmap.
Figure 1.
Panelyze high-level workflow, architecture and outputs. Displays the 5-step workflow, key architectural components, and qualitative and quantitative outputs from Panelyze.
2.2. Generation of diverse synthetic personas and panels
To address PPIE representation gaps, the system leverages the inherent flexibility of software to programmatically model synthetic panels. Unlike real human panels, which are constrained by recruitment logistics, Panelyze allows researchers to configure a comprehensive set of persona attributes to reflect the target population. These configurable attributes span three categories: (1) the nine protected characteristics defined under the Equality Act 2010 (11); (2) socio-demographic variables including occupation, employment status and geographic location; and (3) narrative attributes comprising a free-text lived experience backstory that contextualises how the persona relates to the research domain.
This is achieved through a two-stage grounded generation process. First, to ensure statistical fidelity, the system performs runtime injection of aggregated census statistics [from Scotland Census 2022 (8), England & Wales Census 2021 (9), and US Census 2020 (10)] directly into the LLM system prompts as structured reference data. Rather than embedding raw census tables, the system extracts and formats the relevant distributional statistics for the target geography (e.g., age band proportions, ethnic group percentages, and occupational classification breakdowns) as a compact structured block within the prompt context. A tolerance check is then applied programmatically: the system calculates the attribute distribution across all generated personas and validates that each dimension falls within a predefined variance threshold (±10%) of the corresponding census baseline. If the distribution exceeds this tolerance, persona generation is re-triggered until the panel achieves acceptable demographic fidelity before proceeding to the simulation stage.
Second, the framework incorporates a critical “Human-in-the-Loop” phase before the simulation begins. Researchers have full flexibility to manually review and edit all generated persona attributes, including the protected characteristics, socio-demographic variables, and the AI-generated backstory. This editorial control allows investigators to precisely represent desired voices or specific lived experiences relevant to the research context—for example, deliberately over-sampling individuals with a particular disability status to stress-test accessibility provisions in a proposed protocol. Finally, researchers can select from predefined “Panel Profiles” (e.g., Efficient, Rich Narrative, Risk-Focused, or Innovation-Focused) which parameterise the LLM prompt to modulate narrative depth, discussion length, and the balance of response archetypes (sceptical, inquisitive, supportive) within the simulated discussion.
2.3. Document upload and semantic analysis
The Research Lead uploads the same documentation to Panelyze as that intended for the human panel. The system's semantic parsing engine processes this unstructured text, identifying key study parameters. It extracts core objectives, inclusion/exclusion criteria, and specific research questions to be addressed by the synthetic panel.
2.4. Synthetic discussion generation
Once the panel is generated and the proposal parsed, the platform accesses the prompt library and triggers a simulated 30-minute panel discussion (3,000–5,000 words). This is led by a synthetic PPIE Lead persona and follows a structured debate format: Opening, Initial Reactions, Deeper Exploration, Synthesis, and Closing. The simulation engine introduces a range of response types (sceptical, inquisitive, supportive) and variable contributions from synthetic personas to simulate authentic group dynamics, specifically designed to surface qualitative friction points.
2.5. Co-production artefact generation
Using the transcript from the simulated discussion and the original research proposal, the system creates co-production artefacts to bridge comprehension gaps. Examples include:
Plain English Summaries: Calibrated to NHS standards for accessibility.
Warm-Up Quizzes: Gamified tools for the human panel derived from friction points in the synthetic discussion.
Patient Information Sheets: Addressing any concerns expressed by synthetic panels.
Infographic Descriptions: Visual aids breaking down complex trial designs.
2.6. The “Panelyze Score” algorithm
To translate the qualitative output of a synthetic panel discussion into a reproducible quality metric, the platform computes a composite Panelyze Score (0–100). This metric is anchored to the six UK Standards for Public Involvement (7): Inclusive Opportunities, Working Together, Support & Learning, Communications, Impact, and Governance.
Rather than evaluating the submitted research proposal in isolation, a specialised “Assessor” agent analyses the synthetic panel's discourse to determine how well the proposal's provisions were reflected, challenged, or found absent in the simulated discussion. For each standard, the prompt provides explicit evaluation criteria and classifies what participants revealed in the transcript into three tiers: positive evidence (participants express explicit support backed by concrete proposal details); vague or mixed evidence (the standard is mentioned but lacks specifics or draws concern); and absent evidence (the proposal is silent on the standard or participants identify fundamental gaps). The rubric adopts a “sceptical default” principle, whereby silence on a standard is treated as a failure rather than a neutral omission. Each rationale is required to cite direct participant quotes from the transcript as supporting evidence, ensuring traceability between the qualitative discourse and the quantitative score.
This approach operates on the foundational assumption that census-grounded synthetic personas, when appropriately configured, constitute a reasonable proxy for the perspectives and concerns of individuals sharing the selected demographics and lived experiences—an assumption that warrants empirical validation in future comparative studies.
For each of the six standards, the model assigns an individual integer score (0–100) to every panel member, generating a participant-by-standard scoring matrix. Dimension-level scores are obtained by computing the panel mean for each standard, and the composite Panelyze Score is calculated as the unweighted mean across all six dimensions. This equal weighting is a deliberate design choice to prevent high performance in one domain (e.g., Communications) from masking structural deficits in another (e.g., Governance).
For practical interpretation, scores are mapped to five ordered quality categories: Critical (0–29), Poor (30–49), Moderate (50–69), Good (70–89), and Exceptional (90–100). Any dimension scoring below the Moderate threshold automatically triggers LLM-generated targeted improvement recommendations for the researcher. Ultimately, the Panelyze Score acts as a formative signal capturing what a census-calibrated patient cohort collectively articulates as aligned or deficient; it does not constitute a formal regulatory audit.
3. Results
To validate the operational utility of the Panelyze framework, we deployed the system to evaluate the CAPRIE-2 (Cardiovascular Primary Prevention Research) proposal, a large-scale cardiovascular trial with a ten-year duration focusing on long-term secondary prevention of coronary heart disease (12). Quantitative and qualitative outputs for the CAPRIE-2 validation study are summarised below, with the full dataset including simulated transcript and Panelyze Scores available as Supplementary Material.
3.1. Simulated qualitative inquiry: identification of barriers
While the system generates a numerical score, its primary utility was found in the semantic extraction of qualitative barriers. The synthetic panel acted as a “stress test” for the research proposal, identifying specific socio-economic friction points that would likely impede recruitment in the real world. As shown in Table 1, the system translated technical proposal requirements into human consequences. For example, regarding the study's longitudinal design, the synthetic panel moved beyond generic feedback to identify a “Critical” barrier regarding lost wages for working participants.
Table 1.
Qualitative translation of proposal barriers by synthetic personas. Summarizes technical protocol elements alongside identified socio-economic barriers and verbatim feedback from synthetic personas.
| Protocol element (CAPRIE-2) | Identified barrier (severity) | Synthetic persona “Verbatim” feedback |
|---|---|---|
| Visit schedule: 10-year follow-up requiring clinic visits. | Temporal/financial (critical) | "For working professionals..if it means missing pay, that's a big problem. Maybe financial help for lost earnings could be considered?" |
| Drug terminology: use of “Clopidogrel”. | Cognitive/familiarity (mixed sentiment) | "Clopidogrel sounds more technical and less familiar than Aspirin. Clear explanation..is crucial." |
| Dissemination: use of YouTube for patient info. | Digital Exclusion (Moderate) | "I worry about accessibility..Many of the individuals I support might not have regular access to the internet." |
3.2. Concept testing and terminology validation
The system demonstrated the capacity to answer specific research questions posed by the investigators, validating strategic design choices before human implementation.
Genetic Testing vs. Outcomes: When queried on the proposed decision to forego genetic testing in favour of observing outcomes across ethnicities, the panel consensus was Positive. The synthetic cohort validated the decision as “pragmatic” and “cost-effective”.
Terminology Acceptance: Conversely, the system flagged potential friction regarding the drug name “Clopidogrel”. The panel returned a Mixed Sentiment result, noting that while the science was sound, the terminology was “less familiar than Aspirin” and required an explicit communication strategy.
3.3. Quantitative benchmarking via the Panelyze Score
The system assigned the CAPRIE-2 research proposal a composite Panelyze Score of 79/100, categorising the proposal as “Good”. As illustrated in Figure 2, the proposal performed unevenly across the UK Standards. It achieved its highest ratings in Communications (90/100) and Inclusive Opportunities (85/100), but performance dropped in Working Together (72/100) and Governance (68/100). Notably, the lower Governance score (68/100) was driven by the synthetic panel's observation that the current proposal lacked a dedicated PPIE advisory group.
Figure 2.
Panelyze Score radar chart for CAPRIE-2 research proposal. Displays the performance of the cardiovascular research proposal against the six UK Standards for Public Involvement.
4. Discussion
4.1. Operationalising quality: balancing metrics and narrative
A primary contribution of this work is the operationalisation of the UK Standards for Public Involvement into a dual-output framework. While the Panelyze Score provides a necessary benchmark, the primary value lies in surfacing specific, narrative friction points—such as the financial burden on working participants—that a numerical score alone cannot articulate.
4.2. Augmentation, not replacement
It is a fundamental tenet of this framework that Panelyze synthetic panels are designed to augment, not replace, human involvement. The system can serve as a preparatory filter, allowing research leads to practice and refine the research proposal against a synthetic audience in addition to consultation with existing public involvement panels. Crucially, the synthetic panels amplify voices that are often structurally missing from physical meetings.
4.3. Limitations and conclusion
This proof-of-concept explicitly acknowledges the risks inherent in generative AI, specifically hallucination and algorithmic bias. However, the context of PPIE offers a unique mitigation layer: unlike clinical diagnostics which require factual precision, PPIE is qualitative and sentiment-based. The researcher acts as a mandatory “human-in-the-loop” to vet synthetic outputs for plausibility. Regarding demographic-clinical coherence, the human review phase also serves to identify and correct any implausible mismatches between a persona's synthetic attributes and their simulated health experiences before the discussion proceeds.
Regarding bias, we do not claim to eliminate it, but rather to manage it. We trade the hidden structural bias of human recruitment (e.g., availability for daytime meetings) for the transparent, adjustable parameters of synthetic agents.
The current publication presents validation against a single case study (CAPRIE-2), selected as the only study for which full permissions to share outputs have been granted at this stage. The platform has been applied across approximately ten research proposals to date, and while those additional cases have informed iterative development, their outputs are not reported here pending investigator permissions. However, subsequent comparative work is underway to address this directly. Initial comparisons between Panelyze synthetic panel outputs and feedback from real PPIE members and facilitators have demonstrated significant convergence on key issues, suggesting the synthetic panels identify barriers and concerns that are genuinely recognised by human participants. Notably, both panels also surfaced distinct additional topics not raised by the other, suggesting that synthetic and human panels are complementary rather than substitutive—each adding independent value to the research lead. These preliminary findings further support the case for Panelyze as an augmentation tool and will inform the next phase of platform development.
Ultimately, Panelyze serves as a preparatory simulation environment to stress-test research protocols against diverse demographic perspectives. It utilises in silico modelling to identify barriers early, thereby optimising subsequent human engagement. Future studies are needed to quantify the system's impact on actual research protocols, trial conduct, and recruitment rates (13). Furthermore, the Panelyze Score should be interpreted as a formative benchmarking instrument rather than a psychometrically validated scale; the use of LLM-assigned integer scores as ordinal proxies is an acknowledged methodological limitation that future work should address through structured reliability testing.
In conclusion, Panelyze offers a scalable technological intervention to pressure-test healthcare research proposals and support human PPIE panels. By leveraging SOTA LLMs to simulate diverse perspectives, it facilitates inclusive research design and ensures healthcare innovation is grounded in the needs of the target population.
Acknowledgments
The authors would like to thank Professor Colin Berry (University of Glasgow), Principal Investigator for the CAPRIE-2 study, for providing the research proposal and technical documentation used for system validation, and for his support in piloting the Panelyze framework within a cardiovascular research context. We also thank the staff and community at the NHS Golden Jubilee University National Hospital for their continued support and contribution to this research. The authors are grateful to the members of the University of Glasgow Medical, Veterinary & Life Sciences Public and Patient Involvement and Engagement Group who helped to test and give feedback on the Panelyze interface.
Funding Statement
The author(s) declared that financial support was received for this work and/or its publication. The Open Access publication fee for this article was kindly supported by the NHS Golden Jubilee University National Hospital Research & Development Endowment Fund. This work was otherwise conducted independently of institutional funding or commercial partnership.
Footnotes
Edited by: Hang Dong, University of Exeter, United Kingdom
Reviewed by: Santo Davide Ferrara, University of Padua, Italy
Vishnu Sivarudran Pillai, Kautilya School of Public Policy, India
Mengxuan Sun, University of Aberdeen, United Kingdom
Data availability statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.
Ethics statement
The authors confirm that ethical review and approval was not required for this study in accordance with the local legislation and institutional requirements. The study used entirely synthetic, AI-generated patient personas and did not involve any human or animal subjects, human tissues, or any identifying data.
Author contributions
AS: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. FS: Investigation, Methodology, Resources, Supervision, Writing – review & editing. CM: Conceptualization, Writing – review & editing. TI: Conceptualization, Formal analysis, Investigation, Methodology, Resources, Visualization, Writing – review & editing.
Conflict of interest
Author AS is the sole developer of the Panelyze™ platform and holds the associated Intellectual Property and Trademark rights. This work was conducted independently of any institutional funding or commercial partnership.
The remaining author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that generative AI was used in the creation of this manuscript. The core simulation engine of Panelyze utilises a range of SOTA LLMs including GPT-5 (OpenAI), Claude 4.5 Sonnet (Anthropic), and Gemini 2.5/3 (Google) APIs to generate synthetic persona responses and analyse text. The prompt engineering framework explicitly constrains these models to adhere to defined socio-demographic parameters. All outputs were reviewed by human investigators for hallucination and relevance. Generative AI (Gemini 3) was used to assist in spelling, grammar and format checking in the preparation of this article.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2026.1771729/full#supplementary-material
Synthetic Panel Discussion Transcripts for the CAPRIE-2 Case Study.
Itemised Panelyze Score Breakdown against UK Standards for Public Involvement for the CAPRIE-2 Proposal.
References
- 1.Staniszewska S, Brett J, Mockford C, Barber R. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Br Med J. (2017) 358:j3453. 10.1136/bmj.j3453 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Martin GP. ‘Ordinary people only’: knowledge, representativeness and the publics of public participation in healthcare. Sociol Health Illn. (2008) 30(1):35–54. 10.1111/j.1467-9566.2007.01027.x [DOI] [PubMed] [Google Scholar]
- 3.Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, et al. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol. (2014) 14:42. 10.1186/1471-2288-14-42 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Flory J, Emanuel E. Interventions to improve research participants’ understanding in informed consent for research: a systematic review. JAMA. (2004) 292(13):1593–601. 10.1001/jama.292.13.1593 [DOI] [PubMed] [Google Scholar]
- 5.Nori H, Lee YT, Zhang S, Carignan D, Edgar R, Fusi N, et al. Can generalist foundation models outcompete special-purpose tuning? Case study in medicine. arXiv [preprint]. (2023). 10.48550/arXiv.2311.16452 [DOI] [Google Scholar]
- 6.Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. (2023) 29:1930–40. 10.1038/s41591-023-02448-8 [DOI] [PubMed] [Google Scholar]
- 7.National Institute for Health and Care Research (NIHR). UK Standards for Public Involvement. (2019). Available online at: https://sites.google.com/nihr.ac.uk/pi-standards/standards (Accessed December 15, 2025).
- 8.National Records of Scotland. Scotland’s Census 2022. (2024). Available online at: https://www.scotlandscensus.gov.uk/2022-results/ (Accessed December 15, 2025).
- 9.Office for National Statistics (ONS). Census 2021: England and Wales. (2022). Available online at: https://www.ons.gov.uk/census (Accessed December 15, 2025).
- 10.United States Census Bureau. 2020 Census Results. (2023). Available online at: https://www.census.gov/programs-surveys/decennial-census/decade/2020/2020-census-results.html (Accessed December 15, 2025).
- 11.UK Government. Equality Act 2010. London: HMSO; (2010). Available online at: https://www.legislation.gov.uk/ukpga/2010/15/contents (Accessed December 15, 2025). [Google Scholar]
- 12.Berry C. Aspirin or clopidogrel for long-term secondary prevention of coronary heart disease powered for all-cause mortality: a randomised trial. [Personal communication]. (2025).
- 13.Price A, Albarqouni L, Kirkpatrick J, Clarke M, Liew SM, Roberts N, et al. Patient and public involvement in the design of clinical trials: an overview of systematic reviews. J Eval Clin Pract. (2018) 24:240–53. 10.1111/jep.12805 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Synthetic Panel Discussion Transcripts for the CAPRIE-2 Case Study.
Itemised Panelyze Score Breakdown against UK Standards for Public Involvement for the CAPRIE-2 Proposal.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.