Abstract
Background
Artificial intelligence (AI) is becoming increasingly relevant for public health practice, yet scalable training models and validated usability instruments for AI chatbots remain limited in non-English contexts. This study aimed to assess AI knowledge, attitudes, and practices (KAP) among a sample of Italian public health professionals and to translate, culturally adapt, and psychometrically validate the Italian version of the Chatbot Usability Questionnaire (CUQ-IT).
Methods
We conducted a three-phase study during the 57th Italian National Congress of Hygiene (October 2024): (i) a cross-sectional KAP survey; (ii) a 90-minute theoretical-practical workshop featuring guided prompt engineering and the collaborative development of a Custom GPT for vaccination counseling (VaxSense); and (iii) post-workshop usability evaluation with CUQ-IT. Internal consistency was estimated with Cronbach’s α.
Results
Among 86 eligible respondents, participants reported moderate AI knowledge (mean 2.74±1.05/5), positive attitudes (3.87±0.73/5), but limited routine use (2.88±1.11/5). A statistically significant gender difference in self-rated AI knowledge was identified (p = 0.02). The CUQ-IT demonstrated high internal consistency (Cronbach’s α = 0.881) and robust performance across three domains: Personality, Response Handling, and Navigation Ease. The workshop-developed chatbot, VaxSense, achieved a good overall usability score (mean 66.60±15.29/100).
Conclusions
The findings highlight an “attitude-practice gap” where professionals are conceptually ready for AI but lack operational know-how. The CUQ-IT provides a standardized tool for chatbot evaluation in Italian settings. Furthermore, observed disparities in AI readiness underscore the need for inclusive, hands-on training models that bridge theoretical foundations with practical application to ensure the safe and equitable implementation of generative AI in public health.