Table 1.

Relating primary functionalities of RITA to ethical concerns and potential solution.

Issue & associated function(s)	Specific concerns	Potential approach/solution
Accessibility 4, 5, 10	Novel solution may be unintuitive/Users may be used to different systems / Large volumes of complex data and unclear charts may limit understanding / Wide range of functions and options may increase operational complexity / Novel interface difficult to operate	Utilise communication system (RITA can explain her own operation) / Communication and emotion detection could enable RITA to infer problems and engage responsive tutorials / interface design based on established systems / novel features tagged by system so RITA is ‘aware’ increased support may be needed in these areas
Affordability 1, 5, 7, 12	Biometrics hardware not cost effective / High fidelity, real-time animations may require expensive hardware to run / Large databases may require expensive storage space	Exploit multi-use hardware to limit need for additional biometrics (e.g. camera for face and expression detection; microphone for voice detection/control)
Autonomy / Personal liberty 2, 3, 8–11	User feels that they are losing responsibility for monitoring their own health and wellbeing themselves / User may not wish for their data to be uploaded without consent or may change their mind about what can / cannot be stored / Activity recommendations may reduce user’s sense of ownership over their own life / Users feel patronised by constant reminders and health advice	User-autonomy threshold system (Figure 3) / All automated processes can be identified and cancelled upon request / regular automation review (transparency failsafe) / machine learning for user preferences
Deception 1, 3, 6, 7, 13	Human-like roles and appearance leading to suspicion that RITA is pretending to be human / Automated processes that are not fully disclosed and understood by the user may create suspicions of deception / Collecting any personal data on the user without their understanding and informed consent may create similar issues	Integrated reminders where RITA states she is a machine / transparency failsafe / option for non-human avatar appearance
Attachment / Duty to the system 1, 3, 6, 7, 13	RITA as friend / advocate may lead to attachment / Withdrawal effects if RITA is taken away / Encouraging the user to behave in ways they feel are beneficial to RITA at a detriment to themselves	Variable contact-use time limits between RITA and user/regular appraisal of user independence / increase of user independence a core function
Human carer impact 3, 6, 8–13	RITA functions may limit perceived value of human services / Users may wish to replace human carers with artificial system / Job market impact	RITA function responsive to human carer role / integration and support rather than replacement
Fear / distrust of the system 1–13	Human-like appearance evoking uncanniness and discomfort / Innovative functions appear too futuristic for the user (too removed from their experience of the everyday) / Health and emotion-related functions too vital and personal – inherently instilling distrust	Option for non-human avatar appearance / interface design based on established systems / transparency failsafe
Infantilisation 1, 3, 4, 6, 11	RITA seen as a toy / User could feel talked-down to / effect of reduced autonomy	Avoid ‘gimmicky’ functionality and design choices / RITA presented as a tool / increase of user independence a core function
Intrusiveness 2, 8-13	Invasive biometrics could be disruptive and potentially upsetting / Consistent reminders could become an irritant Coaching and management advice and control could be disruptive and irritating / Recommendations and unrequested advice could become intrusive / constant presence of RITA limits option for solitude	Use integrated (contactless) camera and microphone where possible / machine learning (preferences) to limit unwanted advice / emotion recognition to infer user preferences non-intrusively / Variable contact-use time limits between RITA and user / ‘OFF’ switch
Power 11	User may abuse the advocacy of RITA, using the system to inappropriately interact with third parties (unfairly monopolising their time and resources)	Problem-reporting facility (third parties may raise issues with RITA service support team) / emotion-detection to facilitate automated RITA interventions
Privacy 1, 2, 5, 13	Third party sharing / Emotional and health status monitoring too invasive / Automated collection storing data the user does not want recording / data after death	Highly customisable privacy and monitoring settings / regular review / emotion-detection to infer preferences
Data security 2, 5, 8	All personal data accessible from single point / wireless interception / cloud vulnerabilities / virus & malware risks / accidental security mistakes made by user	Biometric identification (facial/voice / etc.) / strong encryption / supporting secure use central part of system functionality
Reduced human contact 1, 3, 4, 6, 13	Distraction from physical or community-based interaction / User overly reliant on conversation with RITA, discouraging them from pursuing personal relationships	Increase of user independence a core function / person-to-person contact actively encouraged by RITA
Safety 3, 7, 8, 9, 10	No human failsafe should an accident occur / data malfunctions potentially harmful / Emotion interpretation mistakes could cause emotional distress	‘OFF’ switch and user-override facility / physiology and environment monitoring for intelligent threat-detection / option to feed monitoring to human (carer)
Third party communication 1, 3, 6, 10, 11	RITA may accidentally damage relationships with friends and family whilst acting as an advocate	Regular review / emotion-detection to infer preferences
RITA Functionality key: [1] Appearance (human-like character) [2] Automated data collection [3] Carer role [4] Communication device [5] Database / storage [6] Emotional / wellbeing support [7] Emotion detection / interpretation [8] Environment / home monitoring [9] Exercise coach [10] organisational support [11] personal advocacy [12] physiology monitoring / biofeedback [13] real-time conversation (voice detection / speech synthesis)