Table 2.

Findings from studies that investigated user experience and satisfaction

Author year Country Reference	Study type	Sample/data size	Digital triage user	Participants	Key themes and example quotes
Björkman 2018 Sweden44	Descriptive research design using information from online forums using six step 'netnographic' method	Data from 3 Swedish online forums were purposively sampled.	Nurse	General population (users)	General satisfaction/attitudes ‘Where we are, the healthcare advice line is great, I’d rather call them than my primary care center’ Experience of call taker: Patients expressed doubts and mistrust on advice given and credibility of nurses. Feelings that nurses were not well competent/ qualified and relied on google: ‘And seriously, are they real nurses who take the calls at SHD? I almost think it sounds like they’re googling every question they get.’ Safety: Some concerns related to safety and feeling that advice given was not appropriate, for example: a user posted that they were advised to stay at home for a condition that turned out to be serious, ‘When you’re advised to take two paracetamols and go to bed. Not go into the ER. When I was feeling really bad, and called them and described my symptoms, that’s the exact advice I was given. The situation ended with my husband more or less forcing me into the car and driving me to the hospital. By then, my lips were purple and I was having trouble keeping my balance. Once there, they found that both my lungs were filled with 100 s of small blood clots.’ Assertiveness and negotiation: One user posted, ‘If you need help and advice you can always call the healthcare advice line, if you think they’re giving you the ‘wrong’ advice, tell them, and maybe you’ll get better help’ Service working together: a user expressed dissatisfaction where the service did not work well together, ‘There’s no point calling [digital triage service name]. They send you to the ER where they yell at you for being stupid enough to listen to them. [digital triage service name] is a big problem and seems to be at war with the ER’
O'Cathain 2014 England40	Survey	Survey sent to 1200 patients from each of the 4 pilot sites studied, 1769 responded and were included for analysis	Non-clinical call handler	General population (users)	General satisfaction/attitudes Satisfaction levels were good overall (91% very satisfied or satisfied). 73% (1255/1726, 95%CI: 71% to 75%) were very satisfied with the way NHS 111 handled the whole process, 19% (319/1726) were fairly satisfied and 5% (79/1726) were dissatisfied. Two aspects of the service were less acceptable than others: 1) relevance of questions asked and 2) whether the advice given worked in practice. Greater satisfaction with higher urgency advice: Patients more likely to feel the service was helpful if directed to ambulance service (76%), compared with self-care(64%) visit health centre (55%), other service 54%, contact GP (52%). Services working together: Patients more likely to feel the service was helpful if an appointment was arranged for them (71%).
McAteer 2016 Scotland6	Other—mixed methods	Age-stratified and sex-stratified random sample of 256 adults from each of 14 Scottish GP surgeries, final sample was 1190 based on response rate with 601 of those having used the digital triage service. Purposive sampling used for interview group with total of 30 being interviewed.	Non-clinical call handler	General public (users and non-users)	General satisfaction/attitudes: Questionnaire findings: over 80% of those who had used the digital triage service reported being either ’satisfied' or 'very satisfied' - education was the only socioeconomic factor associated with satisfaction (with higher educated participants being less satisfied). Interview findings showed users were broadly satisfied with service. Most common reasons for dissatisfaction related to initial triage questions, for example, ‘I just felt that, she should get me onto a nurse and stop asking me questions, you know, I felt it went on too long’, and the length of time it took to receive visits and not being kept informed.
Rahmqvist 2011 Sweden41	Survey	Random sample of 660 callers, made at one site in October 2008	Nurse	General public (users)	Greater satisfaction with higher urgency advice Patients who were recommended to wait and see, were less likely to be satisfied and more likely to make an emergency visit or an on call doctor. Results reported in relation to callers' agreement with advice: analysed using 3 groups: (1) cases: those who disagreed with nurse advice and felt they needed higher level of care; (2) controls: those who disagreed with nurse advice or felt they needed higher level of care; (3) other callers. Average global patient satisfaction was significantly lower for nurses who served the cases compared with those who had not served the cases
Goode 2004 England43	Interview study	60 interviews	Nurse	General public (users)	General satisfaction/attitudes Results related to feelings that the digital triage service was 'trustworthy', and being able to access care without being a ‘nuisance’. Authors state that some interviewees experienced or predicted deterioration in service quality: ‘They’ll put a bit too much work on their call centres, they’ll be understaffed, then they’ll start becoming hurried or you’ll lose that friendly ‘take as long as you like’ sort of attitude that I experienced…’ Experience of call taker: reassurance Users felt reassured and cared for: ‘I felt like they cared. I was suffering and I felt like they cared. And that’s what I wanted’ ‘For me to be able to ring somebody, you know, and when I did feel in pain, but wasn’t sure whether it was normal or not—well I knew that it wasn’t normal, but is it common? And it was nice just to speak to somebody. And, ‘Okay, yeah, do go to your doctors’, you know, ‘you’re not being silly’
Winneby 2014 Sweden45	Interview study	8 semistructured interviews	Nurse	General public (users)	Experience of call taker: feeling reassured when taken seriously The authors describe findings relating to users feeling reassured on follow-up care required, ‘When the nurse believed and advised them to turn to the care center on duty, having obtained a mandate to go there, gave them a sense of security’. A quote from a participant: ‘Because they [nurses] know more than I do and will refer me if it’s something serious.’ Assertiveness and negotiation ‘Being a nurse, I know what to say and what I’ve done at home. Otherwise they will tell you to ‘drink plenty of fluids’ and 'do this and that'. But now I say that ‘I have drunk a lot” and 'I have medication at home'. It feels as if they [SHD] try to sift out and turn away… you don’t call unless it’s necessary.’
Goode 2004 England42	Interview study	10 interviews	Nurse	General public (users) interviews with men/or that related to men	General satisfaction/attitudes A participant commented on male partner: ‘He thought it was great. He was very impressed. And a male nurse spoke to him as well, which I think he was even more impressed that a man would know what he was talking about…’ The authors describe a male interviewee whose wife called on his behalf ‘He now described NHS Direct as an excellent and much-needed service, which he would continue to use to meet his need for ‘expert’ guidance on the appropriate response to symptoms.’ Assertiveness and negotiation One male participant made a follow up call to NHSDirect regarding his wife, while his wife was waiting for a call back from the service: ‘I simply had one aim at that point, which was to get a doctor out to the house without putting the phone down… everything was pretty much arranged in the one call. It was acknowledged that things were bad and that a doctor would be calling tonight… I guess I was being pretty direct, like, ‘She is sick and she must be seen.’

GP, general practice.