Table 2.

Qualitative feedback in phase 1 prototype usability testing.

Features and content of the LCSDecToola		Feedback from phase 1 prototype testing
		Veterans	Clinicians
Computer based
	Accessed by a URL link on devices: tablet, desktop, laptop, and smartphone	Users varied in preferred device: tablet, laptop, phone	—b
Overview of LCSc: simulated discussion between patient and provider
	Simulated dialog with questions and answers about LCS	Users found this engaging Most recognized that it was a physician and patient discussion and found this engaging The scrolling function was intuitive to most Recognized the format as similar to texting Easy to navigate One did not realize it was a physician–patient discussion	Consider adding audio Clarify who is speaking Dialog seems natural Shorten Define CTd scan Change Nodule to Spot Liked clarification that a false positive is not a mistake Change Doctor to Provider Indicate most nodules are small
Overview of LCS: clickable knowledge boxes
	6 knowledge boxes, each covering a key LCS content area; one must click on all boxes before advancing to the tool	Most found this to be more informative and easier to navigate than simulated dialog Some noted that the repetition of some content in this format reinforced the information that was being conveyed The pictures on each box were engaging	Add a box for what is a CT scan? Navigation may be confusing Symbols would be better than pictures Be careful about using relative risk reduction for mortality benefit Consider the pictorial representation of statistics Add a box for what happens if my scan is abnormal? Agree with bringing up annual screening; include that interval cancers may occur
Pictograph
	Main outcomes from the National Lung Screening Trial displayed in pictograph: lung cancer deaths and deaths averted, false positives, biopsies, and complications	Users (except for 1) understood that the 2 side-by-side pictographs were comparing outcomes between screened and not screened populations Understood dots to represent people and colored dots to represent outcomes Some needed to be guided through the pictograph to understand	Good color contrast Would use with patients Helpful visual aid Describe a major complication Clarify screened and unscreened groups
Value elicitation—rating scale 1 and rating scale 2
	Rating scale 1 response scale: much less likely to much more likely to want screening; rating scale 2 response scale: not at all concerned to extremely concerned	For most users, rating scale 2 was easier to use and demonstrated greater variation in ratings among benefits and potential harms of screening. One user found rating scale 1 to be more relevant and helpful in evaluating these attributes	Less user friendly than the attitudes section Shorten Explain that answers go to the summary page Lacks assessment of cost Prefers scale 2 Carry over stem to each question
Cancer screening attitudes—rating scale
	An assessment of general cancer screening attitudes and beliefs	Questions were intuitive and easy to answer	Reads well Clarify what repeat testing means Clarify why these questions were asked
Veteran-centric content—smoking cessation and mental health
	VAe resources highlighted with the option to request a consultation	Most acknowledged that these were important, and some clicked boxes to request consultations. One user cautioned that raising the issue of anxiety may discourage a veteran from LCS	Provide phone numbers in a handout Note that the risk of lung cancer decreases after smoking cessation Change the description to mental health or behavioral health provider It is important to include smoking cessation to emphasize benefit, even with LCS State that smoking cessation is more effective than LCS in preventing lung cancer deaths Loved mental health access Include information specific to veterans Integrates well with the tool
Enter questions for the provider
	Free text option; questions inserted on the summary sheet	Users all supported this feature	—b
Summary page
	Includes ratings of values or attitudes; able to print, save, or email page	Users all supported this feature	Clarify where the email goes Simplify and shorten Title value responses with “Why I want screening” Clarify that it goes to the provider Improve that format of presenting scale results; use color coding Give suggestions to providers about how to address concerns; goal to distinguish beliefs from misunderstandings Like how it looks; will be helpful to providers
Clinician portal
	Link from entry page to features for use at the point of care: pictograph and value and attitude assessment	—	Name Clinician or Provider rather than Physician portal Add picture with active link to the portal Make more accessible to the clinician Use term save document versus PDF Like that the provider has quick access to patient summary

^aLCSDecTool: lung cancer screening decision tool.

^bNo information emerged for this feature.

^cLCS: lung cancer screening.

^dCT: computed tomography.

^eVA: Veteran Affairs.