Skip to main content
. Author manuscript; available in PMC: 2017 Jul 1.
Published in final edited form as: Comput Methods Programs Biomed. 2016 Apr 8;131:181–189. doi: 10.1016/j.cmpb.2016.03.031

Table I.

Dependent measures on efficiency, effectiveness, engagement and autosuggestion

Measures Data collection Evaluating
dimensions
Methods
Efficiency-related
Completion time Recorded at the
millisecond level by
interfaces
Time length of
completing a
narrative comment
Descriptive
statistics, and t-
test
Keystrokes Recorded by interfaces Keystroke counts of
completing a
comment
Descriptive
statistics, and t-
test
Text generation rate Text length divided by
completion time
The speed of text
generation, at the
unit of
“letters/second”
Descriptive
statistics, and t-
test
Effectiveness-related
Text length Recorded and
calculated at the unit of
letter
Text length (in
letters) of a narrative
comment
Descriptive
statistics, and t-
test
Text chunks As demonstrated in
Figure 1, pressing
<enter> resulted in a
tag in the text segment,
i.e. text chunk
Number of text
chunks in a comment
field
Descriptive
statistics
Chunk length Text length divided by
number of text chunks
Mean length of
text chunks in a
comment
Descriptive
statistics
Reporting
comprehensiveness
A blind review by two
experts; reached an
agreement when score
difference > 1
Number of event
characteristics
described in the text
Expert review,
descriptive
statistics and t-
test
Engagement-related
Non-adherence rate Amount of
unanswered
commentary fields
divided by amount of
commentary fields in
each group
Proportion of
narrative comment
fields that were
ignored
Descriptive
statistics, and
Chi-squared test
AutoSuggestion (AS)-related
Influenced chunks by
AS
Identified text
contained in original
AS
Number of text
chunks that accepted
the text suggested by
AS
Descriptive
statistics
AS influential rate Number of influenced
chunks divided by the
number of total text
chunks in a comment
Percentage of text
chunks contained the
text selected via AS
rather than key in
Descriptive
statistics