Quality Issues of Court Reporters and Transcriptionists for Qualitative Research

Abstract

Transcription is central to qualitative research, yet few researchers identify the quality of different transcription methods. We described the quality of verbatim transcripts from traditional transcriptionists and court reporters by reviewing 16 transcripts from 8 focus group discussions using four criteria: transcription errors, cost and time of transcription, and effect on study participants. Transcriptionists made fewer errors, captured colloquial dialogue, and errors were largely influenced by the quality of the recording. Court reporters made more errors, particularly in the omission of topical content and contextual detail and were less able to produce a verbatim transcript; however the potential immediacy of the transcript was advantageous. In terms of cost, shorter group discussions favored a transcriptionist and longer groups a court reporter. Study participants reported no effect by either method of recording. Understanding the benefits and limitations of each method of transcription can help researchers select an appropriate method for each study.

Developing verbatim transcripts of interviews or group discussions is a core task in much qualitative research. Transcription produces the textual data for analysis and is therefore a critical component of the research process. A verbatim transcript captures participants' own words, language and expressions, which are particularly valued in qualitative research. It is often the language used by participants themselves that helps us to decode behavior, processes and cultural meanings attached to people's perspectives. Therefore, “it is not only the words used by participants that are of interest to qualitative researchers, but perhaps more importantly the meanings and concepts attached to the words, descriptions and expressions that provide a deeper understanding of the research issues within the socio-cultural context of the study participants” (page 23, Hennink, 2008). Furthermore, it is widely accepted in qualitative research that knowledge is ‘created’ through the interactive exchange between a researcher and participant, (Green & Thorogood, 2004; Kvale & Brinkmann, 2009; Oliver, Serovich, & Mason, 2005; Ritchie & Lewis, 2003), and this interaction is also captured in a verbatim transcript. Therefore, generating a good quality verbatim transcript remains fundamental for many types of qualitative data analysis.

In this article, we aimed to describe the quality of research transcripts developed by court reporters and transcriptionists and to identify the benefits and limitations of each approach. We sought to answer the following research question: how effective are court reporters and transcriptionists in developing verbatim transcripts for qualitative research? Transcription is a strong act of representation and the purpose of the research will determine the type of transcription required (Oliver et al., 2005). Linguistic and conversation analysts focus on the structure of speech and therefore the length of pauses, word elongation, emphasis and diction are marked in a transcript. In the sociological tradition, the focus is less on the mechanics of speech and more on the informational content of the interview and the social or cultural meanings attached to this content (Hennink, Hutter, & Bailey, 2011; Kvale & Brinkmann, 2009; McLellan, MacQueen, & Neidig, 2003). In this article, we focused on verbatim transcription for the latter type of analysis.

A verbatim transcript is typically developed by listening to an audio recording and typing the interview dialogue word for word, often with the aid of transcription software to stop, start and slow the recording. Although this remains the most common procedure for developing a verbatim transcript, it is not without limitations. This method of transcription is extremely time and labor intensive (Kvale & Brinkmann, 2009), often resulting in a significant time lag between data collection and completion of transcription for analysis, which can disrupt the iterative process of qualitative research. Tilley (Tilley, 2003; Tilley & Powick, 2002) noted that transcription in academic settings is often done by administrative staff, graduate students or non-research professionals with varying levels of skill, training or supervision, which can lead to poorly transcribed data, or, if multiple transcribers are used, variable quality and content.

There might also be problems because of the recording device (i.e., mechanical failure, battery loss) or the clarity of the recording (i.e., excessive background noise, poor placement of the recorder), all of which can result in missing, incomplete, or incorrect transcription (Easton, McComish, & Greenberg, 2000; MacLean, Meyer, & Estable, 2004; Poland, 1995; Tilley & Powick, 2002). Furthermore, transcriptionists are rarely present at the interview or group discussion and rely solely on interpreting the audio recording, so they might unconsciously influence the data generated (Kvale & Brinkmann, 2009; Padgett, 2012). Transcriptionists might misinterpret a speaker's comment without the aid of nonverbal cues for clarification (for example a speaker might smile or wink when making a comment indicating irony which will be lost to a transcriptionist); use incorrect punctuation that changes the tone or meaning of a comment; or experience difficulties in identifying individual speakers (Bucholtz, 2000; Sandelowski, 1994; Tilley, 2003).

To overcome some of the challenges of transcriptionists, researchers are beginning to use court reporters to develop verbatim transcripts of research interviews and group discussions. Court reporters are trained transcriptionists used to create verbatim records of court proceedings, meetings/events or for closed-caption media steaming. Court reporters for research are present at the interview or group discussion and simultaneously listen to and type dialogue into a stenotype machine using specialized shorthand, which can then be transformed into a transcript in real-time. Proponents of court reporters for generating research data argue there are several benefits of this approach. First, court reporters undergo extensive training and therefore have the skills to produce a more accurate transcript than individuals transcribing from an audiotape (Easton et al., 2000). Second, court reporters are present at the interview so they can record any nonverbal cues made by participants that influence data interpretation, and they can easily see and identify speakers. Third, problems associated with audio equipment or poor recording are eliminated. Finally, transcripts are developed in real-time thereby significantly reducing the time lag experienced with transcriptionists (Scott et al., 2009).

The use of court reporters offers significant potential for generating quick and accurate verbatim transcripts for qualitative research. However, there is little formal evaluation of this method of transcription and no comparative research on the quality of transcription by court reporters versus transcriptionists. Researchers who have reported the use of court reporters for transcription neither offer justification for their choice nor assess the quality of the transcripts developed by court reporters (see the following for examples: [Jennings, Loan, Heiner, Hemman, & Swanson, 2005; Kick, Adams, & O'Brien-Gonzales, 2000; Newhouse, 2005]). Only one published study (Scott et al., 2009) described the perceived benefits and drawbacks of using a court reporter and how they were briefed for the research task.

Methods

Data for the study presented here were collected as part of a larger trial, which aimed to develop a lifestyle intervention to prevent diabetes among South Asians in the United States. Data were collected using focus group discussions with South Asian community members to identify their perceptions and behaviors regarding diet, exercise, obesity and diabetes and to discuss culturally suitable activities for diabetes prevention. Group discussions were held with men and women separately and stratified by age (25-40 years and more than 40 years). Participants were purposively recruited from community venues frequented by members of the South Asian community (i.e., retail and religious centers, community gatherings, festivals and South Asian media) to participate in a group discussion of 60-90 minutes. Data were collected during 2010 and 2011 in Atlanta, Georgia. Both the core study and the ancillary study described in this article were approved by the Emory University Institutional Review Board, and all subjects provided informed consent before any data were collected.

For the purpose of our study, to compare transcription methods, focus group discussions were recorded in two ways: with a digital recorder and by a court reporter. The first method involved capturing the group discussion with two digital recorders (in case of failure of one machine and to check any unclear recording) then using a transcriptionist to develop a verbatim transcript from the recording. Transcriptionists comprised five public health graduate students experienced in qualitative research, three of whom were of South Asian background. All transcriptionists were trained on the requirements for developing a verbatim transcript, such as: fully transcribing the recorded discussion, including speaker identifiers and verbal utterances (e.g., a-ha, yeah), retaining participant's colloquial style of speech and avoiding grammar correction. Transcriptionists developed the verbatim transcripts by listening to the digital recording using Express-scribe transcription software (version 5.5, NCH Software), which can stop, start and reduce the speed of dialogue for transcription. This method of transcription is common in qualitative research and allows transcriptionists to take breaks or for others to listen and clarify unclear sections of dialogue. Speakers were identified by the sound of each voice on the recording; when it was unclear who was speaking, the speaker was denoted by a question mark.

The second method of recording the focus group discussions involved using a court reporter to develop a transcript of the focus group discussion in real-time. Court reporters used in this study were dispatched via an agency therefore we had little control over the selection of specific court reporters. Three different court reporters were used for this study. Before each focus group, court reporters were briefed on the study topic, common words that might be used in the discussion, the types of study participants and potential accents to expect, and general focus group methodology. Court reporters were able to identify different speakers by sight, and each participant had a number on his/her nametag to assist in quickly identifying a speaker. Court reporters used stenograph machines and a personal laptop to create transcripts of the discussions. Although an initial transcript was generated immediately, each court reporter reviewed, edited and formatted the transcript before submitting it to the research team at a later date.

Eight focus group discussions were recorded by both digital recorder and a court reporter, resulting in 16 verbatim transcripts that were used to compare transcription methods. The quality of each method of transcription was assessed using four criteria: transcription errors, cost and time of transcription, and the effect of each method of recording on study participants. The first criterion for assessment was the accuracy of transcription, which was assessed by the type and number of transcription errors found. Transcription errors were defined as any deviation from the digital recording of the group discussion. We first developed a coding scheme that identified the types of transcription errors found (see Table 1), then coded each transcript by these error types and tabulated errors by mode of transcription and characteristics of focus groups to identify any patterns in transcription errors.

Table 1. Definition of Transcription Errors.

Type of Error	Description of Error	Example*
Inaudible Speech
Marked in transcript	Dialogue not transcribed because of unclear recording and marked in transcript as (inaudible)	An inaudible recording might be because of soft speech, mumbling or several participants talking at once
Not marked in transcript	Dialogue not transcribed because of unclear recording but not marked in transcript
Omission of:
Topical discussion	Omission of dialogue between participants on key research topics (e.g., diet, exercise, diabetes)	P1: We use Ghee P3 It improves the taste P2 It's our tradition P4 people like it
Participant acknowledgement	Omission of acknowledgement words from other participants	I see, yes, I agree, and so forth Not for sounds (a-ha, mm-hm)
Moderator acknowledgement	Omission of acknowledgement words from the moderator	I see, yes, I agree, and so forth Not for sounds (a-ha, mm-hm)
Topical word(s)	Omission of up to 3 consecutive words on key research topics	“Some people have a very healthy lifestyle eating many vegetables for their entire lives”
Non-topical words	Omission of up to 3 consecutive words not on key research topics	“I just got my test results back, so it's OK”
Topical phrase	Omission of a whole sentence/phrase (more than 3 consecutive words) on key research topics	“I cook for myself so I can take care of what I am eating much better, but it's a challenge”
Participant comment	Omission of the entire comment from a participant on key research topics. Excludes comments/whispers in the background	P1: I do some exercise every day, sometimes just walking. P2: I start by doing stretches in the morning and then weights in the gym, but not every day. P3: yeah, I just bought my first set of weights, its great
Speaker identifier	Speaker identifier is not marked	P: or P? marked instead of P3:
Verbal utterance	Omission of clear verbal sounds or long pauses by speakers	No indication of (laughter), (shouting) (pause) etc on transcript
Grammatical Changes
Incorrect Word Conjugation	Incorrect conjugations of words	Indian instead of India; heritage instead of hereditary
Grammar Correction	Words added to ‘correct’ speech/grammar instead of verbatim colloquial speech; sentences are completed or repeated words/false starts are removed	“I am going to” instead of “I'm gonna”; “I am not on medication” instead of “Well, yeah, you see, I'm not on medication”
Errors & Additions
Word Added	Addition of less than 3 consecutive words, not on recording	We are meeting Asian people in the community to discuss this problem.
Incorrect Word	Incorrect word(s) used (3 or less consecutive words)	“We go to the gas station” instead of “We go to the filling station”
Incorrect Phrase	Incorrect phrase used (more than 3 consecutive words)	“I have a weight problem” instead of “I have a thyroid problem”

Note:

^*bold text denotes example of missing text, italics denotes addition of words, and ALL CAPS denotes incorrect words

Four types of transcription errors were identified. The first type of transcription error is inaudible speech where dialogue was not transcribed because it was unclear or inaudible on the digital recording and either marked as such by the transcriber or not. The second type of transcription error is omissions, whereby a word, phrase, section of discussion, speaker identifier or verbal utterance was omitted in the transcript but clearly audible on the recording. These errors were categorized by the type of omission (e.g., omitted word, phrase, etc) and whether the omitted word focused on the key research topics (topical omission) or were unrelated to the research issues (non-topical omission). An example of an omitted participant comment by a transcriptionist is shown below.

Court reporter transcript:

• Moderator: Is exercising a problem at all?

• Participant 1 (P1): Indian ladies don't live in nuclear families, we are not, you know, we don't live independently, we have greater responsibilities and cannot go to exercise if there are needs at home.

• Participant 2 (P2): Men can go out anytime, ladies have to stay at home.

Transcriptionist transcript:

• Moderator: Is exercising a problem at all?

• P2: Men can go out anytime, ladies have to stay at home

The third type of transcription error is grammatical changes, where colloquial language, repeated words and false starts were grammatically corrected. For example, a recording might include: “umm, well, people know, most people know that smoking's gonna kill you, but diabetes ain't so obvious,” but is transcribed as “most people know that smoking is going to kill you, but diabetes is not so obvious.” The fourth type of transcription error is errors and additions, where words or phrases were incorrectly transcribed or were added to the transcript but not heard on the recording.

All transcripts were entered into MAXQDA 10 software (VERBI Software, 1989-2012) and coded for errors, which involved reading the transcript while listening to the recording and coding sections of text where specific errors from the codebook were identified. Two researchers piloted coding to check coding consistency, and the codebook was refined before coding the whole data set for transcription errors. Data analysis involved tabulating the frequency of transcription errors by modes of transcription and identifying any pattern in error types. Errors were also examined by characteristics of the focus group participants (e.g., age, gender, group size) to identify further patterns in the type or frequency of transcription errors made.

The second criterion for assessment was the cost of transcription. We kept a record of the cost of developing each transcript. The cost for transcriptionists was calculated by multiplying the number of hours spent on each transcript by the hourly rate for graduate research assistants at Emory University ($12 per hour). The cost for the court reporters was fixed at $300 per group up to eight groups and $265 for any subsequent groups. This charge included attending the focus group discussion for up to two hours and producing the final verbatim transcript. All costs were tabulated for each transcript, averaged and compared by mode of transcription. The time taken to develop a verbatim transcript was a third criterion for assessment. We recorded the turnaround time to develop a transcript as the number of days between the group discussion and receipt of the final transcript from each transcriptionist or court reporter.

The final criterion for assessment was to identify whether participant's contributions to the group discussion were influenced by either the digital recorder or the presence of a court reporter and stenotype machine. Three types of data were collected for this assessment. First, participants were asked at the end of each group discussion how they felt about the presence of the digital recorder and the court reporter; the court reporter was briefed to leave the room before these questions were asked to enable participants to speak freely. Responses were recorded on the digital recorders. Second, participants completed a brief survey after the group discussion, which included several questions on the effect of the mode of recording the group discussion. Participants were asked to rate how they felt about each method of recording the discussion on a scale ranging from “no problem” to “very uncomfortable.” They were also asked: Did having the (digital recorder or stenotype machine) in the room stop you from saying anything in the group? (No/Yes [please explain]). Third, focus group moderators were asked to keep notes on anything they observed or heard regarding participants' reactions to the recording devices that might influence transcription. These notes were used to augment study findings.

Results

Transcription Errors

Both the transcriptionists and court reporters made a range of transcription errors. Although the average number of errors per transcript was similar, the distinction lies in the type and range of errors made in each method of transcription. Table 2 shows the type of transcription errors by the mode of transcription. Overall, the average number of errors made by court reporters and transcriptionists was similar, 269 and 243 respectively.

Table 2. Transcription Errors by Mode of Transcription.

Type of Error	Transcriptionist % (n)		Court Reporter % (n)
Inaudible Speech
Marked in transcript	37%	(727)	4.6%	(100)
Not marked in transcript	0.2%	(4)	8.7%	(189)
Omission of:
Topical discussion	0.05%	(1)	2.3%	(51)
Participant acknowledgement	0.3%	(6)	8.0%	(173)
Moderator acknowledgement			2.6%	(57)
Topical word(s)	0.7%	(15)	15.1%	(328)
Non-topical words	0.4%	(8)	8.1%	(175)
Topical phrase	0.1%	(2)	5.5%	(119)
Participant comment	0.5%	(10)	3.2 %	(71)
Speaker identifier	54.4%	(1060)	1.1%	(24)
Verbal utterance	0.08%	(1)	5.8%	(127)
Grammatical Changes
Incorrect Word Conjugation	0.4%	(9)	2.3%	(51)
Grammar Correction	0.4%	(9)	12.1%	(262)
Errors & Additions
Word Added			0.5%	(12)
Incorrect Word	4.3%	(84)	15.3%	(332)
Incorrect Phrase	0.5%	(10)	2.7%	(60)
Total Errors	100%	(1947)	100%	(2158)
Average errors per transcript	243		269

Errors by transcriptionists were clustered by three types of errors. The most common error was the omission of a speaker identifier, which accounted for 54% of errors by transcriptionists. The second most common error by transcriptionists was related to dialogue not transcribed because of inaudible speech on the recording (37% of errors), although this was marked as such in the transcript. Third were errors in transcribing an incorrect word (4% of errors). Other types of errors by transcriptionists were few, accounting for less than 1% of errors in each category. Few errors among transcriptionists related to grammatical changes in the transcript.

Transcription errors made by court reporters were more diverse than those of transcriptionists with a higher number of errors in most categories. The most common error among court reporters was the use of an incorrect word or the omission of topical words, each accounting for 15% of errors. Another common error was grammar correction whereby court reporters would ‘correct’ a participant's grammar rather than transcribe colloquial speech verbatim; this accounted for 12% of errors. Court reporters' errors also included not marking inaudible speech on the transcript, where dialogue was not transcribed. Most other errors were in the omissions category, such as omitted words, omitted participant acknowledgements, or omitted verbal utterances.

We explored whether transcription errors were clustered within particular transcripts or related to a specific transcriptionist or court reporter. Table 3 shows errors by individual transcriptionist and transcript. There is no clear pattern of transcription errors by transcriptionist, however, errors clustered within two transcripts (focus groups 5 and 8), which were conducted with older men and older women, respectively. More than half of overall errors (57%) by transcriptionists are found in these two transcripts. More than three-quarters of the omitted speaker identifier errors are in these transcripts, and almost half (46%) of the inaudible speech errors were associated with the same transcripts. Therefore, some transcription errors were clearly associated with difficulties in particular group discussions rather than individual transcriptionists. Table 4 shows errors by individual court reporter and transcript. There is no pattern of errors by the different court reporters. Errors are also dispersed across transcripts showing no clear clustering by transcript; however, inaudible speech is higher within transcripts 5 and 8, and these transcripts account for 46% of this type of error among court reporters, suggesting that court reporters also faced challenges in transcribing these two focus group discussions.

Table 3. Transcription Errors by Transcript and Transcriptionist, Percentages (n).

	Transcriptionist Transcript*
	Tr1		Tr 2		Tr 3	Tr 4		Tr5
Type of Error	FGD 2	FGD5	FGD3	FGD8	FGD6	FGD7	FGD9	FGD10	Total
Inaudible Speech
Marked in transcript	58.4% (100)	38.1% (210)	58.8% (20)	21.6% (124)	58.0% (61)	62.8% (66)	28.9% (70)	48.4% (76)	727
Not marked in transcript				0.3% (2)			0.8% (2)		4
Omission of:
Topical discussion						0.8% (1)			1
Participant acknowledgement						2.6% (3)		0.6% (1)	6
Moderator acknowledgement				0.3% (2)
Topical word(s)	1.1% (2)	0.5% (3)			1.9% (2)	4.3% (5)		0.6% (1)	15
Non-topical words					7.6% (8)				8
Topical phrase						0.8% (1)	0.4% (1)		2
Participant comment	1.1% (2)	0.3% (2)				0.8% (1)	0.4% (1)	2.5% (4)	10
Speaker identifier	15.7% (27)	58.1% (320)	32.3% (11)	80.8% (436)	9.5% (10)	30.4% (35)	64.8% (157)	40.7% (64)	1060
Verbal utterance							0.4% (1)		1
Grammatical Changes
Incorrect Word Conjugation	1.1% (2)	0.1% (1)		0.1% (1)	0.9% (1)		1.6% (4)		9
Grammar Correction	0.5% (1)	0.3% (2)			0.9% (1)		0.4% (1)	2.5% (4)	9
Errors & Additions
Word Added									0
Incorrect Word	20.4% (35)	1.4% (8)	8.8% (3)	0.8% (5)	19.1% (20)	1.7% (2)	2.0% (5)	3.8% (6)	84
Incorrect Phrase	1.1% (2)	0.7% (4)		0.1% (1)	0.9% (1)	0.8% (1)		0.6% (1)	10
Total Errors	100% (171)	100% (550)	100% (34)	100% (573)	100% (105)	100% (115)	100% (242)	100% (157)	1947
Average errors/Transcriptionist	360		303		105	178		157

^*Transcriptionists are denoted at TR# and transcripts are denoted as FGD#.

Table 4. Transcription Errors by Transcript and Court reporter, Percentages (n).

	Court Reporter Transcript*
	Cr 1					Cr 2		Cr 3
Type of Error	FGD2	FGD7	FGD8	FGD9	FGD10	FGD3	FGD5	FGD6	Total
Inaudible Speech
Marked in transcript	4.5% (14)	3.8% (13)	7.6% (21)	14.1% (20)	3.2% (7)	3.6% (10)	4.2% (13)	0.6% (2)	100
Not marked in transcript	1.9% (6)	8.6% (29)	21.7% (60)	16.3% (23)	10.7% (23)	1.8% (5)	12.3% (38)	1.1% (5)	189
Omission of:
Topical discussion	3.9% (12)	5.0% (17)	0.3% (1)	2.8% (4)	0.9% (2)	2.9% (8)	2.2% (7)		51
Participant acknowledgement	8.1% (25)	10.9% (37)	7.9% (22)	5.6% (8)	7.9% (51)	7.7% (21)	9.4% (29)	4.5% (14)	173
Moderator acknowledgement	2.2% (7)	1.7% (6)	5.0% (14)	2.8% (4)	7.9% (17)	1.8% (5)	1.3% (4)		57
Topical word(s)	16.2% (50)	18.3% (62)	15.5% (43)	13.4% (19)	15.0% (32)	21.7% (59)	12.7% (39)	7.8% (24)	328
Non-topical words								57.1% (175)	175
Topical phrase	3.9% (12)	5.3% (18)	2.1% (6)	2.8% (4)	2.8% (6)	8.8% (24)	12.3% (38)	3.5% (11)	119
Participant comment	1.3% (4)	2.3% (8)	4.7% (13)	6.3% (9)	2.3% (5)	6.6% (18)	4.5% (14)		71
Speaker identifier		2.3% (8)	1.4% (4)	1.4% (2)	1.4% (3)	1.4%(4)	0.6% (2)	0.3% (1)	24
Verbal utterance	8.1% (25)	6.8% (23)	6.1% (17)	10.6% (15)	6.5% (14)	2.9% (8)	5.5% (17)	2.6% 8	127
Grammatical Changes
Incorrect Word Conjugation	4.5% (14)	1.4% (5)	2.1% (6)	0.7% (1)	2.3% (5)	2.5% (7)	1.3% (4)	2.6% (9)	51
Grammar Correction	1.1% (35)	9.7% (33)	10.8% (30)	11.3% (16)	20.1% (43)	12.5% (34)	12.7% (39)	10.4% (32)	262
Errors & Additions
Word Added	1.6% (5)	1.7% (6)	0.3% (1)						12
Incorrect Word	29.2% (92)	16.0% (54)	12.6% (35)	11.3% (16)	10.3% (22)	17.3% (47)	14.6% (45)	6.8% (21)	332
Incorrect Phrase	1.6% (5)	3.5% (12)	1.0% (3)		5.1% (11)	6.6% (18)	2.9% (9)	0.6% (2)	60
Total Errors	100% (307)	100% (337)	100% (276)	100% (141)	100% (213)	100% (271)	100% (307)	100% (306)	2158
Average errors/Transcriptionist	255					289		306

^*Transcriptionists are denoted at CR# and transcripts are denoted as FGD#.

We explored transcription errors by the age group of study participants (Table 5), whereby younger participants were aged 25-40 years and older participants aged more than 40 years. Results show that for both the transcriptionists and court reporters a greater number of errors were made in transcripts of older study participants. Eighty percent of all errors by transcriptionists were found in transcripts with older study participants, with most errors because of omitted speaker identifiers and inaudible speech. However, inaudible speech was also the most common error made by transcriptionists in transcripts of younger study participants. Fifty five percent of errors made by court reporters were also found in transcripts with older study participants, a smaller proportion than for transcriptionists, and the types of errors were more diverse than among transcriptionists. Moderators of the focus group discussions with older South Asian study participants reported that these groups had the greatest amount of dialogue in Asian languages, with participants translating for one another, which lead to multiple conversations at once, possibly contributing to inaudible speech on the recording. Some transcriptionists noted these side conversations as inaudible on the transcript whereas others noted difficulties in what was heard, for example:

• Unknown Participant (P?): [in Hindi/Gujarati – discussing how she eats toast and chai]

• P?: [discussion with another participant on meal sizes]

• P2: [Inaudible/can't translate]

• P1 and P2: [talking in the background about rice and osaman, a type of very liquid soup]

Table 5. Transcription Errors by Age of Participants: Transcriptionist and Court Reporter.

	Transcriptionist				Court Reporter
	Older FGD		Younger FGD		Older FGD		Younger FGD
Type of Error	%	(n)	%	(n)	%	(n)	%	(n)
Inaudible Speech
Marked in transcript	31.9%	(500)	58.0%	(227)	5.8%	(71)	2.9%	(29)
Not marked in transcript	0.2%	(4)			12.2%	(149)	4.1%	(40)
Omission of:
Topical discussion			0.2%	(1)	1.8%	(22)	2.9%	(29)
Participant acknowledgement	0.1%	(3)	0.7%	(3)	7.9%	(97)	7.89%	(76)
Moderator acknowledgement					3.6%	(44)	1.3%	(13)
Topical word(s)	0.3%	(6)	2.3%	(9)	15.8%	(192)	13.9%	(136)
Non-topical words			2.0%	(8)	0.9%	(12)	20.8%	(203)
Topical phrase	0.06%	(1)	0.2%	(1)	6.4%	(78)	4.2%	(41)
Participant comment	0.4%	(7)	0.7%	(3)	4.8%	(59)	1.2%	(12)
Speaker identifier	63.7%	(997)	18.4%	(72)	1.2%	15)	0.9%	(9)
Verbal utterance	0.06%	(1)			5.8%	(71)	5.7%	(56)
Grammatical Changes
Incorrect Word Conjugation	0.3%	(6)	0.7%	(3)	1.8%	(23)	2.8%	(28)
Grammar Correction	0.4%	(7)	0.5%	(2)	13.3%	(162)	10.2%	(100)
Errors & Additions
Word Added					0.08%	(1)	1.1%	(11)
Incorrect Word	1.7%	(27)	14.5%	(57)	13.6%	(165)	17.1%	(167)
Incorrect Phrase	0.3%	(6)	1.0%	(4)	3.3%	(41)	1.9%	(19)
Total Errors	100%	(1565)	100%	(391)	100%	(1213)	100%	(973)
Average Errors per FGD		313		78		242		194

We also compared transcription errors by the gender of study participants, size of the group discussion, and whether or not the transcriptionist was of South Asian descent (data for each not shown). Results showed that there was little difference in the number or type of transcription errors by the gender of participants. However, the size of the focus group discussion did influence the number of errors made by both transcriptionists and court reporters. Larger groups (10 or more participants) showed a higher average number of errors per group than smaller groups (<10 participants), however the types of errors made remained consistent regardless of group size. We found that transcriptionists of South Asian descent made slightly fewer mean errors per groups than those of non-South Asian descent (213 vs. 292 respectively); inaudible recording and use of an incorrect words accounted for most of the excess errors in transcripts developed by non-South Asian transcriptionists.

Cost and Time of Transcription

The cost of transcription was relatively similar between transcriptionists and court reporters, although transcriptionists' costs were more variable. The cost of transcription by court reporters was fixed by the agency at $300 per group discussion for up to eight groups and $265 for each subsequent group, whereas transcriptionists were paid an hourly rate of $12 per hour. The overall cost of transcription was slightly lower for transcriptionists than court reporters ($2,067 and $2,365 respectively), with an average cost per transcript of $255 and $295 respectively. However, transcriptionist's costs varied by the length of each transcript, so the cost per transcript ranged from $161-$394. The cost differential between transcriptionists and court reporters becomes more marked when examining short and long transcripts. The average cost for transcripts under 30 pages was lower for transcriptionists ($160 transcriptionist and $288 court reporter), whereas for transcripts more than 50 pages the average cost for a transcriptionist was almost $100 more per group discussion than for a court reporter ($394 and $300 respectively).

The turnaround time to develop verbatim transcripts varied slightly between transcriptionists and court reporters. Most transcripts were received one to two months after the group discussion. The average number of days to receive a completed transcript was slightly shorter for transcriptionists (101 days) compared to court reporters (124 days); however, the range of days to receive the transcript (34-170 days for transcriptionists and 60-175 days for court reporters) shows more clearly that court reporters took longer to provide transcripts in this study than did transcriptionists.

Effect of Recording Method on Participants

We identified whether either method of recording influenced participants' contributions to the discussion. Results from both survey questions and group discussions with participants showed little reported difference in how participants viewed each method of recording the discussion, and neither method influenced participants' reported contribution to the discussion. Sixty surveys were completed with participants from eight focus group discussions. The majority of participants reported that they had “no problem” with either the digital recorder (96.7%) or the court reporter's presence (96.7%) during the focus group discussion. Only two participants (3.3%) reported they felt “a little uncomfortable” with the digital recorder and the court reporter, but neither reported that this affected their contribution to the discussion.

During the discussion with participants on each method of recording, they stated that explaining the role of the court reporter was important and that without this they might have felt uncomfortable. Participants also stated that they soon forgot about the court reporter and it did not influence their participation in the discussion. Many participants stated that they were intrigued by how the stenotype machine worked, and some participants spoke with the court reporter afterwards about the technology. For example:

• P1: It [court reporter] didn't bother me. I am just amazed how she does it.

• P2: I looked over at her one time and she was just sitting there with the small keyboard.

• Moderator: Did it bother you at all?

• P2: No, it was intriguing that's all.

Participants also reported no problem with the digital recorder, and that recording the discussion in this way did not influence their participation. However, they indicated that if the topic of discussion was more personal they might have felt uncomfortable. Many participants felt that that it was important to record the discussion to capture their views because the research could have important benefits for the South Asian community.

Discussion

This is the first study to our knowledge to describe transcription errors made by transcriptionists and court reporters. Given the recent interest among qualitative researchers in using court reporters to develop verbatim transcripts, empirical data on the quality of the transcript produced are valuable. Although some transcription errors can be expected regardless of the mode of transcription, it is useful to distinguish the types of errors associated with different methods of transcription and identify how these errors can be reduced.

This study has shown important differences in the type of transcription errors made by transcriptionists and court reporters. Transcriptionists made fewer errors than court reporters, and the types of errors made were different from those of court reporters. In general, errors made by transcriptionists did not affect the topical content of the data being transcribed and the errors might be because of the quality of the recording used. The lower number of errors by transcriptionists might reflect their ability to stop, start and slow down the recorded dialogue and to review transcripts or seek clarification from others on unclear segments. In this study, the lower number of transcription errors might also be because of the characteristics of transcriptionists, who were public health graduate researchers specializing in behavioral sciences and were therefore familiar with the requirements of qualitative research. Furthermore, transcriptionists were members of the study team and invested in the project. Some transcriptionists were also, like study participants, of South Asian descent, which might have reduced the number of errors observed as these transcriptionists likely have increased familiarity with certain South Asian accents and some topics discussed (e.g., foods, activities, etc).

Two types of errors were common among transcriptionists, omitting speaker identifiers and missing dialogue because of inaudible speech in the recording. Transcriptionists are not always present at group discussion and rely on the sound of voices on the audiotape to distinguish each speaker, which is challenging when participants give short replies or only verbal utterances (e.g., “yeah” or “I agree”) or when several participants talk at once or interject. The number of errors because of omitted speaker identifiers might therefore be an inherent limitation of this method of transcription. The effect of this type of error on overall data quality is dependent on the purpose of a study. For example, in some qualitative studies, identifying individual speakers in a group discussion is critical, whereas for others it is sufficient to indicate that different people are speaking but not exactly which participant is speaking.

The other common error made by transcriptionists is missing dialogue because of inaudible speech, which is influenced by the quality of the recording and the moderators' skill in managing a discussion (e.g., ensuring that one person speaks at a time). Most transcriptionists indicated on the transcript where a section of dialogue was inaudible, which was confirmed in our error coding process, so it would be clear during data analysis that a section of dialogue was missing. Finally, errors by transcriptionists were concentrated within two transcripts, indicating that problems with particular group discussions lead to increased transcription errors. When these transcripts were removed from the analysis there were few errors overall made by transcriptionists. Many studies might have a problematic focus group discussion leading to lower quality transcription of that group.

Court reporters made more errors than transcriptionists and errors made had potential to change the topical content of the transcript. The higher number of errors by court reporters might reflect the mode of transcribing in real-time, whereby court reporters need to quickly capture multiple comments and fast-paced discussion, from which there is likely to be some loss of detail or misunderstanding of dialogue. Court reporters are also unable to replay the discussion, unlike a transcriptionist, or to take breaks which might also influence the higher error count.

The type of errors made by court reporters is important. Many errors involved the omission of topical words, phrases or discussions or the transcription of incorrect words that changed the meaning of a participant's comment. These errors lead to a loss of critical detail in the transcription that can influence a researcher's understanding of the issues. The omission of topical content from the transcripts in this study might reflect court reporters' difficulties in understanding accents of participants or being unfamiliar with certain words used in the discussion (e.g., reference to Asian foods, festivals, traditions etc). Another common transcription error made by court reporters is correcting grammar, instead of transcribing participants' colloquial speech. Transcribing colloquial speech might not be seen as important by court reporters (especially those not exposed to qualitative research), or it is possible that some software packages converting court reporter's shorthand into full English words change words to be grammatically correct. Nevertheless, the resulting transcript loses important contextual detail by removing colloquial expressions that hold meaning for participants and help researchers to interpret issues from the participants' perspectives, a critical part of qualitative data analysis, particularly if conducting grounded theory or narrative analysis.

In selecting a method of transcription, researchers need to be aware of the type of errors associated with each method and how this might affect the data needed for a study. Clearly, the purpose of the study will dictate the type of transcript that is needed (MacLean et al., 2004; Sandelowski, 1994). For example, a study using the grounded theory approach will require verbatim transcription of colloquial speech so a transcriptionist might be preferred, whereas another study might use key issues from a transcript to develop items for a survey instrument, whereby court reporters' transcripts might be suitable. Furthermore, researchers using court reporters might consider meeting them prior to the focus group discussion to explain the nature of qualitative research and to ensure they understand the reasons for verbatim transcription, which might minimize some of the transcription errors identified in this study (e.g., grammar correction).

Other considerations relate to the cost and time for transcription. Although an advantage of using court reporters is the potential immediacy of the transcription, there was only little difference in the turnaround time between transcriptionists and court reporters in this study. The authors believe that the longer turnaround time of court reporters in this study was particular to the agency used and not typical of court reporters in general, because agencies used in previous research submitted transcripts within one to two weeks of each group discussion. This study showed that the cost of transcription was influenced by the length of the group discussion. The cost of transcription favors a transcriptionist for shorter group discussions, whereas the fixed costs of a court reporter favored longer group discussions. This suggests that group length might be a criterion for selecting a method of transcription, however, a more intense cost analysis would be required to identify useful cutoffs in group length that indicate which method of transcription would be more cost effective. It should also be noted that costs for court reporters can vary widely, with some agencies charging a rate per page of transcription with fees ranging from $0.50 to $12.95 per page, and others charging additional appearance costs to attend the group discussion.

This study also highlights a number of strategies to improve the quality of transcription. First, because moderators of focus group discussions manage the group discussion, a greater emphasis can be given to encouraging participants to speak loud enough, talk one at a time and avoid having side conversations (Hennink, 2007; Krueger & Casey, 2000). Such attention might lead to a clearer recording of the dialogue with less inaudible segments and data loss. A moderator can also assist in identifying different speakers by asking participants to introduce themselves around the group to enable a transcriptionist to become familiar with different voices and include speaker identifiers on the transcript. Second, where participants are likely to have strong accents or the study is on unfamiliar topics, researchers can provide a list of topical words to expect in the dialogue or provide a sample recording from a pilot group discussion to familiarize accents and reduce data lost. Third, researchers need to ensure that the size of the focus groups remains within that recommended by the methodology, 6-8 participants (Hennink, 2007; Krueger & Casey, 2000), because results of this study showed that errors increased in larger group discussions where there is greater risk of simultaneous discussion and inaudible side conversations that are lost during transcription.

This study has several limitations important in interpreting our findings. Because multiple agencies exist for court reporters, study findings might be influenced by the specific agency used in this study, which was selected because of the limited research budget. The study population of South Asian Americans might also have influenced the type of transcription errors identified. Although study findings are transferable to research among similar ethnic groups or international studies, their transferability to general qualitative research remains limited without further research. This study has provided an initial understanding of transcription quality issues, however, further research is needed to assess whether these issues are evident in research among the general population, to assess the cost effectiveness of transcription methods, to use experimental designs to compare transcription errors by methods, and to explore the perspectives of court reporters on the challenges of developing verbatim transcripts for research.

Conclusion

Verbatim transcription is an important process in qualitative research, which can impact the type of data produced for analysis. In this article, we highlight key differences in transcription errors and the time and cost of using transcriptionists and court reporters. Even though transcriptionists provided transcription with colloquial dialogue, transcription errors were influenced by the quality of the recording used. Court reporters often missed topical dialogue and were less able to produce a verbatim transcript with colloquial language; however the immediacy of the transcript might be advantageous. Both methods of transcription have a place in qualitative research, and their benefits and limitations should be borne in mind when researchers select appropriate method of transcription for individual studies.

Biography

Monique Hennink, PhD, is an Associate Professor of Global Health at Emory University's Rollins School of Public Health, Atlanta, Georgia.

Mary Beth Weber, MPH, PhD is a post-doctoral fellow in the School of Medicine, Emory University, Atlanta, Georgia.