Skip to main content
The Canadian Veterinary Journal logoLink to The Canadian Veterinary Journal
. 2026 Feb 1;67(2):180–187.

Establishment of a sedation threshold score for orthopedic radiographs in dogs and evaluation of inter-rater reliability and accuracy of video-based assessment

Renata H Pinho 1, Daniel Pang 1,✉, Claire Leriquier 1, Dominique Gagnon 1, Javier Benito 1, Mila Freire 1
PMCID: PMC12915448  PMID: 41716508

Abstract

Background

Sedation scales are commonly used to assess sedation levels in dogs, but no threshold scores exist to guide decisions on the need for additional sedatives.

Objective

The objectives were to determine sedation score thresholds for obtaining orthopedic radiographs without restraint, evaluate inter-rater reliability, and compare video and real-time scoring.

Animals and procedure

Dogs (N = 64) sedated for obtaining various orthopedic radiographs were scored using a validated sedation scale, both in real time (1 rater) and via video assessment (3 raters). Sedation threshold scores were determined using receiver operating characteristic curves and the Youden index based on a rater opinion (yes or no) of whether radiographs could be completed without restraint. Two thresholds were calculated: 1 for all radiograph types and 1 specifically for stifle radiographs. Discrimination between adequately and inadequately sedated dogs was evaluated via the area under the curve (AUC). Inter-rater reliability was assessed using the intraclass correlation coefficient, and agreement between scoring methods was analyzed using the Bland-Altman approach.

Results

The threshold sedation scores were ≥ 16/21 (AUC = 0.71) for all radiographs and ≥ 12/21 (AUC = 0.77) for stifle radiographs, both indicating moderate ability to distinguish between adequately and inadequately sedated dogs. The inter-rater reliability of combined scores was very good (intraclass correlation coefficient > 0.81) for all raters, and the mean bias between video and real-time scoring was −0.08.

Conclusion and clinical relevance

The determined threshold scores can assist clinicians in determining whether additional sedation is necessary. The sedation scale demonstrated high reliability and accuracy, particularly when scored via video.

INTRODUCTION

Orthopedic radiographic examinations are routinely completed in the outpatient orthopedic service for diagnostic and follow-up evaluations, and animals are assessed and discharged to their owners shortly after their appointments. As these examinations are usually short-lasting, deep-sedation protocols are often preferred over general anesthesia. Achieving the optimal level of sedation is crucial to ensuring muscle relaxation, minimizing discomfort, and allowing proper positioning with minimal physical restraint. This, in turn, reduces the number of personnel required for restraint and decreases radiation exposure (1). In addition, the choice of chemical restraint for sedation can influence screening results for orthopedic conditions such as hip dysplasia, highlighting the need to establish an objective and appropriate level of sedation (1).

To provide a quantifiable measure to assess sedation in animals, sedation scales have been developed for several species, including dogs (2–4). Using an objective measure allows clinicians to evaluate sedation more effectively in relation to its intended purpose and ensures consistency among researchers in assessing sedation levels across different studies.

The sedation scale for dogs was validated by Wagner (2017), demonstrating excellent internal consistency and good inter-rater reliability when assessed by veterinary technicians (3). Since its validation, this tool has been widely used in several studies to compare different sedation protocols (5–7); however, no threshold score has been established to determine if the desired level of sedation has been achieved. Determining a threshold score can help practitioners establish the suitability and level of sedation necessary for radiographs, allowing for assessment before the animal undergoes diagnostic imaging techniques.

An ideal scenario for determining a threshold score would involve a sedation score with 100% specificity and 100% sensitivity. However, due to the natural variability among animals, this is not achievable. To identify the threshold that provides the optimal balance of sensitivity (i.e., dogs scoring at or above the threshold are truly sedated) and specificity (i.e., dogs scoring below the threshold are truly non-sedated), the receiver operating characteristic curve should be used (8). The area under the curve (AUC) should also be calculated to assess the scale’s discriminatory ability in recognizing sedation (9). These methods were originally developed to assess how well medical test thresholds differentiate sick and healthy individuals, and were later adapted to establish thresholds for pain scales in cats (10).

Although sedation assessment in clinical patients is typically done in person, assessments for research studies are often conducted using video recordings. Video assessment allows for retrospective evaluation of sedation; however, for this method to be as reliable as real-time evaluation, any bias between assessment methods should be minimal.

This study aimed to establish, for a validated sedation scale, a sedation threshold score at which dogs tolerate the manipulation necessary to obtain orthopedic radiographs. Secondary aims were to evaluate inter-rater reliability and to assess the agreement between video and real-time scoring of sedation level in dogs. We hypothesized that a threshold score with a strong discriminatory ability (AUC > 0.90) would be established, the sedation scale would demonstrate very good inter-rater reliability [intraclass correlation coefficient (ICC) > 0.81] when scored by veterinarians, and there would be minimal bias between real-time and video assessment.

MATERIALS AND METHODS

This study used video recordings of different stages of sedation in dogs that were sedated to undergo radiographic examinations as part of a separate study, published elsewhere (7). The study was approved by the Institutional Animal Care and Use Committee (IACUC-CÉUA 19-Rech-1996). Written, informed consent was obtained from the owner of each dog before its enrollment in the original study. Evaluations of the level of sedation were undertaken every 5 min by the same person (a surgery resident), who was unaware of the route of drug administration. The evaluations were video-recorded.

Animals

A total of 64 client-owned dogs were enrolled in the study. These dogs visited the Small Animal Surgery Service of the University of Montreal Teaching Hospital (Saint-Hyacinthe, Quebec) between August 2019 and January 2021 and required sedation for orthopedic radiographic examinations. Types of radiographs included stifle (n = 31), radius/ulna (n = 9), tarsus (n = 7), metatarsus (n = 5), pelvis (n = 4), tibia (n = 2), metacarpus (n = 2), stifle and spine (n = 2), stifle and pelvis (n = 1) and carpus (n = 1). Dogs were included if they were healthy apart from their orthopedic condition, had a body mass > 8 kg, and had been fasted for at least 6 h. Health status was based on history, physical examination, and normal complete blood (cell) count and chemistry profile obtained within 3 mo of the clinic visit. Dogs were excluded if they displayed any signs of aggression during physical examination.

Video recording and assessment

Assessments of the level of sedation at every 5 min during sedation and recovery periods were video-recorded using a camera (X20 4K Action Camera; Campark, Shenzhen, China). A total of 391 videos were generated, randomized, and scored independently by 3 blinded observers: the surgery resident who completed the real-time assessment (Rater A), a rotating intern (Rater B), and an anesthesiologist (Rater C). Sedation was scored based on 7 items, with scores summed to yield a total score (range: 0 to 21) and increasing sedation reflected by higher scores (3). The item categories included general appearance/attitude (0 to 3), spontaneous posture (0 to 4), reaction to noise (clicker actuated at ~150 cm from the head) (0 to 3), eye position (0 to 2), palpebral reflex (0 to 3), jaw and tongue relaxation (0 to 3), and resistance to being placed in lateral recumbency (0 to 3). No modifications were made to the original scale during scoring. The generated scores were used to assess inter-rater reliability and to calculate the agreement between real-time and video scoring.

In addition to applying the sedation assessment scale, the real-time rater (Rater A) provided a binary assessment based on a subjective decision on whether sedation level was sufficient to complete radiographic studies without addition of more sedatives (yes = 0, no = 1).

Study design

Each dog was acclimatized in a quiet room for 15 min before treatment injection. For sedation, 200 μg/m2 of dexmedetomidine (Dexdomitor; Zoetis, Kirkland, Quebec) was administered via one of the following routes: intramuscularly (IM) in the lumbar epaxial muscles, intravenously (IV) in a cephalic vein, or subcutaneously (SC) at the GV20 acupuncture point. The following formula was used to convert meters into kilograms: body surface area (m2) = 0.101 × (body weight in kg)2/3. After radiographs were obtained, atipamezole (2000 μg/m2, Antisedan; Zoetis) was given, either IM in the lumbar epaxial muscles or SC at the GV20 acupuncture point. The routes of administration of dexmedetomidine and atipamezole were determined randomly (using an online tool), creating a randomization list before the beginning of the study. Differences in levels of sedation obtained with the 6 possible treatment combinations (route of administration for sedation/route of administration for recovery: IM/IM, IM/GV20, IV/IM, IV/GV20, GV20/IM, GV20/GV20) were not addressed in this study but are published elsewhere (7).

Following dexmedetomidine administration, the level of sedation was evaluated every 5 min until it was considered subjectively sufficient to transfer the dog to the imaging diagnostic service and conduct the radiographic examination (the dog could be placed in lateral recumbency with limited reaction to stimuli). The number of assessments and recordings varied for each dog, as dogs were considered sufficiently sedated to be transferred to the imaging diagnostic service at different times after sedative administration. A minimum of 2 orthogonal radiographs were obtained, using sandbags to restrain the dog. When the radiographic study was finished, the dog was transported back to the initial sedation room for injection of atipamezole, and recovery was assessed. The level of sedation was evaluated every 5 min, similar to the measurements for the sedation period, until the dog had fully recovered and was able to stand and walk on its own.

Data analysis

Two threshold sedation scores were calculated: 1 for all radiograph types and 1 specifically for stifle radiographs. A receiver operating characteristic curve analysis was used to calculate the sensitivity and specificity of each real-time score provided by Rater A in achieving adequate sedation for radiographs with no restraint. The threshold score for achieving radiographs without restraint was determined based on the Youden index [YI = (sensitivity + specificity) − 1], and the AUC was calculated to indicate the ability of the sedation scale to differentiate scores associated with adequate sedation for radiographs from those indicating insufficient sedation levels.

Inter-rater reliability of each scale item and the sedation assessment score was assessed comparing the video scores provided by the real-time rater (Rater A) and the other raters (Rater B and C), using the ICC. An absolute model was used and both single and average measures were reported, along with the 95% confidence intervals (CI). Interpretation of the ICC was based on Altman (1991), using the following classification: “very good” = 0.81 to 1.0, “good” = 0.61 to 0.80, “moderate” = 0.41 to 0.60, “fair” = 0.21 to 0.40, “poor” < 0.20 (11).

The agreement between sedation scores provided during real-time and video assessments by Rater A was evaluated using the Bland-Altman method. The bias was reported, along with the upper (ULoA) and lower (LLoA) limits of agreement and their corresponding 95% CIs.

RESULTS

Considering all types of radiographs, the highest YI value of 0.33 identified a threshold sedation score of ≥ 16/21, with 64% sensitivity and 69% specificity in differentiating adequate versus inadequate sedation for obtaining radiographs without restraint (Table 1). For stifle radiographs only, the highest YI value of 0.52 identified a threshold sedation score of ≥ 12/21, with 59% sensitivity and 93% specificity in differentiating adequate versus inadequate sedation for obtaining radiographs without restraint (Table 1). The AUC of 0.71 for all types of radiographs (Figure 1 A) and the AUC of 0.77 for stifle radiographs (Figure 1 B) indicated a moderate discriminatory capacity of the sedation assessment score.

TABLE 1.

Sensitivity, specificity, and corresponding Youden index (YI) value for each possible score, considering all types of radiographs and only stifle radiographs.

Score All types of radiographs Stifle radiographs


Specificity Sensitivity YI Specificity Sensitivity YI
0 1 0 0 1 0 0
1 1 0 0 1.00 0.06 0.06
2 1 0.04 0.04 1.00 0.12 0.12
3 1 0.07 0.07 1.00 0.12 0.12
4 1 0.07 0.07 1.00 0.12 0.12
5 0.97 0.07 0.04 1.00 0.18 0.18
6 0.97 0.11 0.08 1.00 0.18 0.18
7 0.97 0.11 0.08 1.00 0.24 0.24
8 0.97 0.14 0.11 1.00 0.29 0.29
9 0.97 0.18 0.15 1.00 0.29 0.29
10 0.97 0.18 0.15 1.00 0.29 0.29
11 0.94 0.18 0.12 0.93 0.47 0.40
12 0.91 0.39 0.31 0.93 0.59 0.52
13 0.86 0.46 0.32 0.93 0.59 0.52
14 0.83 0.46 0.29 0.93 0.59 0.52
15 0.8 0.5 0.3 0.64 0.71 0.35
16 0.69 0.64 0.33 0.43 0.82 0.25
17 0.51 0.79 0.3 0.21 0.94 0.16
18 0.29 0.86 0.14 0.14 0.94 0.08
19 0.14 0.89 0.04 0.00 1.00 0.00
20 0.06 1 0.06 0.00 1.00 0.00
21 0 1 0 0.00 1.00 0.00

Optimal thresholds identified by highest YI value are highlighted in bold font style.

FIGURE 1.

FIGURE 1

Receiver operating characteristic curve for the determined threshold to successfully obtain orthopedic radiographs, and area under the curve (AUC), for all types of radiographs (A) and stifle radiographs (B). AUC interpretation: “low” = 0.50 to 0.70, “moderate” = 0.70 to 0.90, “high” > 0.90 (14).

The ICC values for both single and average measures of the combined sedation assessment score were considered very good for all raters. For the average measures, the items for posture, eyes, recumbency, and general appearance showed very good reliability across all raters. The item for jaw tone had very good inter-rater reliability for Rater B and good reliability for Rater C. The item for noise showed good inter-rater reliability for all raters. The item for palpebral reflex exhibited poor reliability across all raters (Table 2).

TABLE 2.

Inter-rater reliability between the video assessments of Rater A and Raters B and C.

Rater A versus Rater B Rater C
Combined score
 Single 0.91 (0.89 to 0.92) 0.89 (0.87 to 0.91)
 Average 0.95 (0.94 to 0.96) 0.94 (0.93 to 0.95)
Posture
 Single 0.87 (0.84 to 0.89) 0.87 (0.85 to 0.89)
 Average 0.93 (0.91 to 0.94) 0.93 (0.92 to 0.94)
Palpebral
 Single 0.16 (0.06 to 0.26) 0.05 (−0.05 to 0.15)
 Average 0.28 (0.12 to 0.41) 0.10 (−0.09 to 0.26)
Eyes
 Single 0.70 (0.64 to 0.75) 0.71 (0.66 to 0.76)
 Average 0.82 (0.78 to 0.86) 0.83 (0.79 to 0.86)
Jaw
 Single 0.71 (0.65 to 0.75) 0.58 (0.51 to 0.64)
 Average 0.83 (0.79 to 0.86) 0.73 (0.67 to 0.78)
Noise
 Single 0.55 (0.48 to 0.62) 0.51 (0.43 to 0.58)
 Average 0.71 (0.65 to 0.76) 0.68 (0.60 to 0.73)
Recumbency
 Single 0.78 (0.73 to 0.81) 0.68 (0.63 to 0.73)
 Average 0.87 (0.85 to 0.90) 0.81 (0.77 to 0.85)
General
 Single 0.81 (0.77 to 0.84) 0.76 (0.71 to 0.79)
 Average 0.89 (0.87 to 0.91) 0.86 (0.83 to 0.89)

Inter-rater reliability interpretation: “very good” = 0.81 to 1.0, “good” = 0.61 to 0.80, “moderate” = 0.41 to 0.60, “fair” = 0.21 to 0.40, “poor” < 0.20 (11).

The Bland-Altman method revealed a bias of −0.08 (95% CI: −0.20 to 0.04), with video scores underestimating real-time assessment for the combined score, and limits of agreement corresponding to LLoA: −2.45 (95% CI: −2.66 to −2.24) and ULoA: 2.30 (95% CI: 2.09 to 2.50). The scatterplot showed consistent differences in the magnitudes of the measurements, with less pronounced differences between video or real-time assessments when scores were < 5 and > 15 (Figure 2).

FIGURE 2.

FIGURE 2

Bland and Altman plot comparing real-time and video assessment of 391 videos. The dotted lines represent upper limit of agreement (ULoA) and lower limit of agreement (LLoA), the dashed line represents bias, and the solid horizontal line represents no difference between the methods of assessment (difference = 0). Points represent individual observations. The density of overlapping points is indicated by shades of gray: light gray = low density, medium gray = moderate density, dark gray = high density.

DISCUSSION

The results of this study showed that the developed threshold scores can assist practitioners in identifying the best sedation levels for dogs undergoing radiography and that the sedation scale is reliable when used by veterinarians, with accurate results when evaluated using video recordings.

The clinical relevance of our results lies in the novel development of threshold scores for a sedation scale. Although threshold scores are commonly used with pain scales to guide analgesic decisions, their application in managing sedation, particularly for diagnostic radiographs, can be equally valuable. We identified optimal threshold scores of ≥ 16/21 for all types of radiographs and ≥ 12/21 for stifle radiographs. Both thresholds demonstrated moderate discriminatory capacity based on the AUC. Sensitivity was similar for both (64 and 59%, respectively), but the threshold for stifle radiographs showed significantly higher specificity (93%) compared to that for all radiographs (69%). This difference was expected, due to greater variability in factors such as traction, animal positioning, and pain levels when considering multiple radiograph types. Although the sensitivity of the stifle-specific threshold (59%) indicates that not all animals scoring ≥ 12 are adequately sedated, the high specificity (93%) strongly suggests that animals scoring < 12 are unlikely to be adequately sedated and may require additional sedation before imaging. Therefore, reaching a sedation score of 12 would, in most cases, indicate that the animal is likely to tolerate the manipulation required to complete the radiographic examination.

In clinical practice, these thresholds could streamline workflow. For example, animals scheduled for stifle radiographs and scoring ≥ 12 may be transferred directly to the radiography room, whereas those scoring < 12 may require a reassessment (to determine whether the sedation score increases with additional time) or administration of additional sedatives if adequate sedation is not achieved. This approach could optimize patient flow and enhance sedation safety in imaging procedures.

In addition to a threshold score, another key attribute of a measurement instrument is reliability. A reliable scale should produce consistent scores when used by different individuals, regardless of their backgrounds and experiences (12). The “very good” reliability observed for the combined score among veterinarians reinforced the findings of the previous study, in which the scale also demonstrated very good reliability when used by veterinary technicians (3). Although the combined score of the scale demonstrated very good reliability, some individual categories were less reliable. The reliability of the palpebral reflex category was poor, whereas that for response to noise ranged from moderate to good. In addition, jaw tone showed moderate reliability for single measures in 1 rater. Factors contributing to the lower reliability of these items may include insufficient training and experience or unclear or poorly designed descriptors (12). For example, varying interpretations of the “eye position” descriptors were suspected to contribute to the poor reliability observed in this category of the sedation scale for rabbits (4). Another factor potentially affecting the reliability of video assessment scores is image quality. However, this was unlikely to be the case, as the eye position category, which requires high-quality imagery, showed good reliability. If the video quality was sufficient for assessing eye position, it should also have been adequate for evaluating jaw tone and palpebral reflex.

Our results demonstrated an adequate agreement between video scoring and real-time assessment, suggesting that the scale can be effectively applied via video with minimal bias (close to zero). Ensuring accuracy in retrospective video sedation assessments not only supports study blinding but also facilitates involving more raters and allows for later confirmation of results by experienced raters. Although there is no established standard for bias in determining whether an instrument accurately detects sedation, the bias of −0.08 was likely negligible and closely resembled the bias between real-time and video scores of a validated pain scale for rats (−0.07 to −0.14) that is commonly used with video assessment in research studies (13). In more pronounced states, such as absence of or minimal sedation (scores < 5) and high levels of sedation (scores > 15), the bias was less evident, similar to findings from the comparison of real-time and video assessments of the pain scale for rats (13).

A limitation of our study was that it included animals with different clinical conditions and temperaments, each potentially requiring varying levels of manipulation and associated with different pain levels. This variability may have contributed to the low specificity and sensitivity identified, as well as the small AUC for the discriminatory capacity of the thresholds in differentiating sedated from non-sedated dogs (reflecting a moderate discriminatory ability, rather than the high AUC initially hypothesized). Future studies could address this by establishing separate threshold scores for each type of condition and level of pain.

In conclusion, the threshold score can aid clinicians in determining whether an animal is adequately sedated for radiographic imaging or additional sedation is required. The sedation scale is a reliable tool that can be applied both in real time and during video assessment by raters with different backgrounds.

ACKNOWLEDGMENT

We thank Dr. L. Trainor for performing an initial video review. CVJ

Footnotes

This study was funded by the Fonds de santé des animaux de compagnie (internal funds of the Faculty of Veterinary Medicine of the University of Montreal).

Copyright is held by the Canadian Veterinary Medical Association. Individuals interested in obtaining reproductions of this article or permission to use this material elsewhere should contact permissions@cvma-acmv.org.

REFERENCES

  • 1.Malm S, Strandberg E, Danell B, Audell L, Swenson L, Hedhammar A. Impact of sedation method on the diagnosis of hip and elbow dysplasia in Swedish dogs. Prev Vet Med. 2007;78:196–209. doi: 10.1016/j.prevetmed.2006.10.005. [DOI] [PubMed] [Google Scholar]
  • 2.Oliveira ARD, Gozalo-Marcilla M, Ringer SK, et al. Development, validation, and reliability of a sedation scale in horses (EquiSed) Front Vet Sci. 2021;8:611729. doi: 10.3389/fvets.2021.611729. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Wagner MC, Hecker KG, Pang DSJ. Sedation levels in dogs: A validation study. BMC Vet Res. 2017;13:110. doi: 10.1186/s12917-017-1027-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Raulic J, Leung VS, Doss GA, et al. Development and testing of a sedation scale for use in rabbits (Oryctolagus cuniculus) J Am Assoc Lab Anim Sci. 2021;60:549–555. doi: 10.30802/AALAS-JAALAS-21-000002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Wimbish C, Lynch AM, Knych HK, Ueda Y, Messenger KM. Pharmacokinetics of a continuous intravenous infusion of hydromorphone in healthy dogs. Front Vet Sci. 2024;11:1362730. doi: 10.3389/fvets.2024.1362730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.de Carellán Mateo AG, Álvarez ER, Ros C. Comparison of the efficacy of 2 sedative protocols in pediatric dogs undergoing brainstem auditory-evoked response testing. Can Vet J. 2020;61:1175–1180. [PMC free article] [PubMed] [Google Scholar]
  • 7.Leriquier C, Freire M, Llido M, et al. Comparison of sedation with dexmedetomidine/atipamezole administered subcutaneously at GV20 acupuncture point with usual routes of administration in dogs presented for orthopaedic radiographs. J Small Anim Pract. 2023;64:759–768. doi: 10.1111/jsap.13668. [DOI] [PubMed] [Google Scholar]
  • 8.Lasko TA, Bhagwat JG, Zou KH, Ohno-Machado L. The use of receiver operating characteristic curves in biomedical informatics. J Biomed Inform. 2005;38:404–415. doi: 10.1016/j.jbi.2005.02.008. [DOI] [PubMed] [Google Scholar]
  • 9.Streiner DL, Cairney J. What’s under the ROC? An introduction to receiver operating characteristics curves. Can J Psychiatry. 2007;52:121–128. doi: 10.1177/070674370705200210. [DOI] [PubMed] [Google Scholar]
  • 10.Brondani JT, Mama KR, Luna SPL, et al. Validation of the English version of the UNESP-Botucatu multidimensional composite pain scale for assessing postoperative pain in cats. BMC Vet Res. 2013;9:143. doi: 10.1186/1746-6148-9-143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Altman DG. Practical Statistics for Medical Research. London, United Kingdom: Chapman and Hall; 1991. [Google Scholar]
  • 12.Martin P, Bateson PPG. Measuring behaviour: An introductory guide. 3rd ed. Cambridge, United Kingdom: Cambridge University Press; 2007. [Google Scholar]
  • 13.Leung V, Zhang E, Pang DS. Real-time application of the rat grimace scale as a welfare refinement in laboratory rats. Sci Rep. 2016;6:31667. doi: 10.1038/srep31667. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Fischer JE, Bachmann LM, Jaeschke R. A readers’ guide to the interpretation of diagnostic test properties: Clinical example of sepsis. Intensive Care Med. 2003;29:1043–1051. doi: 10.1007/s00134-003-1761-8. [DOI] [PubMed] [Google Scholar]

Articles from The Canadian Veterinary Journal are provided here courtesy of Canadian Veterinary Medical Association

RESOURCES