Abstract
Background: Ultrasound elastography is an imaging technology which can objectively and non-invasively assess tissue stiffness. It is emerging as a useful marker for disease diagnosis, progression and treatment efficacy. Objective: To examine current, published research evaluating the use of ultrasound elastography for the measurement of cutaneous or subcutaneous stiffness and to determine the level of validity and reliability, recommended methodologies and limitations. Methods: MEDLINE, Web of science and Scopus were systematically searched in August 2016 to identify original articles evaluating the use of ultrasound elastography to assess cutaneous stiffness. Relevant studies were then quality evaluated using the Quality Assessment of Diagnostic Accuracy Studies v 2 (QUADAS-2) tool and the Quality Appraisal of Reliability Studies (QAREL). Results: From a total of 688 articles, 14 met the inclusion criteria for full review. Within the 14 studies, elastography was used to evaluate tumors, systemic sclerosis, lymphedema, abscess, and post-radiation neck fibrosis. Only three robust studies demonstrated good interrater reliability, whereas all validity studies had low sample sizes and demonstrated risks of bias. Conclusion: Robust evidence supporting the use of ultrasound elastography as a diagnostic tool in cutaneous conditions is low, however, initial indicators support further research to establish the utility of ultrasound elastography in dermatology.
Keywords: Elastography, sonoelastography, shear-wave, strain imaging, elasticity imaging techniques, objective skin assessment, dermatology, cutaneous
Introduction
Numerous cutaneous conditions are associated with changes in tissue stiffness and fibrosis, including scarring, scleroderma, eczema, psoriasis and skin cancers. These conditions are typically assessed subjectively in clinical practice using both visual and palpation techniques [1]. However, objective, non-invasive methods to quantify the physiological changes in skin are required to improve diagnostic accuracy, to quantify disease severity and progression, and to evaluate the efficacy of treatment.
Elastography is an emerging imaging technology that provides an objective method to map the relative or absolute stiffness of soft tissue. First introduced in 1991 [2], elastography has been integrated into a number of imaging platforms including ultrasound, magnetic resonance imaging [3], optical coherence tomography [4-6], and atomic force microscopy [7]. Ultrasound elastography (UE) is showing the most promise clinically, as a relatively low cost, portable option that is easy to operate, and provides real time images of soft tissue mechanical properties for rapid analysis [3]. UE was first commercialized as an integrated system in 2003 [8], and is now available from all the major ultrasound imaging system manufacturers. It uses non-ionizing radiation and is therefore safe for repeated clinical use [3]. The utility is expanding with UE used to identify tissue stiffness associated with disorders of the thyroid [9], pancreas [10], liver [11], breast [12,13] and musculoskeletal function and dysfunction [14-16]. Recently, expansion of UE to cutaneous research has resulted in studies to identify skin tumors [17,18], systemic sclerosis [19], and abscess induration [20,21].
To evaluate the potential application of UE in the assessment of skin stiffness associated with scarring, this systematic review examined original research that investigated the diagnostic accuracy of UE to measure skin stiffness in a variety of cutaneous diagnostic groups. The aim was to determine the validity and reliability of UE in dermatological conditions, to identify appropriate methodology to minimize result bias, and to identify any perceived limitations of UE. In this review, we investigated the following hypothesis: Ultrasound elastography is a valid and reliable tool for assessing cutaneous stiffness. Based on this premise, we first overview the different types of UE and the methods of data acquisition employed, we then describe the methods and results of the systematic review, as well as the benefits, recommendations and limitations of the use of UE in cutaneous conditions.
Background
Ultrasound elastography (UE)
UE, sometimes referred to as sonoelastography, was developed to characterize the mechanical properties of biological tissues in vivo. As per conventional brightness mode (B-mode) ultrasound (US), acoustic signals are directed into the tissue where they are reflected or back scattered at tissue boundaries that exhibit distinct acoustic properties. This provides a grey scale image of tissue structure. However, UE also introduces a form of mechanical excitation to deform the tissue. This tissue motion is captured by multiple B-mode images. A softer tissue undergoes more strain (regional change in tissue thickness divided by initial thickness) than a stiffer tissue, when subjected to the same magnitude of force. Algorithms to estimate a mechanical parameter, such as strain or elasticity are then employed to map the relative or absolute elasticity into an image, referred to as an elastogram (Figure 1B). Each color pixel in the elastogram is an estimation of the mechanical property or parameter at that location. The elastogram is often visualized as an overlay on the B-mode sonogram, allowing the user to interpret both structural and mechanical properties of the tissue simultaneously. The color scale can vary, as no standard has been universally employed by device manufacturers [22]: in many instances, red represents stiffer tissue and blue represents softer tissues as shown in Figure 1. However, in other instances the scale is reversed [23]. The methods used to generate quantitative information from elastograms differ according to the type of elastography system used.
Types of ultrasound elastography
Measurement modalities currently used in UE include both strain imaging (SI) or shear wave elastography imaging (SWEI) [8]. They both use various forms of mechanical stimulus to deform soft tissues including compression, acoustic radiation force (ARF), vibration [3,24], or physiological movements. Both compression and ARF techniques have been used in dermatology, whereas vibration is used mostly in liver imaging [25], and physiological movements are used in cardiovascular [8], gastrointestinal [23], respiratory [8,26] and musculoskeletal imaging [27,28].
Compression techniques employ a vertical load, applied perpendicular to the skin, usually by means of the ultrasound imaging transducer (Figure 2). This can be achieved in one slow compression (static or quasi static) or by multiple, rapid compressions or vibrations (dynamic) depending on the implementation used in the UE system [29]. Local strain is measured by calculating the degree of tissue deformation with respect to depth, and consequently, it is not a direct measurement of tissue stiffness. The resulting elastogram is therefore qualitative, and provides information regarding the relative differences in strain between adjacent tissues [3]. Compression techniques are highly dependent on both the magnitude and spatial distribution of the applied force, which requires skilled operators to ensure a uniform contact force is applied at an optimal magnitude. Too little compression may not generate enough contrast between the tissues and too much compression can negate the relative differences, due to the nonlinear relationship between stress and strain in soft tissues such as skin [8]. However, a number of UE systems have introduced indicators to signify when optimal compression has been reached [23]. The reporting of strain elastograms can consist of either a subjective description of the color presentation in the elastogram [21], a semi-quantitative numerical scale developed to subjectively rate the heterogeneity and distribution of colors in the elastogram [30-32], or a strain ratio calculated between the area of interest (ROI) and surrounding non-affected tissue within the same field of view [33,34].
In contrast to compression, ARF is an acoustic force ‘push pulse’ (Figure 3) generated by the UE system, providing a more controlled magnitude of force. The push pulse is focused at a specific ROI in the tissue, to induce local tissue deformation on the order of micrometers (1×10-6 meters) [3]. ARF imaging (ARFI) provides an analysis of tissue deformation in response to ARF, however, like SI using compression, ARFI is still a relative measure providing only qualitative elastograms.
In addition to generating a highly focused point of tissue displacement, ARF also generates shear waves orthogonal to the incident sound wave (Figure 3), which ripple through the tissues at velocities typically in the range of 1-10 meters per second. In SWEI, the velocity of the shear waves propagating through the tissues is tracked to provide a quantitative measure of tissue stiffness [35]. An increasing shear wave velocity correlates with increasing tissue stiffness [22]. Different shear wave velocities are assigned different colors to create the elastogram, providing a 2D visual display of tissue mechanical heterogeneity [36]. In addition to producing an image, point shear wave velocity measurements can be obtained for specific ROIs on the elastogram, which can then be used to estimate the Young’s modulus, providing a quantitative measurement of stiffness, measured in kilopascals (kPa) [35,37]. SWEI is less dependent on the level of operator skill [38], however the operator still needs to ensure minimal pressure is applied to reduce interference of shear wave propagation [8].
Methodology
The methodology of this review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement [39,40]. A systematic search of the literature was conducted independently by two authors (H.D and S.A.) in August 2016 using the electronic databases Medline, Web of Science, and Scopus. The purpose of the search was to identify original published research evaluating the diagnostic accuracy of UE to measure the stiffness of skin or subcutaneous adipose tissues. A preliminary search in the Cochrane Library and the Centre for Reviews and Dissemination did not identify any previous systematic reviews on this topic.
Search parameters
The search was limited to peer reviewed papers published in English after 2003, the year UE was first commercialized [23]. Searches were performed of titles, abstracts and keywords to maximise article retrieval. The keywords for the search included: ‘skin’, ‘derma*’, ‘dermis’, ‘cutaneous’, ‘cutis’, ‘subcutaneous’, ‘subcutis’, ‘fat’, ‘adipose’, ‘fascia’, ‘elastography’, sonoelastography, and the mesh term ‘elasticity imaging techniques’. Various combinations of the above terms were included in the searches of the various data bases and refinement tools were utilized within each database (Table 1). The papers were manually screened based on the title and abstract content, and excluded if it obviously did not meet the inclusion criteria. Full text reviews were performed to further refine the results ensuring all included papers met the criteria. Lastly the reference lists of all papers included in the review were manually searched to ensure all related publications were included in this comprehensive review.
Table 1.
Search | Search words | Articles retrieved | ||
---|---|---|---|---|
|
||||
MEDLINE | Web of Science | Scopus | ||
#1 | skin or derma* or dermis or cutaneous or cutis | 761772 | 641499 | 1272045 |
#2 | subcutaneous or subcutis or fat or adipose or fascia | 362231 | 409907 | 670890 |
#3 | elastography or sonoelastography or ‘elasticity imaging techniques’ | 6611 | 9794 | 7800 |
#4 | #1 or #2 | 1089504 | 1026115 | 1880464 |
#5 | #3 AND #4 | 413 | 555 | 454 |
#6 | Following refinements or limitations* | 234 | 262 | 192 |
#7 | After removing duplicates, screening of titles, abstracts and keywords | 54 | +14 | +8 |
Total Articles: 76 |
Refinements or Limitations.
MEDLINE: Refined to include the major headings elasticity imaging techniques (222); skin (36); skin physiological phenomena (24). Scopus was limited to medicine - (370), and excluded: biochemistry, genetics and molecular biology (56); Engineering (47); Physics and Astronomy (32); Materials science (23); Computer Science (18); Agricultural and Biological Sciences (3) and Pharmacology, Toxicology and Pharmaceutics (3); letters (14) and book chapter (2); Review (28); Dentistry (2), neuroscience (1); Veterinary (1); Short survey (2). Web of Science: Refined to include engineering biomedical (79); dermatology (36); Medicine general internal (14); Multidisciplinary science (12); Oncology (11); nursing (2); Radiology nuclear medicine medical imaging (200); Surgery (19); Biophysics (29), limited to articles only.
Inclusion and exclusion criteria
Published articles evaluating the use of UE within human, cutaneous or subcutaneous-adipose tissues were included in the review, in order to evaluate the potential application in assessing burn scars. All study designs were included to ensure all relevant evidence was reviewed, and any novel findings often described in case reports were not omitted. Only studies that evaluated a commercially available, fully integrated UE system capable of measuring both skin and subcutaneous tissues were included to allow for potential replication of study designs. Therefore, papers using transient elastography, or elastography based on other imaging techniques, such as optical coherence tomography, computed tomography or atomic force microscopy were excluded, as were those that evaluated tissue elasticity by manually combining ultrasound with another objective measuring device (e.g. extensometer). Studies testing new (non-commercially available) devices or purely evaluating the engineering, physics or mathematical modelling of elastography were also excluded. Although these studies provide evidence of equipment validity, the scope of this project was to investigate the validity and reliability of the clinical application. Likewise, research conducted exclusively on phantom tissues or on body organs other than the skin and/or subcutaneous adipose tissues (e.g. breast, liver, thyroid, and musculoskeletal) were excluded.
Quality criteria
The papers included for review were assessed for risks of bias using the Quality Assessment of Diagnostic Accuracy Studies - 2 (QUADAS-2) tool (http://www.bristol.ac.uk/social-community-medicine/projects/quadas/quadas-2/) which is a validated and reliable tool to assess the study’s methodological level of bias in four categories: 1) patient selection, 2) index test, 3) reference standard and 4) flow and timing, by using signaling questions to identify areas of either high risk, low risk, unclear or not applicable [41].
In addition, the definitions and recommendations for assessing validity and reliability were guided by the Consensus-based standards for the selection of health measurement Instruments (COSMIN), (http://www.cosmin.nl) and the review of quality was based on the Quality Appraisal of Reliability Studies (QAREL) [42]. A checklist composed of ten questions was adapted from these assessment tools to cover aspects not evaluated in the QUADAS-2 (Table 1).
Data extraction
The data was extracted using a predesigned data extraction document which included the study design, study aim, sample size, diagnostic group, patient demographics, UE system used (index tool) and technical specifications (probes, MHz), the reference tool, methodology, measured values, clinimetric evaluations conducted, analysis used and the reported results, limitations and recommendations.
Data synthesis
Due to the variations in patient diagnostic groups, direct comparison of all the studies was not feasible [43], therefore a descriptive analysis of the studies within each diagnostic group is presented and the overall clinimetric evidence of elastography will be summarized.
Protocol publication and ethics
The protocol was written and submitted a priori to the Edith Cowan School of Medical and Health Sciences Ethics Subcommittee. An ethics declaration was approved on 15/08/2016: project number 16468 DEJONG.
Results
The data bases were searched independently (Table 1). Medline yielded 234, Web of Science 262, and Scopus yielded 192 potential papers for inclusion. These papers were then further screened by titles, keywords and abstracts, with 54 papers identified in Medline, and after duplicates were removed a further 14 were identified in Web of Science and an additional 8 in Scopus. A total of 76 papers underwent a full text screen and a further 52 were removed for not meeting the inclusion criteria, leaving a total of 23 for critical review. Following critical review one paper [44], was excluded as the UE component was used to measure tissue thickness rather than tissue stiffness, three papers using the Tissue Ultrasound Palpation System (TUPS) were excluded as the system is no longer commercially available [45-47], an additional three papers were excluded as they did not document the UE system they used [48-50] and finally one paper used video elastography (not commercially available) which was also excluded [51]. The citations listed within each included paper were also reviewed, however did not generate any additional papers.
A total of fifteen papers published between 2009 and August 2016 were included in the full critical review (Table 2). Eleven papers studied the efficacy of SI and four studied SWEI. Five diagnostic groups were identified including cutaneous/subcutaneous lesions (N=6), systemic sclerosis (N=5), lymphedema (N=2), skin abscess (N=1), and post-irradiation neck fibrosis (N=1). Six studies evaluated the clinimetric properties of their assessment protocols, including interrater reliability (N=4), intra-rater reliability (N=2) and/or test-retest reliability (N=3) whilst others tested criterion validity (N=2), convergent validity (N=6), and/or discriminate validity (N=7).
Table 2.
Diagnostic Group | Main Objective | UE type | Sample size | Elastogram Analysis | Results | Reference |
---|---|---|---|---|---|---|
Cutaneous Melanoma | 1. Criterion validity, Convergent validity | SI | N=39 patients (42 lesions) Total lesions included =37 | 1. Intra dermal descriptive analysis: (low = red, medium = green or high = blue) | 1. Histology: Diagnosis confirmed: 45% tumors were stiff (blue) and 43% moderate (green). | Assessment of Cutaneous Melanoma by Use of Very-High-Frequency Ultrasound and Real-Time Elastography. Botar-Jid et al. 2016 [33] |
2. Sensitivity/specificity | 2. Strain ratio (tumor/dermis & tumor/hypodermis. | 2. UE appearance and US thickness: rs=-0.305, p=0.049. Thicker tumors were stiffer. | ||||
3. Strain ratio average tumor/dermis =1.02 tumor/hypodermis =2.16 | ||||||
Malignant and benign skin tumors | 1. Criterion validity | SI | N=55 patients, (69 lesions) Total lesions included =67 | 1. Intradermal descriptive analysis: Blue =less elastic; red more elastic. | 1. Histology: diagnosis confirmed 43% malignant, 57% benign. | Quantified ultrasound elastography in the assessment of cutaneous carcinoma Dasgeb et al. 2015 [17] |
2. Sensitivity/specificity | 2. Strain ratio (tumor/adjacent ‘green’ tissue | 2. Strain ratio of malignant lesions =3.9-32.2; & Benign lesions =0.01-3.0. | ||||
Mixed tumor of scalp | Describe unique finding. | SI | N=1 | Descriptive analysis of color (red = soft, blue = hard) | Histology confirmed diagnosis. Tumor interior was island shaped with a red (soft) cord-like portion, with a green (medium elasticity) background. Green portion correlated with closely aggregated myoepithelial cells around the sweat gland. The yellow area corresponded to a broad mucinous region, empty space and the chondroid portion. | Ultrasound B-mode and Elastographic Findings of Mixed Tumour of the Skin on the Scalp. Imafuku et al. 2016 [18] |
Angiomatoid fibrous histiocytoma | Describe unique finding | SI | N=1 | Descriptive analysis of color. Blue = stiff, red = soft. | Histology confirmed diagnosis. An area of increased elasticity (soft) within the tumor corresponded to a pseudovascular structure. | Ultrasound B-mode and elastographic findings of angiomatoid fibrous histiocytoma. Hata et al. 2014 [52] |
Melanoma | Describe a unique finding. | SI | N=2 | Descriptive analysis of color. Blue = stiff, red = soft | Histology confirmed diagnosis, Breslow thickness was 1.5 mm and 0.95 mm. The tumor had dark blue areas within lesion, whereas a benign nevus had a green pattern on UE. | Real-time tissue elastography: a helpful tool in the diagnosis of cutaneous melanoma? Hinz et al. 2011 [53] |
Cutaneous T-cell Lymphoma | Describe a unique finding. | SI | N=1 | Descriptive analysis of color. Blue = stiff, red= soft. | Histology confirmed diagnosis, tumor had dark blue appearance, whereas epidermal cyst had a green appearance | Real time tissue elastography for diagnosis of cutaneous T-cell lymphoma. Schmid-Wendher et al. 2011 [54] |
Systemic Sclerosis and diffuse cutaneous systemic sclerosis | 1 Discriminant validity, Convergent validity, Interrater reliablity | SWEI | N=15 patients N=15 matched healthy controls 17 body sites on each participant Total sample: 510 skin sites. | SWV | 1. Patients had increase SWV at 6/17 body sites. mRSS 0, 1 & 2 were significantly higher than healthy controls (p<0.001). | A preliminary study of acoustic radiation force impulse quantification for the assessment of skin in diffuse cutaneous systemic sclerosis. Hou et al. 2015 [38] |
2. Significant differences in SWV between mRSS levels 0 and 1; and between 0 and 2 (both p<0.001). However, level 3 was not significantly different to the other levels. Sum of SWV values from17 sites correlated with total mRSS (r=0.841, p<0.001) | ||||||
3. ICC good for 15/17 body sites (ICC=0.613-0.916), moderate for left middle finger (ICC=0.535) and poor for right middle finger (ICC=0.247). | ||||||
Systemic Sclerosis, Morphea and GVHD | 1. Convergent validity, Discriminant validity, Test retest reliability | ARFI-D and SWEI | N=4 healthy N=12 morphes N=1 SSc N=5 GVHD with contralateral site matched controls Total sample: 22 participants | 1. ARFI-D | Descriptive analysis of selected case examples demonstrating: | Preliminary Results on the Feasibility of Using ARFI/SWEI To assess Cutaneous Sclerotic Diseases. Yun Lee et al. 2015 [35] |
2. SWV | 1. Both SWV and ARFI-D can detect differences between SSc and healthy controls. | |||||
2. SWV more sensitive than ARFI-D for detecting severity. | ||||||
3. SWV showed less errors than ARFI-D | ||||||
4. SWV were more consistent than ARFI-D over time. | ||||||
5. Had inconsistent results over fingers. | ||||||
Systemic Sclerosis (perioral region) | 1. Discriminant validity, Interrater reliability | SI | N=6 female patients N=6 female age-matched controls 4 body sites on each participant | 1. Descriptive analysis and score: Blue score=1 Green score=2 Red score=3. 4 locations measured on each patient and summated for a total score ranging from 4-12 | 1. American College of Rheumatology criteria: Found SSC was predominantly blue with green spots stiffer than controls (p=0.01) | Technical feasibility of real-time elastography to assess the peri-oral region in patients affected by systemic sclerosis Cannao et al. 2014 [30] |
2.Two raters. Cohen’s K=1 (p<0.0-05) for both the summated score and individual measurements. | ||||||
Systemic Sclerosis | 1. Test-retest reliability | SWEI | 1. N=4 patients N=2 controls | SWV | 1. Very good test-retest reliability ICC>0.08, tested 1 week apart. | A preliminary study using virtual touch imaging and quantification for the assessment of skin stiffness in systemic sclerosis Santiago et al. 2016 [19] |
2. Discriminate validity and Convergent validity | 2. N=26 patients N=17 age and gender matched controls. | 2. Patients with SSc showed increased stiffness compared to controls (p<0.01) | ||||
3. Stiffness correlated with mRSS | ||||||
4. Increased stiffness recorded in clinically ‘unaffected’ skin, mRSS*=0. | ||||||
Systemic sclerosis | 1. Discriminant validity, Interrater reliability, Intra rater reliability, Test-retest reliability | SI | N=18 patients N=15 age matched control | 1. Descriptive color scale: red = soft green/pale blue = soft blue = hard | 1. mRSS*: Sclerotic dermis was blue, healthy control dermis was green with pale blue spots. Dorsal fingers obtained inconsistent results. | Ultrasound elastography assessment of skin involvement in systemic sclerosis: lights and shadows. Iagnocco et al. 2010 [55] |
2. two raters: No data presented, stated that both assessors had 100% agreement. | ||||||
3. No data only statement of 100% agreement. | ||||||
4. No data: Images stored and re-evaluated 4 weeks post initial assessment. | ||||||
Lymphedema and LDS. | 1. Convergent validity | SI | N=62 Lymphedema legs N=15 LDS legs | Strain ratio: skin/phantom & scAT/phantom | 1. No difference between ISL stages O, I, II, III of skin or scAT in thigh. | Skin and subcutaneous tissue strain in legs with lymphedema and lipodermatosclerosis. Suehiro et al. 2015 [34] |
2. No difference between ISL stages 0, I, II, III of scAT of calf. | ||||||
3. LDS was stiffer than ISL stages 0, I, II (p<0.05). | ||||||
4. Calf skin was stiffer in ISL stage III compared to stage I and II, but no difference between stage I or O (p<0.05) | ||||||
5. LDS calf was stiffer than ISL 0, I, II (p<0.05). | ||||||
Lymphedema | 1. Protocol validity | SI | 1. 7mm gel standoff | 1. Peak force strain | 1. Optimal manual compression force on 7mm gel standoff =50<F<200gf | Real-time tissue elastography assessment of skin and subcutaneous tissue strains in legs with lymphedema. Suehiro et al. 2014 [56] |
2. Sensitivity in normal controls | 2. N=35 healthy controls | 2. Strain ratio: skin/phantom & scAT/phantom | 2. Normal control demonstrated greatest strain in the ScAT thigh, followed by ScAT calf, skin thigh then skin calf. | |||
3. Discriminate validity | 3. N=15 patients each with affected leg and non-affected control leg. | 3. linear regression | 3. No difference detected between ISL stage II unilateral lymphedema leg and non-affected leg. | |||
Skin and subcutaneous abscess | Discriminate validity. | SI | N=50 patients | Descriptive analysis: red = softest, yellow = soft, green = medium. blue = firm confluent bands large spot small speckles | B-mode ultrasound and drainage of purulent material confirmed diagnosis: UE: abscesses were mixed red and yellow to mixed yellow and green. Tissue Induration: blue with sharp delineation. | Use of ultrasound elastography for skin and subcutaneous abscesses Gaspari et al. 2009 [21] |
Post Irradiation neck fibrosis. | 1. Interrater, Intrarater reliability | SWEI | 1. N=30, 14 patients and 16 controls | The system software calculated a mean stiffness value for the ROI. | 1. Interrater: ICC=0.77-0.94 Intrarater: ICC=0.90-0.99 | Shear Wave Elastography-A New Quantitative Assessment of Post-Irradiation Neck Fibrosis. Liu et al. 2015 [57] |
2. Discriminate validity, convergent validity | 2. N=50, 25 patients and 25 controls. | 2. Irradiated neck tissue stiffer than normal control: 63.9±53.1 vs 15.3±8.37 kPa, p<0.001. | ||||
3. Stiffness progressively became stiffer with time post irradiation. | ||||||
4. US thickness negatively correlated with stiffness (p<0.001) |
ARFI: Acoustic Radiation Force Impulse; ARFI-D: Acoustic Radiation Force Impulse - Displacement; GVHD: Graft versus host disease; ICC: intraclass Correlation; ISL: International Society of Lympedema; LDS: Lipodermatosclerosis; mRSS: modified Rodnan Skin Score; ROI: Region of Interest ; SI: Strain Imaging; ScAT: Subcutaneous Adipose Tissue; SSc: Systemic sclerosis; SWEI: Shear wave elasticity imaging; SWV: Shear Wave Velocity; UE: Ultrasound Elastography.
Quality rating
Reliability
Reliability studies (Table 3) were completed by six of the 14 studies; four assessed interrater reliability [30,38,55,57], two intra-rater reliably [55,57] and three assessed test-retest reliability [19,35,55]. However only three studies had adequate sample sizes: two evaluated SWEI [38,57] and one evaluated SI [55]. The other three studies [19,30,35] had samples sizes below 30 [58].
Table 3.
Studies evaluating Strain elastography | Studies evaluating shear wave elastography | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
|||||||||||
Gaspari 2009 [21] | Iagnocco, 2010 [55] | Cannao, 2014 [30] | Suehiro, 2014 [56] | Dasgeb, 2015 [17] | Suehiro, 2015 [34] | Botar-Jid, 2016 [33] | Santiago, 2016 [19] | Hou, 2015 [38] | Liu, 2015 [57] | Yun Lee, 2015 [35] | |
1. Was interrater reliability of index test reported? | N | Y | Y | N | N | N | N | N | Y | Y | N |
2. Were raters blinded to the findings of other raters? | N/A | Y | Y | N/A | N/A | N/A | N/A | N/A | Y | Y | N/A |
3. Was intra-rater reliability of index test reported? | N | N | N | N | N | N | N | N | N | Y | N |
4. Were raters blinded to their prior findings? | N/A | Y | N/A | N/A | N/A | N/A | N/A | N/A | N/A | Y | N/A |
5. Was test-retest reliability of index test reported? | N | Y | N | N | N | N | N | Y | N/A | N | Y |
6. Was the stability of the variable being measured taken into account when determining the time interval between repeat measures? | N/A | Y | N/A | N/A | N/A | N/A | N/A | Y | N/A | N/A | ? |
7. Were raters blinded to additional cues that could bias results. | N/A | Y | N | N/A | N/A | N/A | N/A | Y | N | Y | N |
8. Was the sample size included in the analysis adequate? | N/A | Y | N | N/A | N/A | N/A | N/A | N | Y | Y | N |
9. For continuous scores: was an intraclass correlation coefficient (ICC) calculated? | N/A | N/A | N/A | N/A | N/A | N/A | N/A | Y | Y | Y | N |
10. For dichotomous/nominal/ordinal scores: was a kappa calculated? | N/A | N | Y | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
Y Yes, N No, ? unclear N/A Not Applicable.
Liu et al. [57] studied the use of SWEI in post-irradiation neck fibrosis, which was conducted on a fair sample size (N=30), including patients (N=14) and healthy controls (N=16). Two operators were blinded to the velocity point measurements displayed on the monitor and to each other’s measurement to test the inter-rater reliability. The first operator then re-imaged the same patients on the same day, to evaluate intra-rater reliability. Both intra- and inter-rater reliability were high (ICC=0.84-0.95 and ICC=0.77-0.94 respectively). A second study evaluated the interrater reliability of SWEI to assess participants diagnosed with systemic sclerosis [38]. Seventeen body sites were measured by two physicians blinded to each other, on a sample of 15 patients and 15 healthy controls. Each body site was independently evaluated for reliability. The interrater reliability was high for 15 of the 17 body sites (ICC=0.613-0.916), moderate for the left middle finger (ICC=0.535) and poor for the right middle finger (ICC=0.247).
Iagnocco et al. [55] evaluated interrater, intra-rater and test-retest reliability on a fair sample size of 32 participants, 18 with systemic sclerosis and 15 healthy controls. Two anatomical sites (volar aspect of forearm and the dorsal aspect of the finger) were evaluated using a dichotomous key (stiff tissue = blue, soft tissue = pale blue, green and red). Two blinded sonographers acquired two images for each site, however it wasn’t clear whether only one image was analyzed or both. The saved images were then evaluated by both sonographers independently to evaluate interrater reliability of image analysis. The evaluation was repeated 4 weeks later to assess test-retest reliability. They found 100% agreement was reported for both interrater and test-retest analyses, however a Cohen’s Kappa was not reported.
The three studies with inadequate sample sizes included Cannao et al. [30], who reported a perfect interrater reliability (Cohen’s Kappa =1) when using SI on six participants with systemic sclerosis and six healthy controls. Two additional studies evaluated test-retest reliability. Santiago et al. [19] reported a high ICC=0.9, on a sample size of six, using SWEI to evaluate systemic sclerosis; and Yun Lee et al. [35] provided a preliminary report of a single selected case study from a sample size of 22 patients with systemic sclerosis, thus it is difficult to know whether these results are representative of the entire cohort.
Validity
Quality assessment (Tables 2 and 4) demonstrated all studies had at least one risk of bias. The primary area of bias was with patient selection, whereby none of the studies justified their sample size or provided power calculations to test their hypothesis. Most studies could be considered exploratory, feasibility or pilot studies for which sample size calculations are not required [59]. However, this means all results regarding validity are taken with caution until larger studies with adequate power calculations are completed. In addition, with only two studies providing adequate interrater reliability, it is uncertain whether the protocols provided consistent results, which can produce a risk of index test bias.
Table 4.
Strain Elastography Studies | Shear Wave Elastography Studies | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
|||||||||||
Gaspari 2009 [21] | Iagnocco, 2010 [55] | Cannao, 2014 [30] | Suehiro, 2014 [56] | Dasgeb, 2015 [17] | Suehiro, 2015 [34] | Botar-Jid, 2016 [33] | Santiago, 2016 [19] | Hou, 2015 [38] | Liu, 2015 [57] | Yun Lee, 2015 [35] | |
Patient Selection | |||||||||||
1. Were patients recruited consecutively or by random sampling? | L | L | ? | ? | L | ? | L | L | L | ? | ? |
2. Was a cross section of the population assessed? | H | H | H | H | L | L | L | L | L | L | H |
3. Did the study avoid inappropriate exclusions? | L | H | ? | ? | L | ? | L | L | L | L | ? |
4. Was the sample size justified with power calculations? | H | H | H | H | H | H | H | H | H | H | H |
5. Could the selection of patients have introduced bias? | L | H | H | H | L | H | H | H | L | L | H |
Index Test | |||||||||||
6. Were the index test results interpreted without knowledge of the results of the reference standard? | L | L | H | ? | L | ? | L | L | L | N/A | ? |
7. Could the conduct or interpretation of the index test introduce bias. | H | H | H | H | H | H | H | H | L | L | H |
Reference Test | |||||||||||
8. Is the reference standard likely to correctly classify the target condition? | L | L | L | L | L | L | L | L | L | L | ? |
9. Were the reference standard results interpreted without knowledge of the results of the index test? | L | L | L | L | L | L | ? | L | L | ? | ? |
Flow and Timing | |||||||||||
10. Was there an appropriate interval between the index test and the reference standard? | L | L | N/A | N/A | L | N/A | L | L | L | N/A | N/A |
11. Did all patients receive the same reference standard? | L | L | L | L | L | L | L | L | L | L | ? |
12. Were all patients included in the analysis? | L | L | L | L | H | L | H | L | L | L | H |
L Low Risk, H High Risk, ? unclear, N/A Not Applicable.
Discussion and recommendations
UE is relatively new technology, with the first commercial release in 2003 and the first cutaneous study published in 2009. Since then, its use in cutaneous research has increased with 50% of the reviewed articles published in or after 2015. The testing of UE in dermatology is still in the early clinical phase with initial exploratory research and feasibility studies defining the potential capabilities, unique findings, limitations and recommendations. Robust evidence for the validity of UE to assess cutaneous stiffness correlating to pathology is yet to be demonstrated and only minimal evidence of interrater and intra-rater reliability was demonstrated. However, the review has highlighted the potential advantages, some limitations and areas for further development both within specific diagnostic groups and for the general use of UE in cutaneous conditions.
The initial findings indicate that UE enhances diagnostic accuracy of skin tumors by detecting differences between pathological and non-pathological tissue stiffness. Six studies, including four case studies, one case-controlled study and one cross sectional study used SI to investigate tumor stiffness prior to diagnosis with histology. The four case studies described the tumor to be visibly stiffer on the elastogram in comparison to the surrounding tissues. The tumors included two melanomas [60], a subcutaneous T-cell lymphoma [54], a mixed tumor of the scalp [18], and an angiomatoid fibrous histiocytoma [52]. In addition, two of the case studies compared the results to non-malignant tumors (epidermal cysts [54] and a nevus [60]) which demonstrated that malignant tumors were relatively stiffer than the surrounding tissue, in comparison to non-malignant tumors.
The case-controlled and cross section studies both calculated strain ratios to evaluate the sensitivity of SI to diagnose malignant skin cancers. Dasgeb et al. [17] assessed 67 clinically suspicious skin lesions prior to biopsy. The strain ratio was calculated between a ROI within the tumor and a ROI in adjacent healthy tissue. The malignant lesions (43% of the sample) had strain ratios equal to or above 3.9 whereas the benign lesions (57% of the sample) were equal to or below 3.0. This is consistent with evidence from breast cancer UE whereby strain ratios were higher in breast malignancies (average 3.04±0.9) compared to benign tumors (average 1.91±0.75) [32,61]. Botar-Jid et al. calculated strain ratios between melanomas and normal dermis, and the melanoma and the hypodermis in 37 participants and found the strain ratios were lower: tumor/dermis (1.02, p=0.002) and the tumor/hypodermis (2.16, p=0.001). However, the malignant lesions assessed by Dasgeb et al. consisted of basal cell carcinomas (N=17), and squamous cell carcinomas (N=12), rather than melanomas, which may explain the differences in results between the two studies. In addition, their chosen methodologies may contribute to the difference: Dasgeb et al. used either a 2 mm or 4 mm ultrasound standoff gel pad, whereas Botar-Jid et al. used ultrasound gel. Being a superficial tissue, skin is more vulnerable to artifacts caused by excessive or inadequate compression of the SI probe [22]. Although both UE systems had quality indicators, neither study assessed the reliability of their protocols prior to their study. A stand-off gel pad is recommended for imaging superficial tissues such as the Achilles tendon [23] as it can disperse the applied forces more evenly, improve the field of view and reduce the risk of artifacts associated with probe contact.
A stand-off gel pad was employed in three other studies [30,34,56] within this review, one assessing systemic sclerosis in the perioral region and two investigating UE evaluation in lymphedema. Cannao et al. [30] evaluated the interrater reliability of their methodology using a 10 mm stand-off gel pad with SI to assess systemic sclerosis. A Cohen’s Kappa of 1 was reported, however it was based on a small sample size of 12. The two studies in lymphedema [34,56] were produced by the same author who assessed the feasibility of using UE to measure skin and subcutaneous adipose tissue (ScAT) stiffness in the thigh and calf of lymphedema patients. The first study developed a protocol to use a 7 mm stand-off gel pad as a coupling medium. The gel pad was repeatedly compressed under various forces which identified the optimal amount of force was between 50 and 200 g to acquire consistent elastograms. In contrast to the previous skin tumor studies, the second part of this study evaluated the stand-off gel pad as a reference standard for strain ratio calculations, rather than adjacent healthy tissue. 35 healthy participants (70 legs) were assessed providing both strain measurements of the skin and scAT and strain-ratio data for the skin/stand-off and the scAT/stand-off. However, the strain ratio measurements were thought to be influenced by the strain in the stand-off gel pad, therefore it was determined that direct strain measurements of the skin and scAT were more suitable calculations compared to the ratios. Further research is required to evaluate the influence of a stand-off gel pad in UE.
Initial evidence of discriminate validity was demonstrated in a study evaluating post-irradiation neck fibrosis using SWEI [57]. In a cross sectional study of 50 people (N=25 patients and 25 gender and age matched controls), the patients post irradiation treatment had a statistically significant higher average Young’s modulus (63.9±53.1 kPa) compared to health controls (15.3±8.37 kPa, p, 0.001). In addition, measurements of subcutaneous adipose (scAT) neck stiffness correlated to time post irradiation (p<0.001), which supported clinical observations and prior research that fibrosis is progressive. Furthermore, Young’s modulus negatively correlated with scAT thickness (r=-0.61, p<0.001 and r=-0.75, p<0.001) suggesting that thinner subcutaneous tissues were also stiffer. These findings support further research into the use of UE as a biomarker for complications of radiotherapy, to assess both the severity and progression of scarring (fibrosis) post treatment and to develop a better understanding of the associations between radiotherapy scarring and other post-irradiation complications, so that fibrosis can be managed more effectively in the clinical setting.
Initial suggestions of discriminate validity were found in all five studies evaluating systemic sclerosis. All studies used the accepted clinical assessment tool, the modified Rodnan Skin Score (mRSS) which classifies skin into four categories (0-3) of severity based on palpation. Patients with systemic sclerosis had stiffer skin compared to healthy controls in studies evaluating SI [30,55] and studies evaluating SWEI [19,35,38]. The methodology however, varied between the studies. Using SI Iagnocco et al. [55] used a dichotomous descriptive scale whereby blue represented stiff tissue and all other colors represented soft tissue. Two blinded raters found all patients with sclerotic skin (N=18) had visibly stiffer skin on the elastogram compared to healthy controls (N=15). However, being a dichotomous scale, it is not sensitive to the severity or progression of the disease. Cannoa et al. [30] developed a three-point scale to visually assess the elastogram of SI, Blue (hard) =1, green (intermediate) =2, and red (soft) =3, and assessed 4 positions on each patient in the perioral region to provide a total score out of 12 for each individual. They found patients with systemic sclerosis (N=6) were consistently stiffer with median total score of 6 (IQR, 4-6), whereas controls (N=6) had a median score of 11 (IQR, 9-11, p ≤ 0.011). This scoring system provides an increased sensitivity compared to the dichotomous scale, and aligns more closely with the 4 point mRSS in scleroderma assessment however, it still doesn’t provide a comprehensive scale to evaluate mixed color variations within the ROI. Development of standardized qualitative assessments tools akin to the breast mass Tsukuba score whereby the color and percentage of color displayed in the elastogram is categorized into a 5-stage scoring system [31,32], may be beneficial for cutaneous conditions. Although variations may be required for individual diagnostic groups, it would be beneficial to have a standard scoring guide to be used for differential diagnostic purposes in all cutaneous conditions. Both reliability and validity can then be evaluated to potentially reduce the risk of bias in future studies.
In contrast to SI, two studies evaluating SWEI demonstrated initial suggestions of both discriminate and convergent validity. Hou et al. [38] assessed 30 participants, 15 patients with systemic sclerosis and 15 healthy controls with SWEI and found statistically significant differences between healthy controls and mRSS levels 0, 1 and 2 (p<0.001) but not mRSS level 3. In addition, SWEI was able to distinguish between mRSS levels 0 and 1 (p<0.001), and between mRSS level 0 and 2 (p<0.001). Again, level 3 mRSS scores were not significantly different to mRSS levels 0, 1, and 2. Santiago et al. [19], found 26 patients with systemic sclerosis had statistically significant stiffer skin compared to the 17 healthy controls in 11 out of 16 anatomical sites tested. In addition, SWV strongly correlated to mRSS in seven of the 16 anatomical sites (left and right forearm, finger, thigh and left hand). Pearson correlation coefficients varied from r=0.525 at the left thigh, to r=0.748 at the left phalanx. Although the sample sizes were low in both studies, these results suggest SWEi has the ability to quantify the stiffness of tissues on a continuous scale correlating to the clinical assessment of severity.
Three studies presented initial evidence that UE can provide preclinical indicators (mRSS level 0) of skin stiffness in patients with systemic sclerosis, [19,38,55]. The extent of skin involvement in systemic sclerosis predicts internal organ involvement and the general outcome for patients [38]. Therefore, UE has the potential to provide both an earlier diagnosis and a more sensitive assessment of disease progression compared to the subjective clinical assessment.
UE provides unique information of tissue change not detectable by other forms of imaging. Hata et al. [52] described a soft central area in an angiomatoid fibrous histiocytoma, visualized on the elastogram which corresponded to a pseudovascular structure (a blood filled space) within the tumor, which was not detected by either color Doppler or US. In addition, neither US or Doppler detected an area of induration surrounding abscesses which was clearly identified in 98% of patients using UE [21]. Continued research by this group (which did not fit the inclusion criteria for this review), demonstrated that the shape of the induration was a predictor of therapy failure which is highly beneficial in determining the treatment options for this patient group [20]. Biological tissues may demonstrate a uniform B-mode US echogenicity, however have very different mechanical properties. The unique ability of UE to image the mechanical properties of tissues can provide unique information, not captured by other imaging modalities.
Lastly, elastography has the ability to provide select information regarding the stiffness of individual tissue layers. The stiffness of skin and subcutaneous adipose tissues were assessed independently in the two studies by Suehiro et al. [34,56], dermal stiffness was isolated in a study by Yun Lee et al. [35], and subcutaneous adipose tissue stiffness was measured independently to both the underlying muscle and the overlying skin in patients post irradiation neck fibrosis [57]. Objective assessment tools used in the past have only been able to assess bulk or gross tissue stiffness with the application of direct manual forces via either extension [62], suction [63-65], torsion [66] or vertical displacement [67,68]. However, each layer in the skin-hypodermal complex is structurally and functional different [69,70], with each independently altering the mechanical property of the tissue as a whole [51,71-73]. With an increasing interest in how the mechanical environment influences tissue healing and disease development [74,75], UE can provide precise and detailed information on select tissues in vivo, both qualitatively and quantitatively, which could assist in the non-invasive research of understanding human pathophysiology of skin disease and dysfunction.
Potential limitations of elastography
A number of potential limitations were identified in the studies included in this review. Firstly, the reliability of UE on fingers was low in two studies. Using SWEI to assess patients with systemic sclerosis, Hou et al. [38] calculated moderate ICC for the dorsal left middle finger (ICC=0.535) and low ICC for the dorsal right middle finger (ICC=0.247), whereas good to high ICC were reported for the other 15 of the 17 body sites tested (ICC=0.613-0.916). Iagnocco et al. also reported difficulty obtaining consistent measures on the fingers, however this didn’t appear to influence the stated 100% reliability. A number of reasons could explain these discrepancies: an uneven, narrow finger surface may not allow a stable, perpendicular positioning of the probe reducing the quality of image acquisition [38], the close proximity of the underlying bone may influence the ultrasound reflectance, generating an artifact [55], or the amount ultrasound gel may have been insufficient to prevent excessive pressure on the superficial tissue. Although SWEI generates its own force to measure tissue stiffness, like SI, it is still sensitive to the amount of contact compression. Increased compression correlated with increased shear wave velocity in both breast and prostate imaging [8], therefore, further research is recommended to evaluate protocol reliability of UE on uneven, superficial tissues with and without a stand-off gel.
An additional limitation of SI was demonstrated when assessing painful conditions [21]. The contact force required to acquire an image increased the experience of pain in some patients with an abscess, which may have resulted in inadequate compression to generate an optimal image. SWEI, however does not require additional compression and therefore may be more appropriate for painful or sensitive skin.
Three studies demonstrated potential limitations using UE in tissues influenced by the presence of fluids. SI was only able to visualize an encapsulated area of fluid within abscess cores 58% of the time, which was clearly visualized with B-mode US [21]. Although pain may have been a factor in some of these cases, cysts within breast tissue appear as black color voids when measured with SWEI [23]. In addition, SI was not effective at discriminating differences in tissue stiffness between legs with lymphedema (a condition with increased free fluid) and healthy controls [34,56]. Although compression techniques may deform tissues, it may not influence tissue strain, as the compression by the transducer is negated by fluid movement [56]. SWEI has however, been used in the assessment of musculoskeletal tissues affected by body fluids [76], therefore, further studies are recommended to evaluate the effects of fluids on UE.
Although SWEI provides objective quantification of tissue stiffness, the results are not yet comparable between different commercial systems. Shear wave velocity is influenced by the magnitude and application of ARF, and the calculation of velocity is influenced by the unique software associated with each UE system. In addition, protocol variations can influence results including the ROI size, shape (either round [57], or square [19]), and ROI position on the elastogram, which can be placed on either the visually stiffest region, softest region or a region of mixed stiffness. Therefore, SWEI does not provide an absolute measure of stiffness [8,23] and transparent methodology is required to determine valid and reproducible research protocols, to allow comparisons between future studies.
Finally, because cutaneous conditions are often highly visible, protocols for assessment should attempt to reduce the risk of bias. Ten of the reviewed studies only utilized a single sonographer/clinician to both acquire and analyze the image, which could result in an overestimation of assessment accuracy. Gaspari et al. [21], on the other hand, utilized one clinician to acquire the image and a second to assess the image, and Liu et al. [57] blinded the assessors to the velocity readouts on the elastogram. Blinding the image assessor is recommended for future studies to reduce the risk of the results being inadvertently influenced by the clinical presentation.
Future research
Further studies are suggested to evaluate the number of images required to provide reproducible data. Various studies reported taking between 3-5 scans in a single evaluation then using either the ‘best scan’ or an average of the measurements to include in their analysis. However, none of the studies reported the degree of variance between the repeated scans. The degree of variance during multiple evaluations within a single session is an important calculation, to determine the degree of acceptable variance using the equipment. If the degree of variation in measurement is greater than the clinically important changes, then the average value may not be appropriate to use. However, if the variation is small, only one scan may be required which would significantly reduce the time burden of image acquisition and analysis. In addition, further research is required to investigate potential artifacts that can influence image interpretation and shear wave speeds.
Limitations of the systematic review
This research was limited to published abstracts and studies conducted on cutaneous and subcutaneous-adipose conditions. However, many studies conducted do not result in publication [77], therefore additional unpublished research may have been conducted yielding results contrary to that presented in this study. In addition, research methodologies conducted in other tissue, organ or diagnostic groups could generate high quality methodologies suitable for the application in cutaneous research, which has been unexplored in this review. Furthermore, only currently available commercialized UE systems were included in the study to allow imminent application in burns research. However, technology is developing at a rapid pace, therefore studies conducted using prototypes of new elastography systems may also yield further evidence of validity and/or reliability. Finally, elastography is also available on platforms other than ultrasound, including optical imaging and magnetic resonance imaging, which may also provide other suitable options for imaging the mechanical properties of cutaneous conditions.
Conclusion
Ultrasound elastography is a non-invasive assessment tool which has the ability to visualize the mechanical differences between healthy and pathological tissues. It is novel in the assessment of cutaneous stiffness and therefore no robust evidence of validity, and only minimal evidence of reliability was found in the available published literature. However, initial studies provide preliminary evidence that both SI and SWEI were able to distinguish between diseased and non-diseased cutaneous tissues, and detect preclinical tissue stiffness in systemic scleroderma. SI was able to demonstrate structural variations in cutaneous lesions not visualized with either Doppler or conventional US. In addition, SWEI was able to discriminate various levels of stiffness severity in systemic sclerosis and post irradiation neck fibrosis, which correlated to clinical assessment. However, a potential limitation was identified with SI for the assessment of tissue stiffness in the presence of body fluids or pain. These initial indicators support the conduct of further studies in cutaneous imaging.
Disclosure of conflict of interest
Brendan Kennedy is a founder and shareholder in OncoRes Medical (Perth, Western Australia, Australia), a start-up company developing optical-based elastography for surgical applications. In addition, he performs funded research for this company.
References
- 1.Doherty JR, Trahey GE, Nightingale KR, Palmeri ML. Acoustic radiation force elasticity imaging in diagnostic ultrasound. IEEE Trans Ultrason Ferroelectr Freq Control. 2013;60:685–701. doi: 10.1109/TUFFC.2013.2617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ophir J, Cespedes I, Ponnekanti H, Yazdi Y, Li X. Elastography: a quantitative method for imaging the elasticity of biological tissues. Ultrason Imaging. 1991;13:111–134. doi: 10.1177/016173469101300201. [DOI] [PubMed] [Google Scholar]
- 3.Li GY, Cao Y. Mechanics of ultrasound elastography. Proc Math Phys Eng Sci. 2017;473:20160841. doi: 10.1098/rspa.2016.0841. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kennedy BF, Liang X, Adie SG, Gerstmann DK, Quirk BC, Boppart SA, Sampson DD. In vivo three-dimensional optical coherence elastography. Opt Express. 2011;19:6623–6634. doi: 10.1364/OE.19.006623. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Es’haghian S, Kennedy KM, Gong P, Li Q, Chin L, Wijesinghe P, Sampson DD, McLaughlin RA, Kennedy BF. In vivo volumetric quantitative micro-elastography of human skin. Biomed Opt Express. 2017;8:2458–2471. doi: 10.1364/BOE.8.002458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kennedy BF, Wijesinghe P, Sampson DD. The emergence of optical elastography in biomedicine. Nature Photonics. 2017;11:215–221. [Google Scholar]
- 7.Achterberg VF, Buscemi L, Diekmann H, Smith-Clerc J, Schwengler H, Meister JJ, Wenck H, Gallinat S, Hinz B. The nano-scale mechanical properties of the extracellular matrix regulate dermal fibroblast function. J Invest Dermatol. 2014;134:1862–1872. doi: 10.1038/jid.2014.90. [DOI] [PubMed] [Google Scholar]
- 8.Shiina T, Nightingale KR, Palmeri ML, Hall TJ, Bamber JC, Barr RG, Castera L, Choi BI, Chou YH, Cosgrove D, Dietrich CF, Ding H, Amy D, Farrokh A, Ferraioli G, Filice C, Friedrich-Rust M, Nakashima K, Schafer F, Sporea I, Suzuki S, Wilson S, Kudo M. WFUMB guidelines and recommendations for clinical use of ultrasound elastography: Part 1: basic principles and terminology. Ultrasound Med Biol. 2015;41:1126–1147. doi: 10.1016/j.ultrasmedbio.2015.03.009. [DOI] [PubMed] [Google Scholar]
- 9.Liu BJ, Li DD, Xu HX, Guo LH, Zhang YF, Xu JM, Liu C, Liu LN, Li XL, Xu XH, Qu S, Xing M. Quantitative shear wave velocity measurement on acoustic radiation force impulse elastography for differential diagnosis between benign and malignant thyroid nodules: a meta-analysis. Ultrasound Med Biol. 2015;41:3035–3043. doi: 10.1016/j.ultrasmedbio.2015.08.003. [DOI] [PubMed] [Google Scholar]
- 10.D’Onofrio M, De Robertis R, Crosara S, Poli C, Canestrini S, Demozzi E, Pozzi Mucelli R. Acoustic radiation force impulse with shear wave speed quantification of pancreatic masses: a prospective study. Pancreatology. 2016;16:106–9. doi: 10.1016/j.pan.2015.12.003. [DOI] [PubMed] [Google Scholar]
- 11.Gallotti A, D’Onofrio M, Romanini L, Cantisani V, Pozzi Mucelli R. Acoustic radiation force impulse (ARFI) ultrasound imaging of solid focal liver lesions. Eur J Radiol. 2012;81:451–455. doi: 10.1016/j.ejrad.2010.12.071. [DOI] [PubMed] [Google Scholar]
- 12.Chen L, He J, Liu G, Shao K, Zhou M, Li B, Chen X. Diagnostic performances of shearwave elastography for identification of malignant breast lesions: a meta-analysis. Jpn J Radiol. 2014;32:592–599. doi: 10.1007/s11604-014-0349-2. [DOI] [PubMed] [Google Scholar]
- 13.Liu B, Zheng Y, Shan Q, Lu Y, Lin M, Tian W, Xie X. Elastography by acoustic radiation force impulse technology for differentiation of benign and malignant breast lesions: a metaanalysis. J Med Ultrason (2001) 2016;43:47–55. doi: 10.1007/s10396-015-0658-9. [DOI] [PubMed] [Google Scholar]
- 14.Akagi R, Yamashita Y, Ueyasu Y. Age-related differences in muscle shear moduli in the lower extremity. Ultrasound Med Biol. 2015;41:2906–2912. doi: 10.1016/j.ultrasmedbio.2015.07.011. [DOI] [PubMed] [Google Scholar]
- 15.Baumer TG, Davis L, Dischler J, Siegal DS, van Holsbeeck M, Moutzouros V, Bey MJ. Shear wave elastography of the supraspinatus muscle and tendon: repeatability and preliminary findings. J Biomech. 2017;53:201–204. doi: 10.1016/j.jbiomech.2017.01.008. [DOI] [PubMed] [Google Scholar]
- 16.Brandenburg JE, Eby SF, Song P, Zhao H, Brault JS, Chen S, An KN. Ultrasound elastography: the new frontier in direct measurement of muscle stiffness. Arch Phys Med Rehabil. 2014;95:2207–2219. doi: 10.1016/j.apmr.2014.07.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Dasgeb B, Morris MA, Mehregan D, Siegel EL. Quantified ultrasound elastography in the assessment of cutaneous carcinoma. Br J Radiol. 2015;88:20150344. doi: 10.1259/bjr.20150344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Imafuku K, Hata H, Kitamura H, Iwata H, Shimizu H. Ultrasound B-mode and elastographic findings of mixed tumour of the skin on the scalp. J Eur Acad Dermatol Venereol. 2016;30:153–155. doi: 10.1111/jdv.12644. [DOI] [PubMed] [Google Scholar]
- 19.Santiago T, Alcacer-Pitarch B, Salvador MJ, Del Galdo F, Redmond AC, da Silva JA. A preliminary study using virtual touch imaging and quantification for the assessment of skin stiffness in systemic sclerosis. Clin Exp Rheumatol. 2016;34:137–141. [PubMed] [Google Scholar]
- 20.Gaspari R, Blehar D, Briones J, Dayno M. Sonoelastographic characteristics of abscess induration associated with therapy failure. J Ultrasound Med. 2012;31:1405–1411. doi: 10.7863/jum.2012.31.9.1405. [DOI] [PubMed] [Google Scholar]
- 21.Gaspari R, Blehar D, Mendoza M, Montoya A, Moon C, Polan D. Use of ultrasound elastography for skin and subcutaneous abscesses. J Ultrasound Med. 2009;28:855–860. doi: 10.7863/jum.2009.28.7.855. [DOI] [PubMed] [Google Scholar]
- 22.Bamber J, Cosgrove D, Dietrich CF, Fromageau J, Bojunga J, Calliada F, Cantisani V, Correas JM, D’Onofrio M, Drakonaki EE, Fink M, Friedrich-Rust M, Gilja OH, Havre RF, Jenssen C, Klauser AS, Ohlinger R, Saftoiu A, Schaefer F, Sporea I, Piscaglia F. EFSUMB guidelines and recommendations on the clinical use of ultrasound elastography. Part 1: basic principles and technology. Ultraschall Med. 2013;34:169–184. doi: 10.1055/s-0033-1335205. [DOI] [PubMed] [Google Scholar]
- 23.Cosgrove D, Piscaglia F, Bamber J, Bojunga J, Correas JM, Gilja OH, Klauser AS, Sporea I, Calliada F, Cantisani V, D’Onofrio M, Drakonaki EE, Fink M, Friedrich-Rust M, Fromageau J, Havre RF, Jenssen C, Ohlinger R, Săftoiu A, Schaefer F, Dietrich CF EFSUMB. EFSUMB guidelines and recommendations on the clinical use of ultrasound elastography. Part 2: clinical applications. Ultraschall Med. 2013;34:238–253. doi: 10.1055/s-0033-1335375. [DOI] [PubMed] [Google Scholar]
- 24.Rahman MM, Islam M, Nargis M, Sarker SC, Hasan MJ, Quddush AR, Sen S. Ultrasound elastography applications. Community Based Medical Journal. 2013;2:76–85. [Google Scholar]
- 25.Ferraioli G, Filice C, Castera L, Choi BI, Sporea I, Wilson SR, Cosgrove D, Dietrich CF, Amy D, Bamber JC, Barr R, Chou YH, Ding H, Farrokh A, Friedrich-Rust M, Hall TJ, Nakashima K, Nightingale KR, Palmeri ML, Schafer F, Shiina T, Suzuki S, Kudo M. WFUMB guidelines and recommendations for clinical use of ultrasound elastography: Part 3: liver. Ultrasound Med Biol. 2015;41:1161–1179. doi: 10.1016/j.ultrasmedbio.2015.03.007. [DOI] [PubMed] [Google Scholar]
- 26.Jeong WK, Lim HK, Lee HK, Jo JM, Kim Y. Principles and clinical application of ultrasound elastography for diffuse liver disease. Ultrasonography. 2014;33:149. doi: 10.14366/usg.14003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Park GY, Kwon DR. Application of real-time sonoelastography in musculoskeletal diseases related to physical medicine and rehabilitation. Am J Phys Med Rehabil. 2011;90:875–886. doi: 10.1097/PHM.0b013e31821a6f8d. [DOI] [PubMed] [Google Scholar]
- 28.Chimenti R, Flemister A, Ketz J, Bucklin M, Buckley M, Richards M. Ultrasound strain mapping of achilles tendon compressive strain patterns during dorsiflexion. J Biomech. 2016;49:39–44. doi: 10.1016/j.jbiomech.2015.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Garra BS. Tissue elasticity imaging using ultrasound. Appl Radiol. 2011;40:24. [Google Scholar]
- 30.Cannaò PM, Vinci V, Caviggioli F, Klinger M, Orlandi D, Sardanelli F, Serafini G, Sconfienza LM. Technical feasibility of real-time elastography to assess the peri-oral region in patients affected by systemic sclerosis. J Ultrasound. 2014;17:265–269. doi: 10.1007/s40477-014-0119-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Barr RG, Nakashima K, Amy D, Cosgrove D, Farrokh A, Schafer F, Bamber JC, Castera L, Choi BI, Chou YH, Dietrich CF, Ding H, Ferraioli G, Filice C, Friedrich-Rust M, Hall TJ, Nightingale KR, Palmeri ML, Shiina T, Suzuki S, Sporea I, Wilson S, Kudo M. WFUMB guidelines and recommendations for clinical use of ultrasound elastography: Part 2: breast. Ultrasound Med Biol. 2015;41:1148–1160. doi: 10.1016/j.ultrasmedbio.2015.03.008. [DOI] [PubMed] [Google Scholar]
- 32.Stachs A, Hartmann S, Stubert J, Dieterich M, Martin A, Kundt G, Reimer T, Gerber B. Differentiating between malignant and benign breast masses: factors limiting sonoelastographic strain ratio. Ultraschall Med. 2013;34:131–136. doi: 10.1055/s-0032-1313168. [DOI] [PubMed] [Google Scholar]
- 33.Botar-Jid CM, Cosgarea R, Bolboacă SD, Şenilă SC, Lenghel LM, Rogojan L, Dudea SM. Assessment of cutaneous melanoma by use of veryhigh-frequency ultrasound and real-time elastography. AJR Am J Roentgenol. 2016;206:699–704. doi: 10.2214/AJR.15.15182. [DOI] [PubMed] [Google Scholar]
- 34.Suehiro K, Morikage N, Murakami M, Yamashita O, Harada T, Ueda K, Samura M, Tanaka Y, Nakamura K, Hamano K. Skin and subcutaneous tissue strain in legs with lymphedema and lipodermatosclerosis. Ultrasound Med Biol. 2015;41:1577–1583. doi: 10.1016/j.ultrasmedbio.2015.01.019. [DOI] [PubMed] [Google Scholar]
- 35.Yun Lee S, Cardones A, Doherty J, Nightingale K, Palmeri M. Preliminary results on the feasibility of using ARFI/SWEI To assess cutaneous sclerotic diseases. Ultrasound Med Biol. 2015;41:2806–2819. doi: 10.1016/j.ultrasmedbio.2015.06.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Dillman JR, Chen S, Davenport MS, Zhao H, Urban MW, Song P, Watcharotone K, Carson PL. Superficial ultrasound shear wave speed measurements in soft and hard elasticity phantoms: repeatability and reproducibility using two ultrasound systems. Pediatr Radiol. 2015;45:376–385. doi: 10.1007/s00247-014-3150-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.McHugh AA, Fowlkes BJ, Maevsky EI, Smith DJ Jr, Rodriguez JL, Garner WL. Biomechanical alterations in normal skin and hypertrophic scar after thermal injury. J Burn Care Rehabil. 1997;18:104–108. doi: 10.1097/00004630-199703000-00003. [DOI] [PubMed] [Google Scholar]
- 38.Hou Y, Zhu QL, Liu H, Jiang YX, Wang L, Xu D, Li MT, Zeng XF, Zhang FC. A preliminary study of acoustic radiation force impulse quantification for the assessment of skin in diffuse cutaneous systemic sclerosis. J Rheumatol. 2015;42:449–455. doi: 10.3899/jrheum.140873. [DOI] [PubMed] [Google Scholar]
- 39.Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. doi: 10.1371/journal.pmed.1000100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. doi: 10.1371/journal.pmed.1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529–536. doi: 10.7326/0003-4819-155-8-201110180-00009. [DOI] [PubMed] [Google Scholar]
- 42.Lucas NP, Macaskill P, Irwig L, Bogduk N. The development of a quality appraisal tool for studies of diagnostic reliability (QAREL) J Clin Epidemiol. 2010;63:854–861. doi: 10.1016/j.jclinepi.2009.10.002. [DOI] [PubMed] [Google Scholar]
- 43. https://www.biomedcentral.com/content/supplementary/1471-2393-14-52-s2.pdf.
- 44.Di Geso L, Filippucci E, Girolimetti R, Tardella M, Gutierrez M, De Angelis R, Salaffi F, Grassi W. Reliability of ultrasound measurements of dermal thickness at digits in systemic sclerosis: role of elastosonography. Clin Exp Rheumatol. 2011;29:926–932. [PubMed] [Google Scholar]
- 45.Zheng Y, Leung S, Mak A. Assessment of neck tissue fibrosis using an ultrasound palpation system: a feasibility study. Med Biol Eng Comput. 2000;38:497–502. doi: 10.1007/BF02345743. [DOI] [PubMed] [Google Scholar]
- 46.Leung SF, Zheng Y, Choi C, Mak S, Chiu S. Quantitative measuement of post-irradiation neck fibrosis based on the young modulus: description of a new method and clinical results. Cancer. 2002;95:656–62. doi: 10.1002/cncr.10700. [DOI] [PubMed] [Google Scholar]
- 47.Lau JC, Li-Tsang CW, Zheng YP. Application of tissue ultrasound palpation system (TUPS) in objective scar evaluation. Burns. 2005;31:445–452. doi: 10.1016/j.burns.2004.07.016. [DOI] [PubMed] [Google Scholar]
- 48.Crisan D, Gheuca Solovastru L, Crisan M, Badea R. Cutaneous histiocytoma-histological and imaging correlations. A case report. Med Ultrason. 2014;16:268–270. [PubMed] [Google Scholar]
- 49.Crişan D, Badea AF, Crişan M, Rastian I, Gheuca Solovastru L, Badea R. Integrative analysis of cutaneous skin tumours using ultrasonogaphic criteria. Preliminary results. Med Ultrason. 2014;16:285–290. doi: 10.11152/mu.201.3.2066.164.dcafb. [DOI] [PubMed] [Google Scholar]
- 50.Park J, Chae IS, Kwon DR. Utility of sonoelastography in differentiating ruptured from unruptured epidermal cysts and implications for patient care. J Ultrasound Med. 2015;34:1175–1181. doi: 10.7863/ultra.34.7.1175. [DOI] [PubMed] [Google Scholar]
- 51.Osanai O, Ohtsuka M, Hotta M, Kitaharai T, Takema Y. A new method for the visualization and quantification of internal skin elasticity by ultrasound imaging. Skin Res Technol. 2011;17:270–277. doi: 10.1111/j.1600-0846.2010.00492.x. [DOI] [PubMed] [Google Scholar]
- 52.Hata H, Natsuga K, Aoyagi S, Homma E, Shimizu H. Ultrasound B-mode and elastographic findings of angiomatoid fibrous histiocytoma. Clin Exp Dermatol. 2014;39:538–539. doi: 10.1111/ced.12314. [DOI] [PubMed] [Google Scholar]
- 53.Hinz T, Wenzel J, Schmid-Wendtner MH. Real-time tissue elastography: a helpful tool in the diagnosis of cutaneous melanoma? J Am Acad Dermatol. 2011;65:424–426. doi: 10.1016/j.jaad.2010.08.009. [DOI] [PubMed] [Google Scholar]
- 54.Schmid-Wendtner MH, Hinz T, Wenzel J, Wendtner CM. Real time tissue elastography for diagnosis of cutaneous T-cell lymphoma. Leuk Lymphoma. 2011;52:713–715. doi: 10.3109/10428194.2010.548537. [DOI] [PubMed] [Google Scholar]
- 55.Iagnocco A, Kaloudi O, Perella C, Bandinelli F, Riccieri V, Vasile M, Porta F, Valesini G, Matucci-Cerinic M. Ultrasound elastography assessment of skin involvement in systemic sclerosis: lights and shadows. J Rheumatol. 2010;37:1688–1691. doi: 10.3899/jrheum.090974. [DOI] [PubMed] [Google Scholar]
- 56.Suehiro K, Nakamura K, Morikage N, Murakami M, Yamashita O, Ueda K, Samura M, Hamano K. Real-time tissue elastography assessment of skin and subcutaneous tissue strains in legs with lymphedema. J Med Ultrason (2001) 2014;41:359–364. doi: 10.1007/s10396-014-0526-z. [DOI] [PubMed] [Google Scholar]
- 57.Liu K, Bhatia K, Chu W, He L, Leung S, Ahuja A. Shear wave elastography-A new quantitative assessment of post-irradiation neck fibrosis. Ultraschall Med. 2015;36:348–354. doi: 10.1055/s-0034-1366364. [DOI] [PubMed] [Google Scholar]
- 58.Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21:651–657. doi: 10.1007/s11136-011-9960-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45:626–629. doi: 10.1016/j.jpsychires.2010.10.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Hinz T, Wenzel J, Schmid-Wendtner MH. Real-time tissue elastography: a helpful tool in the diagnosis of cutaneous melanoma? J Am Acad Dermatol. 2011;65:424–426. doi: 10.1016/j.jaad.2010.08.009. [DOI] [PubMed] [Google Scholar]
- 61.Chang JM, Moon WK, Cho N, Yi A, Koo HR, Han W, Noh DY, Moon HG, Kim SJ. Clinical application of shear wave elastography (SWE) in the diagnosis of benign and malignant breast diseases. Breast Cancer Res Treat. 2011;129:89–97. doi: 10.1007/s10549-011-1627-7. [DOI] [PubMed] [Google Scholar]
- 62.Clark JA, Cheng JC, Leung KS. Mechanical properties of normal skin and hypertrophic scars. Burns. 1996;22:443–446. doi: 10.1016/0305-4179(96)00038-1. [DOI] [PubMed] [Google Scholar]
- 63.Draaijers LJ, Botman YA, Tempelman FR, Kreis RW, Middelkoop E, van Zuijlen PP. Skin elasticity meter or subjective evaluation in scars: a reliability assessment. Burns. 2004;30:109–114. doi: 10.1016/j.burns.2003.09.003. [DOI] [PubMed] [Google Scholar]
- 64.Nedelec B, Correa JA, Rachelska G, Armour A, LaSalle L. Quantitative measurement of hypertrophic scar: intrarater reliability, sensitivity, and specificity. J Burn Care Res. 2008;29:489–500. doi: 10.1097/BCR.0b013e3181710869. [DOI] [PubMed] [Google Scholar]
- 65.Gankande TU, Duke JM, Danielsen PL, Dejong HM, Wood FM, Wallace HJ. Reliability of scar assessments performed with an integrated skin testing device-the dermaLab combo( ®) Burns. 2014;40:1521–9. doi: 10.1016/j.burns.2014.01.025. [DOI] [PubMed] [Google Scholar]
- 66.Boyce ST, Supp AP, Wickett RR, Hoath SB, Warden GD. Assessment with the dermal torque meter of skin pliability after treatment of burns with cultured skin substitutes. J Burn Care Rehabil. 2000;21:55–63. doi: 10.1097/00004630-200021010-00011. [DOI] [PubMed] [Google Scholar]
- 67.Corica G, Wigger N, Edgar D, Wood F, Carroll S. Objective measurement of scarring by multiple assessors: is the tissue tonometer a reliable option? J Burn Care Res. 2006;27:520–523. doi: 10.1097/01.BCR.0000225963.41796.54. [DOI] [PubMed] [Google Scholar]
- 68.Lye I, Edgar DW, Wood FM, Carroll S. Tissue tonometry is a simple, objective measure for pliability of burn scar: is it reliable? J Burn Care Res. 2006;27:82–85. doi: 10.1097/01.bcr.0000194531.93753.c5. [DOI] [PubMed] [Google Scholar]
- 69.Ni Annaidh A, Bruyere K, Destrade M, Gilchrist MD, Ottenio M. Characterization of the anisotropic mechanical properties of excised human skin. J Mech Behav Biomed Mater. 2012;5:139–148. doi: 10.1016/j.jmbbm.2011.08.016. [DOI] [PubMed] [Google Scholar]
- 70.Everett JS, Sommers MS. Skin viscoelasticity: physiologic mechanisms, measurement issues, and application to nursing science. Biol Res Nurs. 2013;15:338–346. doi: 10.1177/1099800411434151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Pierard GE. EEMCO guidance to the in vivo assessment of tensile functional properties of the skin. Part 1: relevance to the structures and ageing of the skin and subcutaneous tissues. Skin Pharmacol Appl Skin Physiol. 1999;12:352–362. doi: 10.1159/000029897. [DOI] [PubMed] [Google Scholar]
- 72.Pailler-Mattei C, Bec S, Zahouani H. In vivo measurements of the elastic mechanical properties of human skin by indentation tests. Med Eng Phys. 2008;30:599–606. doi: 10.1016/j.medengphy.2007.06.011. [DOI] [PubMed] [Google Scholar]
- 73.Yoshitake Y, Miyamoto N, Taniguchi K, Katayose M, Kanehisa H. The skin acts to maintain muscle shear modulus. Ultrasound Med Biol. 2016;42:674–682. doi: 10.1016/j.ultrasmedbio.2015.11.022. [DOI] [PubMed] [Google Scholar]
- 74.Gurtner GC, Dauskardt RH, Wong VW, Bhatt KA, Wu K, Vial IN, Padois K, Korman JM, Longaker MT. Improving cutaneous scar formation by controlling the mechanical environment: large animal and phase I studies. Ann Surg. 2011;254:217–225. doi: 10.1097/SLA.0b013e318220b159. [DOI] [PubMed] [Google Scholar]
- 75.Wong VW, Longaker MT, Gurtner GC. Soft tissue mechanotransduction in wound healing and fibrosis. Semin Cell Dev Biol. 2012;23:981–986. doi: 10.1016/j.semcdb.2012.09.010. [DOI] [PubMed] [Google Scholar]
- 76.Andonian P, Viallon M, Le Goff C, de Bourguignon C, Tourel C, Morel J, Giardini G, Gergelé L, Millet GP, Croisille P. Shear-wave elastography assessments of quadriceps stiffness changes prior to, during and after prolonged exercise: a longitudinal study during an extreme mountain ultra-marathon. PLoS One. 2016;11:e0161855. doi: 10.1371/journal.pone.0161855. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Drucker AM, Fleming P, Chan AW. Research techniques made simple: assessing risk of bias in systematic reviews. J Invest Dermatol. 2016;136:e109–e114. doi: 10.1016/j.jid.2016.08.021. [DOI] [PubMed] [Google Scholar]