Development and Validation of a Composite Skin Quality Scale

Abstract

Background

The increasing demand for skin quality interventions in aesthetic medicine underscores the necessity for objective, evidence-based assessment tools that may be used to evaluate novel interventions or devices.

Objectives

To develop and validate a 5-point photonumeric rating scale for assessing overall skin quality, including radiance, color evenness, and smoothness.

Methods

The IBSA (Institut Biochimique SA) Composite Skin Quality Scale was developed and underwent live validation with 88 real-world patients, chosen to encompass a broad spectrum of skin qualities and Fitzpatrick skin types. Scale validation was performed by board-certified plastic surgeons and dermatologists over 2 rounds, 2 weeks apart. Reliability was assessed through intrarater and interrater agreements, utilizing weighted kappa statistics and intraclass correlation coefficient (ICC). The scale’s ability to discern a clinically relevant 1-grade difference was evaluated with 72 photo pairs.

Results

Combined intrarater reliability results showed weighted kappa values of 0.812 (right side) and 0.815 (left side) and an ICC of 0.903 for both sides, indicating an almost perfect agreement. Interrater reliability ranged from substantial to almost perfect, with kappa coefficients between 0.654 and 0.853 and ICCs between 0.657 and 0.855 across all rater pairs in both rounds. The ability to detect a clinically relevant 1-point difference using the scale was established.

Conclusions

Integrating various key aspects of skin quality, the IBSA Composite Skin Quality Scale is a clinically relevant and highly reliable tool, suitable for skin assessment in clinical studies of new aesthetic technologies and products.

Level of Evidence: 3

The condition of the facial skin is often seen as a reflection of overall health, indicating age, vitality, the presence of disease, nutritional status, and aspects of reproductive health and fertility.^1-3 A clear complexion is commonly perceived as a marker of good health and well-being.^2,3 Because the skin is the primary interface with the external environment, its quality can substantially impact emotional health, life satisfaction, self-image, and social interactions.^3-6 Researchers have consistently shown that the state of one's skin greatly influences perceptions of health, personality, and youthfulness, with even slight variations in skin texture altering perceived attractiveness.^3,4 This is a universal phenomenon, applicable across different ethnicities and to both female and male faces.⁴

High-quality skin is characterized as healthy, youthful, and undamaged. Key characteristics include surface evenness and firmness, tone evenness, and a healthy glow.⁴ Skin surface evenness is influenced by pore size, the presence of fine lines and wrinkles, acne scarring, hair, and overall skin clarity, whereas skin firmness reflects the skin's elasticity, tightness, and hydration. Skin tone evenness encompasses both uniformity in pigmentation and the absence of redness or discoloration. The glow of the skin is described using terms such as radiance, luminosity, brightness, vibrancy, and complexion.⁴

Patient interest in skin quality is both significant and increasing, and approximately one-third of adults who present with aesthetic concerns note that skin quality is a key concern.⁷ In response to growing interest in improving skin quality, a wide variety of minimally invasive aesthetic treatments have been developed and adopted in clinical practice and are undergoing evaluation in clinical trials. These treatments include, but are not limited to, chemical peels, microneedling, and laser treatments, as well as cosmeceuticals, nutraceuticals, and minimally invasive injectables such as botulinum toxin and intradermal fillers.^3,8-17 Importantly, many of these interventions are used in combination or have established effects on measures such as skin laxity but unproven effects on skin quality. To ensure an objective evaluation of these emerging treatments across diverse demographics in clinical trials, the development of inclusive, objective, and standardized assessment methods is essential. In response to this need, the 5-point IBSA (Institut Biochimique SA) Composite Skin Quality Scale was developed and validated, demonstrating its high test–retest reliability and clinical relevance.

METHODS

The IBSA Composite Skin Quality Scale (Figure 1) is a 5-point rating tool designed for evaluating overall skin quality. Each grade includes a scale descriptor and 3 images, composed of 2 real-world patient images and a participant image that has been digitally altered to visually demonstrate each of the 5 grades. The scale was created by 2 independent experts, 1 board-certified dermatologist, and 1 board-certified plastic surgeon, collectively referred to as the “scale developers.” Following development, the scale underwent independent validation involving live participants by the “raters,” a distinct group of board-certified clinicians comprised of 2 dermatologists and 1 plastic surgeon. Prior to validation, the raters were trained by grading sample images and reaching a consensus on rating live patients using the scale. Postvalidation, the same team of raters assessed the scale's clinical relevance. All participants consented to photography, and all patient interactions were carried out according to the principles outlined in the Declaration of Helsinki.

Figure 1.

This 5-point photonumeric scale, developed by IBSA (Institut Biochimique SA), is utilized for the assessment of overall skin quality, capturing features such as radiance, texture, and color evenness. The scale includes 5 grades, each represented by a column. The first row features digitally morphed images derived from a participant's photograph to illustrate each grade. The remaining 2 rows were populated with unmorphed, actual patient images selected for each grade of skin quality severity. This scale is owned by and licensed for use by IBSA.

Scale Development

To develop scale, ∼154 consenting adult participants from both genders and Fitzpatrick skin Types I to VI (with 10%-15% being Type V/VI) participated in the image collection for potential use in the scale. Canfield Scientific, Inc. (Canfield, Parsippany, NJ) conducted the initial image collection by capturing full facial 2-dimensional images. Participants were clean-shaven without tattoos and did not wear makeup or jewelry during the image capture process. Terms and descriptors for each grade on the scale were established by the scale developers. From the pool of captured images, the scale developers selected a patient image to serve as a base model to morph to represent all 5 grades on the scale. With approval by the scale developers, Canfield then morphed the base image to match the description provided for each numeric grade.

Among the photographed participants, the Scale Developers selected 2 representatives per grade as actual real-world participant image examples for the scale. These images were selected to represent diversity in gender, age, and Fitzpatrick skin type. The final scale contains the scale descriptors, morphed images, and actual participant images (Figure 1).

Scale Validation

Trained raters conducted 2 live validation sessions for the scale, spaced 2 weeks apart. A group of 88 patients, chosen to encompass a broad spectrum of skin qualities, participated in the live validation. During the live validation, participants were clean-shaven and did not have tattoos, makeup, or jewelry. Fitzpatrick skin-type assessment was conducted by a clinical rater during the first session of validation. Photographs were taken of both sides of the patients’ faces. In each session, validators approached individual rating stations where participants were stationed and assigned an integer rating from 1 to 5 to both the right and the left sides of each patient's face using a printed copy of the photonumeric scale. Two weeks later, the same patients were evaluated in the second round, but in a randomly altered sequence.

Evaluation of Clinical Relevance

A separate evaluation of clinical relevance was performed based on patient photographs used during the live scale validation. To evaluate the effectiveness of the scale in distinguishing clinically relevant grade differences by trained raters, a collection of photographs used in the live validation was selected to represent each grade on the scale based on assigned ratings, which were determined from the majority scores given by raters during the validation event. Using these images in a subsequent session, 72 photo pairs of 2 different patients were created, categorized into groups based on their original assessments: 12 pairs deemed “not clinically different,” 24 pairs with a 1-point difference, 18 pairs with a 2-point difference, 12 pairs with a 3-point difference, and 6 pairs with a 4-point difference. The 3 raters were then tasked with reviewing all 72 pairs to identify any clinically significant differences in composite skin quality between the 2 patients in each pair. After this review session, each rater independently used the scale to assign ratings to the individual photographs from these pairs, which were presented in a random sequence.

Statistical Analysis

Statistical analyses for the study were carried out using SAS Version 9.4 (SAS Institute, Cary, NC). The test–retest reliability was assessed by quantifying intrarater and interrater reproducibility. Intrarater reliability was determined between Rounds 1 and 2 for individual raters, the median of all raters, and the combined results of all raters, by calculating the percentage of agreement (both exact and within a ≥1-grade difference), the weighted kappa statistic with a 95% CI, and the intraclass correlation coefficient (ICC) with a 95% CI. Similarly, interrater reliability was established for each rater pair and each rater against the median score of all 3 raters, utilizing the same calculations. The weighted kappa statistics and ICC were determined using established methods, in which values >0 and ≤0.2 indicate slight agreement, >0.2 and ≤0.4 indicate fair agreement, >0.4 and ≤0.6 indicate moderate agreement, >0.6 and ≤0.8 indicate substantial agreement, and >0.8 and ≤1.0 indicate almost perfect agreement.^18-20

Clinical relevance was assessed by the absolute differences in the rating scores between each pair of photographs, as determined from the scores given by the 3 independent raters. The differences were compiled for pairs initially classified as clinically different and not clinically different, and reported as descriptive statistics, including the mean, standard deviation, and 95% CI of the mean. Further examination of clinical relevance involved comparing the raters’ consensus with the original classifications of the photo pairs. A consensus was defined as at least 2 of the 3 raters agreeing on the assessment, specifically, whether the pairs were clinically different or not. Proportions reflecting the frequency of such agreement were compiled for all photo pairs, as well as separately for those deemed clinically different and those considered not clinically different.

RESULTS

Live-Patient Validation

Among the 88 patients participating in the live validation, individuals represented both genders and spanned a broad age range, averaging 41 years (Table 1). Although the majority were White, non-Hispanic, the study participants represented a diverse variety of races and ethnicities. All Fitzpatrick skin types were represented, except Type I, with Types V and VI comprising 11 participants (13%). The study also observed a wide range of heights and weights among participants.

Table 1.

Demographics for Live-Patient Scale Validation

Characteristic	N = 88
Age (years)
Mean (SD)	40.7 (13.2)
Median	38.5
Minimum, maximum	20, 63
Height (cm)
Mean (SD)	167.0 (7.9)
Median	166.4
Minimum, maximum	152, 188
Weight (kg)
Mean (SD)	69.7 (12.6)
Median	68.0
Minimum, maximum	48, 104
Sex, n (%)
Male	20 (22.7)
Female	68 (77.3)
Ethnicity, n (%)
Hispanic or Latino	6 (6.8)
Not Hispanic or Latino	82 (93.2)
Race, n (%)
American Indian or Alaska Native	1 (1.1)
Asian	4 (4.5)
Black or African American	10 (11.4)
Native Hawaiian or Other Pacific Islander	1 (1.1)
White	72 (81.8)
Fitzpatrick skin type, n (%)
Type I	0 (0)
Type II	11 (12.5)
Type III	35 (39.8)
Type IV	31 (35.2)
Type V	2 (2.3)
Type VI	9 (10.2)

SD, standard deviation.

The intrarater reliability for the scale was assessed from the agreement between the first and the second rating by the same rater, 2 weeks apart. Results for the right and the left sides of the midface in 87 patients are summarized in Table 2, including the exact match percentage between Rounds 1 and 2, reproducibility within 1 grade, weighted kappa coefficients (95% CI), and the ICC for each of the 3 raters. Both the right and the left facial ratings yielded similar outcomes. Combining data from all 3 raters indicated an almost perfect agreement, with weighted kappa values of 0.812 for the right side and 0.815 for the left side. Similarly, when combined, the ICC for all 3 raters was 0.903 for both the right and the left sides. The percentage of assigned grades within 1 grade was between 87% and 100% for each of the raters.

Table 2.

Test–Retest Reliability by Intrarater Reproducibility

	Percentage exact agreement	Percentage within 1 grade	Weighted kappa coefficient (95% CI)	ICC (95% CI)
Round 1 vs Round 2 (N = 87)
Right face
Rater 1	39.1	90.8	0.690 (0.597–0.783)	0.692 (0.465–0.816)
Rater 2	69.0	100	0.887 (0.845–0.929)	0.888 (0.834–0.925)
Rater 3	62.1	98.9	0.856 (0.801–0.910)	0.857 (0.789–0.904)
Combined	—	—	0.812 (0.771–0.853)	0.903 (0.820–0.944)
Left face
Rater 1	44.8	87.4	0.691 (0.593–0.789)	0.693 (0.456–0.820)
Rater 2	73.6	100	0.905 (0.868–0.941)	0.906 (0.859–0.937)
Rater 3	60.9	97.7	0.848 (0.792–0.903)	0.849 (0.778–0.899)
Combined	—	—	0.815 (0.773–0.856)	0.903 (0.823–0.943)

CI, confidence interval; ICC, intraclass correlation coefficient.

The scale interrater agreement between raters was analyzed for the right and the left sides of the midface in 88 patients, with each side showing similar results. Comparing each rater with the median score assigned by the raters in Round 2 demonstrated consistently high values of ≥0.9 for both the weighted kappa coefficient and the ICC (Table 3). When analyzed across all rater pairs in both rounds, the weighted kappa coefficients ranged from 0.654 to 0.853, and the ICC ranged from 0.657 to 0.855, indicating substantial to almost perfect agreement.

Table 3.

Test–Retest Reliability by Interrater Agreement With the Median Score

	Percentage exact agreement with median	Percentage within 1 grade of median	Weighted kappa coefficient (95% CI)	ICC (95% CI)
Round 2 (N = 87)
Right face
Rater 1	70.1	100	0.900 (0.864–0.936)	0.901 (0.846–0.936)
Rater 2	79.3	98.8	0.917 (0.871–0.963)	0.918 (0.877–0.945)
Rater 3	79.3	100	0.931 (0.899–0.964)	0.932 (0.898–0.955)
Left face
Rater 1	75.9	96.6	0.891 (0.833–0.948)	0.892 (0.837–0.929)
Rater 2	77.0	100	0.924 (0.890–0.959)	0.925 (0.888–0.951)
Rater 3	78.2	98.9	0.920 (0.880–0.960)	0.921 (0.881–0.948)

CI, confidence interval; ICC, intraclass correlation coefficient.

Clinical Relevance

To demonstrate the clinically relevant differences between grades on the scale when utilized by the trained raters, the absolute differences in rating scores were determined between each paired photograph deemed clinically different vs not clinically different according to the original assessments used for photo-pair selections (Table 4). The mean difference and 95% CI in scores were 1.68 (1.52–1.83) for photo pairs deemed as clinically different and 0.67 (0.49–0.85) for pairs deemed as not clinically different according to the original assessments. The 95% CI for the pairs deemed clinically different did not overlap with the CI for the pairs deemed as not clinically different, indicating that the raters were able to accurately rate photographs by utilizing the IBSA Composite Skin Quality Scale, and a 1-point difference on the scale is clinically relevant.

Table 4.

Clinical Relevance by Absolute Differences in Rater Scores for Photo Pairs

	Absolute difference in scores between paired photographs
Original assessments used for photo-pair selections	n	Mean difference in scores (SD)	Minimum, maximum	95% CI of mean
Clinically different pairs	180	1.68 (1.07)	0, 4	1.52–1.83
Not clinically different pairs	36	0.67 (0.53)	0, 2	0.49–0.85

Number of pairs evaluated = 216 = 72 photo pairs × 3 raters. n = number of pairs in each category. CI, confidence interval; SD, standard deviation.

Raters often gave the same assessments of the photo pairs when compared against the original assessments (clinically different vs not clinically different) used in photo-pair selections, indicating that clinically relevant differences on the scale can be detected by raters when evaluating random side-by-side photographs with ≥1 grade or 0 grade difference (Table 5). Approximately 80% of photo pairs were assigned the same assessments by at least 2 out of 3 raters compared with the original assessments.

Table 5.

Clinical Relevance by Proportion of Agreement of Photo-Pair Assessments

	At least 2 raters
Original assessments used for photo-pair selections	Match	No match
Overall (N = 72), n (%)	56 (77.78)	16 (22.22)
Among clinically different pairs (N = 60), n (%)	49 (81.67)	11 (18.33)
Among not clinically different pairs (N = 12), n (%)	7 (58.33)	5 (41.67)

The match column indicates that at least 2 out of 3 raters have selected the same assessment response (different or not different) compared with the original assessment responses used for photo-pair selections.

DISCUSSION

As the demand for skin quality interventions among aesthetic patients grows, the need for evidence-based grading systems becomes increasingly apparent.^3,7 These systems are essential for objectively assessing treatment effectiveness in investigational trials.

Although numerous biophysical techniques are utilized to measure different aspects of skin quality in studies, their lack of standardization and the need for costly technical equipment limit their practical use.^{8-10,12,13,15,21} Although designed for objective measurements, instrumental tools are subject to operator-dependent data acquisition and interpretation, introducing subjectivity that, along with regular calibration needs, can lead to considerable intraobserver and interobserver variabilities.^22-25 Moreover, these techniques tend to focus on specific skin characteristics, such as hydration or elasticity, rather than providing an all-encompassing assessment of skin quality. Existing validated clinical scales are designed to isolate individual skin factors or specifically measure the degree of skin aging.^26-29 Consequently, these objective tools are less effective in evaluating aesthetic treatments designed to concurrently improve multiple skin properties, such as skin resurfacing techniques, hyaluronic acid–based skin revitalizers, and intradermal botulinum toxin injections, highlighting a gap in current assessment methodologies.^15-17

In this context, the validated 5-point IBSA Scale is a notable advancement, effectively addressing the need for objective assessment tools to assess composite skin quality with proven clinical relevance. The scale has demonstrated high test–retest reliability, evidenced by substantial to almost perfect weighted kappa coefficients and ICC values, ensuring accurate evaluations across different raters and occasions. Its clinical relevance is evident by the ability of trained raters to identify significant differences between grades and perceive a 1-point difference.

The IBSA Scale stands out by integrating key aspects that define high-quality skin, such as skin radiance, roughness, pore size, and color uniformity, into a single, comprehensive evaluation tool. Its fully integrated composite design makes it particularly valuable for evaluating aesthetic treatments that target multiple skin properties. This contrasts with previously developed photograph-based skin quality scales that, despite robust validation, focus on a single characteristic such as skin roughness.²⁸ It should be noted, however, that the presented scale assumes variables impacting skin quality change together, which limits its capacity to analyze changes in multiple independent variables contributing to skin quality. Although a more recent scale does address various aspects of skin quality, it isolates 6 specific traits of aging skin by morphing a single image of a White female to satisfy each grade descriptor.²⁷ Although this scale incorporates magnified real photograph sections to complement the morphed scale examples, it lacks diversity in terms of race and ethnicity. Conversely, the IBSA Composite Skin Quality Scale includes diverse images, enhancing its applicability and reliability in real-world practice, as evidenced by its high scores during live validation with a diverse population.

The IBSA Composite Skin Quality Scale has a user-friendly design that displays 3 sets of vivid images for each severity grade, accompanied by detailed descriptions. The top-line photographs are morphed from a base image to represent each grade's descriptors, whereas the remaining part of the scale includes actual patient images, carefully selected to correspond to each specific skin-quality grade and include diversity in gender, age, and Fitzpatrick skin type. Its vertical presentation of facial images within the same grade allows for easy horizontal comparison across different grades, aiding in visualizing the progression of decreasing composite skin quality. This layout enables evaluators to discern the subtle changes in skin associated with each grade in the morphed images, while also referencing a variety of real-world patient images that exemplify each grade. By facilitating a comprehensive understanding of skin quality variations across different grades, the scale effectively assists in setting realistic treatment expectations. Importantly, Grade 5 was intended to represent a transitional stage, above which surgical intervention may be the better course of action. In discussing the findings of our study, it is important to recognize certain limitations. Although our sample was diverse, it might not fully capture the demographic variety of the population seeking skin quality treatments, potentially impacting the generalizability of our results. Future studies could enhance the scale's applicability by conducting further validations to ensure sensitivity and representation across all racial and ethnic groups. Furthermore, the lack of a direct comparison with other existing scales limits evaluation within the broader context of skin quality assessment tools. Although the scale has demonstrated high reliability, the inherent subjectivity of visual assessments could impact interpretation in clinical practice, particularly in the absence of rater training. The subtlety of the differences between each grade necessitates clinician training to ensure accurate discernment. As a result, this tool is primarily intended for research purposes rather than for routine integration into clinical practice. Additionally, although scale validators identified clinically relevant differences between grades, further research into whether patients perceive these 1-point differences as meaningful would be insightful. Future development of rating scales integrating pretreatment and posttreatment images from patients undergoing specific skin quality treatments could also enhance treatment planning and outcomes in clinical settings.

CONCLUSIONS

Developed and validated by board-certified plastic surgeons and dermatologists, the IBSA Composite Skin Quality Scale has demonstrated high test–retest reliability and clinical relevance. Its ability to facilitate accurate patient assessments makes it an invaluable tool for comprehensive skin quality assessment. The scale's user-friendly design integrates both morphed and actual patient images, representative of diverse genders and Fitzpatrick skin types, reflecting its applicability to real-world populations. Clear, vibrant photographs, accompanied by detailed text descriptions, enhance the scale's usability. The IBSA Composite Skin Quality Scale is unique in its integration of key aspects of skin quality, such as radiance, roughness, pore size, and color uniformity, into a single scale, effectively meeting the need for a comprehensive, objective, and reliable tool for skin quality assessment.

Disclosures

Dr Gold is a consultant for IBSA. Dr Lorenc is a consultant for Allergan (Irvine, CA), Galderma (Fort Worth, TX), Merz (Raleigh, NC), Suneva Medical, Inc. (San Diego, CA), and Thermi (Irving, TX) and received honorarium from Canfield Scientific (Parsippany, NJ) for scale development. Dr Bass is an investigator for Cynosure (Westford, MA) and Merz and a consultant for Allergan, Canfield, Cynosure, Endo Pharmaceuticals (Malvern, PA), and Galderma. Dr Bank has received compensation/honorarium as a consultant and principal investigator for clinical trials/projects/scale validation with Allergan, Galderma, Merz, Croma Pharma (Warszawa, Poland), Endo Pharmaceuticals, Evolus (Newport Beach, CA), and Canfield Scientific. Dr Downie is an investigator for Alphaeon Credit (Irvine, CA), BioPharmx (San Jose, CA), Croma Pharma, Endo-Therapeutics, Inc., Evidera PPD (Bethesda, MD), laVív (Fibrocell Science, Exton, PA), Neothetics, Inc. (San Diego, CA), Ranbaxy Laboratories (Mumbai, India), Revance Therapeutics Inc. (Nashville, TN), and Caliway Biopharmaceuticals (New Taipei City, Taiwan); a consultant for Intendis (Bayer, Leverkusen, North Rhine-Westphalia, Germany), Procter & Gamble (Cincinnati, OH), Restorsea (Houston, TX), Theraplex LLC (Memphis, TN), and Valeant Pharmaceuticals (Laval, Canada); a speaker for Co2Lift (Weston, FL), Cutera (Brisbane, CA), Exeltis (Florham Park, NJ), Isdin (Barcelona, Spain), Solta Medical (Bridgewater, NJ), and Stratpharma Switzerland (Basel, Switzerland); an advisory board member for IntraDerm Beauty and Aesthetics (Cape Town, South Africa), Sensus Healthcare (Boca Raton, FL), and Candesant Biomedical (Nashville, TN); a consultant and investigator for Merz, Neauvia (Plan-les-Ouates, Switzerland), and Raziel Therapeutics Ltd (Rehovot, Israel); a consultant and speaker for Aclaris Therapeutics (Wayne, PA), BTL Aesthetics (Newcastle, UK), Galderma, Lifes2good (Galway, Ireland), Nutrafol (New York, NY), Obagi Medicine (Long Beach, CA), Perigee Medical (Park City, Utah), Pfizer (New York, NY), Scientis (Geneva, Switzerland), Sofwave (Yokne'am Illit, Israel), ThermiRF (Austin, TX), and Vi Aesthetics (Los Angeles, CA); a consultant, investigator, and speaker for Allergan/AbbVie, Johnson & Johnson (New Brunswick, NJ) and SkinMedica (Allergan); an advisory board member and speaker for Senté (Carlsbad, CA); a shareholder and speaker for RegimenMD (Houston, TX); and a shareholder in Medimetriks Pharmaceuticals, Inc. (Fairfield, NJ). Drs Bellia, Giori, and Grimolizzi are employees of IBSA Farmaceutici Italia Srl.

Funding

This research was funded by IBSA. The listed IBSA authors participated in the development of this manuscript and reviewed and approved the manuscript prior to journal submission.