Reliability and Construct Validity of a Questionnaire for Assessment of Patient‐Reported Outcomes in Implant Dentistry

ABSTRACT

Objectives

The study aimed to evaluate the reliability and construct validity of a questionnaire assessing patient‐reported outcomes following dental implant rehabilitation.

Material and Methods

A questionnaire was mailed to patients 8 years posttreatment to assess experiences with dental implant therapy, including satisfaction, oral function, perceived pretreatment information and complications. Patients completed the questionnaire again during a clinical examination. Internal consistency was evaluated using Cronbach's alpha and corrected item‐total correlations (CITC), while test–retest reliability was assessed with intraclass correlation coefficients (ICC). Construct‐related validity was examined through associations between questionnaire domains and oral health related quality of life (OHRQoL) measured by OHIP‐14.

Results

A total of 206 patients completed the questionnaire twice. For individual items, ICCs ranged from 0.39 to 0.75 and CITCs from 0.39 to 0.66. The full questionnaire had an alpha of 0.78 and an ICC of 0.78. Construct‐related validity was supported through associations between the questionnaire domains patient satisfaction, pretreatment information and the OHIP‐14 sum scores.

Conclusions

The questionnaire demonstrated acceptable internal consistency reliability, good test–retest performance, and reasonable construct validity.

1. Introduction

Tooth loss has been shown to lower quality of life, including the ability to eat, speak, and participate in social activities (Gerritsen et al. 2010). Caries is the most prevalent cause of tooth loss in individuals up to the mid‐forties, whereas periodontal disease has been reported to be the primary cause of tooth loss thereafter (Chambrone and Chambrone 2006; Hull et al. 1997; Montandon et al. 2012). The choice of retaining or removing teeth often hinges on tooth prognosis. According to the clinical guideline made by the European Federation of Periodontology (EFP) (Herrera et al. 2022), implant rehabilitation may be considered when tooth preservation is no longer possible.

The overall prevalence of adults with dental implants in Sweden has been reported to be 4.7%, with increasing prevalence with age (Berglundh et al. 2024). It is reasonable to assume a similar prevalence in Norway, given the comparable demographics and healthcare systems of the neighboring countries. To effectively plan and evaluate treatment options for patients, it is essential to gain a deeper understanding of the consequences of tooth loss and implant rehabilitation relative to patients' perceptions (Ng and Leung 2006).

Patient‐Reported Outcomes (PROs) are important when assessing effectiveness and quality in health care (Calvert et al. 2013, 2018), and there has been a growing interest in patient‐reported outcome measures (PROMs) in implant dentistry over the last decades (De Bruyn et al. 2015; Feine et al. 2018; Yao et al. 2018). Oral health‐related quality of life (OHRQoL) is a construct that reflects an integral part of general health and well‐being related to oral health (John 2004). OHRQoL aligns with the World Health Organization's definition of general health as a “state of complete physical, mental and social well‐being, and not merely the absence of disease” (He et al. 2017). The Oral Health Impact Profile‐14 (OHIP‐14) is a tool designed to measure OHRQoL (Slade 1997), which has been widely recommended for its reliability, validity, responsiveness, and cross‐cultural consistency (Mounssif et al. 2023). OHIP‐14 is a shortened version of the Oral Health Impact Profile‐49 (OHIP‐49), an instrument designed to assess individuals' perceptions on how oral health conditions affect their well‐being (Slade and Spencer 1994).

A prospective cohort study by Feine et al. (2018) evaluated the impact of implant‐based rehabilitation on OHRQoL in patients with congenitally missing teeth who were rehabilitated with dental implants, using the OHIP‐49 as the measurement tool. The study reported significant improvements in OHRQoL, alongside enhanced satisfaction with dental appearance, masticatory function, and speech. Another prospective study assessed OHRQoL using OHIP‐14 before and after periimplantitis surgery. The study showed consistently low OHIP‐14 scores both before and up to 3 years following surgery, suggesting that surgical treatment of periimplantitis had limited influence on OHRQoL (Rustand et al. 2022). Instruments designed to assess OHRQoL, such as OHIP‐14, may, however, present limitations when the aim is to assess specific oral challenges, such as experiences with dental implants. As an example, it is likely that a patient who had to replace a single molar with an implant, but otherwise presented a healthy full dentition, would be unsatisfied if this implant had to be removed shortly after loading due to periimplantitis or fracture. At the same time, it may be unlikely that such an event/disease may lead to OHRQoL impairment as considered in the OHIP‐14.

Therefore, there is a need for more specific instruments, as there is no accepted standardized measure specifically assessing PROs in patients with dental implants. The lack of standardization complicates the comparison of results across different studies. Furthermore, no valid, theory‐based tool exists to measure patient satisfaction with the outcomes of dental implant treatment (Mauland et al. 2024; Nair et al. 2018). The ongoing lack of a clear, standardized definition of patient satisfaction, which is often assumed to be “common sense”, is a major issue (Yao et al. 2018). While some studies ask about overall satisfaction, others focus on specific aspects such as chewing or speaking. These different approaches can lead to significantly different results. Broad, general questions may elicit overly positive responses, while more targeted questions encourage more thoughtful and accurate feedback (Feine et al. 2018). Hawthorne (2006) has described patient satisfaction as a “multidimensional concept; not yet tightly defined; and part of an apparently yet to be determined complex model”. To complicate things further, the concept is both subjective and related to numerous factors, such as expectations related to treatment and treatment outcomes—and experiences with treatment procedures. Also, sociodemographic variables and time of measurement may influence patient satisfaction (Yao et al. 2018).

These complexities are reflected in the research literature, where a lack of references to a coherent theoretical base is apparent. To improve the quality and reproducibility of PRO assessments following dental implant therapy, there is a need for valid and reliable instruments.

The aim of this study was to evaluate the reliability and construct validity of a questionnaire designed specifically to assess patient‐reported outcomes following implant treatment.

2. Materials and Method

The study protocol was approved by the regional ethics committee (REK 137258/2020) and Norwegian Centre for Research Data. All participants in the study provided informed consent. The reporting of these results follows the STROBE guidelines for observational studies.

2.1. Participants and Procedures

The recruitment of patients for this study has been described in detail in previous studies (Mauland, Beheshti Maal, et al. 2025; Mauland, Sorensen, et al. 2025; Mauland et al. 2024). In short, records of patients who underwent dental implant rehabilitation in 2014 were identified in the Norwegian National Insurance Scheme (NNIS) registry.

NNIS subsidizes dental implant rehabilitation for patients that meet specific dental or health‐related conditions/diagnoses (Mauland et al. 2024). The implant placement must be performed by a dental specialist in oral surgery or periodontology, and the prosthetic treatment by a prosthodontist or a general practitioner who has completed a designated course.

In the year 2014, 3324 patients received subsidized implant rehabilitation in 11 counties in Norway. These 11 counties accounted for 72.1% of the total Norwegian population in 2014. After excluding deceased individuals, those who had moved abroad, and those with unknown addresses, 2980 patients were identified as potential respondents in a survey assessing PROs 8 years after implant treatment. They received a questionnaire and a consent form in November 2021. Among these, 1299 returned the questionnaire, and 940 provided consent to be contacted for an examination. Between March and December 2022, a total of 242 patients underwent a full radiographic and clinical examination (Mauland, Beheshti Maal, et al. 2025; Mauland, Sorensen, et al. 2025). When the patients attended the clinical examination, they were asked to complete the same questionnaire sent to them by mail 1–13 months prior to the examination.

2.2. Questionnaire

The questionnaire comprised 46 items (Mauland et al. 2024) (Table S1). The first part included seventeen items assessing PROs on satisfaction, oral function and pretreatment information. These items, hereafter referred to as Patient‐Reported Outcomes in Dental Implant Therapy‐17 (PRO‐DIT‐17), were developed based on previous studies related to general dental and implant treatment (Adler et al. 2016; Derks et al. 2015; Mauland, Beheshti Maal, et al. 2025; Pjetursson et al. 2005; Simonis et al. 2010). Each item had five Likert‐scale response options: “fully disagree” (1), “partly disagree” (2), “neither agree nor disagree” (3), “partly agree” (4) and “fully agree” (5). Factor analysis on this battery of items revealed three underlying subscales: Patient Satisfaction (8 items), Oral Function (5 items) and Pre‐treatment Information (2 items) (Mauland et al. 2024).

An average sum score was calculated based on all PRO‐DIT‐17 items and for items included in the three subscales: patient satisfaction, oral function and pretreatment information.

Three items assessed patients' experiences with dental implant complications, using the response options “never”, “yes, once”, “yes, multiple times,” and “I don't know.”

Two items assessed smoking behavior before and after implant therapy using the response categories “nonsmoker,” “sometimes,” “< 10 cigarettes/day,” and “≥ 10 cigarettes/day”. Eight items collected background and demographic information such as the number of implants placed (“1”, “2–3”, “4–5”, “≥ 6 implants”, or “no implants any longer”), country in which treatment was performed, treatment cost (“≤ 2000€”, “2000€‐7000€”, “≥ 7000€”), sex (“male”, “female”), age, level of education (“none”, “primary school”, “high school”, “≤ 4 years of higher education”, “> 4 years of higher education”), and nationality.

The questionnaire underwent a two‐step pretesting process before it was distributed to the study sample (Mauland et al. 2024). First, it was reviewed by a panel of experts in periodontology, implantology, and survey methodology to ensure content validity and clarity. Following this, it was pilot tested in 10 patients with dental implants. Revisions were made based on feedback from both the expert review and the patient piloting.

2.3. Statistical Analyses

Descriptive statistics included mean and standard deviation for continuous variables and relative frequencies for categorical variables. Assessment of internal consistency reliability of the PRO‐DIT‐17, its three subscales and OHIP‐14 included calculation of Corrected Item‐Total Correlations (CITC) and Cronbach's alphas. CITC values greater than 0.30 were interpreted as satisfactory, while those below 0.20 were considered unreliable (Bernstein 2010). Cronbach's alpha values above 0.70 were considered acceptable, and values above 0.80 were regarded as good indicators of reliability (Field 2024).

Test–retest reliability was evaluated using the intraclass correlation coefficient (ICC) for each PRO‐DIT‐17 subscale, each individual item and the total score for both PRO‐DIT‐17 and OHIP‐14. ICC values were interpreted as follows: ≥ 0.81 as excellent, 0.61–0.80 as good, 0.41–0.60 as moderate, and ≤ 0.40 as poor (Landis and Koch 1977). Following Koo and Li (2016), we used a two‐way mixed‐effects ICC analysis with absolute agreement, presenting single‐measure ICCs for individual items and average‐measure ICCs for subscales and total scores. This specification was chosen because the measurement occasions (test and retest) were fixed, exact agreement between measurements was required, and the reliability of the mean of the two administrations was of interest.

Cohen's weighted kappa was used to assess test–retest reliability of the self‐reported complication variables. Weighted kappa applied quadratic weights to account for the ordinal nature of the response options. “Do not know” responses and missing values were excluded pairwise from the analysis. Interpretation of Kappa values followed Landis and Koch's guidelines, ranging from slight to almost perfect agreement (Cohen 1960; Landis and Koch 1977). Weighted kappa values were interpreted as follows: < 0.00 poor, 0.00–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial and 0.81–1.00 as almost perfect agreement.

As described in the introduction, OHIP‐14 has been used for assessing impacts of implant‐based rehabilitation on patients' OHRQoL. In the present study, OHIP‐14 sum score was hypothesized to be associated with sum scores of the three PRO‐DIT‐17 subscales (satisfaction, pretreatment information and oral function). Associations were examined with Pearson correlation analyses as a test of construct validity. Crosstabulations with Chi‐square tests were used to analyze categorical variables in the complication items. Missing responses were handled using pairwise deletion, and the number of missing responses was reported for both test and retest measurements, as well as the total number of missing cases per item.

All analyses were performed at the patient level. A significant level of p < 0.05 was applied. Statistical computations were conducted using SPSS software (version 29.0; IBM SPSS Statistics, Armonk, NY, USA).

3. Results

Some questionnaire replies were missing from the test (n = 31) or retest (n = 5) of those who underwent a clinical examination (n = 242), resulting in a total sample of 206 subjects who completed the questionnaire twice. The exact date of the first questionnaire completion was not known, but all replies were obtained by March 1st, 2022, and all clinical examinations were completed by the end of 2022. Therefore, the time range between completing the questionnaire the first time (test) and second time (retest) was between 1 and 13 months for all participants.

3.1. Sample Characteristics

The study included 206 participants with a mean age of 62.4 years (SD 15.3) (Table 1). Forty‐five percent were female and 55% had completed higher education. Regarding smoking habits following implant treatment, the majority reported they had never smoked. The 206 patients carried 587 dental implants. The most common implant counts were 1 implant (31.6%), 2 implants (24.8%), and 3 implants (13.1%), and the remaining participants presented ≥ 4 implants or no implant(s). Most patients (59.2%) carried single implant retained crown(s), 30.1% carried partial arch implant supported bridge(s), and the remaining patients (10.7%) missed all teeth in one or both jaws. Out of the seven edentulous patients, all were rehabilitated with fixed implant supported prostheses. In addition, three of these patients had implant supported overdentures.

TABLE 1.

Characteristics of the study sample.

	Total population n = 206
Age
Mean ± SD	62.4 ± 15.3 years
Median	66
Range	26–91
Sex (female, %)	45%
Nationality Norwegian (%)	92.7%
Higher education (university/college, %)	55%
Number of teeth ± SD
Smoking after implant treatment (%)	20.1 (±7.5)
Never	82.5%
Occasionally	6.8%
< 10	6.3%
≥ 10	4.4%
Self‐funded cost of implant treatment (%)
< €2000	27.2%
€2000–7000	56.3%
> €7000	16.5%
Number of implants (%)
0 remaining implants	0.5%
1 implant	31.6%
2 implants	24.8%
3 implants	13.1%
4 implants	12.1%
5 implants	3.4%
6 implants	6.8%
7 implants	2.4%
8 implants	1.0%
9 implants	1.0%
10 implants	1.5%
11 implants	1.9%
Distribution of implants (n = 587)
Anterior (1–3) / Posterior (4–7)	209 (35.6%)/378 (64.4%)
Maxilla/Mandibula	435 (74.1%) / 152 (25.9%)
Type of implant‐supported prosthesis	Frequency (%)
Single crown (s)	122 (59.2%)
Bridge, partial arch	62 (30.1%)
Bridge, full arch maxilla	14 (6.8%)
Bridge, full arch mandible	1 (0.5%)
Edentulous	7 (3.4%)

3.2. Scale Measurement Properties of Patient‐Reported Outcomes in Dental Implant Treatment‐17 (PRO‐DIT‐17)

Internal consistency reliability measured by Cronbach's alpha was 0.73 for Patient Satisfaction, 0.79 for Oral Function, and 0.73 for Pre‐treatment Information (Table 2). Cronbach's alpha for the total PRO‐DIT‐17 scale was 0.78. CITCs for individual items ranged from 0.39 to 0.66.

TABLE 2.

Scale measurement properties of Patient‐Reported Outcomes in Dental Implant Treatment‐17 (PRO‐DIT‐17).

Item	Full sample n = 206				Test	ICC (95% CI)	Retest	Total missing
	Mean (test)	SD	CITC	Cronbach's alpha	Missing		Missing
Patient satisfaction	4.48	(0.77)		0.73		0.73 (0.64–0.79)
PRO‐DIT‐1	4.56	(0.88)	0.59		1	0.60 (0.50–0.68)	2	3
PRO‐DIT‐2	4.66	(0.76)	0.39		1	0.39 (0.27–0.50)	2	3
PRO‐DIT‐3	4.55	(0.80)	0.55		1	0.55 (0.44–0.64)	2	3
PRO‐DIT‐13	4.64	(0.80)	0.54		2	0.54 (0.43–0.63)	4	6
PRO‐DIT‐14	4.56	(0.85)	0.49		1	0.49 (0.37–0.59)	3	4
PRO‐DIT‐15	4.08	(1.16)	0.49		1	0.48 (0.37–0.58)	3	4
PRO‐DIT‐16	4.48	(0.93)	0.51		1	0.52 (0.41–0.61)	3	4
PRO‐DIT‐17	4.47	(0.97)	0.55		1	0.55 (0.44–0.64)	3	4
Oral function	3.46	(0.99)		0.79		0.79 (0.72–0.84)
PRO‐DIT‐4	3.73	(0.80)	0.66		3	0.66 (0.57–0.73)	5	7
PRO‐DIT‐5	3.97	(1.06)	0.56		1	0.56 (0.46–0.65)	2	3
PRO‐DIT‐6	3.06	(1.20)	0.46		3	0.46 (0.34–0.56)	6	9
PRO‐DIT‐7	3.60	(1.30)	0.64		3	0.64 (0.54–0.71)	5	8
PRO‐DIT‐8	3.11	(1.24)	0.61		3	0.61 (0.51–0.69)	5	8
Pretreatment information	3.68	(1.10)		0.73		0.73 (0.64–0.79)
PRO‐DIT‐11	3.48	(1.30)	0.45		3	0.62 (0.50–0.71)	4	7
PRO‐DIT‐12	3.91	(1.32)	0.61		3	0.75 (0.68–0.81)	3	6
Remaining items
PRO‐DIT‐9	4.20	(0.90)	0.51		3	0.43 (0.31–0.54)	6	9
PRO‐DIT‐10	1.75	(1.25)	−0.39		1	0.51 (0.40–0.60)	3	4
TOTAL PRO‐DIT‐17	3.91	(0.60)	a	0.78	10	0.78 (0.70–0.83)	13	22

^a Corrected Item–Total Correlation (CITC) is not applicable for the total score; value shown for consistency only.

Test–retest reliability, assessed by ICCs was 0.73 (0.64–0.79) for Patient Satisfaction, ICC = 0.79 (0.72–0.84) for Oral Function, and ICC = 0.73 (0.64–0.79) for Pre‐treatment Information. The total PRO‐DIT‐17 yielded an ICC of 0.78 (0.70–0.83), with each individual item ranging from 0.39 to 0.75. The highest ICCs were observed for PRO‐DIT‐12 (ICC = 0.75) and the lowest for PRO‐DIT‐2 (ICC = 0.39).

Previous factor analysis of the PRO‐DIT‐17 items revealed factor loadings below 0.50 for two items (PRO‐DIT‐9, cleansability and PRO‐DIT‐10, side effects) (Mauland et al. 2024). PRO‐DIT‐9 showed an acceptable CITC (0.51), whereas PRO‐DIT‐10 demonstrated poor fit with a negative CITC (−0.39).

Mean item scores ranged from 3.06 to 4.66, with the highest ratings observed for patient satisfaction items and lower scores for items within oral function, reflecting greater variability in functional outcomes.

A post hoc analysis was done to assess whether the ICC varied according to the extent of implant rehabilitation and the number of teeth. First, the sample was divided into three groups: (1) only single crown(s) (n = 122), (2) partial‐arch fixed bridge(s) (n = 62), and (3) full‐arch fixed bridge(s) also including edentulous patients (n = 22) (Table 1). No significantly different ICC was however observed across groups for the total PRO‐DIT‐17, the three subscales satisfaction, oral function, and pretreatment information, or the OHIP‐14 sum score. Secondly, the sample was stratified according to number of teeth: (1) ≥ 20 teeth (n = 125) or (2) < 20 teeth (n = 81). A significantly higher ICC was observed for those with < 20 teeth (0.91 (0.86–0.95)) as compared to those with ≥ 20 teeth (0.67 (0.52–0.77)) for the total PRO‐DIT‐17, but no difference was observed for the subscales nor the OHIP‐14 sum score.

3.3. Scale Measurement Properties of OHIP‐14

Internal consistency reliability measured by Cronbach's alpha was 0.89. CITCs for individual items ranged from 0.20 to 0.74 (Table 3).

TABLE 3.

Scale measurement properties of OHIP‐14.

Item	Full sample (test) n = 206					Retest	Missing	Total missing
	Mean	SD	CITC	Cronbach's alpha	Missing	ICC (95% CI)
OHIP‐1	0.42	(0.76)	0.50		1	0.50 (0.39–0.60)	1	2
OHIP‐2	0.24	(0.56)	0.20		0	0.20 (0.06–0.33)	3	3
OHIP‐3	0.09	(0.37)	0.68		0	0.67 (0.58–0.74)	1	1
OHIP‐4	0.14	(0.51)	0.53		1	0.53 (0.42–0.62)	0	1
OHIP‐5	0.28	(0.74)	0.74		0	0.74 (0.67–0.79)	0	0
OHIP‐6	0.28	(0.68)	0.53		0	0.53 (0.42–0.62)	0	0
OHIP‐7	0.30	(0.76)	0.42		0	0.41 (0.29–0.52)	3	3
OHIP‐8	0.28	(0.68)	0.32		0	0.31 (0.18–0.43)	3	3
OHIP‐9	0.23	(0.55)	0.40		0	0.40 (0.28–0.51)	1	1
OHIP‐10	0.27	(0.60)	0.28		1	0.28 (0.15–0.40)	3	4
OHIP‐11	0.67	(0.88)	0.67		0	0.67 (0.59–0.74)	1	1
OHIP‐12	0.15	(0.46)	0.54		0	0.54 (0.43–0.63)	1	1
OHIP‐13	0.17	(0.47)	0.62		0	0.60 (0.51–0.68)	2	2
OHIP‐14	0.09	(0.32)	0.70		0	0.68 (0.60–0.75)	1	1
Total OHIP	0.75	(0.39)	a	0.89	1	0.84 (0.79–0.88)	1	2

^a Corrected Item–Total Correlation (CITC) is not applicable for the total score; value shown for consistency only.

Test–retest reliability assessed by ICC for the OHIP‐14 sum score was 0.84 (0.79–0.88), whereas ICCs for individual items ranged from 0.20 to 0.68, with most items demonstrating moderate to substantial agreement.

The highest mean scores were observed for OHIP‐11 (pain or discomfort from mouth and teeth) and OHIP‐1 (discomfort from eating food), while OHIP‐3 (had to interrupt meals) and OHIP‐14 (unable to function in daily life) rendered the lowest means.

3.4. Self‐Reported Complications

Weighted Cohen's Kappa values indicated substantial to almost perfect test–retest agreement for the self‐reported complications items. Agreement was 0.80 for general complications (C‐1, SE = 0.04, 95% CI: 0.72–0.89), 0.76 for technical complications (C‐2, SE = 0.06, 95% CI: 0.65–0.87), and 0.83 for biological complications (C‐3, SE = 0.04, 95% CI: 0.75–0.92) (Table 4).

TABLE 4.

Test–retest reliability of self‐reported complications.

Item	Test					Retest
% n = 206	Never	Yes, once	Yes, several times	Do not know	Missing response	Never	Yes, once	Yes, several times	Do not know	Missing response	Weighted kappa	Standard error (SE)	Significance	Total missing
C‐1 Have you ever had any complications with implant treatment?	71.8 (n = 148)	14.6 (n = 30)	10.2 (n = 21)	2.6 (n = 7)	n = 7	68.9 (n = 142)	20.4 (n = 42)	8.7 (n = 18)	1.9 (n = 4)	n = 4	0.80 (0.72–0.89)	0.04	< 0.001	n = 9
C‐2 Have you experienced loosening or fracture of the implant‐supported prosthesis?	74.8 (n = 154)	17.0 (n = 35)	7.3 (n = 15)	0.8 (n = 2)	n = 2	69.9 (n = 144)	20.4 (n = 42)	7.3 (n = 15)	1.9 (n = 4)	n = 5	0.76 (0.65–0.87)	0.06	< 0.001	n = 6
C‐3 Have you experienced signs of inflammation in the soft tissues surrounding the implant(s)?	62.6 (n = 129)	13.6 (n = 28)	14.1 (n = 29)	9.7 (n = 20)	n = 20	59.7 (n = 123)	11.2 (n = 23)	13.6 (n = 28)	15.5 (n = 32)	n = 32	0.83 (0.75–0.92)	0.04	< 0.001	n = 41

Regarding response distributions, the majority selected «never» for C‐1/general (71.8% at test, 68.9% at retest) and C‐2/technical (74.8% at test, 70.2% at retest). C‐3/biological showed more variation, with “Do not know” response increasing from 9.7% to 15.5% from test to retest, respectively.

Table 5 presents the test–retest reliability across the three complication‐related items. All items showed statistically significant differences in response distribution over time (p < 0.001). Item C‐1 (general) had the lowest response stability (58% identical answers) and the highest proportion of low expected cell counts.

TABLE 5.

Chi‐square test assessing test–retest reliability of complication‐related items.

Item	$x^2$	df	p	% identical responses	% cells < 5
C‐1	207.2	9	< 0.001	58%	62.5%
C‐2	227.5	9	< 0.001	72%	56.3%
C‐3	191.1	9	< 0.001	67%	59.4%

Note: Chi‐square (x ²) measures how much the observed values differ from the expected values. Degree of freedom (df) represents the number of independent comparisons being made. % identical responses represent the percentage of participants who gave the exact same answer at both time points. % cells with expected frequency < 5 shows how many cells in the crosstab have too few expected observations.

Table 6 presents the associations between general complications (C‐1) and specific complications C‐2 (technical) and C‐3 (biological) at the initial test. Both comparisons showed statistically significant associations. C‐1 vs. C‐2: $x^2$ = 144.24 p < 0.001 and C‐1 vs. C‐3: $x^2$ = 41.42 p < 0.001.

TABLE 6.

Chi‐square test results: Associations between general complications (C‐1), technical (C‐2) and biological (C‐3) complications.

Comparison	$x^2$	df	p	% of cells with expected count < 5
C‐1 vs. C‐2	144.24	9	< 0.001	56.3%
C‐1 vs. C‐3	41.42	9	< 0.001	62.5%

3.5. Construct Validity

Table 7 presents the relationship between the OHIP‐14 total score and the PRO‐DIT‐17 subscales. Pearson correlations showed statistically significant negative associations between the OHIP‐14 total score and the PRO‐DIT‐17 subscales patient satisfaction (r = −0.55, p < 0.001) and pretreatment information (r = −0.23, p < 0.001). No statistically significant association was found for the subscale oral function (r = −0.10, p = 0.154).

TABLE 7.

Associations between PRO‐DIT‐17 subscales and OHIP‐14 total score.

PRO‐DIT‐17 subscales	Pearson correlation (r)	95% CI	Significance	N
Patient satisfaction	−0.55**	−0.70 to −0.34	< 0.001	206
Oral function	−0.10	−0.24 to 0.06	0.154	206
Pretreatment information	−0.23**	−0.39 to −0.03	< 0.001	206

Abbreviation: OHIP‐14, Oral Health Impact Profile (14 items).

**p < 0.01 = statistically significant.

4. Discussion

This study aimed to evaluate the reliability and construct validity of a questionnaire designed to assess patient‐reported outcomes following dental implant rehabilitation. The instrument showed generally good internal consistency and test–retest reliability and reasonable construct validity. These findings suggest that PRO‐DIT‐17 is a promising, stable, and credible tool for measuring implant‐specific outcomes.

Reliability is a fundamental property of any PROM, commonly assessed through internal consistency and test–retest reliability. It ensures that observed changes represent true effects of an intervention rather than measurement error or instrument instability (Terwee et al. 2007). CITC values ranged from 0.39 to 0.66, all exceeding the accepted minimum threshold of 0.30. This implies that each item demonstrated a meaningful correlation with the overall scale and contributed positively to the internal consistency reliability. Cronbach's alpha values for each subscale were between 0.73 to 0.79 and the total PRO‐DIT‐17 Cronbach's alpha was 0.78, indicating an acceptable level of internal consistency among the items (Terwee et al. 2007). A high Cronbach's alpha is often observed in scales with many items, such as for the total PRO‐DIT‐17, since the coefficient is influenced by the total number of items count. In contrast, subscales with fewer items typically result in lower alpha values (Terwee et al. 2007).

The findings further indicate that the instrument provides consistent responses over time. At the item level, ICCs ranged from 0.39 to 0.75, reflecting moderate to substantial agreement (Landis and Koch 1977). This range is consistent with expectations for PROs, where individual perceptions may vary between assessments, particularly in domains that are sensitive to changes in health status or treatment stage. The stability of most items supports the use of PRO‐DIT‐17 in both clinical and research settings.

The full PRO‐DIT‐17 demonstrated good test–retest reliability, with an ICC of 0.78, which was slightly lower but still comparable to the ICC of OHIP‐14 (0.84). ICCs for the PRO‐DIT‐17 subscales ranged from 0.73 to 0.79, indicating consistently high test–retest reliability across the domains of patient satisfaction, oral function, and pretreatment information. In contrast, while OHIP‐14 also showed strong overall test–retest reliability, individual item ICCs varied more widely, ranging from 0.20 to 0.68, with some items falling below acceptable thresholds (OHIP‐2: 0.20; OHIP‐10: 0.28; OHIP‐8: 0.31). The narrower and higher ICC range at both the item and subscale levels suggests that PRO‐DIT‐17 yields more stable responses over time, supporting its reliability in clinical and research settings where patient perceptions may fluctuate. These findings underscore its suitability as a reliable alternative or complement to OHIP‐14, particularly in the present context, which requires sensitive, domain‐specific tracking of PROs.

The higher test–retest reliability observed for biological complication items (C‐3) compared to technical complications items (C‐2) may seem unexpected, given the often less noticeable and asymptomatic nature of biological complications. Technical issues tend to be more apparent to patients, whereas biological symptoms are often subtle, leading to more uncertainty and higher rates of “do not know” responses. There was also a substantially higher proportion of missing responses for biological complications (C‐3), which is consistent with greater patient uncertainty associated with these items. It has been documented that periimplantitis is often present without visible signs or symptoms tangible to the patient, which may influence the precision of self‐reporting biological complications (Romandini et al. 2021).

Although C‐1 (general complications) correlated with both C‐2 (mechanical complications) and C‐3 (biological complications), its stronger association with C‐2 suggests that patients may primarily link “complications” with mechanical issues. These findings emphasize the importance of precise, well‐defined terminology in patient‐reported outcome measures. The relatively high proportion of “Do not know” responses, particularly for C‐1, resulted in sparse category counts, which likely contributed to its lower test–retest stability compared with C‐2 and C‐3. Although C‐3 showed the highest proportion of “do not know” responses, its test–retest reliability remained high, suggesting that missing or uncertain responses primarily affected response propensity rather than response consistency.

While the PRO‐DIT‐17 subscales patient satisfaction and pretreatment information showed statistically significant correlations with the OHIP‐14 total score, the oral function subscale did not. This is likely due to the latter subscale's focus on temporal rather than static changes. These results suggest that PRO‐DIT‐17 may capture both shared and distinct dimensions of OHRQoL, providing support for its validity as a tool for assessing patient‐reported impacts following implant rehabilitation.

The separate consideration of the PRO‐DIT‐9 and PRO‐DIT‐10 items is consistent with best practice in questionnaire validation, where items with low factor loadings are typically removed to strengthen scale structure. Although PRO‐DIT‐9 showed an acceptable CITC, the weak factor loading indicated that it did not adequately represent the underlying construct. PRO‐DIT‐10 was problematic in both measures, displaying a low factor loading and a negative CITC, suggesting inconsistency with the scale. Despite these findings, we ultimately chose to retain both items. This decision was informed by the fact that the removal of PRO‐DIT‐9 and PRO‐DIT‐10 had minimal impact on the overall internal consistency and reliability metrics. Specifically, Cronbach's alpha and ICC values remained largely unchanged when comparing the 17‐item version with the reduced 15‐item version. Given the statistical similarity in reliability outcomes, we prioritized content coverage and construct comprehensiveness, recognizing that these items may still capture relevant aspects of the construct not fully represented by other items.

The present study has some important limitations. The most obvious limitation is the long‐ and varying time interval between test and retest. Recommendations by (Terwee et al. 2007) suggest 1–2 weeks as optimal to minimize both carryover and true change effects. While the extended interval of 1–13 months might better capture real‐world variability, it also introduces greater risk of status change between tests. If the time interval is too short, it may produce carryover effects due to memory, while a longer interval increases the chances that a change in status could occur (Allen and Yen 1979).

The participants in the present study were heterogeneous with respect to their implant rehabilitation but also related to other oral aspects, where diseases, symptoms, follow‐up, discomfort, and functional status potentially may change substantially over a short period of time. Such changes may influence retest scores independently of measurement error and therefore lead to an underestimation of the true test–retest reliability. Thus, the heterogeneity of the retest window may imply that some patients were retested during a stable phase whereas others were retested following clinical deterioration. Consequently, the stability estimates must be interpreted with caution, as they may reflect clinical change in some patients.

The “preinformation” subscale consisted of only two items, which represent a limitation in terms of construct breadth. While two‐item factors can demonstrate acceptable reliability, their interpretation should be made with caution. Raubenheimer (2004) have reported that a minimum of three items per factor is generally recommended, although multidimensional scales may be identified with as few as two items per factor.

Another potential limitation is that while the questionnaire was designed specifically for issues related to dental implants, some patients may have difficulties distinguishing between natural teeth and dental implants, potentially influencing their responses. However, it is worth noting that the number of self‐reported implants has been clinically validated in a previous study on the same sample (Mauland, Sorensen, et al. 2025), and a very strong correlation (r = 0.91) between self‐reported and clinical counts was observed. Also, a significant association was observed between self‐reported biological and technical complications and periimplantitis and clinically assessed technical complications.

A strength of this study is that it includes a diverse sample which is also larger than those included in comparable test–retest studies (He et al. 2017). Nevertheless, for future research, a fixed time interval should be established for administering the same questionnaires. This would allow for more precise assessment of whether reliability remains stable, improves, or declines over time. Finally, although conceptual and theoretical work to further develop valid measures of PROs, patient satisfaction in particular, has been called for over the last 3–4 decades (e.g., Sitzia and Wood 1997), there is a lack of references to a coherent theoretical base in the literature. Thus, future research could benefit from including more profound investigations of the construct validity of instruments assessing PROs, including patient satisfaction, in various contexts. Such research should include an exploration of various dimensions of the phenomena of interest, and the weights given to them, through qualitative research in the target population(s).

5. Conclusion

The PRO‐DIT‐17 demonstrated generally good internal consistency reliability and test–retest performance. Assessment of construct validity further supported its applicability in assessing patient‐reported outcomes following implant treatment in both clinical and research settings.

Author Contributions

Anders Verket: conceptualization, investigation, funding acquisition, writing – review and editing, methodology, project administration, data curation, supervision, formal analysis. Erik Klepsland Mauland: conceptualization, methodology, investigation, writing – review and editing.

Funding

This study was funded in part by a research grant 2021–1625 from the ITI Foundation and in part by the authors' institutions.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Table S1: clr70110‐sup‐0001‐TableS1.docx.

Data S1: clr70110‐sup‐0002‐Supinfo.docx.

Data S2: clr70110‐sup‐0003‐Supinfo1.zip.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.