Validation of a fornix depth measurer: a putative tool for the assessment of progressive cicatrising conjunctivitis

Abstract

Background/aims

Documentation of conjunctival forniceal foreshortening in cases of progressive cicatrising conjunctivitis (PCC) is important in ascertaining disease stage and progression. Lower fornix shortening is often documented subjectively or semi-objectively, whereas upper forniceal obliteration is seldom quantified. Although tools such as fornix depth measurers (FDMs) have been described, their designs limit upper fornix measurement. The purpose of this study was to custom-design a FDM to evaluate the upper fornix and to assess variability in gauging fornix depth.

Methods

A polymethylmethacrylate FDM was constructed using industry-standard jewellery computer software and machinery. Two observers undertook a prospective independent evaluation of central lower fornix depth in a heterogeneous cohort of patients with clinically normal and abnormal conjunctival fornices both subjectively and by using the FDM (in mm). Upper central fornix depth was also measured. Agreement was assessed using Bland–Altman plots.

Results

Fifty-one eyes were evaluated. There was 100% intraobserver agreement to within 1 mm for each observer for lower fornix measurement. The mean difference in fornix depth loss using the FDM between observer 1 and 2 was 1.19%, with 95% confidence of agreement (±2SD) of −15% to +20%. In total, 86% (44/51) of measurements taken by the two observers agreed to within 10% of total lower fornix depth (ie, ±1 mm) versus only 63% (32/51) of the subjective measurements. Mean upper fornix difference was 0.57 mm, with 95% confidence of agreement of between −2 and +3 mm.

Conclusions

This custom-designed FDM is well tolerated by patients and shows low intraobserver and interobserver variability. This enables repeatable and reproducible measurement of upper and lower fornix depths, facilitating improved rates of detection and better monitoring of progression of conjunctival scarring.

Introduction

Sequential documentation of forniceal foreshortening in cases of progressive cicatrising conjunctivitis (PCC), such as ocular mucous membrane pemphigoid (OcMMP), is important in assessing stage and progression of disease.^1–3 The Foster staging relies on subjective evaluation of subepithelial fibrosis and extent of symblepharon formation,³ and the system described by Mondino and Brown^{2 4} describes grading of percentage shrinkage of the lower fornix. A modification encompassing both systems was described by Tauber et al,¹ who proposed that counting the number of, and percentage horizontal obliteration of the lower fornix by symblephara could potentially improve detection of disease progression.

We have previously shown that lower fornix shortening is documented either subjectively or semi-objectively by utilising a slit-light beam, the degree of upper forniceal obliteration is seldom quantified.⁵ Furthermore, we have shown that, at the initial visit to tertiary referral centres, Foster's staging of disease is undertaken in 100% of patients' lower fornix, but only 78% of patients had quantification of forniceal shrinkage. This is probably related to the difficulty in assessing lower fornix depth accurately without the aid of a made-for-purpose tool.⁶

Measurement of the fornices using devices such as the fornix depth measurer (FDM) has previously been described, but their design restricts accurate upper fornix measurements.^{7 8} Specifically, the depth and curvature of the upper fornix dictates that the ideal FDM must be sufficiently long and curved to enable comfortable and accurate assessment of it. We previously used in routine clinical practice an FDM based on that designed by Schwab et al,⁷ but its design was suboptimal (figure 1). In 2004, a modification of the Schwab FDM was designed at Moorfields Eye Hospital for a clinical trial⁹ and is currently being used in an epidemiological study. In 2007, we wanted to redesign and evaluate a bespoke comfortable prototype that could provide an accurate tool for improving forniceal sac documentation in the outpatient clinic setting, with potential for wider-scale commercial manufacture. In this study we describe this custom-designed FDM constructed specifically to facilitate evaluation of the depth of the upper conjunctival fornix, and we assess intraobserver and interobserver variability in gauging the extent of the upper and lower fornices by validating the FDM and comparing it with subjective assessment of fornix shrinkage.

Figure 1.

Colour photographs illustrating the evolution of the fornix depth measurer (FDM). (A) The original adaptation of an FDM described by Schwab et al⁷ (produced by our local prosthetics department). (B) An FDM constructed at Moorfields Eye Hospital, which is an elongated polymethylmethacrylate modification of the Schwab FDM using a hand-made plaster cast (designed by VS, Scott Hau and David Carpenter, ocular prosthetist at Moorfields Eye Hospital, specifically to facilitate upper fornix depth measurement for use in a clinical trial). For comparison, (C) illustrates the computer-designed bespoke FDM prototype. This is an elongated, biconcave design with engraved markings to a precision of 2 μm/step, and increments expressed at 2 mm intervals on both the main body of the FDM and the narrower ‘handle’. The markings on the handle facilitate upper fornix measurement and the ability to measure the fornix in the presence of symblephara (D). The accuracy and reproducibility of the computer-generated design and jewellery precision engraving provides potential for commercial manufacture.

Materials and methods

Design of a bespoke FDM

A polymethylmethacrylate FDM was designed using industry-standard jewellery computer software (CAD V5, 3Design, Brussieu, France, 2007). The virtual model was then exported as an. STL file into a program to set the cutting parameters for the milling machine (Modela Player V4; Roland DG, Shizuoka, Japan, 2002) (figure 2A). The virtual model was cut with a machine to a precision of 2 μm/step, and increments were expressed at 2 mm intervals at both ends of the FDM: the main ‘body’ and the ‘handle’. This design feature enables measurement of the fornix in the context of multiple symblephara, which could potentially hinder the smooth passage of the FDM to the limits of the conjunctival sac. The FDM measured 25 mm × 5 mm and was moulded to a biconcave shape to ensure ease of insertion and comfort (figures 1C and 2B).

Figure 2.

(A) A polymethylmethacrylate fornix depth measurer (FDM) was constructed using industry-standard jewellery software and machinery. A screenshot of the 3design prototype is shown in (A). The final prototype is illustrated in (B). Increments are expressed at 2 mm intervals to a precision of 2 μm/step, and the FDM was heat moulded to a biconcave shape for comfort. The FDM was applied after instillation of one drop of 0.4% oxybuprocaine hydrochloride (C). Patients were asked to look in the opposite direction to the placement of the FDM, and the central conjunctival fornix was measured to the eyelid margin, defined as the posterior lip of the meibomian gland orifice.

Patients

An evaluation of the bespoke FDM was undertaken through a prospective, masked, independent assessment of central lower fornix depth by two observers (GPW, TS), following the Tenets of the Declaration of Helsinki. This was undertaken as a service evaluation at the Birmingham and Midland Eye Centre (BMEC). The validation of the FDM was conducted on a heterogeneous group of patients comprising clinically normal and abnormal conjunctival fornices presenting to BMEC. This heterogeneity enabled a wide range of fornix depths to be tested with the bespoke FDM.

The cohort consisted of 51 eyes of 26 patients with a median age of 64 years (range 42–100), of whom 65% (17/26; 33/51 eyes) had an identifiable cause of cicatrising conjunctivitis (OcMMP, 10; dry eyes, five (including three with Sjögren's syndrome); Stevens–Johnson syndrome, two) and 35% (18/51) had no evidence of conjunctival scarring (age-related macular degeneration, four; uveitis, three; peripheral ulcerative sclerokeratitis, two).

After instillation of one drop of 0.4% oxybuprocaine hydrochloride, patients were asked to look in the opposite direction to the placement of the FDM (upper fornix, down-gaze; lower fornix, up-gaze) in order to protect the cornea, ensure consistency in readings, and circumvent variability in the eyelid position in primary gaze. The central conjunctival fornix was measured to the eyelid margin, defined as the posterior lip of the meibomian gland orifice (figure 1C). All FDM readings were taken in triplicate, with the first measurement taken used for interobserver comparison. A semi-objective estimation of lower fornix conjunctival shrinkage was also performed by gauging the central lower fornix depth, measured from the inferior fornix to the eyelid margin with the aid of a vertical 1 mm wide slit-lamp beam with illumination and observation axes in a coaxial position (NB a subjective assessment of the upper fornix is impossible). The FDM was sterilised by soaking the device in 0.05% sodium dichloroisocyanurate solution for 5 min between patients (as per the BMEC infection control policy for reusable tonometer heads). Patients were also asked about their tolerance to the FDM.

Calculations and statistical analyses

The percentage loss of lower fornix for both methods of measurement (subjective and objective) was calculated using the equation:

$$\left\{\right[\text{fornix depth}\left(\text{FD}\right)\text{age}-\text{FDM measurement}]/\text{FD age}\}\times 100=\%\text{loss of fornix}$$

A correction factor for age was implemented, as the lower forniceal depth is known to progressively shorten with age.⁷ The ‘FD age’ values were derived from published age-specific lower fornix depths in normal eyes detailed in table 1.⁷ For example, for a patient aged 80 years with a lower fornix measurement of 6 mm, the calculated percentage shrinkage is as follows:

$$\left\{\right[10.2-6]/10.2\times 100\}=41.2\%$$

Table 1.

Age-specific normal values for the conjunctival lower fornix

Age	Mean depth of normal lower fornix (mm) (‘100%’)	10% of normal lower fornix (mm)
40–49	11.9	1.19
50–59	11.3	1.13
60–69	11.0	1.10
70–79	10.6	1.06
80+	10.2	1.02

Adapted from Schwab et al.⁷

It is important to note that it was not possible to calculate percentage loss of the upper fornix because the normal range of age-specific upper forniceal depths is currently unknown.

Intraobserver and interobserver agreement was assessed using Bland–Altman plots of differences in measurements versus mean measurements using Excel for Macintosh (Microsoft Office 2008).^{10 11} The mean difference in observations and the 95% limits of agreement (the mean difference ±2 standard deviations)¹⁰ were calculated using SPSS V16.0 for Macintosh. A continuity correction was applied to the 95% limits of agreement to take account of the fact that Bland–Altman plots assume that the variables measured are continuous.

A 10% threshold or ‘tolerance’ was chosen as an allowance for intraobserver variation. Agreement was also evaluated by determining the percentage of observations that agreed to within the 10% ‘allowance’ for both observers.

Results

Lower fornix assessment

Intraobserver variation

Triplicate measurements of FDM readings of the same anatomical position by each observer (central lower fornix) showed exact agreement of 86% (42/49) and 89% (41/46) of measurements within observer 1 and observer 2, respectively. When allowing for 1 mm ‘tolerance’ (approximating to 10% of the normal lower fornix, see table 1), 100% of intraobserver observations fell within 1 mm for both observers.

Interobserver variation

Interobserver variation between the subjective and objective measurements of the central lower fornix by the two observers was also assessed. Assessment of the lower fornix shrinkage was expressed as a percentage for both subjective and objective estimations, the latter using the age correction factor described in the methods.⁷

The mean difference in calculated percentage fornix depth using measurements obtained from the FDM by observer 1 and 2 was 1.19%, and, with a continuity correction, the 95% limits of agreement (±2SD) were narrower for interobserver objective (FDM) measurements than for those obtained subjectively (−15% and + 20%) (figure 3). The interobserver agreement within the 10% allowance (ie, approximately ±1 mm) of total lower fornix depth was 86% (44/51) (figure 4).

Figure 3.

Bland–Altman plots showing interobserver variation in lower fornix assessment ((A) objective and (C) subjective). Some data points are identical and therefore overlay each other on the figures. The percentage difference in assessment between observer 1 and 2 is plotted against the mean percentage loss of fornix for each eye. If there was a completely normal fornix, this is represented as 0% loss of fornix on the x axis. Note the increase in the 95% limits of agreement (±2SD) for subjective assessment (arrowed), demonstrated also by the histograms (B,D).

Figure 4.

Difference in objective (A) and subjective (B) lower fornix assessment between observer 1 and 2. The number of individuals that agree to within a 10% ‘allowance’ are boxed and are higher for objective measurements (86% (44/51)) than for subjective measurements (63% (32/51)).

In contrast, the interobserver mean difference in subjective estimation of percentage fornix depth was −1.86%, and, with a continuity correction, the 95% limits of agreements (±2 SD) were between −30% and +25% (figure 3). Only 63% (32/51) of the subjective measurements taken by the two observers agreed to within a 10% allowance of total lower fornix depth (figure 4).

These data highlight that the FDM afforded greater consistency in fornix depth measurement by each observer (intraobserver variation) and between observers (interobserver variation).

Upper fornix assessment

Intraobserver variation

Triplicate measurements (to assess intraobserver variation) of FDM readings of the same anatomical position of the central upper fornix by each observer showed minimal variation (ie, identical objective measurements) in 88% (45/51) and 70% (33/47) of measurements by observer 1 and observer 2, respectively. There are no data regarding normal upper fornix depth in the published literature, preventing calculations for age-based corrections and percentage fornix depth foreshortening and tolerance threshold.

Interobserver variation

Interobserver variation of the upper fornix showed a mean difference in fornix depth measurement using the FDM for observer 1 and 2 of 0.57 mm, with 95% limits of agreement (±2SD) of −2 and + 3 mm (figure 5). The absence of normal upper fornix values precludes evaluation of the 10% allowance; however, 84.3% (43/51) of upper fornix measurements were within 1SD of the mean difference of 0.57 mm (+2 to −1 mm of the mean with a continuity correction).

Figure 5.

(A) Bland–Altman plot showing interobserver variation in upper fornix assessment. As there are no defined limits for the upper fornix, the calculations are in mm. The mm difference in assessment between observer 1 and 2 is plotted against mean mm measurement for each patient. The 95% limits of agreement are vertically arrowed and also represented by a histogram (B). (C) Difference in upper fornix assessment between observer 1 and 2; the 10% allowance cannot be calculated in the absence of normal upper fornix values, but 71% (36/51) of measurements were within 1 mm, and 92% (47/51) of observations were within 2 mm of each other.

Patient comfort and tolerance

The FDM was well tolerated by patients, with only a few (three) experiencing mild discomfort during upper fornix measurement, despite repeated measurements. None of these patients reported prolonged discomfort or pain.

Discussion

PCC comprises a group of disorders characterised by progressive scar formation in response to conjunctival inflammation affecting the stromal layers of the conjunctiva.¹² These include ocular immunobullous diseases such as OcMMP, Stevens–Johnson syndrome and toxic epidermal necrolysis, lichen planus, linear IgA disease, paraneoplastic pemphigus and epidermolysis bullosa. Other causes include graft-versus-host disease, Sjögren's syndrome, acne rosacea and those associated with topical therapy where progression is reported to be more insidious with less destructive clinical sequelae.

Because of the sight-threatening consequences of OcMMP, the ocular phenotype of MMP is regarded as ‘high risk’,¹³ particularly as disease progression is more aggressive in younger patients¹⁴ and 50% of patients continue to have progression of cicatrising disease in the apparent absence of inflammation.¹⁵

Determining progression in PCC is a challenging aspect of patient management. When considering OcMMP, for example, the most common of the acquired immunobullous diseases that cause PCC,^{12 16} progression may occur at any stage of disease,^{2 17} which can be aggressive early in the disease course,¹⁴ but, importantly, is often independent of clinically identifiable inflammation.¹⁵ Determining disease progression in the cicatrising conjunctivitides therefore relies upon accurate documentation of disease, in particular conjunctival shrinkage of the fornices. Although the Mondino staging system is considered to be more sensitive than Foster's staging system¹⁵ and is integrated into the system proposed by Tauber et al,¹ Mondino fornix depth measurement is reported to be undertaken in only 78% of new patients in tertiary referral centres compared with 100% documentation of the Foster's system.

Vigilant assessment, quantification and documentation of forniceal foreshortening is mandatory for enabling accurate patient follow-up, currently not achieved by subjective or semi-objective assessments of the fornix. These approaches are limited by inconsistency and poor reproducibility and reliability, but are overcome by the development and implementation of fornix depth measuring devices for the assessment and progression of diseases that cause conjunctival scarring.^{7 8 18} Our aim was to design an FDM based upon the original concept of Schwab et al,⁷ but with additional upper fornix depth capability. Specifically, our custom-made FDM was found to be comfortable and accurate. Designed and made using industry-standard computer software and machinery, this FDM prototype was modelled mathematically taking into account the curvature of the globe necessary for comfortable measurement of the upper fornix. We tested our bespoke FDM on patients with a range of fornix depths in the presence or absence of conjunctival fibrosis. Our findings showed low intraobserver and interobserver variability, enabling repeatable and reproducible measurements of both upper and lower fornix depths, highlighting its potential in facilitating both accurate and robust clinical documentation of disease stage. This FDM prototype is currently being optimised with further modifications to improve comfort and fit.

Crucially, the conjunctival cicatrising process is not confined to the lower fornix, and sight-threatening sequelae secondary to subtarsal fibrosis, upper lid entropion or lash trauma commonly ensue. These clinicopathological processes are not taken into consideration by ocular staging systems that calculate percentage obliteration of the lower fornix,^{1 2 4} although are reflected in the staging system described by Foster where fornix shortening or symblephara of any degree throughout the conjunctival surface are thought to be important.³ The decreased sensitivity in relying solely upon this system, or those that include direct measurement of the lower fornix alone, increase the risk of type 2 errors (false negatives) in determining disease progression, which are considerably rescinded by the use of fornix depth measuring devices for the upper fornix.

The larger depth of the upper fornix and difficulty of access explains the omission of detailed upper fornix pathology in currently recognised scoring systems.^{1 2 4} Perhaps not surprisingly therefore, there appear to be few data in the ophthalmic literature regarding the anthropology and normal depth of the upper fornix. Kawakita and colleagues⁸ have recently discussed the use of a non-curved 150×2 mm FDM in Japanese patients with Stevens–Johnson syndrome and healthy volunteers. They found that the mean superotemporal and superonasal upper fornix depths were 14.1±2.5 mm in normal people. Our findings show a median central upper fornix depth of 16 mm in a caucasian group of patients, even in the presence of recognised cases of cicatrising conjunctivitis. The variation in central upper fornix depth among healthy populations using anthropological ethnography together with differences in age remains unresolved. This has recently been highlighted in the context of other diseases affecting the size of the upper fornix such as the giant fornix syndrome described by Rose.¹⁹ These data illustrate the need for population-based studies of normal age-based upper fornix depths to facilitate calculations of percentage shrinkage in conjunctival scarring diseases. Such studies are currently being undertaken at BMEC and Moorfields Eye Hospital.

In summary, our custom-designed FDM was well tolerated by patients in this study, with only three experiencing short-lived mild discomfort during assessment of the upper fornix. This FDM shows low intraobserver and interobserver variability, enabling repeatable and reproducible measurements of lower fornix depths. We believe that the custom design of an FDM using industry-standard jewellery software and machinery, curved to fit the globe, provides an accurate and comfortable means of assessing lower fornix depth. Furthermore, it offers the potential to measure upper fornix depth, currently not routinely carried out in clinical practice, thereby improving both the detection and monitoring of progressive conjunctival fibrosis in this group of devastating disorders.