Challenges in measuring and valuing productivity costs, and their relevance in mood disorders

Abstract

Lost productivity is often excluded from economic evaluations, which may lead to an underestimation of the societal benefits of treatment. However, there are multiple challenges in reliably estimating and reporting productivity losses. This article explores the main challenges, ie, selecting an appropriate valuation method (ie, human capital, friction cost, or multiplier), avoiding double counting, and accounting for equity. It also discusses the use of presenteeism instruments and their application in clinical trials, with a specific focus on their relevance in individuals with mood disorders. Further research and discussion is required on the development of reliable techniques for measuring and valuing productivity changes due to presenteeism.

Introduction

The measurement and valuation of productivity loss remains a much debated topic in the field of economic evaluation. Health technology assessment organizations in many countries impose restrictions on the inclusion of productivity costs in health economic evaluations (Table 1), questioning both their relevance and the available methodology. The National Institute for Health and Care Excellence in England and Wales states that the perspective on costs should be that of the National Health Service and Personal Social Services; that is, productivity costs and costs borne by patients and carers that are not reimbursed by the National Health Service and Personal Social Services should be excluded from analysis of both reference and non-reference cases.¹ Guidelines from other countries, eg, Australia and Canada, adopt a wider perspective, allowing productivity costs to be included if productivity is likely to be substantially affected by a new health technology, and provided that these costs are evaluated and reported separately.^2,3 In contrast, Dutch and Swedish guidelines specifically call for the inclusion of costs related to lost productivity.^4,5 The method by which productivity costs should be measured also differs between countries. Swedish guidelines recommend the use of the traditional human capital (HC) method, which values lost productivity in terms of gross earnings, whereas the Canadian, Dutch, and Australian authorities recommend the friction cost (FC) approach, which focuses on the time required to restore the lost productivity due to the absent worker.⁶

Table 1.

Productivity costs by health technology assessment agency

Country	Primary perspective	Secondary perspective	Productivity costs	Method of valuation	Notes
United Kingdom (National Institute for Health and Care Excellence1)	NHS and PSS	Societal perspective	Not included	NA	For the reference case, the perspective on costs should be that of the NHS and PSS. For technologies for which a substantial proportion of the costs are expected to be incurred outside of the NHS and PSS, information on costs to other government bodies may be reported separately from the reference case analysis. However, productivity costs and costs borne by patients and carers that are not reimbursed by the NHS or PSS are not included in either the reference case or non-reference case analyses.
Canada (Canadian Agency for Drugs and Technologies in Health2)	Publicly funded health care system	Public payer, societal perspective	May be included	FC approach	The primary perspective of the analysis should be that of the publicly funded health care system. The costs associated with adopting a wider perspective should be reported separately where it is likely that they have an impact on the results of the analysis; for example, when an intervention permits patients to return to work sooner than otherwise, costs shift to patients and their families, or result in savings or additional costs to other public sector agencies.
Australia (PBAC3)	Health care budget perspective	Societal perspective	May be included	FC approach preferred	The PBAC mainly considers the costs of providing health care resources. It may also consider costs and cost offsets of non-health care resources, but these might not be as influential in decisionmaking as health care resources. If a claim is made for a change in non-health care resource costs or a change in non-health outcomes (eg, production changes), a supplementary analysis must be presented with these included.
The Netherlands (College vor Zorgverzekeringen4)	Societal perspective		Included	FC approach	The pharmacoeconomic evaluation should be performed and reported from a societal perspective, Separate analyses for productivity costs should be performed and reported.
Sweden (Tandvårds-och läkemedelsförmånsverket5)	Societal perspective		Included	HC method	All relevant costs associated with treatment and illness should be identified, quantified and evaluated. The production loss for treatment and sickness should also be included (estimated using the HC method).

Abbreviations: FC, friction cost; HC, human capital; NA, not applicable; NHS, National Health Service; PBAC, Pharmaceutical Benefits Advisory Committee; PSS, personal social services; SHI, Statutory Health Insurance.

Differing perspectives on productivity costs

Economic evaluations may be conducted from a variety of perspectives, depending on the objectives of the decision-maker. The societal perspective allows for the most comprehensive cost assessment, because the economic evaluation may include not only lost productivity in paid work, but also lost productivity in non-work-related tasks, such as the provision of informal care. The government payer perspective allows consideration of transfer payments to sick employees and lost tax revenues from diminished or forgone wages. Productivity costs from the employer perspective are relevant in countries where employers pay a proportion of workers’ health insurance. Such costs would account for losses in employee-generated revenue during the period of diminished productivity, and for employee replacement costs, inclusive of recruitment and training. A health service perspective limits the loss of revenue or replacement costs to health care workers only. However, a non-governmental third-party payer perspective completely precludes the use of productivity costs, because compensation resulting from lost productivity due to ill health is seldom provided by a third-party payer. Finally, from a patient perspective, productivity costs represent the difference between an employee’s wages forgone and the sickness benefit received.

Importance of productivity costs in mood and anxiety disorders

There are four main reasons why productivity costs are of particular relevance in mood disorders, such as depression and bipolar disorder. First, they account for a large proportion of the total cost burden in mood disorders.^7,8 For example, Sobocki et al reported that costs due to production loss represented 65% of the total cost of treating Swedish patients (n = 398) for depression.⁸

Second, individuals suffering from mood disorders reportedly lose, on average, more work days per year than individuals suffering from other chronic conditions. Munoz et al compared productivity losses in patients with different psychiatric conditions (ie, depression, mania, agoraphobia, social phobia, general anxiety disorder, panic disorder, and post-traumatic stress disorder) with those in patients with chronic non-psychiatric conditions, eg, arthritis and hypertension.⁹ Patients suffering from depression and panic disorder were found to lose the most work days (mean 25.51 days, 95% confidence interval [CI] 16.53–34.5, and 20 days, 95% CI 3.02–36.97, respectively). The average number of days lost by patients suffering from other chronic conditions was lower (mean 6.89, 95% CI 3.04–10.74) than for five of the seven psychiatric conditions of interest.⁹

Third, employees with mood disorders have greater productivity losses due to presenteeism than do employees with other chronic conditions, eg, arthritis and back pain.¹⁰ Presenteeism has been defined as “reduced on-the-job productivity due to employee health”; it can be measured as the sum of employees’ reduced work output, errors on the job, or failure to meet company production standards.¹¹ Wang et al used experience sampling to collect comparative data on moment-in-time work performance in employees with either major depression or other chronic conditions (eg, allergies, arthritis, back pain, headache, high blood pressure, asthma).¹⁰ The objective was to assess two dimensions of presenteeism, ie, task focus (concentration and day-dreaming) and productivity (quality, speed, and efficiency). The results showed that major depression was the only condition that was statistically significant with respect to decrements in both productivity (P < 0.05) and task focus (P < 0.10).¹⁰

Finally, in individuals with mood disorders, presenteeism alone can account for 50%–80% of total lost productivity.¹² In the 2006 US National Comorbidity Survey Replication, presenteeism accounted for between 54% (major depressive disorder) and 67% (bipolar disorder) of total lost productivity.¹³ Stewart et al demonstrated that, among depressed individuals, presenteeism accounted for 82.1% of total lost productivity (4.6 hours per worker per week out of a total lost productive time of 5.6 hours).¹²

Many health technology assessment organizations exclude productivity costs from health economic evaluations, either by choosing to conduct the analysis from a third-party payer perspective or because of concerns about whether productivity costs can be measured appropriately.¹⁴ The choice of perspective is primarily the value judgment of the health technology assessment agency or government in a given jurisdiction, so the objective of this paper is to explore the second methodologic issue. Preclusion of productivity costs from economic evaluations in mood disorders may lead to an underestimation of the societal benefits of treatment, or may give only a partial picture of the economic implications of a treatment strategy. To facilitate the assessment of productivity savings in future economic evaluations conducted in the area of mood disorders, this paper explores the methodologic challenges related to the measurement and valuation of productivity costs, and proposes solutions to these challenges.

Measurement of presenteeism

Productivity loss due to absenteeism can be measured by simply counting the number of days absent; however, measuring productivity loss due to presenteeism is more complex. To determine an employee’s reduced productive output due to ill health, one first needs to establish the employee’s normal productive output. Several instruments have been developed to measure presenteeism.¹⁵ Some of these instruments can be used to monetize productivity; for example, the Health and Work Performance Questionnaire, the Health and Labor Questionnaire, and the Work Productivity and Activity Impairment Questionnaire. Others are not amenable to monetization; for example, the Endicott Work Productivity Scale and the Stanford Presenteeism Scale. Brooks et al¹⁶ highlighted several points about the conversion of presenteeism into lost productivity or monetary units. Although some instruments produce outputs suitable for conversion to time or money directly from the questionnaire, others do not. The conversion process depends on the type of output resulting from each instrument. For example, the main output can be expressed as a “percent impairment while working due to health problem” (as in the Work Productivity and Activity Impairment Questionnaire), a “rating of overall job performance on the days worked during the past 4 weeks” (as in the Health and Work Performance Questionnaire), or a “number of hours to catch up on all the work not performed over the last month” (as in the Health and Labor Questionnaire). All of these measures can be converted directly into monetary units.¹⁶ The Work Limitation Questionnaire is a presenteeism instrument that has been used extensively in scientific research.¹⁶ However, the conversion of presenteeism scores from the Work Limitation Questionnaire into lost productivity or monetary units is complex, because the main output includes four scales, each of which is weighted to calculate the “productivity loss score”. This score represents a percent reduction in output over the previous 2 weeks compared with the output of a healthy (ie, not limited) employee.

Another challenge in measuring presenteeism is the lack of objective measures, given that most presenteeism instruments rely on self-report.¹⁶ There is concern that self-reported measures may overestimate time spent working compared with results obtained from official workplace data on absenteeism, time away from desk, and electronic continuous performance data specific to the workplace.^17,18 Lerner et al¹⁹ have provided valuable insights into the relationship between self-reported assessment and productivity obtained through an objective measure, and the impact of depressive symptoms on this relationship. The authors evaluated the productivity of employees at a large company, estimating self-reported presenteeism using the Work Limitation Questionnaire, and obtaining objective employee-level productivity data from the company. The study established a link between self-reported and objectively measured productivity loss, and found that the relationship was not modified in patients with depressive symptoms. Further research should be carried out to validate these findings in other settings.

Finally, additional challenges arise when the aim is to measure productivity loss in a randomized clinical trial setting. It is rare to detect between-treatment differences in presenteeism using self-report instruments for several reasons: the instruments lack sensitivity, there is often a lack of statistical power, and there are variations in aspects specific to the trial design, eg, trial duration and time points of assessment.²⁰

Proposed solution

When aiming to measure productivity loss in the context of clinical trials, it is important to:

• Ensure a sample size that is large enough to demonstrate any between-treatment differences.²⁰ This could be achieved by powering the study based on an endpoint that is associated with changes in work productivity. For example, Drummond et al, in undertaking an economic evaluation alongside a clinical trial of a new drug for schizophrenia, powered the study to detect a clinically important difference in acute exacerbations requiring hospitalization, because it was thought these events would have a major impact on resource use and work loss.²¹

• Minimize recall bias by using shorter recall periods or patient diaries for recording of work loss.²²

• Use objective measures of presenteeism alongside self-report, or select presenteeism instruments that are associated with objectively measured productivity loss.

In the context of health technology assessment, productivity arguments are most effective when they are based on observational studies carried out in a relevant setting. Study design and setting should be discussed directly with payers before initiation, to ensure that payer needs are met. This could be achieved through a process of early dialogue with pricing and reimbursement agencies.²³ Early engagement has been undertaken for drugs in a variety of therapeutic areas; however, thus far, there are no published examples of its application to drugs for mood disorders.

Valuation of productivity loss

Human capital, friction cost, or multiplier method

Another question related to the valuation of productivity loss is which method should be employed (Table 2). The HC method uses gross wages (including employee benefits) to estimate productivity costs, assuming that wages are a proxy measure of employee output.²⁴ This method is grounded in economic theory, and is based on the assumption that companies employ labor until the marginal value of a worker’s productivity equals the marginal cost of labor, or the worker’s wage.²⁵ However, in the real world, an absent worker may be replaced on an interim basis by a less suitable colleague, or they may need to be replaced permanently, thus incurring recruitment and training costs.²⁶ The FC approach attempts to compensate for this by measuring lost productivity due to illness based on the employee replacement cost, rather than the employee wage. In other words, the FC approach operates on the premise that employee wages are not a true proxy for lost productivity, and that wages are likely to be the upper bound estimate of lost productivity due to illness. The FC approach hypothesizes that some of the absent employee’s tasks may be completed by colleagues, or that the employee in question will somehow make up a portion of the lost productivity once the person returns to work. In addition, the FC approach can include costs associated with replacing an employee that are borne by the employer, such as advertising, recruitment, and training.²⁷ Some policymakers view this as a more realistic approach to measuring lost productivity due to absenteeism.

Table 2.

Human capital method and friction cost approach

	Human capital method	Friction cost approach
Valuation	Measure of potential value of production loss due to illness	Measure of actual value of production loss due to illness
Reflection of reality	Fails to account for the possibility that absent workers may be replaced	Assumption that absent employee may be replaced is only conditionally valid
Scope	Broad scope: includes cost of lost productivity due to sickness, disability, early retirement and presenteeism, as well as lost non-work time and informal carers’ time	Narrow scope: includes only productivity loss as measured by an employee replacement cost
Feasibility of assessment	Simple calculation using wages as a proxy measure of employee output	More data-demanding, requiring data on disease-specific employment rates and job vacancy duration

The FC approach is inherently more difficult to apply than the HC method, because the former requires more data and more complex calculations; for example, the FC approach requires data on disease-specific employment rates and job vacancy duration (ie, the friction period). This information is likely to be difficult to obtain from standard national data sources.²⁶ The HC method incorporates a broader view of lost productivity than the FC method, including lost non-work time and informal carers’ time, in addition to lost production in paid employment. In theory, the cost of lost productivity due to disability, early retirement, and presenteeism may also be included when the HC valuation is used.²⁸ The FC approach does not consider these types of non-work-related productivity costs. Moreover, there is, as yet, no consensus on how best to accommodate presenteeism within this method, because it is assumed that lower productivity due to illness will be compensated for by others, in which case productivity loss due to presenteeism could be valued at less than an employee’s wage. To date, no empiric studies of productivity loss have explicitly included presenteeism costs using a FC valuation. Wieser et al took an FC approach to estimating the cost of lower back pain, but calculated the presenteeism element using the HC method.²⁹

When assessing short-term absence, the difference in productivity costs between the two methods will depend on the length of the friction period. The closer the friction period is to the period of absenteeism, the closer the FC and HC estimates will be, assuming that both estimates use the same hourly or daily estimate of the value of employee productivity. For longer term absences (ie, due to disability or mortality), the difference between the FC and HC estimates will be greater (Table 3).

Table 3.

Human capital approach and friction cost method in studies evaluating productivity cost for mortality, disablement, and/or work absence

	Mortality		Disablement		Work absence
	HC	FC	HC	FC	HC	FC
Kaitelidou et al39	€9.9 million	€302,513	€264 million	€5.7 million	€4.9 million	€4.3 million
Koopmanschap and van Ineveld40	2,072 million DFL	60 million DFL	2,615 million DFL	19 million DFL	624 million DFL	353 million DFL
Koopmanschap et al6	8 billion DFL	0.15 billion DFL	49.1 billion DFL	0.15 billion DFL	23.8 billion DFL	9.2 billion DFL
Lopez-Bastida et al41	Cancer: €122.4 million CVD: €80.8 million	Cancer: €2.5 million CVD: €1.7 million			Cancer: € 13.3 million CVD: €30.8 million	Cancer: €13.2 million CVD: €23.6 million
Oliva et al42	Breast: €113.1 million Cervical: €21.7 million	Breast: €2.3 million Cervical: €393,000	Breast: €159.3 million Cervical: €20.6 million	Breast: €5.4 million Cervical: €310,800	Breast: €16.4 million Cervical: €1.2 million	Breast: €3.9 million Cervical: €432,000

Abbreviations: DFL, Dutch Florin (The Netherlands currency); CVD, cardiovascular disease; FC, friction cost approach; HC, human capital method.

In contrast with the HC method and FC approach, the multiplier approach assumes that a worker’s productivity is greater than the person’s wage, given that the absence or reduced at-work productivity of any employee affects the productivity of other employees in a group setting.^30,31 This method may more accurately describe the real-world situation of lost productivity, in which employee replacement may not always be possible, and the absence or reduction in productivity of one employee is likely to adversely affect the productivity of one or more coworkers.

Nicholson et al identified three factors that result in absenteeism and presenteeism costs being greater than an employee’s wage: the employer being unable to find a suitable replacement worker; the fact that production occurs in a team setting; and the fact that there is time sensitivity, such that postponed output will result in a reduction in price or revenue.³⁰

Proposed solution

Lost productivity may be measured using one of three methods, ie, the FC approach, the HC method, or the multiplier method. The most appropriate method depends on the disease, proposed outcomes, and time horizon (eg, whether it is relevant to consider early retirement or disability). When considering mood disorders, where presenteeism represents approximately 80% of lost productivity,¹² the HC method could be used (ie, by assessing the proportion of the employee’s production that is lost). The FC approach does not easily lend itself to the valuation of presenteeism, because the worker is not normally replaced. Rather, one would need to explore the impact on other workers’ productivity, through their compensation for the sick employee’s loss of production. In this situation, the FC approach would be similar to the multiplier approach, but would be data-intensive, requiring job-specific information about the impact on an employee’s coworkers if the person was absent from work or present but suffering from ill health.

Equity considerations

Equity is an important issue in the context of productivity loss valuation. According to economic theory, the gross wage (inclusive of benefits) is an appropriate unit of value to measure productivity. The theory is that companies will continue to employ labor until the marginal cost of that labor is equal to the marginal revenue; hence, the gross wage rate is equal to the productivity gains provided by the marginal employee.¹⁴ There is concern that if wages are used to quantify productivity, evaluators might value the health of high-income earners over that of low-income earners, of workers over non-workers, and of men over women.³² A recent empiric study related to the treatment of borderline personality disorder reported significant variation in productivity costs depending on whether an adjustment was made for those identified as “work-disabled” at the start of the study.³³ Although there may be practical justification for excluding productivity losses of non-workers, doing so would probably result in a higher valuation of treatment aimed at working people. Such an approach would be hard to justify as equitable.

When using the FC approach, the impact on equity considerations is complex, even if average wage rates are used to value production losses, because the estimates of production loss are partly driven by the duration of the friction period. The friction period will vary by job and by industry, but is likely to be shorter for jobs performed by low-paid workers, because such workers tend to be easier to replace. This means that the FC approach will tend to give lower estimates for productivity gains for health care interventions for low-paid workers.

Proposed solution

Equity issues could be avoided by applying a general wage rate for all working individuals, or by reporting productivity gains in non-monetary units (eg, number of days or hours gained as a result of the new treatment).³⁴

Double counting

Empirical research has suggested an association between work productivity and quality of life, such that diminished productivity is associated with lower quality of life.³⁵ In a study that used the EQ-5D visual analog scale (scores ranging from 100 [best imaginable health state] to 0 [worst imaginable health state]), respondents who did not report a loss in productivity following their return to work had an average score of 81, compared with 74 for those who had experienced or were still experiencing reduced productivity following their return.³⁵ Lamers et al³⁶ reported that the correlation between productivity and quality of life was relatively small when comparing the quality of life scores between patients with no reported productivity losses, those with mild efficiency losses at work, and those absent for a short period (the observed average EQ-5D scores were 0.70, 0.68, and 0.65, respectively) (scores on the EQ-5D descriptive system ranging from 1 [best possible health state] to 0 [dead]).

If we assume that individuals take the impact of ill health on income into account when valuing health states, lost productivity will implicitly be included on the outcomes side of a cost-effectiveness ratio; consequently, explicit inclusion of associated costs on the cost side may lead to double counting.³² The Washington Panel on Cost-Effectiveness Analysis explored this idea, and determined that productivity costs were captured in an individual’s assessment of his or her own income losses due to illness. The panel concluded that productivity losses should be captured in the denominator (effects) of a cost-effectiveness analysis.³⁷

Conversely, several studies have shown that individuals do not spontaneously include income effects when directly valuing health states.³⁸ Of seven available studies, six found that 40% or fewer respondents included income effects without explicitly being asked to do so. These results suggest that productivity costs are not captured in health state valuations, and that productivity loss should be included on the cost side of a cost-effectiveness ratio. A more recent online survey of 300 individuals from the Dutch general population investigated whether health state outcome measures already incorporate productivity losses due to illness; it found that 49% of respondents had spontaneously included income effects in health state valuations. However, the authors noted that spontaneous inclusion of income effects did not significantly affect health state valuations at the aggregate level. These findings suggest that the inclusion of productivity costs in the numerator of the cost-effectiveness ratio may not lead to double counting.³⁸

Proposed solution

To ensure that inclusion of monetized productivity costs in health economic analyses is not associated with double counting, Drummond et al and Gold et al recommend “purging” health state valuations of income effects (by eliciting health state preferences under the assumption of full compensation for lost earnings).^32,34 If this approach is not feasible, results should be presented with and without productivity costs.

Conclusion

The importance of productivity loss in mood disorders, combined with the complexity of the issue, means that it is important to be able to make an accurate estimation of presenteeism associated with mood disorders. Ultimately, the perspective and approach taken will depend on the objective of the decision-maker. Nevertheless, to facilitate the assessment of productivity savings in future economic evaluations, research is needed into the validity of existing instruments and algorithms in relevant patient populations. Improvements in measurement methods will help to reduce policymakers’ uncertainty as to whether productivity changes should be included in economic evaluations in the health sector.

More discussion among policymakers is needed to reach some kind of consensus about including productivity changes in economic evaluations. If policymakers are to be convinced of the need to include estimates of productivity changes, further progress in measurement techniques is required. A particular focus should be the measurement of productivity changes due to presenteeism, which is important in conditions such as mood disorders. Since the FC approach is favored by some policymakers, attempts should be made to develop an appropriate FC methodology for estimating productivity costs due to presenteeism.