A Control Engineering Approach for Designing an Optimized Treatment Plan for Fibromyalgia

Abstract

Control engineering offers a systematic and efficient means for optimizing the effectiveness of behavioral interventions. In this paper, we present an approach to develop dynamical models and subsequently, hybrid model predictive control schemes for assigning optimal dosages of naltrexone as treatment for a chronic pain condition known as fibromyalgia. We apply system identification techniques to develop models from daily diary reports completed by participants of a naltrexone intervention trial. The dynamic model serves as the basis for applying model predictive control as a decision algorithm for automated dosage selection of naltrexone in the face of the external disturbances. The categorical/discrete nature of the dosage assignment creates a need for hybrid model predictive control (HMPC) schemes. Simulation results that include conditions of significant plant-model mismatch demonstrate the performance and applicability of hybrid predictive control for optimized adaptive interventions for fibromyalgia treatment involving naltrexone.

I. INTRODUCTION

Conventional medical practice is based on treatment plans designed for a standard response that does not necessarily incorporate individual characteristics or optimization procedures. In behavioral health, the term adaptive interventions is used to describe operationalized, individually-tailored strategies for the prevention and treatment of chronic, relapsing disorders [1]. Instead of relying solely on a clinician’s intuition for assigning dosages, adaptive interventions employ decision rules and repeated assessments of participant response to improve outcomes. Control systems engineering principles applied to adaptive interventions have been proposed as enablers for more efficacious treatments that minimize waste, increase compliance, and enhance intervention potency [2], [3].

Fibromyalgia (FM) is a disorder characterized primarily by chronic widespread pain. Other important symptoms of FM include fatigue, sleep irregularities, bowel abnormalities, anxiety, and mood dysfunction. The causes for FM are uncertain, unknown or disputed; due to its chronic nature, it has been difficult to single out a specific type of treatment for this disease. There is a good evidence to suggest that naltrexone has a neuroprotective role and may be a potentially effective treatment for diseases like FM [4], [5].

This paper is intended to demonstrate how control engineering can positively impact the treatment of fibromyalgia. The approach is based on a secondary analysis of information collected from a previously conducted clinical trial using naltrexone for the treatment of FM. We approach this problem from a systems and controls point-of-view: first, we apply system identification techniques to develop models from daily diary reports completed by intervention participants. These diary reports include self-assessments of outcomes of interest (e.g., general pain symptoms, sleep quality) and additional external variables that affect these outcomes (such as stress, anxiety, and mood). The dynamical systems model serves as the basis for applying model predictive control as a decision algorithm for dosage selection of naltrexone in the face of the external disturbances. The categorical/discrete-event nature of the dosage assignment creates a need for hybrid model predictive control (HMPC) schemes, which we contrast with its continuous counterpart. Instead of conventional tuning using weight matrices, a multiple degree-of-freedom formulation is evaluated that enables the user to adjust the speed of setpoint tracking, measured disturbance rejection and unmeasured disturbance rejection independently in the closed loop system.

The paper is organized in following sections: Section II describes the clinical data and the system identification methodology. In Section III, we show the MPC formulation and present the application of both continuous and hybrid control for delivering adaptive interventions under uncertainty. Summary and conclusions are presented in Section IV.

II. Using System Identification to Model FM Intervention Dynamics

In light of the unknown dynamics of FM, we evaluate an empirical modeling approach where input-output data from a clinical trial is used to build a model describing the effect of drug and external factors on FM symptoms. This model then serves as the basis for control design.

A. The Data

The data for this study has been taken from clinical trials conducted by the Systems Neuroscience and Pain Lab in the Stanford University School of Medicine. The study was conducted in two phases: a single blind pilot study on 10 participants and a double blind full study on 30 participants (with longer protocol). The time series is split into baseline, placebo, drug and washout phases with the number of data points ranging from 98 to 154 sampled daily (T = 1). Participants entered their responses in a handheld computer to questions like “Overall, how well did you sleep last night?” on a scale of 0–100 as well as visited a clinic every two weeks to undergo a series of physical sensory tests. The daily diary data consists of one primary endpoint “Overall, how severe have your FM symptoms been today?” [FM sym] and 13 secondary endpoints: fatigue, sadness, stress, mood, anxiety, satisfaction with life, overall sleep quality, trouble with sleep, ability to think, headaches, average daily pain, highest pain and gastric symptoms [4]. We classify these variables as follows:

Outputs: We are primarily interested in understanding the magnitude and speed at which naltrexone affects various FM symptoms during the intervention. Hence typical symptoms like pain, fatigue, sleep disturbance correspond to dependent variables in the system which we classify as outputs.

Inputs: Drug and placebo are classified as the primary inputs in this analysis, as they are introduced externally to the system and can be manipulated by the clinician. In addition to these primary inputs, there are other exogenous or disturbance variables affecting the outputs. Variables in the self-reports such as anxiety, stress, and mood are treated as measured disturbance inputs that when coupled with the primary inputs can help better explain the output variance and ultimately improve the overall estimated model % fit.

Fig. 1 shows data for selected variables for a representative participant who will be the focus of the results described in the remainder of the paper.

Fig. 1.

Selected variables associated with the naltrexone intervention of FM, as shown for a representative participant. The naltrexone drug concentration was 4.5 mg.

B. System Identification Procedure

The modeling process undertaken in this study can be summarized in three subparts as follows:

1. Data preprocessing. Initially the data is pre-processed for missing entries. To reduce the high frequency content in the time series, a three-day moving average filter is applied.

2. Discrete-time modeling using multi-input ARX models. The filtered data is fitted to a parametric model. We rely on multi-input ARX-[n_a n_b n_k] models

   $$A(q)y(t)=\sum _{i=1}^{n_u}{B}_i(q)u_i(t-n_{k_i})+e(t)$$ (1)

   where n_u represents the number of inputs, n_a, n_b and n_k are model orders, e(t) is the prediction error, and $A(q)=1+{\sum }_{j=1}^{n_a}{a}_j{q}^{-j}$ and $B_i(q)={\sum }_{j=1}^{n_{b_i}}{b}_j{q}^{-j+1}$ are polynomials in q, the forward shift operator. ARX models are computationally simple to estimate and can be consistently estimated provided the inputs are persistently exciting and the model structure is sufficiently high. In our examination of multiple participants, ARX-[441] models were the highest order needed, and in many cases ARX-[2 2 1] models were suitable (as determined by classical prediction-error validation criteria, per [6]).

   The procedure for the choice of input signals is to begin with drug and placebo, which are expected to contribute significantly to FM symptoms for all participants. Additional input variables are then introduced sequentially to improve the goodness of fit. Consequently, while increasing the number of inputs improves the overall fit, an exceptionally high fit may not necessarily imply a highly predictive model. Proper judgement on the choice of input variables that adequately describes the data across all participants must be made. The protocol applied in this study did not allow for a crossvalidation data set.

3. Simplification to a continuous time model. The step responses from the ARX model are individually fit to a parsimonious continuous second-order model structure of the form

   $$G(s)=\frac{K_p({\tau }_{a}s+1)}{{\tau }^2{s}^2+2\zeta \tau s+1}$$ (2)

   From (2) important dynamical system information such as gain, time constant, overshoot, rise and settling times for each input can be obtained.

The use of prediction-error models, and ARX models in particular, is justified because we can rely on well-established bias relations to obtain insight. For purposes of illustration, consider a system described by one manipulated input (e.g., drug), one measured disturbance input (e.g, anxiety) and noise with plant and estimated models as follows:

$$y(t)=p(q)u(t)+p_d(q)d(t)+H(q)\nu (t)$$ (3)

$$=\stackrel{\sim }{p}(q)u(t)+{\stackrel{\sim }{p}}_d(q)d(t)+{\stackrel{\sim }{p}}_e(q)e(t)$$ (4)

The one-step-ahead prediction error can be written as

$$e(t)={\stackrel{\sim }{p}}_e{(q)}^{-1}(y(t)-(\stackrel{\sim }{p}(q)u(t)+{\stackrel{\sim }{p}}_d(q)d(t)))$$ (5)

Using Parseval’s theorem, we can write the filtered prediction-error power spectrum as:

$${\mathrm{\Phi }}_{e_F}(\omega )=\frac{{\mid L\mid }^2}{{\mid p_e\mid }^2}\{{\mid p-\stackrel{\sim }{p}\mid }^2{\mathrm{\Phi }}_u(\omega )+{\mid p_d-{\stackrel{\sim }{p}}_d\mid }^2{\mathrm{\Phi }}_d(\omega )+2Re((p-\stackrel{\sim }{p})(p_d-{\stackrel{\sim }{p}}_d)){\mathrm{\Phi }}_{ud}(\omega )+{\mid H\mid }^2{\sigma }_{\nu }^2\}$$ (6)

where ν(t) is assumed to be uncorrelated with u(t) and d(t); L(q) is the prefilter. From (6) it is possible to obtain insights into how input power, model structure, cross-correlation between signals, and other factors can influence the goodness-of-fit in the identification process.

C. Analysis for a representative participant

In this subsection, we focus on the application of the system identification modeling procedure to a participant from the pilot study with data as seen in Fig. 1. It was noted that inputs power spectrum bandwidth was approximately 0.6 radians/day. As per (6), we minimize Φ_ud (ω) by selecting inputs which are not significantly cross-correlated. For example, headache and gastric variables have high degree of cross-correlation, and are also correlated with the FM symptoms (output). Adding them as inputs did not yield good estimates. In comparison, anxiety and mood variable are essentially uncorrelated and offer good estimates when included as inputs.

The multi-input ARX-[2 2 1] models applied to the representative participant (with respective input(s) and FM symptoms treated as the primary output) are as follows:

1. Model 1 (Drug)

2. Model 2 (Drug, Placebo)

3. Model 3 (Drug, Placebo, Anxiety)

4. Model 4 (Drug, Placebo, Anxiety, Stress)

5. Model 5 (Drug, Placebo, Anxiety, Stress, Mood)

Fig. 2 shows the the step responses resulting for the ARX models for the specific case of the naltrexone drug input. The final model has a gain of −2.47, indicating a nearly 2.5 point drop in the pain report per mg dose of naltrexone. The negative gain for drug allows us to classify this participant as a responder to treatment. A rise time (T_r) of slightly over 5 days, and a 98% settling time (T_s) of nearly 11.5 days characterizes the naltrexone response for this participant. Table I shows how including additional inputs improved the goodness-of-fit for this participant.

Fig. 2.

ARX model step responses for the drug-FM symptoms.

TABLE I.

Model estimate summary for the drug-FM model. % fit corresponds to the multi-input ARX model structure used.

Model	%fit	Kp, τ, ζ, τa	Tr(days)	Ts(days)
1	46.5	−12.03, 5.67, 4.14, 21.3	75.5	139.69
2	59.2	−0.91, 3.5, 2.67, 44.4	0.43	75.06
3	64.7	−1.02, 2.09, 1.5, 15.3	0.43	25.6
4	71.8	−3.11, 1.62, 1.24, 0.22	7.53	14.38
5	73.9	−2.47, 1.57, 1.26, 1.96	5.12	11.49

Fig. 3 shows the corresponding fit for Model 5 which explains 74% of the output variance. Beyond the five inputs noted, adding more variables did not improve the fit significantly and resulted in overparameterization. For some participants, additional inputs like sadness and headache, as well as ARX models with higher orders gave good fits.

Fig. 3.

Comparison of estimated versus measured FM symptoms output for ARX Model 5 (with drug, placebo, anxiety, stress, mood as inputs).

Table II summarizes the transfer functions for all inputs (manipulated and disturbance) for the Model 5 structure. For all these transfer functions the settling times and rise times (with the exception of Mood-FM) are essentially similar. The positive gain for the placebo input indicates that in the case of this participant, the administration of placebo has a detrimental effect. The large magnitude of the placebo gain is in part a consequence of how the input signal is coded (1 when present and 0 when not). Examining the gains for the measured disturbance models (anxiety, stress, and mood), these correspond to 0.86, 2.29, and −0.091, respectively. The positive values for the anxiety and stress gains agree with the clinical observation for how these variables worsen FM symptoms. The low magnitude of the mood gain, coupled with the relatively small contribution of this input to the percent variance described by the model (approximately 2%) indicates the low importance of this variable as a contributor to FM symptoms. Table II also includes the model resulting from the effect of drug to overall sleep. The positive gain in this transfer function demonstrates improved sleep quality with drug administration; however, the fact that τ_a < 0 for this model denotes the presence of inverse response.

TABLE II.

Step response tabulation for various inputs-FM continuous models as well as the drug-overall sleep (Drug-OSleep) model.

Model	Kp, τ, ζ, τa	Tr(days)	Ts(days)
Drug-FM	−2.47, 1.57, 1.26, 1.96	5.12	11.49
Placebo-FM	45.81, 1.57, 1.26, 1.15	6.59	13.06
Anxiety-FM	0.86, 1.57, 1.26, 0.24	7.45	14.24
Stress-FM	2.29,1.57, 1.26, 0.49	7.31	13.94
Mood-FM	−0.091, 1.57, 1.26, 4.67	0.8	11.93
Drug-OSleep	4.98, 2.13, 1.04, −3.35	7.06	15.83

III. Model Predictive Control of a naltrexone intervention for Fibromyalgia

In a control engineering approach to adaptive interventions, the controller assigns dosages to each participant as dictated by model dynamics, problem constraints, and disturbances (both measured and unmeasured). In this work, we use Model Predictive Control (MPC) as the algorithmic framework for making these systematic dosage assignments. This control technology effectively combines feedback-feedforward control action by online optimization of a cost function using a receding horizon and is particularly suited for designing treatment regimens.

An important consideration in adaptive interventions is that intervention dosages can assume only discrete levels, and therefore it is necessary to consider hybrid algorithms [7]. A Mixed Logical Dynamical (MLD) framework is used to represent linear hybrid systems which are systems with real and integer states, inputs and constraints [8] as shown:

$$x(k+1)=Ax(k)+B_{1}u(k)+B_2\delta (k)+B_{3}z(k)+B_{d}d(k)$$ (7)

$$y(k)=Cx(k)+d^{\prime }(k)+\nu (k)$$ (8)

$$E_2\delta (k)+E_{3}z(k)\le E_5+E_{4}y(k)+E_{1}u(k)-E_{d}d(k)$$ (9)

where x and u represent states and inputs of the system. y is the output and d, d′ and ν represent measured disturbances, unmeasured disturbances and measurement noise signals respectively. δ and z are discrete and continuous auxiliary variables that are introduced in order to convert logical/discrete decisions into their equivalent linear inequality constraints. The effect of all unmeasured disturbances is lumped as d′ in the measurement equation. Details of the controller formulation examined in this work (both continuous and hybrid) can be found in [7].

In this work, we rely on a three-degree-of-freedom (3 DoF) approach to tune our controller. The 3 DoF tuning methodology enables performance requirements associated with setpoint tracking, anticipated measured disturbance rejection and unmeasured disturbance rejection to be adjusted independently ([9], [10]) by varying parameters α_r, α_d and f_a respectively. These parameters can be adjusted between values 0 and 1; they in turn alter the response of Type I filter (f(q, α_i)) as

$$f(q,{\alpha }_i)=\frac{(1-{\alpha }_i)q}{q-{\alpha }_i}\forall {\alpha }_i\in [0,1),i=\{r,d\}$$ (10)

which supplies a filtered signal to the controller (for setpoint tracking (α_r) and measured disturbance rejection (α_d)) or adjust the observer gain (K_f) as

$$K_f={\left[\begin{array}{ccc}0& {(f_a)}^2& f_a\end{array}\right]}^T\forall f_a\in (0,1]$$ (11)

for unmeasured disturbance rejection. Hence the controller can be tuned for slower rejection of measured disturbances, e.g., by more extensive filtering of the disturbance signals.

The cost function used in this work can be described as:

$$\underset{\{{[u(k+i)]}_{i=0}^{m-1},{[\delta (k+i)]}_{i=0}^{p-1},{[z(k+i)]}_{i=0}^{p-1}\}}{min}J\triangleq \sum _{i=1}^p{\left|\right|(y(k+i)-y_r)\left|\right|}_{Q_y}^2$$ (12)

such that mixed integer constraints of (9) hold true and:

$$y_{min}\le y(k+i)\le y_{max},1\le i\le p$$ (13)

$$u_{min}\le u(k+i)\le u_{max},0\le i\le m-1$$ (14)

$$\mathrm{\Delta }{u}_{min}\le \mathrm{\Delta }u(k+i)\le \mathrm{\Delta }{u}_{max},0\le i\le m-1$$ (15)

where p is the prediction horizon, m is the control horizon, y_r is the reference and Q_y is the penalty weight on the error.

A. Case Study

In this work, we demonstrate a drug dosage assignment problem on the representative participant from Section II. A continuous model from the estimated ARX Model 5 structure is used as the nominal model. Let the drug dosages be on four levels: u(k) ∈ {0,4.5,9,13.5} mg. The system characterized by discrete inputs can be represented logically as:

$${\delta }_i(k)=1\iff z_i(k)=13.5-(i-1)\times 4.5$$ (16)

$$u(k)=\sum _{i=1}^4{z}_i(k)\sum _{i=1}^4{\delta }_i(k)=1;i\in \{1,2,3,4\}$$ (17)

These conditions and implications (⇔) are then converted into inequality constraints as represented in (9). The control system aims at performing the following three functions:

1. Setpoint tracking. Drug dosages are assigned to take an outcome of interest (such as FM symptoms or overall sleep quality) to a desired goal.

2. Measured disturbance rejection. The controller manipulates drug dosages to mitigate the effect from reported external influences (e.g., anxiety) using estimated disturbance models.

3. Unmeasured disturbance rejection. The controller manipulates drug dosages to mitigate the effect of unknown and unmodeled external influences.

This control system functionality is used to evaluate the nominal performance of the proposed algorithm. For comparative study, we evaluate both hybrid and continuous solutions to the drug assignment problem. The parameters for simulation are as follows: the controller horizons are p = 25 and m = 15, Q_y = 1 and the sampling time is T = 1 day. The sampling time is a potential design variable, as it can be changed for different scenarios (although it is kept constant in this paper). In a real-life setting, patients can enter their daily diary reports to an information system which can supply endpoint values in real-time to the controller.

The control results can be grouped in two categories: 1) evaluation of nominal performance for tracking and disturbance rejection, and 2) evaluation of robust performance under plant-model mismatch. The FM symptoms variable serves as the primary outcome in the analysis, while anxiety (assumed to be reported daily by the participant) serves as the measured disturbance signal. All other disturbances are set equal to zero.

1. Nominal Performance: As a representative example, we consider an intervention that seeks to reduce FM symptoms by 11% from an initial baseline condition of 50%. A step measured disturbance d(k) = 12.9 is applied at day t = 25, while an unmeasured disturbance d′(k) = 11 is applied at day t = 45.

   The 3 DoF formulation allows us to tune the controller independently for setpoint tracking and disturbance rejection as shown in Fig. 4. We evaluate the following tuning parameter value: (α_r, α_d, f_a) = (0.4, 0.4, 0.4) where the controller is de-tuned or the speed of response has been reduced as compared to if no filtering is applied. The hybrid MPC result is compared with respective continuous MPC cases (i.e. where u(t) can take any real value in its domain). For setpoint tracking, α_r can be adjusted to suit the expected response. For example, a clinician may want to see that a desired goal is achieved within a specified time limit. Similarly, the response to disturbances can be varied by α_d and f_a to suit the conditions at hand. By implementing filtering action, the dosage assignments are more gradual in both cases and in the case of hybrid control, dosages are increased in steps. Perfect compensation of measured disturbances is possible in the continuous case by the anticipative feedforward action of the controller but because of the limitations imposed by discrete dosage levels, the hybrid controller is unable to perform in this manner. However, FM symptom deviations from setpoint are not substantial whether u(t) is continuous or discrete and it can noted how the hybrid MPC follows the response from continuous MPC.

2. Robust Performance: The case of plant-model mismatch is presented to demonstrate the robustness of the proposed formulation when there are significant differences between the nominal controller model and the plant. Robust performance is evaluated via simulation for the following cases:

   1. When a fixed nominal model is used for all participants. A nominal model is used as a basis by the controller to assign dosages for different plants. This case can be understood in two ways: first, in the classical interpretation where the estimated model is an approximation of the true system and second, when the nominal model represents an average or representative model, with a single, fixed controller assigning dosages to different participants within this population.

   2. When the true plant serves as the nominal model for each participant. For each scenario considered in the previous case, we supply the true plant as the nominal model to the controller. This case can be understood as when accurate modeling (through system identification or otherwise) has been performed for each individual in a population.

Fig. 4.

Performance of hybrid MPC (four levels) with tuning parameters ((α_r, α_d, f_a) = (0.4, 0.4, 0.4)) for setpoint tracking (t = 0), measured (t = 25) and unmeasured (t = 45) disturbance rejection; compared to continuous MPC.

We compare and contrast the performance between these two scenarios to see the effect of model accuracy for achieving desired performance. The tuning parameters used are (α_r, f_a) = (0.4, 0.4) and are kept constant in all simulations. We present five scenarios of mismatch in the plant dynamics represented by (2), in the presence of both a setpoint change of −8.5% and an unmeasured disturbance d′ = 2.5 occuring at t = 0. We apply no measured disturbance (d = 0). From many possible combinations, we use the following mismatch scenarios: 1) nominal model (no mismatch); 2) ΔK = (−14.8%); 3) ΔK = (14.8%); 4) ΔK,Δζ,Δτ = (−14.8%,−16.6%,259%); 5) ΔK,Δζ,Δτ = (14.8%,79.3%,191%).

We note that under significant model uncertainties, the hybrid controller manages to adapt dosages for best possible performance for both cases under consideration. For mismatch scenario 2 and 3, an offset due to gain mismatch is observed as seen in Fig. 5a. Also for scenario 3, Fig. 5b in the presence of an accurate nominal model, the hybrid controller tries best to minimize error and we see an oscillatory output. For scenario 4, Fig. 5a, we see more oscillatory responses with offset due to reduced ζ. For scenario 5, Fig. 5a, we see more sluggish response due to increased ζ and/or τ. In comparison, when accurate nominal models are available, we observe a much tighter control for scenarios 4 and 5, Fig. 5b. It should also be noted that due to hybrid dynamics, the controller can only choose an input on allowed levels and hence we may observe offset and oscillatory responses. From these results, we conclude that this MPC formulation can show acceptable performance in the presence of model error, although better results can be obtained by increasing model accuracy per participant. The proper tradeoff between modeling effort and controller performance will depend on clinical requirements. Details of robustness analysis with additional cases can be found in [11].

Fig. 5.

Robustness evaluation using parameter perturbations on drug-FM model. Controller tuning parameters are: (α_r, f_a) = (0.4, 0.4).

IV. Summary and Conclusions

In this paper we demonstrate how by using system identification and hybrid MPC, it is possible to design an adaptive intervention that assigns appropriate dosage levels of naltrexone as a treatment for fibromyalgia. The approach described in this work generates models from experimental data and considers hybrid dynamics in an MLD framework. We performed a secondary data analysis to estimate parsimonious models from data available through clinical trials. Since any modeling effort will not result in an exact description of the real system, the controller formulation should be robust enough to handle plant-model mismatch and unknown disturbances. Additionally, successful handling of disturbances or external influences is critical from a standpoint of an effective treatment plan. From simulation results, we show that this MPC setup maintains the outcome variable at goal in presence of disturbance and uncertainties.