Compensator design for improved counterbalancing in high speed atomic force microscopy

Abstract

High speed atomic force microscopy can provide the possibility of many new scientific observations and applications ranging from nano-manufacturing to the study of biological processes. However, the limited imaging speed has been an imperative drawback of the atomic force microscopes. One of the main reasons behind this limitation is the excitation of the AFM dynamics at high scan speeds, severely undermining the reliability of the acquired images. In this research, we propose a piezo based, feedforward controlled, counter actuation mechanism to compensate for the excited out-of-plane scanner dynamics. For this purpose, the AFM controller output is properly filtered via a linear compensator and then applied to a counter actuating piezo. An effective algorithm for estimating the compensator parameters is developed. The information required for compensator design is extracted from the cantilever deflection signal, hence eliminating the need for any additional sensors. The proposed approach is implemented and experimentally evaluated on the dynamic response of a custom made AFM. It is further assessed by comparing the imaging performance of the AFM with and without the application of the proposed technique and in comparison with the conventional counterbalancing methodology. The experimental results substantiate the effectiveness of the method in significantly improving the imaging performance of AFM at high scan speeds.

INTRODUCTION

The ability to image nano-scale features at high imaging rates using AFMs will open many new opportunities by providing the possibility of novel scientific observations. Nano-manufacturing and semiconductor industry would also greatly benefit from such a technology, as it provides the means for online quality control and process refinement. However, these applications have been limited due to the significantly slow speed of the current atomic force microscopes. The typical imaging rate of the current commercial AFMs is in the order of minutes, limiting their application to static or quasi-static observations. The operation of AFM is based on the movement of a probe with respect to sample where an image of the sample surface is obtained on serial-point-collection bases.¹ As such, higher imaging speed necessitates faster movement of the probe/sample. Higher scan speed leads to the excitation of unwanted lateral dynamics of the microscope. In addition, faster movement of the probe with respect to sample requires faster out of plane motion of the sample/probe in order to achieve a constant interaction force between the two, leading to the excitation of the out of plane dynamics of the microscope. The vibrations associated with such dynamics appear as shadows and ripples in the acquired images, severely undermining their reliability. Furthermore, the out of plane dynamics adversely affect the AFM control performance, possibly leading to the instability of AFM scanner movement. To avoid such phenomenon, AFM users are forced to settle with lower controller gains (in the case of commonly used PI controllers) and hence compromise the intended constant interaction force between the probe and the sample.

During the past decade a considerable amount of research effort has focused on the improvement of the AFM imaging speed.^{2, 3, 4, 5, 6} An important thrust towards the design of faster AFMs has been to capture dynamic processes of biological macromolecules,^{2, 3, 4} mineral dissolution and growth⁵ and polymer crystallization⁶ in real time. These efforts have mainly focused on scanner bandwidth enhancement for faster in-plane (referred to as x and y directions),^{7, 8, 9, 10, 11} and out of plane (referred to as z direction) movement^{2, 12} of the sample/probe. Probe enhancement for improved sensitivity and speed has also been under investigation.^{3, 4, 13, 14, 15} The advancements of the scanner speed in x, y, and z directions have been brought about through improvements in both the scanner mechanical design and the scanner control.

The research on the controls approach to improved imaging speed has led to the development of various techniques.¹⁶ To improve the z control Schitter et al.¹² proposed H_∞ feedforward and feedback control schemes. In another effort,² Kodera et al. applied active damping to lower the Q factor of the first z piezo resonance. As reported, through this technique the bandwidth of the z scanner could be extended to the first resonance of the z piezo-actuator. Although, the controls approach to high speed atomic force microscopy has shown to be effective, it also suffers from some limitations. In atomic force microscopy, the topography data are derived from the control signal which imposes constraints on the design of the z control scheme. For high order z-controllers where the dynamics of the AFM structure and the actuator in the z direction are compensated, the topography can no longer be directly derived from the control signal.^{17, 18} In addition, control techniques are unable to extend the bandwidth of AFM scanners beyond their natural mechanical bandwidth. As such, modifying the mechanical design of the scanner is another important approach which should be undertaken in parallel with control methodologies.

A major boost in the scanning bandwidth has been realized through improving the mechanical design of the AFMs. This is achieved by replacing the conventional piezo tubes having relatively low-frequency resonances with piezo stacks^{3, 8, 14, 19} (Figs. 18). In these designs, independent piezo stack actuators move the sample in the x, y, and z directions. In one approach, independent actuators are stacked on top of each other. In this design, the actuations in different directions are completely decoupled, however, the fast scan axis is predefined.^{3, 20} In another class, referred to as rigid scanners,^{19, 21} the three sets of piezo stack actuators for three scan directions (x, y, and z) are implemented on a flexure based nano-positioner.^{14, 19, 21} In this design, the actuators are to a large extent decoupled while operating in a parallel fashion. Hence, the fast scan axis can be chosen freely. Schematic of the piezo stack based rigid scanner design is illustrated in Fig. 1. In this research we build upon the improvements obtained through the piezo stack based rigid scanner designs and propose a technique to further improve the imaging bandwidth. The proposed technique is based on a feed-forward controlled counter actuation (CA) mechanism.

Figure 1.

Schematics of AFM operation (rigid, piezo-stack based design).

Figure 8.

AFM Scanner used in the experiments.

As observed by Ando et al.³ in these scanner designs, the structural resonance in the z direction occurs at a lower frequency compared to the resonance of the z piezo stack itself. Consequently, further improvements in the imaging bandwidth could be obtained by properly compensating for such resonances. A simple approach to suppress the structural vibrations is to use a counterbalancing piezo-actuator.^{2, 3, 21} As shown in Fig. 2a, in this approach, an additional piezo actuator is installed on the opposite side of the z-piezo platform, actuated by the same control signal driving the z-piezo. The role of the additional piezo actuator (hereafter referred to as counter-z or cz piezo) is to counteract the momentum transferred to the base (where the z piezo is mounted, see Fig. 2a) by the z piezo and hence keep the structure stationary.

Figure 2.

(a) A section view of the scanner demonstrating the conventional counterbalancing methodology, (b) a 3 DOF model of the z piezo, counter-z piezo and the base, and (c) block diagram corresponding to part (a).

Although, the counterbalancing technique has proven to be effective, it suffers from some limitations. The main objective of the counterbalancing is to suppress the structural vibrations, i.e., corresponding to the base where the z-piezo sits. However, a more robust approach would be to aim for the compensation of dynamics observed on top of the z-piezo where the sample rests. The counterbalancing technique is not only unable to compensate for the dynamics associated with the z-piezo but also intensifies the corresponding vibrations, potentially leading to the deterioration of imaging performance. Furthermore, the differences between the dynamic properties of z and counter-z piezos, e.g., damping, spring constant, carried masses, etc. would limit the effective cancellation of structural vibrations. This is especially important in practice, as for example, it is not easily possible to alter the mass carried by the cz piezo in accord with the variations of the sample and the sample stage.

This research aims to devise a technique to compensate for these limitations. In the proposed methodology, the signal driving the cz piezo is not the same as the one driving the z piezo. The cz actuation signal is obtained by properly filtering the AFM controller output via a linear filter, hereafter referred to as CA compensator. The CA compensator parameters are estimated through an optimization step.

The paper hereafter is organized as follows. The counterbalancing method is investigated and the corresponding limitations are elaborated in Sec. 2. The proposed feed-forward controlled counter actuation technique is introduced in Sec. 3. This section also presents the CA compensator parameter estimation methodology and the corresponding algorithmic steps. The proposed method is experimentally evaluated in Sec. 4. Concluding remarks are given in Sec. 5.

CONVENTIONAL COUNTERBALANCING: PRINCIPLE OF OPERATION AND LIMITATIONS

Figure 1 schematically presents the basic components of our rigid, piezo stack-based AFM. The x and y piezos move the sample in the lateral (in plane) direction while the z piezo moves the sample in the vertical (out of plane) direction. A complete description on the AFM operation and the corresponding modes can be found in Ref. 22. Figure 2a presents a section view of the rigid scanner where the counter z piezo can also be observed. As explained earlier and as shown in this figure, in the conventional counterbalancing technique both the z and counter z (cz) piezos are driven by the AFM controller output.

Figure 2b demonstrates a 3 DOF model of the counterbalancing mechanism where, (m_b, m_z, and m_cz), (k_b, k_z, and k_cz), and (c_b, c_z, and c_cz), represent the equivalent point masses, spring and damping constants of the base, z piezo and the counter-z piezo (cz piezo), respectively. A block diagram representation of Fig. 2a is shown in Fig. 2c, where G_z(ω) and G_cz(ω) represent the transfer functions from the excitation voltage to the response on top of the z piezo corresponding to the actuation of z and cz piezos, respectively. To give a heuristic explanation on the operation and the limitations of the counter balancing technique we evaluate three transfer functions, (a) Q_z(s) from f_z(t) to h(t) = h_z(t), (b) Q_cz(s) from f_cz(t) to h(t) = h_cz(t), and (c) Q_{z, cz}(s) corresponding to the simultaneous actuation of z and cz piezos, i.e., from f_z(t) = f_cz(t) = f(t) to h(t) = h_{z, cz}(t). The subscripts denote the excited piezo (e.g., z, cz, or both) and the response is always considered on top of the z piezo where the sample rests unless otherwise noted. We have

$$Q_{\mathrm{z}}\left(s\right)=\frac{N_{\mathrm{z}}\left(s,m_{\mathrm{b}},m_{\mathrm{c}z},k_{\mathrm{b}},k_{\mathrm{c}z},c_{\mathrm{b}},c_{\mathrm{c}z}\right)}{D(s,m_{\mathrm{b}},m_{\mathrm{z}},m_{\mathrm{c}z},k_{\mathrm{b}},k_{\mathrm{z}},k_{\mathrm{c}z},c_{\mathrm{b}},c_{\mathrm{z}},c_{\mathrm{c}z})}.$$ (1a)

Figure 3a shows the Bode plot of this transfer function (blue, solid line) for tentative values of dynamic pro- perties, m_b = 6 g, m_cz = m_z = 2g, k_b = 2.5 × 10⁷ N/m, k_z = k_cz = 1 × 10⁸ N/m, ζ_b = 0.03, ζ_z = ζ_cz = 0.06, where ζ denotes the damping ratio. This transfer function features three pairs of complex poles, at 8 kHz, 35 kHz, and 46 kHz, and two pairs of complex zeros at 9 kHz and 41 kHz. The characteristics of the poles and zeros, respectively, at 8 kHz and 9 kHz are mainly ruled by the total mass (m_b + m_z + m_cz) and the base structural properties. The pair of complex poles at 35 kHz is associated with the dynamics of the z piezo and the pairs of zeros and poles, respectively, at 41 kHz and 46 kHz are associated with the addition of the counter-z piezo to the opposite side of the base.

Figure 3.

Bode plot of the transfer functions Q_z(s) (solid blue line), Q_{z, cz}(s) (red dashed line) and Q_cz(s) (green dotted line), corresponding to two cases where (a) the dynamic properties of the z and cz piezos perfectly match, and (b) the dynamic properties of the z piezo are deviated from those of the cz piezo.

For the case where the counter-z piezo is actuated by the same signal as that driving the z-piezo, the transfer function, Q_{z, cz}(s), from the actuation force f_z(t) = f_cz(t) = f(t), to the response of the z piezo h(t) = h_{z, cz}(t), can be written as (for details of Eqs. 1a, 1b see Appendix A.):

$$Q_{\mathrm{z},cz}\left(s\right)=\frac{N_{\mathrm{z},cz}\left(s,m_b,m_{cz},k_z,k_b,k_{cz},c_z,c_b,c_{cz}\right)}{D(s,m_b,m_z,m_{cz},k_b,k_z,k_{cz},c_b,c_z,c_{cz})}.$$ (1b)

As noted above, both the transfer functions Q_z(s) and Q_{z, cz}(s) have a common denominator, D(s). In other words, actuation of the counter-z piezo does not affect the location of the poles. Furthermore, the numerator of both Q_z(s) and Q_{z, cz}(s)are independent of the mass carried by the z piezo, i.e., zero locations are not affected by the mass of the z-piezo. The operation of the conventional counterbalancing technique can be explained based on the above-mentioned properties and the Bode plots of Fig. 3. For the case where the dynamic properties of the z and the cz piezos are similar (balanced case), the simultaneous actuation of the cz piezo moves the two pairs of zeros to match the pairs of poles at 8 kHz and 46 kHz, cancelling both pairs. Now consider a case where the equivalent point mass of the z piezo is deviated from that of cz piezo (unbalanced case). According to the above, although changing the z piezo mass shifts the poles of Q_{z, cz}(s) away from their original locations, the zeros of Q_{z, cz}(s) remain intact. As such, the pole-zero cancellation does not take place and hence counterbalancing loses effectiveness. Though interpreting the effect of deviating the spring constant and the damping characteristics of the z piezo from those of the cz piezo is not as straightforward, a similar argument applies. To demonstrate the effect of mass unbalance, the mass of the z piezo is deviated from that of the cz piezo, $m_{\mathrm{z}}^{\prime }=2m_{\mathrm{z}}$ = 4g. Considering the small weight of the piezo stack actuators used in high speed rigid scanners, the sample and sample stage masses are comparable to the z piezo mass. Hence, the given deviation is realistic. The Bode plots associated with balanced and unbalanced cases corresponding to Q_z(s) and Q_{z, cz}(s) (solid blue and dashed red lines, respectively) are shown in Figs. 3a, 3b, respectively.

Another limitation of the counterbalancing approach is the intensification of the vibrations associated with the z piezo dynamics. This is true even when the dynamic propertied of the z and cz piezos are similar. This phenomenon can be observed from the Bode plot of Q_{z, cz}(s) in both Figs. 3a, 3b (red dashed line) where the magnitude of the peak associated with the poles at 35 kHz (corresponding to the z-piezo dynamics) is increased. The Bode plots of Q_cz(s) corresponding to the balanced and unbalanced cases are also shown in Figs. 3a, 3b (green dotted line). As evident from these Bode plots, on a frequency range bounded by the zeros of Q_z(s), Q_cz(s) is in phase with Q_z(s). As such, on this frequency range the dynamics induced by the actuation of z and cz piezos are superposed leading to the intensification of the corresponding vibrations. This phenomenon may lead to inferior AFM performance, especially when the dynamic properties of the z and the counter-z piezos do not match. In this case, the counterbalancing technique is not only unable to properly suppress the base structural vibrations but also intensifies the dynamics associated with the z piezo.

THE NOTION OF COUNTER ACTUATION COMPENSATOR

In the following, we propose a method to compensate for the limitations of the conventional counterbalancing explained in Sec. 2. The method consists of generating a more effective control signal for the counter piezo actuator. This is shown in Fig. 4a. The output signal from the controller is processed through a compensator that generates the proper input for the cz piezo. This filtering step aims to compensate for the adverse effect of the unbalance between the mechanical properties of the z and cz piezos in suppressing the base structural vibrations. Furthermore, the proposed compensator effectuates the suppression of the dynamics associated with the z piezo actuator. Figure 4b demonstrates a block diagram representation of this technique.

Figure 4.

(a) Schematics of the proposed counter actuation based on CA compensator, and (b) the corresponding block diagram.

CA compensator structure and design strategy

Consider a filter structure k(q, θ) = B(q, θ_b)/A(q, θ_a), a discrete representation of k(ω, θ), with θ a p-parameter vector, θ = {θ_a, θ_b}, θ_a = {a₁, a₂, …, a_n}, θ_b = {b₁, b₂, …, b_m}, and p = m + n, where A(q) = 1 + a₁q⁻¹ + ⋅⋅⋅ + a_nq⁻ⁿ and B(q) = b₁ + b₂q⁻¹ + ⋅⋅⋅ + b_mq^−m. q is a forward shift operator such that for a discrete time signal x(n + 1) = qx(n) and similarly x(n + 1) = qx(n). Also denote the discrete equivalents of G_{z, cz}(ω), G_z(ω), and G_cz(ω) byG_{z, cz}(q), G_z(q), and G_cz(q), respectively.

The discrete frequency response of the system shown in Fig. 4b is

$$G_{\mathrm{z},cz}\left(e^{j\omega },\theta \right)=\frac{H_{\mathrm{z},cz}(e^{j\omega },\theta )}{V\left(e^{j\omega }\right)},$$

where H_{z, cz}(e^jω, θ) and V(e^jω) denote the frequency domain representations of discrete data h_{z, cz}(n, θ) and ν(n), respectively. ν(n) is a discrete actuation voltage band-limited to [0, ω₀] and h_{z, cz}(n, θ) is the z piezo response measured via the cantilever deflection when both z and cz piezos are active. Obviously, this transfer function is dependent on the parameter vector θ. To emphasize this, we have included θ as an argument in G_{z, cz}(q, θ). We can write

$$G_{\mathrm{z},cz}\left(q,\theta \right)=G_{\mathrm{z}}\left(q\right)+k\left(q,\theta \right)G_{\mathrm{c}z}\left(q\right).$$ (2)

Also consider the ideal response of the scanner when the z piezo is actuated with the voltage, v(n), (when the cz piezo is inactive) as β* v(n), and the associated transfer function as

$$G_{\mathrm{z}}^{\mathrm{i}deal}\left(q\right)={\beta }^{*},$$

where β* is an unknown scaling parameter. Furthermore, let us define a transfer function, $G_{\mathrm{z}}^{\mathrm{d}ev}(q,\theta )$, hereafter referred to as ideal deviation transfer function, representing the deviation of the z piezo transfer function G_z(q) from a uniform frequency response over [0, ω₀] as

$$G_{\mathrm{z}}^{\mathrm{d}ev}\left(q\right)=G_{\mathrm{z}}\left(q\right)-G_{\mathrm{z}}^{\mathrm{i}deal}\left(q\right)=G_{\mathrm{z}}\left(q\right)-{\beta }^{*},$$ (3)

or equivalently

$$G_{\mathrm{z}}\left(q\right)=G_{\mathrm{z}}^{\mathrm{i}deal}\left(q\right)+G_{\mathrm{z}}^{\mathrm{d}ev}\left(q\right).$$ (4)

Substituting Eq. 4 into Eq. 2 gives

$$G_{\mathrm{z},cz}\left(q,\theta \right)=G_{\mathrm{z}}^{\mathrm{i}deal}\left(q,\theta \right)+G_{\mathrm{z}}^{\mathrm{d}ev}\left(q\right)+k\left(q,\theta \right)G_{\mathrm{c}z}\left(q\right).$$ (5)

A block diagram representation of this equation is given in Fig. 5. As evident from this figure, for a compensator parameter vector, θ = θ*, selected in such a way that h_cz(n, θ*) can suppress $h_{\mathrm{z}}^{\mathrm{d}ev}\left(n\right)$, then h_{z, cz}(n, θ*) would represent the ideal response, $h_{\mathrm{z}}^{\mathrm{i}deal}\left(n\right)$. By extracting the portion of Fig. 5 enclosed by dashed line and rearranging the transfer functions k and G_cz, the process of compensator parameter estimation can be schematically stated as shown in Fig. 6.

Figure 5.

Block diagram representation of Eq. 5.

Figure 6.

CA compensator parameter estimation by minimizing {e(n, θ)} representing the deviation of the z piezo response from ideal when both z and cz piezos are actuated.

In other words, given two datasets {h_cz(n); n = 1, …, N} and $\left\{h_{\mathrm{z}}^{\mathrm{d}ev}\left(n\right);n=1,...,N\right\}$ and with

$$e\left(n,\theta \right)=h_{\mathrm{z}}^{\mathrm{d}ev}\left(n\right)+k\left(q,\theta \right)h_{\mathrm{c}z}\left(n\right),$$ (6)

$$V\left(\theta \right)=\frac{1}{N}\sum _{n=1}^N{\left(e\left(n,\theta \right)\right)}^2,$$ (7)

we have

$$\widehat{\theta }=\arg \underset{\theta }{min}\left(V\left(\theta \right)\right),$$ (8)

where $\widehat{\theta }$ denotes the estimated CA compensator parameter vector, h_cz(n) is the z piezo response to counter-z piezo actuation, and $h_{\mathrm{z}}^{\mathrm{d}ev}\left(n\right)$ is the deviation of the z piezo response from the ideal response (ideal deviation response) in the absence of cz actuation. h_cz(n) can be measured directly by exciting the cz piezo and measuring the cantilever deflection response. However, $h_{\mathrm{z}}^{\mathrm{d}ev}\left(n\right)$ should be extracted from h_z(n). The process of extracting the ideal deviation response is given in Appendix B. The order of the compensator can be selected by comparing the minimum obtainable cost function as given by Eq. 7 for various orders while taking into account the imposed hardware limitations.

Denoting $y\left(n\right)=h_{\mathrm{z}}^{\mathrm{d}ev}\left(n\right)$ and u(n) = −h_cz(n), the objective is to estimate y(n) in the least square error sense (LS) by properly filtering u(n). Hence, the CA compensator parameter estimation methodology given by Eqs. 6, 7, 8, can be interpreted as an output-error system identification problem.²³ As such, hereafter we refer to the datasets u(n) and y(n) as virtual input and virtual output, respectively. The flowchart of the proposed compensator design technique is given in Fig. 7.

Figure 7.

Flowchart of the proposed compensator design technique.

Pre-processing the virtual input, u(n), and virtual output, y(n)

The CA compensator design based on the raw datasets u(n) and y(n) is not an appropriate strategy and proper pre-processing steps are required to assure acceptable performance. A frequency domain interpretation of the LS based compensator design (Eq. 8) can help us to design necessary pre-processing steps. Let us assume that there exists an objective transfer function k(q, θ*), such that

$$y\left(n\right)=k\left(q,{\theta }^{*}\right)u\left(n\right),$$ (9)

leading to zero cost function, i.e., V(θ) = 0. We can show that²³ (Appendix C):

$$\begin{array}{ccc}\hfill V\left(\theta \right)& \approx & \frac{1}{N}\sum _{n=1}^N{\left|\frac{Y\left(2\pi n/N\right)}{U\left(2\pi n/N\right)}-k\left(2\pi n/N,\theta \right)\right|}^2{\left|U\left(2\pi n/N\right)\right|}^2,\hfill \\ & \approx & \frac{1}{N}\sum _{n=1}^N|k\left(2\pi n/N,{\theta }^{*}\right)\hfill \\ & & {\left.-k\left(2\pi n/N,\theta \right)\right|}^2{\left|U\left(2\pi n/N\right)\right|}^2.\hfill \end{array}$$ (10)

According to Eq. 10, minimizing (Eq. 8) the sum square error given by Eq. 7 is approximately equivalent to the minimization of the weighted sum of the squared error between the estimated transfer function $k\left(2\pi n/N,\widehat{\theta }\right)$ and the objective transfer function k(2πn/N, θ*). The weight given to each frequency is determined by the energy of u(n), i.e., |U(2πn/N)|² at that specific frequency. The above result has a very important implication on the performance of the CA compensator.

Firstly, the Y(2πn/N)/U(2πn/N) ratio would be of unlimited bandwidth as the compensator aims to achieve ideal response in the output. According to Eq. 10, this will result in infinite bandwidth of the resulting compensator leading to increased image noise. Furthermore, the weighting function, |U(2πn/N)|², needs to be modified to further suppress the low SNR (signal to noise ratio) frequency bands of the acquired data. Assuming that both the datasets, u(n) and y(n), are corrupted by additive white noise, the Y(2πn/N)/U(2πn/N) ratio has a higher SNR over the frequency range where the virtual input has a higher energy. The factors that determine the spectral energy distribution of u(n) are twofold, (a) the frequency content of the excitation voltage, ν(n), and (b) the frequency range on which the transfer function, G_cz(q), has dominant response. As shown in Appendix D, G_cz(q) has bandpass characteristics. The passband of this transfer function is [ω_b, ω_z], where ${\omega }_{\mathrm{z}}\approx \sqrt{k_{\mathrm{z}}/m_{\mathrm{z}}}$ and ${\omega }_{\mathrm{b}}\approx \sqrt{k_{\mathrm{b}}/m_{\mathrm{b}}+m_{\mathrm{c}z}+m_{\mathrm{z}}}$ are the natural frequencies associated with the z piezo and the base structure, respectively. This result matches the simulation results of Fig. 3 (green dotted line). As such, assuming a wideband actuation signal ν(n), the high energy frequency range of the virtual input, u(n), falls in [ω_b, ω_z].

The weighting function modification can be implemented via a bandpass filtering step. Bandpass filtering the u(n) and y(n) by an identical filter F(2πn/N), will not affect the ratio Y(2πn/N)/U(2πn/N).²³ As such, it can be observed that via filtering, the frequency weighting function is modified to |U^′(2πn/N)|² = |U(2πn/N)F(2πn/N)|². The lower and upper cutoff frequencies of the bandpass filter can be roughly selected based on the prior knowledge about the resonance frequencies of the base structure and the z piezo, i.e., ([ω_b, ω_z]) or by directly inspecting the spectral content of u(n). As such, through bandpass filtering, the weighting function can be decreased to a small value out of the band [ω_b, ω_z] forcing the compensator to match only the in-band behavior of the objective transfer function. However, one should note that following the bandpass filtering step the out-of-band behavior of the compensator is still unknown. This may lead to unacceptable practical results. For example a high gain at low frequency range may lead to the saturation of the digital filter. This can be avoided by penalizing the out of band compensator gain as given below:

$$\begin{array}{ccc}\hfill V\left(\theta \right)& \approx & \frac{1}{N}\left(\sum _{n\in \mathrm{i}nband}\left|\frac{Y\left(2\pi n/N\right)}{U\left(2\pi n/N\right)}\right.\right.\hfill \\ & & {\left.-k\left(2\pi n/N,\theta \right)\right|}^2{\left|U\left(2\pi n/N\right)\right|}^2\hfill \\ & & \left.+\sum _{n\in outofband}{\left|k\left(2\pi n/N,\theta \right)\right|}^2{\left|{\lambda }_n\right|}^2\right),\hfill \end{array}$$

where λ_n is the penalizing gain for frequency 2πn/N. The penalizing term forces roll off of the compensator towards higher, ω > ω_z, and lower, ω < ω_b, frequencies. A time domain interpretation of the penalizing term and a schematic of the proposed compensator design technique are given in Appendix E.

EXPERIMENTAL EVALUATION

In this section, we evaluate the effectiveness of the proposed method via experiments. This evaluation is in terms of dynamic performance behavior of the scanner and imaging capability.

Experimental setup

The AFM used in this research is described in Ref. 21 and is shown in Fig. 8. The sample movement in the x, y, and z directions is provided by piezo stack actuators. As shown in the figure, the scanner motion in each one of the x and y directions is achieved by two piezos in a pull-push configuration. This is primarily because the piezo stacks are able to provide force when expanding. In this scanner design, ball bearings are used to decouple the motion between the x and y piezos. Furthermore, the scanner center stage, which hosts the z and counter-z piezos, is enclosed by a flexing frame in order to limit the out of plane vibrations. The counter z piezo is directly underneath the z piezo platform. In these experiments, a micro-cantilever, model TM300-A from SensaProbes, with 300 kHz resonance frequency is brought in contact with a mica sample. The cantilever deflection is the only measured signal through an optical system, hence eliminating the need for any additional sensors. This characteristic of the proposed technique makes it quite versatile and implementable on the AFM scanners without any modifications. To observe the effectiveness of the proposed scheme, in the following experiments the dynamic unbalance between the z and the counter piezos is increased by resting a mass of 1 g on the sample holder.

CA compensator design

The z and counter-z piezos are excited independently with a wideband random binary sequence excitation voltage. The resulting deflection signal is acquired at 250 kHz sampling rate to measure h_z(n) and u(n) = −h_cz(n), respectively. The empirical transfer function estimate (ETFE) of the measured data, h_z(n) and u(n), are presented in Figs. 9a, 9b, respectively. The scaling parameter β* is then estimated and the ideal deviation response y(n) is extracted from the deflection signal h_z(n)using Eqs. B3, B4 (Appendix B). From Fig. 9b the high SNR frequency band of u(n) is detected to be around 4–73 kHz (enclosed by vertical lines). As such, both y(n) and u(n) are bandpass filtered using an FIR filter designed based on Parks-McClellan filter design algorithm,²⁴ with a passband of 4 kHz–73 kHz to obtain y^′(n), u^′(n). Harmonics are then added to u^′(n) to obtain u_f(n) as suggested by Eq. E2 (Appendix E) to force the out of band roll-off of the compensator. The datasets y^′(n) and u_f(n) are then used to estimate a 10th order linear compensator via Eq. 8. The designed compensator is then implemented on a NI PXI-7851R FPGA in transposed, direct-form II, second order sections format.²⁵ The Bode plot of the designed CA compensator is illustrated in Fig. 10.

Figure 9.

Smoothed empirical transfer function estimates with the excitation voltage to (a) z piezo, and (b) cz piezo, as input and the measured deflection signal as output.

Figure 10.

Bode plot of the designed CA compensator.

Dynamic performance

In this subsection, the dynamic performance of the scanner, in the presence of the compensator, is evaluated. To assess the performance of the compensator, the z piezo is excited by step inputs and the deflection signal is measured for two separate cases as shown in Figs. 11a, 11b. Figure 11a shows the difference in performance between the conventional counterbalancing method in blue (solid line) and no-balancing in black (dotted line). The figure shows that there are two dominant modes of vibration as predicted by the ETFE of Fig. 9a. One mode is at 31 kHz, due to the z piezo dynamics, and the other at 7.7 kHz which is related to the base structure dynamics. The response with no balancing, in black (dotted line), shows the worst performance. As shown in Fig. 11a (blue, solid line), with the application of counterbalancing although there is a slight reduction on the level of vibrations associated with the lower frequency mode (base structural dynamics), the higher frequency mode (z piezo dynamics) of oscillations are intensified. On the other hand, Fig. 11b shows in red (solid line), the response corresponding to the case where the CA compensator is applied. A significant decrease in the level of vibrations is observed. This improvement in performance is attributed to the cancellation of the vibrations associated with the base (7.7 kHz) as well as significant reduction in the z piezo dynamics (31 kHz).

Figure 11.

Deflection signal in response to an step signal actuating the z piezo, (a) black, dotted line: no counter-balancing is used; blue, solid line: the cz piezo is driven by the same signal as that driving the z piezo, and (b) black, dotted line: no counter-balancing is used; red, solid line: the CA compensator is applied.

Figure 12 presents the empirical transfer function estimates associated with the cases described in Fig. 11. One can observe that the conventional counter balancing approach, in blue (dashed line), reduces slightly the peaks associated with the low frequency resonance and anti-resonance. However, the higher frequency modes are intensified. These behaviors of the conventional counter balancing approach were predicted analytically in Sec. 2. The same figure shows a significant improvement in performance due to the CA compensator, in red (solid line). This is characterized by a much higher bandwidth and proper compensation for the induced vibrations due to z piezo and the scanner base.

Figure 12.

Empirical transfer function estimates associated with the three cases represented in Fig. 11, black dotted line: the cz piezo is not active, blue dashed line: the cz piezo is driven by the same signal as that driving the z piezo (conventional counterbalancing), red solid line: the CA compensator is applied.

Imaging performance

In this section, the performance of the proposed method is evaluated by using the AFM to image several samples of freshly cleaved and sanded mica. Figure 13 show a typical result of the topography and the deflection images taken at 60 lines/sec when the conventional counterbalancing (left) and the designed CA compensator (right) are applied. One can see from Fig. 13 (right) that with the application of the CA compensator the shadows and ripples appearing in the left-side, error and height images, are completely removed, significantly improving the quality of the images.

Figure 13.

Topography (top) and deflection (bottom) AFM images, taken from freshly cleaved and sanded mica at 60 lines/sec with conventional counterbalancing technique (left), and CA compensator applied (right), for similar PID gains.

CONCLUSION

The nanoscale resolution and versatility of atomic force microscope makes it an ideal candidate for real-time observation of nanoscale processes. However, to this end the limited imaging speed of these devices needs to be improved. The new generation of rigid nano-positioners has shown great capabilities for high speed AFM imaging. The counterbalancing technique, currently implemented in these scanners, aims to suppress the structural vibrations and hence improve the image quality at high speeds. However, this technique is built upon the assumption of similar mechanical properties of the z and counter-z piezos. This requirement is difficult to maintain in practice. Furthermore, even at its highest performance capabilities, the counterbalancing technique not only fails to compensate for the z piezo dynamics but also can intensify such vibrations, potentially leading to inferior imaging performance. In this research we illustrated that by proper actuation of the counter-z piezo one can compensate for these limitations. This was achieved by modifying the AFM controller output via a CA compensator. The compensator parameters were estimated based on the measured deflection signals, eliminating the need for any additional sensors. The proposed technique was evaluated on the dynamic response of the z piezo as well as the images obtained from several samples of freshly cleaved and sanded mica featuring sharp profile edges. These results corroborated the effectiveness of the proposed technique in significantly improving the image quality at high scan speeds.

APPENDIX A: DETAILS ON EQS. 1a, 1b

$$Q_{\mathrm{z}}\left(s\right)=\frac{N_z\left(s,m_{\mathrm{b}},m_{\mathrm{c}z},k_{\mathrm{b}},k_{\mathrm{c}z},c_{\mathrm{b}},c_{\mathrm{c}z}\right)}{D(s,m_{\mathrm{b}},m_{\mathrm{z}},m_{\mathrm{c}z},k_{\mathrm{b}},k_{\mathrm{z}},k_{\mathrm{c}z},c_{\mathrm{b}},c_{\mathrm{z}},c_{\mathrm{c}z})},$$

where,

$$\begin{array}{ccc}\hfill N_{\mathrm{z}}\left(s\right)& =& \left[m_{\mathrm{b}}{s}^2+\left(c_{\mathrm{b}}+c_{\mathrm{c}z}\right)s+(k_{\mathrm{b}}+k_{\mathrm{c}z})\right]\hfill \\ & & \times \left(m_{\mathrm{c}z}{s}^2+c_{\mathrm{c}z}s+k_{\mathrm{c}z}\right)-{\left(c_{\mathrm{c}z}s+k_{\mathrm{c}z}\right)}^2,\hfill \\ \hfill D\left(s\right)& =& \left(m_{\mathrm{z}}{s}^2+c_{\mathrm{z}}s+k_{\mathrm{z}}\right)[m_{\mathrm{b}}{s}^2+\left(c_{\mathrm{z}}+c_{\mathrm{b}}+c_{\mathrm{c}z}\right)s\hfill \\ & & +(k_{\mathrm{z}}+k_{\mathrm{b}}+k_{\mathrm{c}z})]\left(m_{\mathrm{c}z}{s}^2+c_{\mathrm{c}z}s+k_{\mathrm{c}z}\right)\hfill \\ & & -{\left(c_{\mathrm{c}z}s+k_{\mathrm{c}z}\right)}^2\left(m_{\mathrm{z}}{s}^2+c_{\mathrm{z}}s+k_{\mathrm{z}}\right)\hfill \\ & & -{\left(c_{\mathrm{z}}s+k_{\mathrm{z}}\right)}^2\left(m_{\mathrm{c}z}{s}^2+c_{\mathrm{c}z}s+k_{\mathrm{c}z}\right),\hfill \end{array}$$

and

$$Q_{z,cz}\left(s\right)=\frac{N_{z,cz}\left(s,m_{\mathrm{b}},m_{\mathrm{c}z},k_{\mathrm{z}},k_{\mathrm{b}},k_{\mathrm{c}z},c_{\mathrm{z}},c_{\mathrm{b}},c_{\mathrm{c}z}\right)}{D(s,m_{\mathrm{b}},m_{\mathrm{z}},m_{\mathrm{c}z},k_{\mathrm{b}},k_{\mathrm{z}},k_{\mathrm{c}z},c_{\mathrm{b}},c_{\mathrm{z}},c_{\mathrm{c}z})},$$

where,

$$\begin{array}{ccc}\hfill N_{\mathrm{z},cz}\left(s\right)& =& \left[m_{\mathrm{b}}{s}^2+\left(c_{\mathrm{b}}+c_{\mathrm{c}z}\right)s+(k_{\mathrm{b}}+k_{\mathrm{c}z})\right]\hfill \\ & & \times \left(m_{\mathrm{c}z}{s}^2+c_{\mathrm{c}z}s+k_{\mathrm{c}z}\right)-{\left(c_{\mathrm{c}z}s+k_{\mathrm{c}z}\right)}^2\hfill \\ & & +m_{\mathrm{c}z}{s}^2\left(c_{\mathrm{z}}s+k_{\mathrm{z}}\right).\hfill \end{array}$$

APPENDIX B: EXTRACTION OF THE IDEAL DEVIATION RESPONSE, y(n) FROM THE MEASURED DEFLECTION SIGNAL, h_z(n)

As mentioned in Sec. 3 the ideal response of the z piezo can be written as a scaled version of the input voltage, i.e., β* v(n). Hence, for a given scaling parameter β* and the excitation voltage v(n), the ideal deviation response of the z piezo can be obtained as

$$y\left(n\right)=\left(G_{\mathrm{z}}\left(q\right)-{\beta }^{*}\right)v\left(n\right).$$

As such, to extract the ideal deviation response the scaling parameter β* should be calculated. This can be done by evaluating the mean of the truncated impulse response associated with G_z(q). We denote the truncated impulse response associated with the transfer function G_z(q) by an L × 1 vector g_z. Then we can write:

$$h_{\mathrm{z}}\left(n\right)=g_{\mathrm{z}}^{*}v\left(n\right),$$ (B1)

where * reflects the convolution operation. From the above equation²³ for the given dataset h_z(n) and RBS excitation signal v(n), n = 1, …, N, the impulse response, g_z, can be obtained through the following:

$$g_{\mathrm{z}}=R_v^{-1}{R}_{hv},$$ (B2)

where R_v is the autocorrelation matrix for the dataset v(n) and R_hv the cross correlation matrix for v(n) and h_z(n). Then we can write:

$$\widehat{\beta }=\sum _{n=1}^L{g}_{\mathrm{z}}\left(n\right),$$ (B3)

where $\widehat{\beta }\approx {\beta }^{*}$ is an estimate of the scaling parameter. Hence, we have

$$y\left(n\right)=h_{\mathrm{z}}\left(n\right)-\widehat{\beta }v\left(n\right).$$ (B4)

APPENDIX C: FREQUENCY DOMAIN INTERPRETATION OF THE LEAST SQUARES BASED COMPENSATOR PARAMETER ESTIMATION

In the following we give a frequency domain interpretation of the CA compensator parameter estimation based on Eq. 8. Given two datasets {u(n); n = 1, …, N} and {y(n); n = 1, …, N}, with {e_θ(n); n = 1, …, N} given by Eq. 6, we can write:

$$E_{\theta }\left(\omega =2\pi n/N\right)=\frac{1}{\sqrt{N}}\sum _{k=1}^N{e}_{\theta }\left(n\right)e^{-2\pi kni/N}.$$ (C1)

Then from the Parseval's theorem²³ we have

$$\begin{array}{c}\hfill V_{N,\theta }=\frac{1}{N}\sum _{n=1}^N{e}_{\theta }^2\left(n\right)=\frac{1}{N}\sum _{n=1}^N{\left|E_{\theta }\left(\omega =2\pi n/N\right)\right|}^2.\end{array}$$ (C2)

On the other hand, denoting $w\left(n\right)=k\left(q,\widehat{\theta }\right)u\left(n\right)$, one can show²³

$$\begin{array}{c}\hfill W\left(2\pi n/N\right)=k\left(2\pi n/N,\widehat{\theta }\right)U\left(2\pi n/N\right)+R\left(2\pi n/N\right),\end{array}$$ (C3)

where W and U are the DFT representations of u(n) and w(n), given by $U\left(2\pi n/N\right)=1/\sqrt{N}{\sum }_{k=1}^{N}u\left(k\right)e^{-2\pi kni/N}$ and $W\left(2\pi n/N\right)=1/\sqrt{N}{\sum }_{k=1}^{N}w\left(k\right)e^{-2\pi kni/N}$, respectively, and $\left|R\left(2\pi n/N\right)\right|\le C/\sqrt{N}$ where c is a constant. From Eqs. 6, C3, for large datasets, i.e., N ≫ 1 Eq. C1 can be written as

$$\begin{array}{c}\hfill E_{\theta }\left(\omega =2\pi n/N\right)\approx Y\left(2\pi n/N\right)-k\left(2\pi n/N,\widehat{\theta }\right)U\left(2\pi n/N\right).\end{array}$$ (C4)

Substituting Eq. C4 into Eq. C2, we have

$$\begin{array}{ccc}\hfill V_{N,\theta }& \approx & \frac{1}{N}\sum _{n=1}^N\left|Y\left(2\pi n/N\right)\right.\hfill \\ & & -k\left(2\pi n/N,\widehat{\theta }\right)U\left(2\pi n/N\right){|}^2.\hfill \end{array}$$ (C5)

On the other hand, similar to Eq. C3, for Eq. 9 we can write²³

$$Y\left(2\pi n/N\right)=k\left(q,{\theta }^{*}\right)U\left(2\pi n/N\right)+R\left(2\pi n/N\right),$$

which for large N can be written as

$$k\left(q,{\theta }^{*}\right)\approx \frac{Y\left(2\pi n/N\right)}{U\left(2\pi n/N\right)}.$$ (C6)

Using Eq. C6 in Eq. C5, we have

$$\begin{array}{ccc}\hfill V_{N,\theta }& \approx & \frac{1}{N}\sum _{n=1}^N{\left|\frac{Y\left(2\pi n/N\right)}{U\left(2\pi n/N\right)}-k\left(2\pi n/N,\widehat{\theta }\right)\right|}^2{\left|U\left(2\pi n/N\right)\right|}^2,\hfill \\ & \approx & \frac{1}{N}\sum _{n=1}^N{\left|k\left(q,{\theta }^{*}\right)-k\left(2\pi n/N,\widehat{\theta }\right)\right|}^2{\left|U\left(2\pi n/N\right)\right|}^2.\hfill \end{array}$$ (C7)

APPENDIX D: THE BANDWIDTH OF THE TRANSFER FUNCTION FROM THE cz ACTUATION TO z PIEZO RESPONSE

In the following we provide an approximate evaluation of the bandwidth of the transfer function, P(s) = H_cz(s)/D_cz(s), from the counter z piezo elongation to the vibration response of the z piezo, h_cz(t), where $d_{\mathrm{c}z}\left(t\right)=h_{\mathrm{c}z}^{\prime }\left(t\right)-x\left(t\right)$ (see Fig. 2b) and f_z(t) = 0. As such, it is inherently assumed that the counter-z piezo has a bandwidth on the same order or wider than that of the z-piezo.

Referring to the 3 DOF model of Fig. 2b, we can write

$$m_{\mathrm{c}z}{\ddot{h}}_{\mathrm{c}z}^{\prime }=-f_{\mathrm{c}z}\left(t\right)+k_{\mathrm{c}z}\left(x-h_{\mathrm{c}z}^{\prime }\right)+c_{\mathrm{c}z}\left(\dot{x}-{\dot{h}}_{\mathrm{c}z}^{\prime }\right),$$ (D1)

$$\begin{array}{ccc}\hfill m_{\mathrm{b}}\ddot{x}& =& f_{\mathrm{c}z}\left(t\right)-k_{\mathrm{c}z}\left(x-h_{\mathrm{c}z}^{\prime }\right)-k_{\mathrm{b}}x+k_{\mathrm{z}}\left(h_{\mathrm{c}z}-x\right)\hfill \\ & & +c_{\mathrm{z}}\left({\dot{h}}_{\mathrm{c}z}-\dot{x}\right)-c_{\mathrm{b}}\dot{x}-c_{\mathrm{c}z}\left(\dot{x}-{\dot{h}}^{\prime }\right),\hfill \end{array}$$ (D2)

$$m_{\mathrm{z}}{\ddot{h}}_{\mathrm{c}z}=-k_{\mathrm{z}}\left(h_{\mathrm{c}z}-x\right)-c_{\mathrm{z}}\left({\dot{h}}_{\mathrm{c}z}-x\right),$$ (D3)

where f_cz(t) represents the internal expansion force generated by the counter-z piezo.

From D1, D2 we can write:

$$X\left(s\right)=\frac{\left(k_{\mathrm{z}}+c_{\mathrm{z}}s\right)H_{\mathrm{c}z}\left(s\right)-m_{\mathrm{c}z}{s}^2{D}_{\mathrm{c}z}\left(s\right)}{\left(m_{\mathrm{b}}+m_{\mathrm{c}z}\right)s^2+\left(c_{\mathrm{z}}+c_{\mathrm{b}}\right)s+\left(k_{\mathrm{b}}+k_{\mathrm{z}}\right)},$$ (D4)

where X(s), H_cz(s), and D_cz(s) are the Laplace transformations of x, h_cz, and d_cz, respectively.

Using D3, D4, we have

$$\begin{array}{ccc}\hfill P\left(s\right)& =& \frac{H_{\mathrm{c}z}\left(s\right)}{D_{\mathrm{c}z}\left(s\right)}=\frac{-\left(k_{\mathrm{z}}+c_{\mathrm{z}}s\right)m_{\mathrm{c}z}{s}^2}{\left(m_{\mathrm{z}}{s}^2+c_{\mathrm{z}}s+k_{\mathrm{z}}\right)\left[\left(m_{\mathrm{b}}+m_{\mathrm{c}z}\right)s^2+\left(c_{\mathrm{z}}+c_{\mathrm{b}}\right)s+\left(k_{\mathrm{b}}+k_{\mathrm{z}}\right)\right]-{\left(k_{\mathrm{z}}+c_{\mathrm{z}}s\right)}^2}\hfill \\ & \approx & \frac{-k_{\mathrm{z}}{m}_{\mathrm{c}z}{s}^2}{\left(m_{\mathrm{z}}{s}^2+c_{\mathrm{z}}s+k_{\mathrm{z}}\right)\left(\left(m_{\mathrm{b}}+m_{\mathrm{c}z}+m_{\mathrm{z}}\right)s^2+c_{\mathrm{b}}s+k_{\mathrm{b}}\right)},\hfill \end{array}$$ (D5)

where in the above approximation, it is assumed that m_b ≫ m_cz, m_z.

According to Eq. D5, the transfer function, P(s), from the elongation excitation of the counter-z piezo to the vibration response measured on top of the z-piezo, has bandpass characteristics. The passband of this transfer function is [ω_b, ω_z], where ${\omega }_{\mathrm{z}}\approx \sqrt{k_{\mathrm{z}}/m_{\mathrm{z}}}$ and ${\omega }_{\mathrm{b}}\approx \sqrt{k_{\mathrm{b}}/\left(m_{\mathrm{b}}+m_{\mathrm{c}z}+m_{\mathrm{z}}\right)}$ are the natural frequencies associated with the z piezo and the base structure, respectively.

APPENDIX E: TIME DOMAIN INTERPRETATION OF THE OUT-OF-BAND COMPENSATOR GAIN PENALIZATION

Following the bandpass filtering pre-processing step, the sum square error V(θ) can be approximated by

$$\begin{array}{ccc}\hfill V\left(\stackrel{`}{e}\right)& \approx & \frac{1}{N}\left(2\sum _{n=1}^{n_L-1}{\left|\frac{{\varepsilon }_{{\mathrm{y}}^{\prime }}\left(2\pi n/N\right)}{{\varepsilon }_{{\mathrm{u}}^{\prime }}\left(2\pi n/N\right)}-k\left(2\pi n/N,\theta \right)\right|}^2|{\varepsilon }_{{\mathrm{u}}^{\prime }}\left(2\pi n/N\right){|}^2+{\left|\frac{{\varepsilon }_{{\mathrm{y}}^{\prime }}\left(2\pi \right)}{{\varepsilon }_{{\mathrm{u}}^{\prime }}\left(2\pi \right)}-k\left(2\pi ,\theta \right)\right|}^2{\left|{\varepsilon }_{{\mathrm{u}}^{\prime }}\left(2\pi \right)\right|}^2\right)\hfill \\ & & +\frac{2}{N}\sum _{n=n_{\mathrm{L}}}^{n_{\mathrm{H}}}{\left|\frac{Y\left(2\pi n/N\right)}{U\left(2\pi n/N\right)}-k\left(2\pi n/N,\theta \right)\right|}^2{\left|U\left(2\pi n/N\right)\right|}^2\hfill \\ & & +\frac{1}{N}\sum _{n=n_{\mathrm{H}}+1}^{N-n_{\mathrm{H}}-1}{\left|\frac{{\varepsilon }_{{\mathrm{y}}^{\prime }}\left(2\pi n/N\right)}{{\varepsilon }_{{\mathrm{u}}^{\prime }}\left(2\pi n/N\right)}-k\left(2\pi n/N,\theta \right)\right|}^2{\left|{\varepsilon }_{{\mathrm{u}}^{\prime }}\left(2\pi n/N\right)\right|}^2,\hfill \end{array}$$

where 2πn_L/N = ω_L and 2πn_H/N = ω_H, are the lower and upper cutoff frequencies of the bandpass filter, respectively. ${\varepsilon }_{{\mathrm{y}}^{\prime }}$ and ${\varepsilon }_{{\mathrm{u}}^{\prime }}$ represent the out of band frequency content of the bandpass filtered y(n) and u(n), (denoted by y^′(n) and u^′(n), respectively), and as a result $|{\varepsilon }_{{\mathrm{y}}^{\prime }}|,|{\varepsilon }_{{\mathrm{u}}^{\prime }}|=1$. By adding a constant ${\lambda }_n\gg |{\varepsilon }_{{\mathrm{y}}^{\prime }}\left(2\pi n/N\right)|,|{\varepsilon }_{{\mathrm{u}}^{\prime }}\left(2\pi n/N\right)|$ to ${\varepsilon }_{{\mathrm{u}}^{\prime }}\left(2\pi n/N\right)$ for {n ∈ {[1, …, n_L − 1] ∪ [N − n_L + 1, …, N] ∪ [n_H + 1, N − n_H − 1]}}, we can write:

$$\begin{array}{ccc}\hfill V\left(\theta \right)& \approx & \frac{1}{N}\left(2\sum _{n=1}^{n_{\mathrm{L}}-1}|k\left(2\pi n/N,\widehat{\theta }\right){|}^2{\left|{\lambda }_n\right|}^2+{\left|k\left(2\pi ,\widehat{\theta }\right)\right|}^2{\left|{\lambda }_n\right|}^2\right)\hfill \\ & & +\frac{2}{N}\sum _{n=n_{\mathrm{L}}}^{n_{\mathrm{H}}}{\left|\frac{Y\left(2\pi n/N\right)}{U\left(2\pi n/N\right)}-k\left(2\pi n/N,\widehat{\theta }\right)\right|}^2\hfill \\ & & \times {\left|U\left(2\pi n/N\right)\right|}^2+\frac{1}{N}\sum _{n=n_{\mathrm{H}}+1}^{N-n_{\mathrm{H}}-1}\hfill \\ & & \times |k\left(2\pi n/N,\widehat{\theta }\right){|}^2{\left|{\lambda }_n\right|}^2.\hfill \end{array}$$ (E1)

Through the minimization of the cost function given by Eq. E1 the squared out of band gain of the estimated compensator is penalized with weight |λ_n|². As such, the out of band roll off of the compensator is forced. In the time domain, for ${\lambda }_{N-n}={\lambda }_n^{*}$, this can be implemented by adding harmonics to u^′(n) as given below:

$$\begin{array}{ccc}\hfill u_{\mathrm{f}}\left(n\right)& =& u^{\prime }\left(n\right)+\sum _{l=0}^{n_{\mathrm{L}}-1}2{\lambda }_l\sin \left(\frac{2\pi ln}{N}\right)\hfill \\ & & +\sum _{l=n_{\mathrm{H}}+1}^{Q}2{\lambda }_l\sin \left(\frac{2\pi ln}{N}\right)+{\lambda }^{\prime }\sin \left(\pi \right),\hfill \end{array}$$ (E2)

where * denotes complex conjugate, Q = (N − 1)/2, λ′ = 0 for N odd and Q = N/2 − 1, λ′ = λ_N/2 for N even.

A larger λ_n would lead to a steeper out-of-band roll-off of the compensator. Selection of penalizing weight function is quite flexible and may vary depending on the system noise and required roll off. However, as a rule of thumb λ_n may be selected according to the following:

$$\left\{\begin{array}{c}{\lambda }_n=c_1\ge 2\left|U\left(2\pi n_{\mathrm{L}}/N\right)\right|\mathrm{for}n<n_{\mathrm{L}},\\ {\lambda }_n=c_2\ge 2\left|U\left(2\pi n_{\mathrm{H}}/N\right)\right|\mathrm{for}n>n_{\mathrm{H}}.\end{array}\right.$$ (E3)

A schematic summary of the steps in the proposed compensator design technique is given in Fig. 14.

Figure 14.

Schematic representation of the proposed compensator design technique.