Second-law-like inequalities with information and their interpretations

Our quantity of interest is the time-dependent velocity v_t of a trapped, underdamped Brownian particle of mass m, coupled to a thermal reservoir at temperature T with viscous damping coefficient $\gamma$, evolving according to the Langevin equation [34]

$$\begin{eqnarray}&&m{{\dot{v}}_{t}}=-\gamma {{v}_{t}}+{{f}_{t}}+{{\xi }_{t}},\end{eqnarray} \tag{ 2 }$$

where f_t is an externally controlled force, and ${{\xi }_{t}}$ is zero-mean Gaussian white noise with covariance $\langle {{\xi }_{t}}{{\xi }_{s}}\rangle =2\gamma T\delta (t-s)$. Starting here we set Boltzmannʼs constant to unity, ${{k}_{{\rm B}}}=1$.

In the absence of control, f_t = 0, the velocity v_t relaxes to an equilibrium Boltzmann distribution ${{p}_{{\rm eq}}}(v)\propto {\rm exp} [-m{{v}^{2}}/(2T)]$. In the following, we will vary f_t using feedback in order to cool the particle, that is damp its thermal fluctuations, thereby reducing its kinetic temperature ${{T}_{{\rm kin}}}=m\langle {{v}^{2}}\rangle \lt T$. Before we get to that, it is helpful to first review the energetics and thermodynamics of a driven, underdamped Brownian particle without feedback, so that we can appreciate the differences that arise in the presence of feedback.

To this end, we require the Fokker–Planck equation associated with (2) for the time-dependent probability density p_t(v) [35],

$$\begin{eqnarray}&&{{\partial }_{t}}{{p}_{t}}(v)=-{{\partial }_{v}}J_{t}^{v}(v),\end{eqnarray} \tag{ 3 }$$

$$\begin{eqnarray}&&J_{t}^{v}(v)=-\frac{1}{m}(\gamma {{v}_{t}}-{{f}_{t}}){{p}_{t}}(v)-\frac{\gamma T}{{{m}^{2}}}{{\partial }_{v}}{{p}_{t}}(v),\end{eqnarray} \tag{ 4 }$$

where we have introduced the (probability) current J^v_t. Anticipating our discussion of the thermodynamics, we divide the current into its irreversible half, which is anti-symmetric under time-reversal, and its reversible half, which is time-reversal symmetric, as [35–37]

$$\begin{eqnarray}&&J_{t}^{{\rm irr}}(v)=-\frac{\gamma }{m}v{{p}_{t}}(v)-\frac{\gamma T}{{{m}^{2}}}{{\partial }_{v}}{{p}_{t}}(v)\end{eqnarray} \tag{ 5 }$$

$$\begin{eqnarray}&&J_{t}^{{\rm rev}}(v)=\frac{{{f}_{t}}}{m}{{p}_{t}}(v).\end{eqnarray} \tag{ 6 }$$

Key to this splitting is treating the force f_t as even under time reversal, as typically assumed for a force arising from an external potential. With this identification, the irreversible portion of the current $J_{t}^{{\rm irr}}$ arises solely due to the forces imparted on the particle by its surroundings: the friction, $-\gamma {{v}_{t}}$, and the fluctuating force, ${{\xi }_{t}}$.

Moving on to the thermodynamics, we have from stochastic energetics an unambiguous identification of the heat flow into the system as the work done by the thermal reservoir on the particle [22, 36, 38, 39], which on average reads

$$\begin{eqnarray}&&\dot{Q}=\int mvJ_{t}^{{\rm irr}}(v){\rm d}v.\end{eqnarray} \tag{ 7 }$$

It notably only depends on the irreversible current arising from the forces due to the thermal reservoir. The particleʼs (internal) energy is its average kinetic energy

$$\begin{eqnarray}&&E=\left\langle \frac{1}{2}mv_{t}^{2} \right\rangle =\int \frac{1}{2}m{{v}^{2}}{{p}_{t}}(v){\rm d}v.\end{eqnarray} \tag{ 8 }$$

By differentiating E with time and substituting in the Fokker–Planck equation (3), we are able to identify the extracted work rate via the first law of thermodynamics $\dot{E}=-{{\dot{W}}_{{\rm ext}}}+\dot{Q}$,

$$\begin{eqnarray}&&{{\dot{W}}_{{\rm ext}}}=-\langle {{f}_{t}}{{v}_{t}}\rangle ,\end{eqnarray} \tag{ 9 }$$

as the average power delivered against the external force f_t.

From stochastic thermodynamics, we also have the (irreversible) entropy production rate [36, 37, 39]

$$\begin{eqnarray}&&{{\dot{S}}_{{\rm i}}}=\dot{S}(v)-\frac{{\dot{Q}}}{T}=\frac{{{m}^{2}}}{\gamma T}\int \frac{{{[J_{t}^{{\rm irr}}(v)]}^{2}}}{{{p}_{t}}(v)}{\rm d}v\geqslant 0,\end{eqnarray} \tag{ 10 }$$

where we have the traditional splitting into the time variation of the systemʼs Shannon entropy $S(v)=-\int {{p}_{t}}(v){\rm ln} {{p}_{t}}(v){\rm d}v$,

$$\begin{eqnarray}&&\dot{S}(v)=-\int J_{t}^{v}(v){{\partial }_{v}}{\rm ln} {{p}_{t}}(v){\rm d}v,\end{eqnarray} \tag{ 11 }$$

and the reversible entropy exchange with the environment

$$\begin{eqnarray}&&{{\dot{S}}_{{\rm env}}}=-\frac{{\dot{Q}}}{T}.\end{eqnarray} \tag{ 12 }$$

Notably, the entropy production only depends on the irreversible current, since it is a measure of the time-reversal symmetry breaking of the dynamics [36]. This property is what allowed us to pullout the contribution due to the heat, which is also only a function of the irreversible current.

Our main focus in this paper is feedback cooling, where we vary f_t in response to measurements of the velocity. Following [23], we consider a feedback protocol where we measure the velocity v_t obtaining outcomes y_t with some error, and then feed back those measurements by applying a force ${{f}_{t}}=-a{{y}_{t}}$ that acts as an additional friction, extracting work. A simple way to incorporate measurement error is to add to our read-out of v_t Gaussian white noise ${{\eta }_{t}}$ of zero mean and covariance $\langle {{\eta }_{t}}{{\eta }_{s}}\rangle ={{\sigma }^{2}}\delta (t-s)$, with ${{\sigma }^{2}}$ quantifying the measurement uncertainty: for example as ${{y}_{t}}={{v}_{t}}+{{\eta }_{t}}$. However, white noise fluctuations are very violent. To make the problem more tractable, we smooth over the noise by applying a low-pass filter with time constant τ to the measurements: ${{y}_{t}}=(1/\tau )\int _{0}^{t}{{{\rm e}}^{-(t-s)/\tau }}({{v}_{s}}+{{\eta }_{s}}){\rm d}s$ [40]. We are therefore led to the following modified dynamics including measurement and feedback [23]

$$\begin{eqnarray}\begin{array}{rcl} m{{{\dot{v}}}_{t}} & = & -\gamma {{v}_{t}}-a{{y}_{t}}+{{\xi }_{t}} \\ \tau {{{\dot{y}}}_{t}} & = & -({{y}_{t}}-{{v}_{t}}-{{\eta }_{t}}), \\ \end{array}\end{eqnarray} \tag{ 13 }$$

where a is the feedback gain. It is important to note at this point that y_t is merely a model of measurement outcomes. We are not making any assumption about the physical system that records the measurements, nor implements the feedback in response.

In general, the joint system relaxes to a time-independent, nonequilibrium steady state, where heat is continuously being extracted as work to maintain the particle at the cooled kinetic temperature. This is the scenario we focus on in the following.

To discuss the energetics, we need the equivalent description of the dynamics in (13) in terms of the Fokker–Planck equation for the time-dependent probability density ${{p}_{t}}(v,y)$,

$$\begin{eqnarray}&&{{\partial }_{t}}{{p}_{t}}(v,y)=-{{\partial }_{v}}J_{t}^{v}(v,y)-{{\partial }_{y}}J_{t}^{y}(v,y),\end{eqnarray} \tag{ 14 }$$

with (probability) currents

$$\begin{eqnarray}\begin{array}{rcl} J_{t}^{v}(v,y) & = & -\frac{1}{m}(\gamma v+ay){{p}_{t}}(v,y)-\frac{\gamma T}{{{m}^{2}}}{{\partial }_{v}}{{p}_{t}}(v,y) \\ J_{t}^{y}(v,y) & = & -\frac{1}{\tau }(y-v){{p}_{t}}(v,y)-\frac{{{\sigma }^{2}}}{2{{\tau }^{2}}}{{\partial }_{y}}{{p}_{t}}(v,y). \\ \end{array}\end{eqnarray} \tag{ 15 }$$

Again we can split the velocity current J^v_t into irreversible and reversible pieces, as in (5),

$$\begin{eqnarray}&&J_{t}^{{\rm irr}}(v,y)=-\frac{\gamma }{m}v{{p}_{t}}(v,y)-\frac{\gamma T}{{{m}^{2}}}{{\partial }_{v}}{{p}_{t}}(v,y),\end{eqnarray} \tag{ 16 }$$

$$\begin{eqnarray}&&J_{t}^{{\rm rev}}(v,y)=-\frac{a}{m}y{{p}_{t}}(v,y).\end{eqnarray} \tag{ 17 }$$

This splitting singles out the irreversible current as solely due to the thermal reservoir as before (see (5)), which is required to correctly link the heat and entropy production in the following. Again, this division relies on choosing ${{f}_{t}}=-a{{y}_{t}}$ as time-reversal symmetric, just as in the preceding section.

Our focus is the steady state solution, which due to the linear, Gaussian dynamics is the Gaussian probability density [34],

$$\begin{eqnarray}&&{{p}_{{\rm s}}}(v,y)=\frac{1}{\sqrt{{{(2\pi )}^{2}}|{\boldsymbol{ \Sigma }} |}}{\rm exp} \left[ -\frac{1}{2}(v,y)\cdot {{{\boldsymbol{ \Sigma }} }^{-1}}\cdot {{(v,y)}^{T}} \right],\end{eqnarray} \tag{ 18 }$$

where the steady-state covariance matrix is

$$\begin{eqnarray}{\boldsymbol{ \Sigma }} =\left( \begin{array}{ccccccccccccccc} \sigma _{{\rm v}}^{2} & {{\sigma }_{{\rm vy}}} \\ {{\sigma }_{{\rm vy}}} & \sigma _{{\rm y}}^{2} \\ \end{array} \right),\end{eqnarray} \tag{ 19 }$$

and the associated steady-state currents are $J_{{\rm s}}^{v}$ and $J_{{\rm s}}^{y}$. The entries of ${\boldsymbol{ \Sigma }}$ can be determined by plugging (18) into (14), as detailed for a more general model in [23]; however their precise expressions are unilluminating and therefore relegated to appendix A. We do observe that the reduced distribution of the velocity ${{p}_{{\rm s}}}(v)=\int {{p}_{{\rm s}}}(v,y){\rm d}y$ is also Gaussian. Therefore, it has the same structure as an equilibrium distribution, but with a smaller variance, or a cooler effective temperature [23]

$$\begin{eqnarray}&&{{T}_{{\rm kin}}}=T\frac{1+(a/\gamma )(a{{\sigma }^{2}}/(2T))+(1+a/\gamma )(\gamma \tau /m)}{1+a/\gamma +(1+a/\gamma )(\gamma \tau /m)}\leqslant T,\end{eqnarray} \tag{ 20 }$$

where the inequality is only satisfied in the regime of good cooling, $a{{\sigma }^{2}}\leqslant 2T$. Otherwise too much measurement noise is fed back into the velocity, effectively heating it.

Again from stochastic energetics the heat current is identified as the energy lost due to the irreversible current arising from the thermal noise [22, 38, 39]

$$\begin{eqnarray}&&\dot{Q}=\int mvJ_{t}^{{\rm irr}}(v,y){\rm d}v{\rm d}y=\int mvJ_{t}^{{\rm irr}}(v){\rm d}v,\end{eqnarray} \tag{ 21 }$$

which importantly only depends on the velocity as in (5), since the measurement and feedback do not affect the interaction with the thermal environment. In a similar way as before (9) the extracted work rate is

$$\begin{eqnarray}&&{{\dot{W}}_{{\rm ext}}}=a\langle {{y}_{t}}{{v}_{t}}\rangle ,\end{eqnarray} \tag{ 22 }$$

due to the correlations between the feedback force and the particle. In the steady state, ${{\dot{W}}_{{\rm ext}}}$ can be simplified using the defining equations for the elements of the covariance matrix ${\boldsymbol{ \Sigma }}$ in appendix A,

$$\begin{eqnarray}&&{{\dot{W}}_{{\rm ext}}}=a{{\sigma }_{{\rm vy}}}=\frac{1}{{{\tau }_{{\rm v}}}}\left( T-{{T}_{{\rm kin}}} \right),\end{eqnarray} \tag{ 23 }$$

in terms of the velocityʼs relaxation rate, $1/{{\tau }_{{\rm v}}}=\gamma /m$. When the feedback is successful, and we have reduced the kinetic temperature ${{T}_{{\rm kin}}}\lt T$, we must be extracting work, ${{\dot{W}}_{{\rm ext}}}\gt 0$, recovering the results of [23].

We finally will require the fluctuating-trajectory solutions of (13) up to time t, $v_{0}^{t}=\{{{v}_{s}}\}_{s=0}^{t}$ and $y_{0}^{t}=\{{{y}_{s}}\}_{s=0}^{t}$. We can obtain the probability densities for these trajectories by discretizing time and then using the usual procedure for obtaining path-integral densities, which we sketch in appendix B. The joint density $\mathcal{P}[v_{0}^{t},y_{0}^{t}]$ can be conveniently expressed in terms of two probability densities

$$\begin{eqnarray}&&\hat{\mathcal{P}}[y_{0}^{t}|v_{0}^{t},{{y}_{0}}]\propto {\rm exp} \left[ -\int _{0}^{t}{\rm d}s\;\frac{{{(\tau {{{\dot{y}}}_{s}}+{{y}_{s}}-{{v}_{s}})}^{2}}}{2{{\sigma }^{2}}} \right],\end{eqnarray} \tag{ 24 }$$

suitably normalized, and

$$\begin{eqnarray}&&\hat{\mathcal{P}}[v_{0}^{t}\mid y_{0}^{t},{{v}_{0}}]\propto {\rm exp} \left[ -\int _{0}^{t}{\rm d}s\;\frac{{{(m{{{\dot{v}}}_{s}}+\gamma {{v}_{s}}+a{{y}_{s}})}^{2}}}{4\gamma T/{{m}^{2}}} \right],\end{eqnarray} \tag{ 25 }$$

as

$$\begin{eqnarray}&&\mathcal{P}[v_{0}^{t},y_{0}^{t}]=\hat{\mathcal{P}}[v_{0}^{t}\mid y_{0}^{t},{{v}_{0}}]\hat{\mathcal{P}}[y_{0}^{t}\mid v_{0}^{t},{{y}_{0}}]p({{v}_{0}},{{y}_{0}}),\end{eqnarray} \tag{ 26 }$$

with initial probability density $p({{v}_{0}},{{y}_{0}})$. It cannot be under emphasized that each $\hat{\mathcal{P}}$ is not the conditional probability of the feedback process, i.e., $\hat{\mathcal{P}}[y_{0}^{t}\mid v_{0}^{t},{{y}_{0}}]\ne \mathcal{P}[y_{0}^{t}\mid v_{0}^{t},{{y}_{0}}]=\mathcal{P}[v_{0}^{t},y_{0}^{t}\mid {{y}_{0}}]/\mathcal{P}[v_{0}^{t}\mid {{y}_{0}}]$, since v_t and y_t influence each other when there is feedback [8]. Instead, we can understand $\hat{\mathcal{P}}[y_{0}^{t}\mid v_{0}^{t},{{y}_{0}}]$ by first imagining that we fix the entire velocity trajectory v₀^t, and then evolve y_t alone according to (13). This procedure has no feedback and the probability to observe a particular measurement trajectory is exactly $\hat{\mathcal{P}}[y_{0}^{t}\mid v_{0}^{t},{{y}_{0}}]$. A similar interpretation holds for $\hat{\mathcal{P}}[v_{0}^{t}\mid y_{0}^{t},{{v}_{0}}]$ as well. This distinction between $\hat{\mathcal{P}}$ and $\mathcal{P}$ will become important in section 3.1 when we introduce the transfer entropy rate.

The first information measure we discuss is the transfer entropy rate from v_t to y_t. The transfer entropy is a directional measure of information, which quantifies in an information-theoretic manner how much the dynamics (or more specifically the transition probabilities) of y_t are influenced by v_t [14]. For our continuous stochastic process, it reads

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm v}\to {\rm y}}}=\mathop{{\rm lim} }\limits_{t\to \infty }\frac{1}{t}\int \mathcal{D}[v_{0}^{t}]\mathcal{D}[y_{0}^{t}]\;\mathcal{P}[v_{0}^{t},y_{0}^{t}]{\rm ln} \frac{\hat{\mathcal{P}}[y_{0}^{t}\mid v_{0}^{t},{{y}_{0}}]}{\mathcal{P}[y_{0}^{t}\mid {{y}_{0}}]}\geqslant 0.\end{eqnarray} \tag{ 27 }$$

In appendix B, we justify this expression by discretizing the evolution and then utilizing the well-developed theory for repeated, discrete feedback [3, 5, 7, 16, 27, 41]. When no measurements are taking place, the dynamics of y_t is independent of v_t, $\hat{\mathcal{P}}[y_{0}^{t}\mid v_{0}^{t},{{y}_{0}}]=\mathcal{P}[y_{0}^{t}\mid {{y}_{0}}]$, and the transfer entropy rate is zero. On the other hand, the more influence the velocity has on the measurement outcomes the larger the transfer entropy rate. Furthermore, when there is only one measurement the transfer entropy simplifies to the mutual information [8]. An alternative, equivalent expression for the transfer entropy rate in the context of continuous feedback has been introduced by Sandberg et al [11]. A similar analysis was performed by Fujitani and Suzuki for discrete Markov processes [6, 27]. The transfer entropy rate in feedback systems described by continuous-time, discrete Markov processes has been extensively studied in [10, 16, 41, 42].

To compare ${{\dot{I}}_{{\rm v}\to {\rm y}}}$ with the other information measures, we calculate its value in our model of feedback cooling. The calculation is facilitated by noting that for stationary Gaussian processes, as we have, integrals of the form (27) can be conveniently expressed in terms of the power spectra—Fourier transforms of the correlation functions. For (27), we demonstrate in appendix C that it can be formulated as

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm v}\to {\rm y}}}=-\frac{1}{4\pi }\int _{-\infty }^{\infty }{\rm ln} \frac{{{{\hat{C}}}_{{\rm yy}|{\rm v}}}(\omega )}{{{C}_{{\rm yy}}}(\omega )}\;{\rm d}\omega ,\end{eqnarray} \tag{ 28 }$$

where ${{C}_{{\rm yy}}}(\omega )$ is the power spectrum of y_t, and ${{\hat{C}}_{{\rm yy}|{\rm v}}}(\omega )$ is the Fourier transform of the variance of y_t given a fixed trajectory v₀^t. We have carried out the integral in appendix D with the result

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm v}\to {\rm y}}}=\frac{\gamma }{2m}\left( \sqrt{1+\frac{2T/\gamma }{{{\sigma }^{2}}}}-1 \right)=\frac{1}{2{{\tau }_{{\rm v}}}}\left( \sqrt{1+{\rm SNR}}-1 \right).\end{eqnarray} \tag{ 29 }$$

New information is acquired at the relaxation rate of v_t, $\gamma /m=1/{{\tau }_{{\rm v}}}$; that is we learn new information about v_t only as fast as v_t changes enough to detect. In addition, the transfer entropy rate does not depend on the feedback parameters a and $\tau$, but only on the measurement accuracy ${{\sigma }^{2}}$ through the dimensionless signal-to-noise ratio ${\rm SNR}=(2T/\gamma )/{{\sigma }^{2}}$, which quantifies the relative size of the measurement accuracy to the thermal diffusion of the velocity. As a result, for perfect measurements without error, $\sigma =0$, the ${\rm SNR}$ diverges and with it the transfer entropy rate. Thus, error-free measurement corresponds to infinite information, consistent with the notion that infinite information is required to localize a continuous variable with perfect precision.

We next consider the information flow, whose origin is in the exchange of information between the velocity and the auxiliary measurement device implementing the control. It was first considered in the context of interacting diffusion processes [17], but subsequently has been introduced in the analysis of the thermodynamics of continuously-coupled, discrete stochastic systems [16, 18, 43]. When the coupling is not continuous, but each system takes turns evolving, the information flow simplifies to the mutual information [18–20]. In order to facilitate connections to the other information measures, we sketch in this section the basic arguments leading to the information flow, following the program outlined in [18], and then calculate its value in our feedback cooling model.

First, we must note that strictly speaking this approach requires that y_t be the degree of freedom of a physical system, not simply an abstract measurement outcome. Still, in this section we would like not to comment on the precise thermodynamics of y_t, taking it only as a generic thermodynamic system. We will come back to its precise interpretation in section 4 when we compare the physics underlying the different information measures.

The key insight in this approach is that the (irreversible) entropy production of the joint system of v_t and y_t can be divided as

$$\begin{eqnarray}&&{{\dot{S}}_{{\rm i}}}=\dot{S}_{{\rm i}}^{v}+\dot{S}_{{\rm i}}^{y}\geqslant 0,\end{eqnarray} \tag{ 30 }$$

with positive contributions arising due to the irreversible current in the v-direction (16),

$$\begin{eqnarray}&&\dot{S}_{{\rm i}}^{v}=\frac{{{m}^{2}}}{\gamma T}\int \frac{{{[J_{t}^{{\rm irr}}(v,y)]}^{2}}}{{{p}_{t}}(v,y)}\;{\rm d}v{\rm d}y\geqslant 0,\end{eqnarray} \tag{ 31 }$$

and separately from y_t, $\dot{S}_{{\rm i}}^{y}$. The next step is to perform the traditional splitting of $\dot{S}_{{\rm i}}^{v}$ into the variation of the Shannon entropy due to v_t (see (11)),

$$\begin{eqnarray}&&\dot{S}(v)=-\int J_{t}^{v}(v,y){{\partial }_{v}}{\rm ln} {{p}_{t}}(v){\rm d}v{\rm d}y,\end{eqnarray} \tag{ 32 }$$

and the heat $\dot{Q}$ (21) as

$$\begin{eqnarray}&&\dot{S}_{{\rm i}}^{v}=\dot{S}(v)-\frac{{\dot{Q}}}{T}+{{\dot{I}}_{{\rm flow}}}\geqslant 0.\end{eqnarray} \tag{ 33 }$$

The additional contribution due to the influence of y_t is an information-theoretic piece

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm flow}}}=-\int J_{t}^{v}(v,y){{\partial }_{v}}{\rm ln} \frac{{{p}_{t}}(v,y)}{{{p}_{t}}(v){{p}_{t}}(y)}\;{\rm d}v{\rm d}y,\end{eqnarray} \tag{ 34 }$$

which is (minus) the variation of the mutual information³

$$\begin{eqnarray}&&I({{v}_{t}};{{y}_{t}})=\int {{p}_{t}}(v,y){\rm ln} \frac{{{p}_{t}}(v,y)}{{{p}_{t}}(v){{p}_{t}}(y)}\;{\rm d}v{\rm d}y\end{eqnarray} \tag{ 35 }$$

between v_t and y_t, due to the fluctuations of v_t [44]. The mutual information $I({{v}_{t}};{{y}_{t}})$ is a measure of correlations, quantifying how knowledge of the measurement outcomes reduces uncertainty in the velocity. While ${{\dot{I}}_{{\rm flow}}}$ may be positive or negative, in the regime of good cooling where we are extracting work, we will always have ${{\dot{I}}_{{\rm flow}}}\geqslant 0$. In the steady state, $J_{{\rm s}}^{v}=0,$ and $\dot{Q}={{\dot{W}}_{{\rm ext}}}$, so that (33) reduces to [16, 18]

$$\begin{eqnarray}&&{{\dot{W}}_{{\rm ext}}}\leqslant T{{\dot{I}}_{{\rm flow}}},\end{eqnarray} \tag{ 36 }$$

in the form of (1).

Employing the steady-state solution in (18), we have for the steady-state information flow

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm flow}}}=\frac{\gamma }{m}\left( \frac{T}{m}\frac{\sigma _{{\rm y}}^{2}}{|{\boldsymbol{ \Sigma }} |}-1 \right)\geqslant 0,\end{eqnarray} \tag{ 37 }$$

where $|{\boldsymbol{ \Sigma }} |$ denotes the determinant of ${\boldsymbol{ \Sigma }}$. Unfortunately, we have been unable to formulate a more transparent expression in general. Even still, the information rate again only grows as fast as the relaxation rate of the velocity $\gamma /m=1/{{\tau }_{{\rm v}}}$.

For the feedback cooling of a Brownian particle without errors an entropy pumping bound has been introduced by Kim and Qian [21]. This approach has subsequently been developed by Ge [45] and extended to the setup in (13) by Munakata and Rosinberg [22–24], which we discuss in this section.

The entropy pumping approach is based on a coarse graining of the Fokker–Planck equation (14). Following [23], we formally integrate out y_t from (14) to obtain the reduced Fokker–Planck equation

$$\begin{eqnarray}&&{{\partial }_{t}}{{p}_{t}}(v)={{\partial }_{v}}\left( \frac{1}{m}\left( \gamma v+\tilde{f}_{t}^{{\rm fb}}(v) \right){{p}_{t}}(v)+\frac{\gamma T}{{{m}^{2}}}{{\partial }_{v}}{{p}_{t}}(v) \right)=-{{\partial }_{v}}{{\tilde{J}}_{t}}(v),\end{eqnarray} \tag{ 38 }$$

where we have identified an effective feedback force

$$\begin{eqnarray}&&\tilde{f}_{t}^{{\rm fb}}(v)=a\int y{{p}_{t}}(y|v){\rm d}y.\end{eqnarray} \tag{ 39 }$$

Furthermore, we treat $\tilde{f}_{t}^{{\rm fb}}$ as time-reversal symmetric, as we would expect for an external force [23]. In which case, we single out from the coarse-grained current the irreversible current exactly as for the no-feedback case (5),

$$\begin{eqnarray}&&{{\tilde{J}}_{t}}(v)=J_{t}^{{\rm irr}}(v)-\frac{1}{m}\tilde{f}_{t}^{{\rm fb}}(v).\end{eqnarray} \tag{ 40 }$$

This will allow us to connect the entropy production in the environment with the heat.

Equation (38) is not a closed equation for p_t (v); the measurement dynamics are required to solve it. Nevertheless, the entropy pumping approach is to treat (38) as a thermodynamically consistent equation for p_t (v) with an effective external force $\tilde{f}_{t}^{{\rm fb}}$. In this case, the entropy balance is developed in analogy to the no-feedback setup, as in (10),

$$\begin{eqnarray}&&\mathop{\tilde{S}}\limits^{.}_{{\rm i}}^{v}=\frac{{{m}^{2}}}{\gamma T}\int \frac{{{[J_{t}^{{\rm irr}}(v)]}^{2}}}{{{p}_{t}}(v)}{\rm d}v=\dot{S}(v)-\frac{{\dot{Q}}}{T}+{{\dot{I}}_{{\rm pump}}}\geqslant 0,\end{eqnarray} \tag{ 41 }$$

where the second equality follows by substituting in the definition of the coarse-grained current ${{\tilde{J}}_{t}}(v)$ in (40). Here, $\dot{S}(v)=-\int {{\tilde{J}}_{t}}(v){{\partial }_{v}}{\rm ln} {{p}_{t}}(v){\rm d}v$ is equivalent to the expression for the rate of change of the systemʼs Shannon entropy including feedback in (32), and the additional entropy pumping term arises due to the coarse-grained feedback force

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm pump}}}=\int {{p}_{t}}(v){{\partial }_{v}}\frac{1}{m}\tilde{f}_{t}^{{\rm fb}}(v){\rm d}v.\end{eqnarray} \tag{ 42 }$$

As pointed out in [23], the feedback force is proportional to the minimum mean square error estimate of y_t given v_t. Other than that though, there does not appear to be a crisp interpretation of the entropy pumping as a form of information, like for the transfer entropy rate and information flow.

Using the steady-state distribution in (18), we have for the steady-state entropy pumping [23]

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm pump}}}=\int {{p}_{{\rm s}}}(v){{\partial }_{v}}\frac{1}{m}\tilde{f}_{{\rm s}}^{{\rm fb}}(v){\rm d}v=\frac{a}{m}\frac{{{\sigma }_{{\rm vy}}}}{\sigma _{{\rm v}}^{2}}=\frac{1}{{{\tau }_{{\rm v}}}}\left( \frac{T-{{T}_{{\rm kin}}}}{{{T}_{{\rm kin}}}} \right)\geqslant 0,\end{eqnarray} \tag{ 43 }$$

with positivity guaranteed when there is cooling $T\geqslant {{T}_{{\rm kin}}}$.

Another information measure that has aroused some attention is the mutual information rate between the entire v₀^t and y₀^t trajectories [15, 46]. For continuous stochastic processes, the trajectory mutual information rate is [44]

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm traj}}}=\mathop{{\rm lim} }\limits_{t\to \infty }\frac{1}{t}I(v_{0}^{t};y_{0}^{t})=\mathop{{\rm lim} }\limits_{t\to \infty }\frac{1}{t}\int \mathcal{D}[v_{0}^{t}]\mathcal{D}[y_{0}^{t}]\;\mathcal{P}[v_{0}^{t},y_{0}^{t}]{\rm ln} \frac{\mathcal{P}[v_{0}^{t},y_{0}^{t}]}{\mathcal{P}[v_{0}^{t}]\mathcal{P}[y_{0}^{t}]}\geqslant 0.\end{eqnarray} \tag{ 44 }$$

It quantifies how much the uncertainty about the entire velocity trajectory v₀^t is reduced given knowledge of the entire measurement trajectory y₀^t, and vice versa, as it is symmetric.

The ${{\dot{I}}_{{\rm traj}}}$ bound on the extracted work follows readily once we observe a close connection between the trajectory mutual information and the transfer entropy pointed out in [16]; by substituting $\mathcal{P}$ with $\hat{\mathcal{P}}$ (26) in ${{\dot{I}}_{{\rm traj}}}$, it follows that

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm traj}}}={{\dot{I}}_{{\rm v}\to {\rm y}}}+{{\dot{I}}_{{\rm y}\to {\rm v}}},\end{eqnarray} \tag{ 45 }$$

after identifying the transfer entropy rate from y_t to v_t, ${{\dot{I}}_{{\rm y}\to {\rm v}}}\geqslant 0$, defined analogously to ${{\dot{I}}_{{\rm v}\to {\rm y}}}$ (27). The positivity of the transfer entropy implies that

$$\begin{eqnarray}&&{{\dot{W}}_{{\rm ext}}}\leqslant T{{\dot{I}}_{{\rm v}\to {\rm y}}}\leqslant T{{\dot{I}}_{{\rm traj}}},\end{eqnarray} \tag{ 46 }$$

giving (1) for the trajectory information, which is always weaker than the transfer entropy bound.

The trajectory information rate has been studied in numerous contexts and has a well-known expression in terms of power spectra [47–50] that we recall in appendix C,

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm traj}}}=-\frac{1}{4\pi }\int _{-\infty }^{\infty }{\rm ln} \left( 1-\frac{|{{C}_{{\rm vy}}}(\omega ){{|}^{2}}}{{{C}_{{\rm vv}}}(\omega ){{C}_{{\rm yy}}}(\omega )} \right){\rm d}\omega .\end{eqnarray} \tag{ 47 }$$

in appendix D, we perform this integral to find

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm traj}}}=\frac{\gamma }{2m}\left( \sqrt{1+\frac{2T/\gamma }{{{\sigma }^{2}}}}-1 \right)+\frac{1}{2\tau }\left( \sqrt{1+\frac{{{a}^{2}}{{\sigma }^{2}}}{2\gamma T}}-1 \right).\end{eqnarray} \tag{ 48 }$$

Comparing with (45), we have as a byproduct the transfer entropy rate from y_t to v_t,

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm y}\to {\rm v}}}=\frac{1}{2\tau }\left( \sqrt{1+\frac{{{a}^{2}}{{\sigma }^{2}}}{2\gamma T}}-1 \right).\end{eqnarray} \tag{ 49 }$$

A final bound on the extracted work is simply to maximize ${{\dot{W}}_{{\rm ext}}}$ in (23) with respect to the measurement parameters a and $\tau$. While the result is not general, remarkably for linear Guassian processes it has a close connection with the transfer entropy rate, as first noticed by Sandberg et al [11]. Using standard calculus methods, the extracted work is bounded above by its maximal value

$$\begin{eqnarray}&&{{\dot{W}}_{{\rm ext}}}\leqslant \dot{W}_{{\rm ext}}^{{\rm max} }={{T}_{{\rm kin}}}{{\dot{I}}_{{\rm v}\to {\rm y}}},\end{eqnarray} \tag{ 50 }$$

akin to (1), for parameter values

$$\begin{eqnarray}&&{{a}^{*}}=\gamma \left( \sqrt{1+\frac{2T/\gamma }{{{\sigma }^{2}}}}-1 \right),\qquad {{\tau }^{*}}=0.\end{eqnarray} \tag{ 51 }$$

The optimal measurement has no low-pass filtering: it is immediately fed back into the particle to control it. Remarkably, the extracted work is again bounded by the transfer entropy rate, except multiplied by the cooled kinetic temperature of the particle, instead of T.

3.6.1. Quantitative comparison of information measures

To better understand the relationship between all of these information measures, we plot them all together with ${{\dot{W}}_{{\rm ext}}}$ in figure 1 as a function of the feedback gain a and measurement error ${{\sigma }^{2}}$ in the range where cooling occurs (${{\dot{W}}_{{\rm ext}}}\geqslant 0$). As expected, each information measure bounds the extracted work.

Zoom In Zoom Out Reset image size

The most striking feature of figure 1 is the hierarchy of information measures

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm traj}}}\geqslant {{\dot{I}}_{{\rm v}\to {\rm y}}}\geqslant {{\dot{I}}_{{\rm flow}}}\geqslant {{\dot{I}}_{{\rm pump}}},\end{eqnarray} \tag{ 52 }$$

apart from $\dot{W}_{{\rm ext}}^{{\rm max} }$, which does not actually have a generic information interpretation. In fact, this ranking holds quite generally. We have already seen that ${{\dot{I}}_{{\rm traj}}}\geqslant {{\dot{I}}_{{\rm v}\to {\rm y}}}$ in section 3.4 when discussing the second-law-like inequality for the trajectory information. The middle inequality, ${{\dot{I}}_{{\rm v}\to {\rm y}}}\geqslant {{\dot{I}}_{{\rm flow}}}$, has been demonstrated by Hartich et al [16] for continuous-time, discrete Markov jump processes. For diffusion processes, a similar conclusion was reached by Allahverdyan et al [17] except for a slightly different transfer entropy rate that uses only the most recent measurement, which upper bounds the transfer entropy rate considered here, as pointed out in [16]. Nevertheless, the proof for jump processes in [16] can be carried over to diffusion processes, once their evolution is discretized. The last inequality between the information flow and the entropy pumping also is generic. This follows by bounding the steady-state entropy production of v_t in the information-flow description (31) using a coarse-graining inequality [23] to connect it to the coarse-grained, entropy-pumping approach

$$\begin{eqnarray}&&\begin{array}{l} \dot{S}_{{\rm i}}^{v}=-\frac{{\dot{Q}}}{T}+{{{\dot{I}}}_{{\rm flow}}}=\frac{{{m}^{2}}}{\gamma T}\int \frac{{{[J_{{\rm s}}^{{\rm irr}}(v,y)]}^{2}}}{{{p}_{{\rm s}}}(v,y)}{\rm d}v{\rm d}y \\ \qquad \qquad \qquad \qquad \,\geqslant \;\frac{{{m}^{2}}}{\gamma T}\int \frac{{{[J_{{\rm s}}^{{\rm irr}}(v)]}^{2}}}{{{p}_{{\rm s}}}(v)}{\rm d}v=-\frac{{\dot{Q}}}{T}+{{{\dot{I}}}_{{\rm pump}}}=\mathop{\tilde{S}}\limits^{.}_{{\rm i}}^{v}, \\ \end{array}\end{eqnarray} \tag{ 53 }$$

where we have employed the entropy balance of entropy pumping in (41). Clearly

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm flow}}}\geqslant {{\dot{I}}_{{\rm pump}}}.\end{eqnarray} \tag{ 54 }$$

As a lower bound on all other information measures, the entropy pumping can be given an information-theoretic interpretation, which till now has been lacking, as a minimal information requirement for successful feedback cooling.

An alternative perspective on this hierarchy of information measures comes from considering the efficiency of work extraction

$$\begin{eqnarray}&&\varepsilon =\frac{{{{\dot{W}}}_{{\rm ext}}}}{T\dot{I}}\leqslant 1.\end{eqnarray} \tag{ 55 }$$

By utilizing the smaller information measures, we will estimate higher efficiencies, even without changing the measurement or feedback procedure. This conclusion is somewhat surprising, since it makes the notion of efficiency somewhat arbitrary. We will come back to this observation later, after discussing the physical origins of the different information measures.

We also see in figure 1 that the transfer entropy rate and the trajectory mutual information diverge as the measurement error tends to zero, ${{\sigma }^{2}}\to 0$; whereas the other measures remain finite. Munakata and Rosinberg have also observed that the entropy pumping displays a nontrivial structure, attaining a maximum at the maximum cooling rate [23]. Figure 1 demonstrates that ${{\dot{I}}_{{\rm flow}}}$ displays a similar structure, but its maximum does not quite correspond to the maximum cooling. Most likely, this discrepancy arises due to the effect of coarse-graining.

3.6.2. Optimal control and the Kalman–Bucy filter

Interestingly, closer connections exist between the information flow, transfer entropy rate, and maximum extractable work that are revealed by re-examining our feedback problem from the perspective of optimal control theory.

The feedback cooling we have been addressing is a special case of a classic problem in optimal control theory: the characterization of feedback controllers that minimize quadratic performance objectives of the form

$$\begin{eqnarray}&&\mathcal{J}=\langle v_{t}^{2}\rangle +\rho \langle f_{t}^{2}\rangle ,\end{eqnarray} \tag{ 56 }$$

where $\rho \gt 0$ is a constant parameter used to tune the trade-off between keeping small fluctuations in v_t and applying a strong control force f_t, for example [51]. For the special case of cooling, we have been focused on minimizing $\langle v_{t}^{2}\rangle$ alone, which corresponds to $\rho \to 0$.

Assuming linear dynamics and Gaussian noise, the optimal feedback controller with access to noisy measurements ${{v}_{t}}+{{\eta }_{t}}$ can be written in the form

$$\begin{eqnarray}\begin{array}{rcl} m{{\mathop{{\hat{v}}}\limits^{.}}_{t}} & = & -\gamma {{{\hat{v}}}_{t}}-G{{{\hat{v}}}_{t}}+K({{v}_{t}}+{{\eta }_{t}}-{{{\hat{v}}}_{t}}) \\ {{f}_{t}} & = & -G{{{\hat{v}}}_{t}}, \\ \end{array}\end{eqnarray} \tag{ 57 }$$

where ${{\hat{v}}_{t}}$ is the abstract dynamical state of the controller, and G and K are carefully chosen constants. According to the separation principle [51, 52], these parameters G and K can be determined as the solutions to two independent optimization problems: the optimal gain G is obtained by minimizing $\mathcal{J}$, temporarily assuming there is no measurement noise, $\sigma =0$; whereas the optimal K is obtained by minimizing the estimation error, see below, and is independent of the tuning parameter $\rho$. While the exact expression for the optimal gain G is of little interest to us here, we do note that it tends monotonically to infinity as $\rho \to 0$. This makes intuitive sense, since $\rho \to 0$ means we only care about minimizing the variance $\langle v_{t}^{2}\rangle$ and assess no cost for large control forces $\langle f_{t}^{2}\rangle$. On the other hand, optimal filtering theory selects an optimal K by minimizing the estimation error

$$\begin{eqnarray}&&{{\mathcal{E}}_{t}}\equiv \mathop{{\rm min} }\limits_{K}\langle {{({{v}_{t}}-{{\hat{v}}_{t}})}^{2}}\rangle ,\end{eqnarray} \tag{ 58 }$$

given all the past measurements $(v+\eta )_{0}^{t}$. The steady-state optimum, achieved for

$$\begin{eqnarray}&&K=\gamma \left( \sqrt{1+\frac{2T/\gamma }{{{\sigma }^{2}}}}-1 \right),\end{eqnarray} \tag{ 59 }$$

is

$$\begin{eqnarray}&&\mathcal{E}=\frac{{{\sigma }^{2}}}{m}K={{\sigma }^{2}}\frac{\gamma }{m}\left( \sqrt{1+\frac{2T/\gamma }{{{\sigma }^{2}}}}-1 \right).\end{eqnarray} \tag{ 60 }$$

Thus, ${{\hat{v}}_{t}}$ represents the best estimate of v_t given all past measurements. In fact, no other filter, nonlinear or otherwise, can produce a better estimate than the one described here, which is known as the Kalman–Bucy filter [51, 53].

Remarkably, the optimal controller (57) with Kalman–Bucy filter can always be realized using the feedback cooling dynamics in (13) by a simple rescaling

$$\begin{eqnarray}&&{{\hat{v}}_{t}}=\left( \frac{K}{\gamma +K+G} \right)\;{{y}_{t}},\end{eqnarray} \tag{ 61 }$$

and choosing the parameters a and τ as

$$\begin{eqnarray}&&{{a}_{{\rm KB}}}=\frac{GK}{\gamma +K+G},\qquad {{\tau }_{{\rm KB}}}=\frac{m}{\gamma +K+G}.\end{eqnarray} \tag{ 62 }$$

This mapping allows us to investigate our information measures from a new point of view by replacing y_t with the optimal ${{\hat{v}}_{t}}$.

For starters, maximal cooling, which coincides with the maximum extracted work $\dot{W}_{{\rm ext}}^{{\rm max} }$ (50), is obtained when $G\to \infty$, in which case the optimal controller (62) becomes

$$\begin{eqnarray}&&{{a}_{{\rm KB}}}=K=\gamma \left( \sqrt{1+\frac{2T/\gamma }{{{\sigma }^{2}}}}-1 \right),\qquad {{\tau }_{{\rm KB}}}=0,\end{eqnarray} \tag{ 63 }$$

recovering a* and ${{\tau }^{*}}$ in (51) as expected.

The optimal controller also extracts the maximum amount of information⁴ . To see this, first note that optimality of the estimate ${{\hat{v}}_{t}}$ implies that the estimation error is stochastically orthogonal to the estimate $\langle {{\hat{v}}_{t}}({{v}_{t}}-{{\hat{v}}_{t}})\rangle =0$ for all t [51]. This property greatly simplifies the steady-state covariance matrix

$$\begin{eqnarray}{\boldsymbol{ \Sigma }} =\left( \begin{array}{ccccccccccccccc} \sigma _{{\rm v}}^{2} & {{\sigma }_{{\rm v\hat{v}}}} \\ {{\sigma }_{{\rm v\hat{v}}}} & \sigma _{{{\rm \hat{v}}}}^{2} \\ \end{array} \right)=\left( \begin{array}{ccccccccccccccc} \sigma _{{{\rm \hat{v}}}}^{2}+\mathcal{E} & \sigma _{{{\rm \hat{v}}}}^{2} \\ \sigma _{{{\rm \hat{v}}}}^{2} & \sigma _{{{\rm \hat{v}}}}^{2} \\ \end{array} \right),\end{eqnarray} \tag{ 64 }$$

where the variance of the estimate is simply

$$\begin{eqnarray}&&\sigma _{{{\rm \hat{v}}}}^{2}=\frac{{{K}^{2}}{{\sigma }^{2}}}{2m(\gamma +G)}.\end{eqnarray} \tag{ 65 }$$

Note optimal cooling is achieved by $G\to \infty$, forcing $\sigma _{{{\rm \hat{v}}}}^{2}\to 0$, so that fluctuations in the velocity $\sigma _{{\rm v}}^{2}=\mathcal{E}$ are only caused by estimation error. Furthermore, by exploiting the structure of ${\boldsymbol{ \Sigma }}$ in (64), the expression for the steady-state information flow (37) greatly simplifies,

$$\begin{eqnarray}&&{{\dot{I}}_{{\rm flow}}}=\frac{\gamma }{m}\left( \frac{T}{m}\frac{\sigma _{{{\rm \hat{v}}}}^{2}}{|{\boldsymbol{ \Sigma }} |}-1 \right)=\frac{\gamma }{m}\left( \frac{T}{m\mathcal{E}}-1 \right)=\frac{\gamma }{2\;m}\left( \sqrt{1+\frac{2T/\gamma }{{{\sigma }^{2}}}}-1 \right)={{\dot{I}}_{{\rm v}\to {\rm y}}},\end{eqnarray} \tag{ 66 }$$

for all G. This is a very interesting observation, supporting the claimed optimality of the Kalman–Bucy filter. We already know that ${{\dot{I}}_{{\rm flow}}}\leqslant {{\dot{I}}_{{\rm v}\to {\rm y}}}$. What we see here is that the class of controllers given by (57), i.e., with K fixed (59) and G free, saturates the bound, maximizing the information flow. Hence, a controller with a small gain G (zero even) only uses information to create an optimal estimate of the process, whereas a high gain cools as well. To gain further insight into equality (66), we have to look at the transfer entropy rate and information flow from a different perspective. Namely, the transfer entropy rate can also be defined as the rate of growth of the mutual information between v_t and the entire trajectory of measurement outcomes y₀^t, that is the change in $I({{v}_{t}};y_{0}^{t})$. On the other hand, the information flow is the rate of growth of the mutual information between v_t and just the most recent measurement y_t, that is the change in $I({{v}_{t}};{{y}_{t}})$. The inequality ${{\dot{I}}_{{\rm flow}}}\leqslant {{\dot{I}}_{{\rm trans}}}$ is then related to the simple idea that the entire trajectory of measurements contains more information than just the last. Now, it is known that the Kalman–Bucy filter ${{\hat{v}}_{t}}$ is a sufficient statistic for the conditional distribution of v_t given the measurements $y_{0}^{t}$ [54]. In other words, everything useful in a collection of measurements for predicting v_t is contained in just ${{\hat{v}}_{t}}$, or in terms of the mutual information $I({{v}_{t}};\hat{v}_{0}^{t})=I({{v}_{t}};{{\hat{v}}_{t}})$. This equality translated into rates implies (66).

In figure 2, we illustrate how the extracted work depends on G, and how the maximum is asymptotically achieved. In addition, we see that ${{\dot{I}}_{{\rm flow}}}={{\dot{I}}_{{\rm v}\to {\rm y}}}$ holds for all G. We can also conclude that with certain choices of a and τ (namely ${{a}_{{\rm KB}}}$ and ${{\tau }_{{\rm KB}}}$ in (62)) our original setup (13) can always saturate ${{\dot{I}}_{{\rm flow}}}\leqslant {{\dot{I}}_{{\rm v}\to {\rm y}}}$, which is indeed observed in figure 1 for $a\approx 2$.

Zoom In Zoom Out Reset image size

Let us start with the simpler measurement scenario corresponding to the information flow ${{\dot{I}}_{{\rm flow}}}$. Actually, we have already touched on its physical interpretation when we introduced it in section 3.2. Recall, there we considered the measurement outcomes y_t to correspond to a physical degree of freedom of an auxiliary system. We now clarify that interpretation by taking y_t to be the position of a secondary, harmonically-trapped, overdamped Brownian particle. To be thermodynamically consistent, the origin of the measurement noise must be a thermal reservoir, which requires imposing the Fluctuation-Dissipation theorem [39]

$$\begin{eqnarray}&&{{\sigma }^{2}}=2\tau T.\end{eqnarray} \tag{ 67 }$$

We have chosen the temperature of the measurement device to be the same as the controlled system, which is the customary choice. From this point of view, (13) is the equation of motion for an overdamped Brownian particle with viscous damping coefficient τ trapped in a harmonic potential $V(y,v)={{(y-v)}^{2}}/2$ of unit spring constant, centered about the velocity, as illustrated in figure 3. Alternatively, such a coupling can be implemented in an electric circuit as was presented in [11]. The result is that the position of the measurement oscillator y_t feels a fluctuating force making it track the velocity v_t, thereby establishing and maintaining correlations. Roughly speaking, the measurement oscillator is constantly learning new information at a rate ${{\dot{I}}_{{\rm flow}}}$, which keeps getting rewritten in the value of its position.

Zoom In Zoom Out Reset image size

When introducing the information flow, we divided the entropy production into two positive contributions (30), one due to the velocity $\dot{S}_{{\rm i}}^{v}$, and another due to the measurements $\dot{S}_{{\rm i}}^{y}$. When studying the extracted work ${{\dot{W}}_{{\rm ext}}}$, we focused on $\dot{S}_{{\rm i}}^{v}$. However, a similar analysis also holds for $\dot{S}_{{\rm i}}^{y}$, which verifies that the y-system must consume at least ${{\dot{I}}_{{\rm flow}}}$ free energy to sustain the correlations that promote feedback. Observing that as a position y_t is even under time-reversal (consistent with our previous analysis in section 2.1), we develop its thermodynamics by splitting its current J^y_t (15) into irreversible and reversible portions as

$$\begin{eqnarray}\begin{array}{rcl} J_{t}^{{\rm irr},y}(v,y) & = & -\frac{1}{\tau }y{{p}_{t}}(v,y)-\frac{T}{\tau }{{\partial }_{y}}{{p}_{t}}(v,y), \\ J_{t}^{{\rm rev},y}(v,y) & = & \frac{v}{\tau }{{p}_{t}}(v,y). \\ \end{array}\end{eqnarray} \tag{ 68 }$$

Notice that here the irreversible current is the time-symmetric contribution, since y_t is even under time-reversal [36]. Then, repeating the analysis in section 3.2, we have that in the steady state [16–18]

$$\begin{eqnarray}&&\dot{S}_{{\rm i}}^{y}=\frac{\tau }{T}\int \frac{{{[J_{{\rm s}}^{{\rm irr},y}(v,y)]}^{2}}}{{{p}_{{\rm s}}}(v,y)}{\rm d}v{\rm d}y=\frac{{{{\dot{Q}}}^{y}}}{T}-{{\dot{I}}_{{\rm flow}}}\geqslant 0,\end{eqnarray} \tag{ 69 }$$

where ${{\dot{Q}}^{y}}=-\int yJ_{{\rm s}}^{{\rm irr},{\rm y}}(v,y){\rm d}v{\rm d}y=(\sigma _{{\rm y}}^{2}-T)/\tau$ is the heat flow rate into yʼs reservoir. Thus, in the steady state

$$\begin{eqnarray}&&{{\dot{Q}}^{y}}\geqslant T{{\dot{I}}_{{\rm flow}}}.\end{eqnarray} \tag{ 70 }$$

In order to track the velocity, yʼs environment continually absorbs heat at a rate ${{\dot{Q}}^{y}}$. We verify this bound in figure 4, where ${{\dot{Q}}^{y}}$ is plotted with ${{\dot{I}}_{{\rm flow}}}$. The minimum ${{\dot{Q}}^{y}}=T{{\dot{I}}_{{\rm flow}}}$ is reached when the measurement device operates reversibly in the limit $\tau \ll {{\tau }_{{\rm v}}}$, so that y_t rapidly relaxes to its instantaneous equilibrium centered about v_t: ${{p}_{{\rm s}}}(y|v)\propto {\rm exp} [-{{(y-v)}^{2}}/(2T)]$.

Zoom In Zoom Out Reset image size

In addition, we have already argued that the entropy pumping lower bounds the information flow, ${{\dot{I}}_{{\rm flow}}}\geqslant {{\dot{I}}_{{\rm pump}}}$ (54). As a result, ${{\dot{I}}_{{\rm pump}}}$ offers a weaker lower bound on the energy required for an auxiliary system to provide the entropy-pumping feedback, ${{\dot{Q}}^{y}}\geqslant T{{\dot{I}}_{{\rm pump}}}$, which is verified in figure 4 as well.

The transfer entropy rate can also be understood as the minimum free energy required to measure, but with an alternative measurement scenario. In the previous section, the information flow was shown to bound the energy required to constantly rewrite a single memory with each new measurement y_t. By contrast, the setup for the transfer entropy rate is much closer to that envisioned by Landauer and Bennett in their thermodynamics of computation [2, 55]: each measurement is recorded separately in one of a collection of memories through a specific driven thermodynamic process; one example of which was recently described in [25].

The central idea is that each measurement outcome is recorded in a distinct memory. Therefore, to track the system over any finite time interval, say from time s = 0 to t, we require an infinite number of memories in which to record the infinity of measurements. However, this is difficult to analyze. So to proceed, we discretize time as ${{s}_{k}}=k\Delta s$, with $k=0,\ldots ,N$ and $\Delta s=t/N$, where the measurement outcome at time s_k is denoted simply as ${{y}_{k}}\equiv {{y}_{{{s}_{k}}}}$, and similarly ${{v}_{k}}\equiv {{v}_{{{s}_{k}}}}$. To store these measurement outcomes, we imagine a collection of N auxiliary memories with phase space positions m_k, prepared initially in positions ${{m}_{k,0}}$ distributed according to ${{\rho }_{0}}({{m}_{k,0}})$. The measurement is a thermodynamic process during a time interval of length θ in which the kth memory is manipulated, with the velocity fixed, in such a way to reproduce the correlations with ${{v}_{k-1}}$ embodied in the measurement statistics of y_k. In other words, we demand that the statistics of the kth memory after the measurement are ${{m}_{k,\theta }}\sim {{y}_{k}}$ (equality in distribution).

To see how these ideas play out in our model system, consider the discretized version of the Langevin equation (13)

$$\begin{eqnarray}&&{{y}_{k}}={{y}_{k-1}}-\frac{\Delta s}{\tau }({{y}_{k-1}}-{{v}_{k-1}})+\frac{1}{\tau }\Delta {{\eta }_{k-1}},\end{eqnarray} \tag{ 71 }$$

where the $\Delta {{\eta }_{k}}$ are independent Gaussian random variables of zero mean and covariance $\langle \Delta {{\eta }_{k}}\Delta {{\eta }_{l}}\rangle ={{\sigma }^{2}}\Delta s{{\delta }_{kl}}$. Equation (71) is a rule that tells us how the measurement outcome y_k at time s_k depends on the velocity ${{v}_{k-1}}$ as well as the past measurement outcome ${{y}_{k-1}}$ stored in a previous memory. Such measurements that depend on past outcomes are sometimes called non-Markovian measurements [42]. Specifically, y_k is characterized by a Gaussian probability density

$$\begin{eqnarray}\begin{array}{rcl} P({{y}_{k}}|{{y}_{k-1}},{{v}_{k-1}}) & = & \frac{1}{\sqrt{2\pi {{\Omega }^{2}}}}{\rm exp} \left\{ -\frac{{{[{{y}_{k}}-{{\mu }_{k}}({{v}_{k-1}},{{y}_{k-1}})]}^{2}}}{2{{\Omega }^{2}}} \right\} \\ {{\mu }_{k}}({{v}_{k-1}},{{y}_{k-1}}) & = & {{y}_{k-1}}-\frac{\Delta s}{\tau }({{y}_{k-1}}-{{v}_{k-1}}),\qquad {{\Omega }^{2}}=\frac{{{\sigma }^{2}}\Delta s}{{{\tau }^{2}}}, \\ \end{array}\end{eqnarray} \tag{ 72 }$$

roughly centered about the velocity with a variance depending on the measurement error. Now, in view of our previous discussion, we desire a physical system to act as a memory and a measurement process that prepares that system in a statistical state with the probability density in (72). A natural choice is an overdamped harmonic oscillator coupled to a thermal reservoir at temperate T. Initially each memory oscillator is prepared in equilibrium with an arbitrary initial spring constant k₀ centered about zero, as illustrated in figure 5. Since each measurement is performed in sequence, it is attractive to visualize the phase spaces of the N measurement oscillators aligned in a row, or tape. Then one by one we couple each measurement oscillator to the system as well as past memories, so as to establish correlations. The density in (72) suggests that the measurement protocol for the kth oscillator should be the quasistatic turn-on of an interaction that shifts the center of the harmonic oscillator to ${{\mu }_{k}}$—which includes interactions with the past memories—while simultaneously expanding the spring constant to ${{k}_{1}}=T/{{\Omega }^{2}}$, which results in the interaction potential

$$\begin{eqnarray}&&V({{m}_{k}},{{m}_{k-1}},{{v}_{k-1}})=\frac{T}{2{{\Omega }^{2}}}{{[{{m}_{k}}-{{\mu }_{k}}({{v}_{k-1}},{{m}_{k-1}})]}^{2}}\end{eqnarray} \tag{ 73 }$$

as depicted in figure 5. As a result, upon completion of the kth measurement the memoryʼs position ${{m}_{k,\theta }}$ has settled into an equilibrium distribution ${{\rho }_{\theta }}({{m}_{k,\theta }}|{{m}_{k-1,\theta }},{{v}_{k-1}})\propto {\rm exp} \left[ -V({{m}_{k,\theta }},{{m}_{k-1,\theta }},{{v}_{k-1}})/T \right]$ equivalent to (72). To complete the measurement, we must freeze the state of the memory to lock in the correlations, and remove the interactions by turning off V. One possible, though admittedly idealized, option is to instantaneously set V = 0, and then immediately turn off the dynamics of the measurement oscillator—perhaps by quenching the temperature to zero—so that the oscillator can no longer move. By repeating this sequence of actions on each new memory, we store a collection of measurement outcomes, each in a different physical memory. Now to be precise each measurement has to be performed instantaneously from the point of view of the velocity. This merely means that the time-scale of the evolution of the individual memories must be much faster than the velocity, $\theta \ll {{\tau }_{{\rm v}}}$, so that the measurement is completed before the velocity changes appreciably [25]. However, this assumption is not unreasonable, since measurements are usually assumed to read out the instantaneous state of the system.

Zoom In Zoom Out Reset image size

Having described how we can mimic the measurement statistics in a physical situation, we now address the thermodynamics from a general point of view, applying the methods of [19, 20, 25]. Our analysis is based on the following second-law-like inequality that relates the work performed in an isothermal process to the increment in the nonequilibrium free energy [56, 57]: for a thermodynamic system with microscopic states z, the work W performed along an isothermal process during which the systemʼs probability density transitions from p(z) to ${{p}^{\prime }}({{z}^{\prime }})$ is bounded as

$$\begin{eqnarray}&&W-\Delta \mathcal{F}({{z}^{\prime }})\geqslant 0,\end{eqnarray} \tag{ 74 }$$

where $\Delta \mathcal{F}({{z}^{\prime }})=\mathcal{F}({{z}^{\prime }})-\mathcal{F}(z)$ is the change in the nonequilibrium free energy $\mathcal{F}(z)=U(z)-TS(z)$ defined in terms of the average internal energy U(z) and entropy $S(z)=-\int p(z){\rm ln} p(z){\rm d}z$. The nonequilbirium free energy is a natural extension of the equilibrium free energy to systems characterized by an arbitrary probability density, since it reduces to the equilibrium free energy for systems in equilibrium.

We begin by focusing on the work done during the kth measurement, W_k, during which the kth memory becomes correlated with not only the velocity ${{v}_{k-1}}$ but all the past memories $m_{0}^{k-1}=\{{{m}_{l,\theta }}\}_{l=0}^{k-1}\sim \{{{y}_{l}}\}_{l=0}^{k-1}$ through the velocity which depends on the entire past. Applying (74), we have

$$\begin{eqnarray}&&{{W}_{k}}-\Delta \mathcal{F}({{m}_{k,\theta }}|m_{0}^{k-1},{{v}_{k-1}})\geqslant 0,\end{eqnarray} \tag{ 75 }$$

where $\Delta \mathcal{F}({{m}_{k,\theta }}|m_{0}^{k-1},{{v}_{k-1}})=\mathcal{F}({{m}_{k,\theta }}|m_{0}^{k-1},{{v}_{k-1}})-\mathcal{F}({{m}_{k,0}}|m_{0}^{k-1},{{v}_{k-1}})$ is the change in the nonequilibrium free energy of the kth memory, corresponding to the change in the conditional density from ${{\rho }_{0}}({{m}_{k,0}}|m_{0}^{k-1},{{v}_{k-1}})={{\rho }_{0}}({{m}_{k,0}})$—due to the initial independence of each memory—to ${{\rho }_{\theta }}({{m}_{k,\theta }}|m_{0}^{k-1},{{v}_{k-1}})={{\rho }_{\theta }}({{m}_{k,\theta }}|{{m}_{k-1,\theta }},{{v}_{k-1}})$. We single out the new correlations by introducing the mutual information between ${{m}_{k,\theta }}$ and ${{v}_{k-1}}$ conditioned on all the past measurements as $I({{m}_{k,\theta }};{{v}_{k-1}}|m_{0}^{k-1})=S({{m}_{k,\theta }}|m_{0}^{k-1})-S({{m}_{k,\theta }}|m_{0}^{k-1},{{v}_{k-1}})$ [44]. Substituting in this definition, (75) becomes

$$\begin{eqnarray}&&{{W}_{k}}-\Delta \mathcal{F}({{m}_{k,\theta }}|m_{0}^{k-1})\geqslant TI({{m}_{k,\theta }};{{v}_{k-1}}|m_{0}^{k-1}),\end{eqnarray} \tag{ 76 }$$

where $\Delta \mathcal{F}({{m}_{k,\theta }}|m_{0}^{k-1})$ is the change in free energy conditioned on just the past memories: ${{\rho }_{0}}({{m}_{k,0}})\to {{\rho }_{\theta }}({{m}_{k,\theta }}|m_{0}^{k-1})$. Summing over all measurements, we find

$$\begin{eqnarray}&&W-\Delta \mathcal{F}(m_{1}^{N}|{{m}_{0}})\geqslant TI_{{\rm v}\to {\rm y}}^{N},\end{eqnarray} \tag{ 77 }$$

where $W=\sum _{k=1}^{N}{{W}_{k}}$ is the work to perform all N measurements, $\Delta \mathcal{F}(m_{1}^{N}|{{m}_{0}})=\sum _{k=1}^{N}\Delta \mathcal{F}({{m}_{k,\theta }}|m_{0}^{k-1})$ is the change in entire tapeʼs free energy, and we have identified the discrete version of the transfer entropy [10]

$$\begin{eqnarray}&&I_{{\rm v}\to {\rm y}}^{N}=\mathop{\sum }\limits_{k=1}^{N}I({{m}_{k,\theta }};{{v}_{k-1}}|m_{0}^{k-1}),\end{eqnarray} \tag{ 78 }$$

which is reviewed in appendix B. Importantly, by construction, the statistics of each memory reproduce the statistics of the measurement outcomes, so equivalently

$$\begin{eqnarray}&&I_{{\rm v}\to {\rm y}}^{N}=\mathop{\sum }\limits_{k=1}^{N}I({{y}_{k}};{{v}_{k-1}}|y_{0}^{k-1})=\left\langle {\rm ln} \frac{\hat{\mathcal{P}}[y_{0}^{N}\mid v_{0}^{N},{{y}_{0}}]}{\mathcal{P}[y_{0}^{N}\mid {{y}_{0}}]} \right\rangle .\end{eqnarray} \tag{ 79 }$$

Taking the limit as the number of measurements go to infinity while the time between them goes to zero, we obtain

$$\begin{eqnarray}&&\dot{W}-\dot{\mathcal{F}}=\mathop{{\rm lim} }\limits_{t\to \infty }\frac{1}{t}\left[ \mathop{{\rm lim} }\limits_{\Delta s\to 0}W-\Delta \mathcal{F}(m_{1}^{N}|{{m}_{0}}) \right]\geqslant T{{\dot{I}}_{{\rm v}\to {\rm y}}}.\end{eqnarray} \tag{ 80 }$$

Thus, the transfer entropy rate is the minimum rate at which free energy is consumed to write to the memories. The slow protocol that we described previously saturates this bound, since it is quasistatic and therefore thermodynamically reversible.

At this point, it is worthwhile to make a connection to a class of Maxwell-demon models that exploit a tape of low entropy, auxiliary systems or cells, similar to what we have just described [26, 58–62]. Apart from the study in [62], these models use an ideal tape that has no internal energy, and therefore cannot exchange energy with the system, but only entropy; a setup sometimes referred to as an information reservoir [26, 60]. Under these conditions, a second-law-like inequality has been predicted that shows that the extracted work is bounded by the increase in entropy of each individual auxiliary system, ignoring the correlations between the different cells. Our memories, on the other hand, have internal energy and therefore the natural thermodynamic quantity to consider is the free energy instead of the entropy. Therefore to fit our measurement model into this tape-model framework, we must relate our information bound on the work to measure to a bound that ignores the correlations. To this end, we start with the bound for the energy to measure $W-\Delta \mathcal{F}(m_{1}^{N}|{{m}_{0}})\geqslant TI_{{\rm v}\to {\rm y}}^{N}$ in (77), which includes through $\Delta \mathcal{F}$ all the correlations between different memories. By noting that ignoring correlations and conditioning increases the entropy, $H(m_{1}^{N}|{{m}_{0}})\leqslant {{\sum }_{k}}H({{m}_{k,\theta }})$ [44], we can conclude that ignoring the correlations decreases the free energy $\mathcal{F}(m_{1}^{N}|{{m}_{0}})\geqslant {{\sum }_{k}}\mathcal{F}({{m}_{k,\theta }})$. As a result, we have from (77) and the initial independence of each memory the series of inequalities

$$\begin{eqnarray}&&W-\mathop{\sum }\limits_{k=1}^{N}\Delta \mathcal{F}({{m}_{k,\theta }})\geqslant TI_{{\rm v}\to {\rm y}}^{N}\geqslant {{W}_{{\rm ext}}}.\end{eqnarray} \tag{ 81 }$$

For the ideal tape with no internal energy this reduces to ${{\sum }_{k}}\Delta H({{m}_{k,\theta }})\geqslant {{W}_{{\rm ext}}}$ recovering the ideal-tape bound [26, 58–62] in our setup. Equation (81) may lead us to conclude that the bound on the extracted work from the tape-model framework, $W-\sum _{k=1}^{N}\Delta \mathcal{F}({{m}_{k,\theta }})$, is weaker than that provided by the transfer entropy. However, this would be too hasty, because these tape models allow a more general interaction between the tape cells and the system. Whereas, in our setup the memory evolution is assumed to occur separately with the velocity fixed, the tape models consider a dynamics where the memory (or cell) would be allowed to evolve simultaneously with the velocity. From this point of view, the measurement model we have presented is a special case of these more general tape models, and it is exactly our assumption that the velocity is frozen during measurement that allows us to tighten the tape-model bound using the transfer entropy. Further comparisons of such tape models with other information measures and more traditional statements of the second law can be found in [25, 26].

Finally, it should be noted that the preceding second law analysis can be viewed as a specific implementation of the information flow framework (outlined in sections 3.2 and 4.1) applied to a nonautonomously driven auxiliary memory composed of a sequence of many subsystems, see [18].

The transfer entropy rate and information flow both bound the energy consumed during measurement. However, each measurement scenario is distinct, and in general each of these information measures will not bound the energy consumption for the otherʼs measurement scenario. An example where ${{\dot{I}}_{{\rm v}\to {\rm y}}}\gt {{\dot{Q}}^{y}}/T$ is possible is presented in [16]; thus, the transfer entropy rate does not generally lower bound the amount of heat dissipated by a single memory being constantly rewritten. Our model corroborates this observation, as verified in figure 4 by the crossing of ${{\dot{Q}}^{y}}/T$ and ${{\dot{I}}_{{\rm v}\to {\rm y}}}$. The one exception is if the the controller implements the Kalman–Bucy filter (57). In which case, the equality of the information measures, ${{\dot{I}}_{{\rm flow}}}={{\dot{I}}_{{\rm v}\to {\rm y}}}$, implies that there is a unique lower bound to the energy required for measurement.

To conclude this section, we take a broader perspective. Our observation that the transfer entropy rate and information flow both represent the minimum (free) energy consumed (or alternatively the entropy produced) in the auxiliary memory to create that information, suggests that it is reasonable to interpret some second-law-like inequalities as actually telling us something about the thermodynamics of the system and its surroundings, where the surroundings include the measurement device. This allows us to incorporate information into the standard statement of the second law of thermodynamics through a kind of information reservoir on equal footing with other traditional thermodynamic reservoirs, similar to what was suggested for tapes in [26, 60]

$$\begin{eqnarray}&&{{\dot{S}}_{{\rm i}}}=\dot{S}+{{\dot{S}}_{{\rm env}}}=\dot{S}-\frac{{\dot{Q}}}{T}+\dot{I}\geqslant 0,\end{eqnarray} \tag{ 82 }$$

which is equivalent to (1) in the steady state. Here, $\dot{I}$ represents the minimum entropy produced in the environment that allows for feedback, with the minimum attained for reversible measurement. The appropriate choice of $\dot{I}$—transfer entropy rate or information flow—depends on which type of information reservoir we wish to use. From this point of view, the efficiency introduced in (55) is a true measure of energetic efficiency that quantifies how faithfully the energy supplied by a reversible memory is extracted back out as work.