An alternating iterative minimisation algorithm for the double-regularised total least square functional

It is well known that the study of local behaviour of nonsmooth functions can be achieved by the concept of subdifferentiality which replaces the classical derivative at non-differentiable points.

The first-order necessary condition based on subdifferentiability is stated as the following: if $(\bar{k},\bar{f})$ minimises the functional $J_{\alpha ,\beta }^{\delta ,\varepsilon }$ then

$$\begin{eqnarray}&&(0,0)\in \partial J_{\alpha ,\beta }^{\delta ,\varepsilon }\left( \bar{k},\bar{f} \right).\end{eqnarray} \tag{ 9 }$$

We denote the set of all subderivatives of the functional $J_{\alpha ,\beta }^{\delta ,\varepsilon }$ at (k, f) by $\partial J_{\alpha ,\beta }^{\delta ,\varepsilon }(k,f)$ and we name it the subdifferential of $J_{\alpha ,\beta }^{\delta ,\varepsilon }$ at (k, f). For a quick revision on subdifferentiability we refer to the apppendix.

The first result gives us the derivative of a bilinear operator B.

Lemma 3.1. Let B be a bilinear operator and assume that (2) holds. Then the Fréchet derivative of B at $(k,f)\in \mathcal{U}\times \mathcal{V}$ is given by

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} B^{\prime} (k,f)(u,v) & = & B(k,v)+B(u,f) \\ {} & = & {{A}_{k}}v+{{C}_{f}}u. \\ \end{array}\end{eqnarray*}$$

Moreover, the derivative is Lipschitz continuous with constant $\sqrt{2}C$.

Proof. We have to show

$$\begin{eqnarray*}&&B(k+u,f+v)=B(k,f)+B^{\prime} (k,f)(u,v)+o(\parallel (u,v)\parallel ).\end{eqnarray*}$$

Since B is bilinear, we have

$$\begin{eqnarray*}&&B(k+u,f+v)-B(k,f)=B(k,v)+B(u,f)+B(u,v),\end{eqnarray*}$$

and we observe $\parallel B(u,v)\;\parallel =o(\parallel (u,v)\parallel )$: As B fulfills (2), we have

$$\begin{eqnarray*}&&\frac{\parallel B(u,v)\parallel }{\parallel (u,v)\parallel }\leqslant \frac{C\parallel u\parallel \parallel v\parallel }{{{(\parallel u{{\parallel }^{2}}+\parallel v{{\parallel }^{2}})}^{1/2}}}\leqslant \frac{C}{\sqrt{2}}{{(\parallel u\parallel \parallel v\parallel )}^{1/2}},\end{eqnarray*}$$

which converges to zero as $(u,v)\to 0$.

We further observe

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} B^{\prime} (k,f)(u,v)-B^{\prime} \left( \tilde{k},\tilde{f} \right)(u,v) & = & B(k,v)+B(u,f)-\left( B(\tilde{k},v)+B(u,\tilde{f}) \right) \\ {} & = & B(u,f-\tilde{f})+B(k-\tilde{k},v) \\ \end{array}\end{eqnarray*}$$

which implies

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} \left\|B^{\prime} (k,f)(u,v)-B^{\prime} \left( \tilde{k},\tilde{f} \right)(u,v)\right\| & = & \left\|B(u,f-\tilde{f})+B(k-\tilde{k},v)\right\| \\ {} & \leqslant & \left\|B(u,f-\tilde{f})\right\|+\left\|B(k-\tilde{k},v)\right\| \\ {} & \leqslant & C\parallel u\parallel \left\|f-\tilde{f}\right\|+C\left\|k-\tilde{k}\right\|\parallel v\parallel \\ \end{array}\end{eqnarray*}$$

Using the inequality ${{(a+b)}^{2}}\leqslant 2({{a}^{2}}+{{b}^{2}})$ we get

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} \left\|B^{\prime} (k,f)(u,v)-B^{\prime} \left( \tilde{k},\tilde{f} \right)(u,v)\right\|{^{2}} & \leqslant & 2{{C}^{2}}\left( \parallel u{{\parallel }^{2}}\left\|f-\tilde{f}\right\|{^{2}}+\left\|k-\tilde{k}\right\|{^{2}}\parallel v{{\parallel }^{2}} \right) \\ {} & \leqslant & 2{{C}^{2}}\left( \parallel u{{\parallel }^{2}}+\parallel v{{\parallel }^{2}} \right)\left( \left\|k-\tilde{k}\right\|{^{2}}+\left\|f-\tilde{f}\right\|{^{2}} \right) \\ {} & = & 2{{C}^{2}}\parallel (u,v){{\parallel }^{2}}\left\|(k-\tilde{k},f-\tilde{f})\right\|{^{2}} \\ \end{array}\end{eqnarray*}$$

and thus

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} \left\|B^{\prime} (k,f)-B^{\prime} (\tilde{k},\tilde{f})\right\| & = & \mathop{{\rm sup} }\limits_{\parallel (u,v)\parallel =1}\left\|B^{\prime} (k,f)(u,v)-B^{\prime} \left( \tilde{k},\tilde{f} \right)(u,v)\right\| \\ {} & \leqslant & \sqrt{2}C\left\|(k-\tilde{k},f-\tilde{f})\right\|. \\ \end{array}\end{eqnarray*}$$

□

Note that the adjoint operator ${{(B^{\prime} (k,f))}^{*}}$ of the Frechét derivative $B^{\prime} (k,f)$ exists and is a bounded linear operator whenever both $\mathcal{H}$ and $\mathcal{U}\times \mathcal{V}$ are Hilbert spaces.

In order to analyse the optimality condition (9) we shall compute the subdifferential of a functional over two variables. As pointed out in [6, proposition 2.3.15] for a general function h the set-valued mapping $\partial h:\mathcal{U}\;\rightrightarrows \;{{\mathcal{U}}^{*}}$ the set $\partial h({{x}_{1}},{{x}_{2}})$ and the product set ${{\partial }_{1}}h({{x}_{1}},{{x}_{2}})\times {{\partial }_{2}}h({{x}_{1}},{{x}_{2}})$ are not necessarily contained in each other. Here, ${{\partial }_{i}}h$ denotes the partial subgradient with respect to x_i for $i=1,2$. However this is not the case for the functional we are interested in as will be shown in the following theorem.

Theorem 3.2. Let $J:\mathcal{U}\times \mathcal{V}\to \bar{\mathbb{R}}$ be a functional with the structure

$$\begin{eqnarray}&&J(u,v)=\varphi (u)+Q(u,v)+\psi (v),\end{eqnarray} \tag{ 10 }$$

where Q is a (nonlinear) differentiable term and $\varphi :\mathcal{U}\to \bar{\mathbb{R}}$, $\psi :\mathcal{V}\to \bar{\mathbb{R}}$ are proper convex functions, $u\in {\rm dom}\;\varphi$ and $v\in {\rm dom}\;\psi$ . Then

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} \partial J(u,v) & = & \{\partial \varphi (u)+Q_{u}^{\prime }(u,v)\}\times \{\partial \psi (v)+Q_{v}^{\prime }(u,v)\} \\ {} & = & \{{{\partial }_{u}}J(u,v)\}\times \{{{\partial }_{v}}J(u,v)\}. \\ \end{array}\end{eqnarray*}$$

Proof. In general the subdifferential of a sum of functions does not equal the sum of its subdifferentials. However, if Q is differentiable, $\varphi$ and ψ are convex some inclusions and even equalities hold true (combining [6, proposition 2.3.3; corollary 3; proposition 2.3.6]), as for instance,

$$\begin{eqnarray*}&&\partial J(u,v)=\partial (\varphi (u)+\psi (v))+\partial Q(u,v).\end{eqnarray*}$$

Since Q is differentiable, calling the previous results, the (partial) subderivative is unique [6], proposition 2.3.15] and therefore

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} \partial Q(u,v) & = & {{\partial }_{u}}Q(u,v)\times {{\partial }_{v}}Q(u,v) \\ {} & = & \left( Q_{u}^{\prime }(u,v),Q_{v}^{\prime }(u,v) \right). \\ \end{array}\end{eqnarray*}$$

Note that the subderivative of the sum of two separable convex functionals satisfies

$$\begin{eqnarray*}&&\partial (\varphi (u)+\psi (v))=(\partial \varphi (u),\partial \psi (v))\end{eqnarray*}$$

see [18, corollary 2.4.5].

Altogether, we can compute the subderivative as follows

$$\begin{eqnarray}\begin{array}{ccccccccccccccc} \partial J(u,v) & = & (\partial \varphi (u),\partial \psi (v))+\left( Q_{u}^{\prime }(u,v),Q_{v}^{\prime }(u,v) \right) \\ {} & = & \{{{\partial }_{u}}\varphi (u)+Q_{u}^{\prime }(u,v)\}\times \{{{\partial }_{v}}\psi (v)+Q_{v}^{\prime }(u,v)\}. \\ \end{array}\end{eqnarray} \tag{ 11 }$$

The last implication of this theorem,

$$\begin{eqnarray*}&&\partial J(u,v)=\{{{\partial }_{u}}J(u,v)\}\times \{{{\partial }_{v}}J(u,v)\}\end{eqnarray*}$$

follows straightforward by the definition of partial subderivative and (11). □

Please note that the above proof holds for all definitions of subdifferentials introduced in the appendix, as for convex functionals all the definitions are equivalent, and for differentiable (possibly nonlinear) terms the subdifferential is a singleton and the subderivative equals the derivative. Based on theorem 3.2 we can now calculate the derivative of the functional which is the gist for building up the upcoming algorithm; please give heed to the structure of (10) and the proposed functional $J_{\alpha ,\beta }^{\delta ,\varepsilon }$:

Corollary 3.3. Let $J_{\alpha ,\beta }^{\delta ,\varepsilon }$ the functional defined in (8), then

$$\begin{eqnarray*}&&\partial J_{\alpha ,\beta }^{\delta ,\varepsilon }(k,f)=\{C_{f}^{*}({{C}_{f}}k-{{g}_{\delta }})+\gamma (k-{{k}_{\epsilon }})+\beta \zeta \}\times \{A_{k}^{*}({{A}_{k}}f-{{g}_{\delta }})+\alpha {{L}^{*}}Lf\}\end{eqnarray*}$$

where $\zeta \in \partial \mathcal{R}(k)$.

Proof. The result follows straightforward from lemma 3.1 and theorem 3.2. Observe that the sum $C_{f}^{*}({{C}_{f}}k-{{g}_{\delta }})+\gamma (k-{{k}_{\epsilon }})+\beta \zeta$ is well-defined in the Hilbert space $\mathcal{U}$, since the subderivative $\zeta \partial \mathcal{R}(k)$ is also an element of $\mathcal{U}$.□

Up to now, we did not specify the functional $\mathcal{R}$, it is only required to be convex and lower semi-continuous. We are particularly interested in, e.g. the L_p norm or the weighted ℓ_p norm, denoted by $\mathcal{R}(k)=\parallel k{{\parallel }_{w,p}}$. Its subdifferential is given in section 4. An easy way to compute the subderivatives of functionals $\mathcal{R}$ with a specific structure is given by the following lemma.

Lemma 3.4. [(3, lemma 4.4)] Let $\mathcal{H}={{L}_{2}}(\Omega ,d\mu )$ where Ω is a σ-finite measure space. Let $\mathcal{R}:\mathcal{H}\to (-\infty ,+\infty ]$ be defined by

$$\begin{eqnarray}\mathcal{R}(u)=\left\{ \begin{array}{ccccccccccccccc} {{\int }_{\Omega }}h(u)d\mu & {\rm if}\;{\rm the}\;{\rm integral}\;{\rm is}\;{\rm finite} \\ \infty & {\rm else,} \\ \end{array} \right.\end{eqnarray} \tag{ 12 }$$

where $h:\mathbb{C}\to \mathbb{R}$ is a convex function. Then $\xi \in \mathcal{H}$ is an element of $\partial \mathcal{R}(u)$ if and only if $\xi (x)\in \partial h(u(x))$ for almost every $x\in \Omega$ (with the identification $\mathbb{C}={{\mathbb{R}}^{2}}$).

Coordinate descent methods are based on the idea that the minimisation of a multivariable function can be achieved by minimising it along one direction at a time. It is a simple and surprisingly efficient technique. The coordinates can be chosen arbitrarily with any permutation, but one can also replace them by block coordinates (for more details see [16] and references therein). This method is closely related to coordinate gradient descent (CGD), Gauss–Seidel and SOR methods, which was studied previously by several authors and described in various optimisation books, e.g. [1, 14]. In the unconstrained setting the method is called alternating minimisation (AM) when the variables are split into two blocks.

The computation of a solution of dbl-RTLS is not straightforward, as determining the minimum of the functional (8) with respect to both parameters is a nonlinear and nonconvex problem over two variables. Nevertheless we shall overcome this problem by applying some coordinate descent techniques.

In the following we shall denote the dbl-RTLS functional by J instead of $J_{\alpha ,\beta }^{\delta ,\varepsilon }$, as the parameters of the functionals are kept fix for the minimisation process.

In the AM algorithm, the functional is minimised iteratively with two alternating minimisation steps. Each step minimises the problem over one variable while keeping the second variable fixed:

$$\begin{eqnarray}&&{{f}^{n+1}}\in \mathop{{\rm arg} \;{\rm min} }\limits_{f\in V}J(k,f|{{k}^{n}})\end{eqnarray} \tag{ 13a }$$

$$\begin{eqnarray}&&{{k}^{n+1}}\in \mathop{{\rm arg} \;{\rm min} }\limits_{k\in U}J(k,f|{{f}^{n+1}}).\end{eqnarray} \tag{ 13b }$$

The notation $J(k,f|u)$ means we minimise the function J with u fixed, where u can be either k or f. Thus we minimise in each cycle the functionals

$$\begin{eqnarray*}&&J(k,f|{{k}^{n}})=\left\|{{A}_{{{k}^{n}}}}f-{{g}_{\delta }}\right\|{^{2}}+\alpha \parallel Lf{{\parallel }^{2}},\end{eqnarray*}$$

and

$$\begin{eqnarray*}&&J(k,f|{{f}^{n+1}})=\left\|{{C}_{{{f}^{n+1}}}}k-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|k-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(k).\end{eqnarray*}$$

We highlight some important facts:

• 1.  
  For each subproblem, the considered operators are linear, and the functional is convex. Thus a local minimum is global.
• 2.  
  The first step is a standard quadratic minimisation problem.

First we will show a monotonicity result for the sequence ${{\{({{k}^{n}},{{f}^{n}})\}}_{n}}$ of iterates:

Proposition 3.5. The functional J is non-increasing on the AM iterates,

$$\begin{eqnarray*}&&J({{k}^{n+1}},{{f}^{n+1}})\leqslant J({{k}^{n}},{{f}^{n+1}})\leqslant J({{k}^{n}},{{f}^{n}}).\end{eqnarray*}$$

Proof. The iterates are defined as

$$\begin{eqnarray*}&&{{f}^{n+1}}\in \mathop{{\rm arg} \;{\rm min} }\limits_{f\in V}\;J(k,f|{{k}^{n}})\end{eqnarray*}$$

and

$$\begin{eqnarray*}&&{{k}^{n+1}}\in \mathop{{\rm arg} \;{\rm min} }\limits_{k\in U}\;J(k,f|{{f}^{n+1}}).\end{eqnarray*}$$

Therefore,

$$\begin{eqnarray*}&&J({{k}^{n}},{{f}^{n+1}})\leqslant J({{k}^{n}},f)\qquad \forall f\in V\end{eqnarray*}$$

and

$$\begin{eqnarray*}&&J({{k}^{n+1}},{{f}^{n+1}})\leqslant J(k,{{f}^{n+1}})\qquad \forall k\in U,\end{eqnarray*}$$

and in particular, setting $f={{f}^{n}}$ and $k={{k}^{n}}$,

$$\begin{eqnarray*}&&\begin{array}{ccccccccccccccc} J({{k}^{n}},{{f}^{n+1}})\leqslant J({{k}^{n}},{{f}^{n}}) \\ J({{k}^{n+1}},{{f}^{n+1}})\leqslant J({{k}^{n}},{{f}^{n+1}}), \\ \end{array}\end{eqnarray*}$$

and

$$\begin{eqnarray*}&&J({{k}^{n+1}},{{f}^{n+1}})\leqslant J({{k}^{n}},{{f}^{n+1}})\leqslant J({{k}^{n}},{{f}^{n}}).\end{eqnarray*}$$

□

The existence of a minimiser of the functional J has already been proven in [2, theorem 4.2]. The goal of the following results is to prove that the sequence generated by the alternating minimisation algorithm has at least a subsequence which converges towards to a critical point of the functional. Throughout this section, let us make the following assumptions.

Assumption A. 

• B is strongly continuous, i.e. if $({{k}^{n}},{{f}^{n}})\rightharpoonup (\bar{k},\bar{f})$ then $B({{k}^{n}},{{f}^{n}})\to B(\bar{k},\bar{f})$.
• The adjoint of the Fréchet derivative B' of B is strongly continuous, i.e. if $({{k}^{n}},{{f}^{n}})\rightharpoonup (\bar{k},\bar{f})$ then $B^{\prime} {{({{k}^{n}},{{f}^{n}})}^{*}}z\to B^{\prime} {{(\bar{k},\bar{f})}^{*}}z$, $\forall z\in \mathcal{D}(B^{\prime} )$

Additionally to the standard norm for the pair $(k,f)\in \mathcal{U}\times \mathcal{V}$

$$\begin{eqnarray*}&&\parallel (k,f){{\parallel }^{2}}=\parallel k{{\parallel }^{2}}+\parallel f{{\parallel }^{2}}\end{eqnarray*}$$

we define the weighted norm for given $\gamma \gt 0$ as

$$\begin{eqnarray*}&&\parallel (k,f)\parallel _{\gamma }^{2}=\gamma \parallel k{{\parallel }^{2}}+\parallel f{{\parallel }^{2}}.\end{eqnarray*}$$

Proposition 3.6. For given regularisation parameters $0\lt \alpha$ and $\beta$, the sequence ${{\{({{k}^{n+1}},{{f}^{n+1}})\}}_{n+1}}$ of iterates generated by the AM algorithm has at least a weakly convergent subsequence $({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})\rightharpoonup (\bar{k},\bar{f})$, and its limit fulfils

$$\begin{eqnarray}&&J(\bar{k},\bar{f})\leqslant J(\bar{k},f)\quad {\rm and}\quad J(\bar{k},\bar{f})\leqslant J(k,\bar{f})\end{eqnarray} \tag{ 14 }$$

for all $f\in \mathcal{V}$ and for all $k\in \mathcal{U}$.

Proof. As the iterates of the AM algorithm can be characterised as the minimisers of a reduced dbl-RTLS functional, see (13a), (13b) we observe

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} \alpha \left\|L{{f}^{n+1}}\right\|{^{2}}+\gamma \left\|{{k}^{n}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}({{k}^{n}}) & \leqslant & J({{k}^{n}},{{f}^{n+1}}) \\ {} & = & \mathop{{\rm min} }\limits_{f}\;J(k,f|{{k}^{n}}) \\ {} & \leqslant & J({{k}^{n}},0) \\ {} & = & \left\|{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|{{k}^{n}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}({{k}^{n}}) \\ \end{array}\end{eqnarray*}$$

and

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} \alpha \left\|L{{f}^{n+1}}\right\|{^{2}}+\gamma \left\|{{k}^{n+1}}-{{k}_{\epsilon }}\right\|{^{2}} & \leqslant & J({{k}^{n+1}},{{f}^{n+1}}) \\ {} & = & \mathop{{\rm min} }\limits_{k}\;J(k,f|{{f}^{n+1}}) \\ {} & \leqslant & J(0,{{f}^{n+1}}) \\ {} & = & \left\|{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|{{k}_{\epsilon }}\right\|{^{2}}+\alpha \left\|L{{f}^{n+1}}\right\|{^{2}}. \\ \end{array}\end{eqnarray*}$$

Keeping in mind that the operator L is continuously invertible, the first inequality gives

$$\begin{eqnarray*}&&\left\|{{f}^{n+1}}\right\|{^{2}}\leqslant \frac{1}{\left\|{{L}^{-1}}\right\|{^{2}}\alpha }\left\|{{g}_{\delta }}\right\|{^{2}}.\end{eqnarray*}$$

Using the second estimate above and the standard inequality $\parallel a+b{{\parallel }^{2}}\leqslant 2(\parallel a{{\parallel }^{2}}+\parallel b{{\parallel }^{2}})$ we have

$$\begin{eqnarray*}&&\gamma \left\|{{k}^{n+1}}\right\|{^{2}}\leqslant 2\left\|{{g}_{\delta }}\right\|{^{2}}+4\gamma \left\|{{k}_{\epsilon }}\right\|{^{2}}.\end{eqnarray*}$$

Thus, the sequence ${{\{({{k}^{n+1}},{{f}^{n+1}})\}}_{n+1}}$ is bounded

$$\begin{eqnarray*}\begin{array}{ccccccccccccccc} \left\|({{k}^{n+1}},{{f}^{n+1}})\right\|_{\gamma }^{2} & = & \gamma \left\|{{k}^{n+1}}\right\|{^{2}}+\left\|{{f}^{n+1}}\right\|{^{2}} \\ {} & \leqslant & 2\left\|{{g}_{\delta }}\right\|{^{2}}+4\gamma \left\|{{k}_{\epsilon }}\right\|{^{2}}+\frac{1}{{{c}^{2}}\alpha }\left\|{{g}_{\delta }}\right\|{^{2}} \\ {} & = & \left( 2+\frac{1}{\left\|{{L}^{-1}}\right\|{^{2}}\alpha } \right)\left\|{{g}_{\delta }}\right\|{^{2}}+4\gamma \left\|{{k}_{\epsilon }}\right\|{^{2}} \\ \end{array}\end{eqnarray*}$$

and by Alaoglu's theorem, it has a weakly convergent subsequence ${{\{({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})\}}_{{{n}_{j}}+1}}\rightharpoonup (\bar{k},\bar{f})$ .

Since ${{f}^{{{n}_{j}}+1}}$ minimises the functional $J({{k}^{{{n}_{j}}}},f)$ for a fixed kⁿ_j, it holds

$$\begin{eqnarray*}&&J({{k}^{{{n}_{j}}}},{{f}^{{{n}_{j}}+1}})\leqslant J({{k}^{{{n}_{j}}}},f)\quad \forall f\in \mathcal{V}\end{eqnarray*}$$

and thus

$$\begin{eqnarray*}&&\left\|B({{k}^{{{n}_{j}}}},{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\alpha \left\|L{{f}^{{{n}_{j}}+1}}\right\|{^{2}}\leqslant \left\|B({{k}^{{{n}_{j}}}},f)-{{g}_{\delta }}\right\|{^{2}}+\alpha \parallel Lf{{\parallel }^{2}}.\end{eqnarray*}$$

Using the fact that J is w-lsc and the strong continuity of B, we observe

$$\begin{eqnarray}&&\begin{array}{ccccccccccccccc} \left\|B(\bar{k},\bar{f})-{{g}_{\delta }}\right\|{^{2}}+\alpha \left\|L\bar{f}\right\|{^{2}} \\ \quad \leqslant \mathop{{\rm lim} \;{\rm inf} }\limits_{{{n}_{j}}\to \infty }\left\{ \left\|B({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\alpha \left\|L{{f}^{{{n}_{j}}+1}}\right\|{^{2}} \right\} \\ \quad \leqslant \mathop{{\rm lim} \;{\rm inf} }\limits_{{{n}_{j}}\to \infty }\left\{ \left\|B({{k}^{{{n}_{j}}}},{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\alpha \left\|L{{f}^{{{n}_{j}}+1}}\right\|{^{2}} \right\} \\ \quad \leqslant \mathop{{\rm lim} \;{\rm inf} }\limits_{{{n}_{j}}\to \infty }\left\{ \left\|B({{k}^{{{n}_{j}}}},f)-{{g}_{\delta }}\right\|{^{2}}+\alpha \parallel Lf{{\parallel }^{2}} \right\} \\ \quad \leqslant \mathop{{\rm lim} \;{\rm sup} }\limits_{{{n}_{j}}\to \infty }\left\|B({{k}^{{{n}_{j}}}},f)-{{g}_{\delta }}\right\|{^{2}}+\alpha \parallel Lf{{\parallel }^{2}} \\ \quad =\mathop{{\rm lim} }\limits_{{{n}_{j}}\to \infty }\left\|B({{k}^{{{n}_{j}}}},f)-{{g}_{\delta }}\right\|{^{2}}+\alpha \parallel Lf{{\parallel }^{2}} \\ \quad \mathop{=}\limits^{(A1)}\left\|B(\bar{k},f)-{{g}_{\delta }}\right\|{^{2}}+\alpha \parallel Lf{{\parallel }^{2}} \\ \end{array}\end{eqnarray} \tag{ 15 }$$

Therefore,

$$\begin{eqnarray*}&&J(\bar{k},\bar{f})\leqslant J(\bar{k},f)\quad \forall f\in \mathcal{V}.\end{eqnarray*}$$

The second inequality in (14) is proven similarly: since ${{k}^{{{n}_{j}}+1}}$ minimises the functional $J(k,{{f}^{{{n}_{j}}+1}})$ for fixed ${{f}^{{{n}_{j}}+1}}$ it is

$$\begin{eqnarray*}&&J({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})\leqslant J(k,{{f}^{{{n}_{j}}+1}})\quad \forall k\in \mathcal{U},\end{eqnarray*}$$

which is equivalent to

$$\begin{eqnarray*}&&\begin{array}{ccccccccccccccc} \left\|B({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|{{k}^{{{n}_{j}}+1}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}({{k}^{{{n}_{j}}+1}}) \\ \quad \leqslant \left\|B(k,{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|k-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(k). \\ \end{array}\end{eqnarray*}$$

Again, we observe

$$\begin{eqnarray}&&\begin{array}{ccccccccccccccc} \left\|B(\bar{k},\bar{f})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(\bar{k}) \\ \quad \leqslant \mathop{{\rm lim} \;{\rm inf} }\limits_{{{n}_{j}}\to \infty }\left\{ \left\|B({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|{{k}^{{{n}_{j}}+1}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}({{k}^{{{n}_{j}}+1}}) \right\} \\ \quad \leqslant \mathop{{\rm lim} \;{\rm inf} }\limits_{{{n}_{j}}\to \infty }\left\|B(k,{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|k-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(k) \\ \quad =\mathop{{\rm lim} }\limits_{{{n}_{j}}\to \infty }\left\|B(k,{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|k-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(k) \\ \quad =\left\|B(k,\bar{f})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|k-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(k), \\ \end{array}\end{eqnarray} \tag{ 16 }$$

and thus

$$\begin{eqnarray*}&&J(\bar{k},\bar{f})\leqslant J(k,\bar{f}),\quad \forall k\in \mathcal{U}.\end{eqnarray*}$$

□

In summary, the AM algorithm yields a bounded sequence ${{\{({{k}^{n+1}},{{f}^{n+1}})\}}_{n}}$ and hence a weakly convergent subsequence. The next two results extend the convergence on the strong topology, for both ${{\{{{k}^{{{n}_{j}}+1}}\}}_{{{n}_{j}}}}$ and ${{\{{{f}^{{{n}_{j}}+1}}\}}_{{{n}_{j}}}}$, respectively.

Proposition 3.7. Let ${{\{({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})\}}_{{{n}_{j}}}}$ be a weakly convergent (sub-) sequence generated by the AM algorithm (13), where ${{k}^{{{n}_{j}}+1}}\rightharpoonup \bar{k}$ and ${{f}^{{{n}_{j}}+1}}\rightharpoonup \bar{f}$. Then there exists a subsequence ${{\{{{k}^{{{n}_{{{j}_{m}}}}+1}}\}}_{{{n}_{{{j}_{m}}}}}}$ of ${{\{{{k}^{{{n}_{j}}+1}}\}}_{{{n}_{j}}}}$ such that ${{k}^{{{n}_{{{j}_{m}}}}+1}}\to \bar{k}$ and $0\in {{\partial }_{k}}J(\bar{k},\bar{f})$.

Proof. Inequalities (16) in the proposition 3.6's proof reads

$$\begin{eqnarray*}&&\begin{array}{ccccccccccccccc} \mathop{{\rm lim} \;{\rm inf} }\limits_{{{n}_{j}}\to \infty }\left\{ \left\|B({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|{{k}^{{{n}_{j}}+1}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}({{k}^{{{n}_{j}}+1}}) \right\} \\ \quad =\left\|B(k,\bar{f})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|k-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(k). \\ \end{array}\end{eqnarray*}$$

for any k. Setting $k=\bar{k}$ yields in particular

$$\begin{eqnarray*}&&\begin{array}{ccccccccccccccc} \mathop{{\rm lim} \;{\rm inf} }\limits_{{{n}_{j}}\to \infty }\left\{ \left\|B({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|{{k}^{{{n}_{j}}+1}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}({{k}^{{{n}_{j}}+1}}) \right\} \\ \quad =\left\|B(\bar{k},\bar{f})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(\bar{k}). \\ \end{array}\end{eqnarray*}$$

As the limes inferior exists, we can in particular extract a subsequence ${{({{k}^{{{n}_{{{j}_{m}}}}+1}},{{f}^{{{n}_{{{j}_{m}}}}+1}})}_{{{n}_{{{j}_{m}}}}}}$ of ${{({{k}^{{{n}_{j}}+1}},{{f}^{{{n}_{j}}+1}})}_{{{n}_{j}}}}$ such that

$$\begin{eqnarray}&&\begin{array}{ccccccccccccccc} \mathop{{\rm lim} }\limits_{{{n}_{{{j}_{m}}}}\to \infty }\left\{ \left\|B({{k}^{{{n}_{{{j}_{m}}}}+1}},{{f}^{{{n}_{{{j}_{m}}}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|{{k}^{{{n}_{{{j}_{m}}}}+1}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}({{k}^{{{n}_{{{j}_{m}}}}+1}}) \right\} \\ \quad =\left\|B(\bar{k},\bar{f})-{{g}_{\delta }}\right\|{^{2}}+\gamma \left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(\bar{k}). \\ \end{array}\end{eqnarray} \tag{ 17 }$$

For the sake of notation simplicity we denote for the remainder of the proof the index ${{n}_{{{j}_{m}}}}+1$ by $m+1$. By (A1) we observe

$$\begin{eqnarray*}&&\mathop{{\rm lim} }\limits_{m\to \infty }\left\|B({{k}^{m+1}},{{f}^{m+1}})-{{g}_{\delta }}\right\|{^{2}}\mathop{=}\limits^{(A1)}\left\|B(\bar{k},\bar{f})-{{g}_{\delta }}\right\|{^{2}}\end{eqnarray*}$$

As all summands in (17) are positive, we have thus and

$$\begin{eqnarray}\begin{array}{ccccccccccccccc} \mathop{{\rm lim} }\limits_{m\to \infty }\left\{ \gamma \left\|{{k}^{m+1}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}({{k}^{m+1}}) \right\} & = & \gamma \mathop{{\rm lim} }\limits_{m\to \infty }\left\|{{k}^{m+1}}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathop{{\rm lim} }\limits_{m\to \infty }\mathcal{R}({{k}^{m+1}}) \\ {} & = & \gamma \left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}+\beta \mathcal{R}(\bar{k}). \\ \end{array}\end{eqnarray} \tag{ 18 }$$

Now let us show that ${{k}^{m+1}}$ converges strongly. As the sequence converges weakly, it is enough to show

$$\begin{eqnarray*}&&\mathop{{\rm lim} }\limits_{m\to \infty }\left\|{{k}^{m+1}}\right\|{^{2}}=\left\|\bar{k}\right\|{^{2}}\end{eqnarray*}$$

Equivalently, we can also show ${{{\rm lim} }_{m\to \infty }}\left\|{{k}^{m+1}}-{{k}_{\epsilon }}\right\|{^{2}}=\left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}$. Again due to the weak convergence of ${{k}^{m+1}}$ it is sufficient to prove

$$\begin{eqnarray*}&&\mathop{{\rm lim} \;{\rm sup} }\limits_{m\to \infty }\left\|{{k}^{m+1}}-{{k}_{\epsilon }}\right\|{^{2}}\leqslant \left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}.\end{eqnarray*}$$

Let us assume that

$$\begin{eqnarray*}&&\mu :=\mathop{{\rm lim} \;{\rm sup} }\limits_{m\to \infty }\left\|{{k}^{m+1}}-{{k}_{\epsilon }}\right\|{^{2}}\gt \left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}.\end{eqnarray*}$$

holds. Rewriting (18) yields

$$\begin{eqnarray}\begin{array}{ccccccccccccccc} \beta \ \mathop{{\rm lim} \;{\rm sup} }\limits_{m\to \infty }\left\{ \mathcal{R}({{k}^{m+1}}) \right\} & = & \gamma \left( \left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}-\mathop{{\rm lim} \;{\rm sup} }\limits_{m\to \infty }\left\|{{k}^{m+1}}-{{k}_{\epsilon }}\right\|{^{2}} \right)+\beta \mathcal{R}(\bar{k}) \\ {} & = & \gamma \left( \left\|\bar{k}-{{k}_{\epsilon }}\right\|{^{2}}-\mu \right)+\beta \mathcal{R}(\bar{k}) \\ {} & \lt & \beta \mathcal{R}(\bar{k}). \\ \end{array}\end{eqnarray} \tag{ 19 }$$

However, since $\mathcal{R}$ is w-lsc, we observe

$$\begin{eqnarray*}&&\mathcal{R}(\bar{k})\leqslant \mathop{{\rm lim} \;{\rm inf} }\limits_{m\to \infty }\;\mathcal{R}({{k}^{m+1}})\leqslant \mathop{{\rm lim} \;{\rm sup} }\limits_{m\to \infty }\;\mathcal{R}({{k}^{m+1}}),\end{eqnarray*}$$

which is in contradiction to (19). Thus we have shown the convergence of ${{k}^{m+1}}$ to $\bar{k}$ in norm.

The last part of this proof focus on the convergence of the partial subdifferential of J with respect to k.

Since ${{k}^{m+1}}$ solves the sub-minimisation problem (13b), the optimality condition reads as $0\in {{\partial }_{k}}J({{k}^{m+1}},{{f}^{m+1}})$, or equivalently, there exists an element

$$\begin{eqnarray}&&\xi _{k}^{m+1}:=-\frac{1}{\beta }\left( C_{{{f}^{m+1}}}^{*}({{C}_{{{f}^{m+1}}}}{{k}^{m+1}}-{{g}_{\delta }})+\gamma ({{k}^{m+1}}-{{k}_{\epsilon }}) \right)\end{eqnarray} \tag{ 20 }$$

such that $\xi _{k}^{m+1}\in \partial \mathcal{R}\left( {{k}^{m+1}} \right)\subset \mathcal{U}$; see corollary 3.3.

Now, on the limit, $0\in {{\partial }_{k}}J(\bar{k},\bar{f})$, means that

$$\begin{eqnarray*}&&\bar{\xi }:=-\frac{1}{\beta }\left( C_{{\bar{f}}}^{*}({{C}_{{\bar{f}}}}\bar{k}-{{g}_{\delta }})+\gamma (\bar{k}-{{k}_{\epsilon }}) \right)\quad {\rm and}\quad \bar{\xi }\in \partial \mathcal{R}\left( {\bar{k}} \right)\end{eqnarray*}$$

holds, i.e. the right hand-side of (20) converges and the limit of the sequence of subderivatives belongs also to the subdifferential set $\partial \mathcal{R}\left( {\bar{k}} \right)$.

The first part of the statement above can be seen by using condition (A2). Whereas the second part is obtained by the assumption that $\mathcal{R}$ is a convex functional, because in this case the Fenchel subdifferential coincides with the limiting subdifferential, which is a strong-weakly closed mapping (see appendix).

The strong convergence for the second variable is obtained as follows.

Proposition 3.8. Let $\{m\}$ be a subsequence of $\mathbb{N}$ such that the (sub-) sequence ${{\{({{k}^{m+1}},{{f}^{m+1}})\}}_{m}}$ generated by AM algorithm (13) satisfies ${{k}^{m+1}}\to \bar{k}$ and ${{f}^{m+1}}\rightharpoonup \bar{f}$. Then there is a subsequence of ${{\{{{f}^{m+1}}\}}_{m}}$ such that ${{f}^{{{m}_{j}}+1}}\to \bar{f}$ and $0\in {{\partial }_{f}}J(\bar{k},\bar{f})$.

Proof. Similarly as the previous theorem, by setting $f=\bar{f}$ at (15) in the proposition 3.6's proof we obtain

$$\begin{eqnarray*}&&\begin{array}{ccccccccccccccc} \mathop{{\rm lim} \;{\rm inf} }\limits_{m\to \infty }\left\{ \left\|B({{k}^{m+1}},{{f}^{m+1}})-{{g}_{\delta }}\right\|{^{2}}+\alpha \left\|L{{f}^{m+1}}\right\|{^{2}} \right\} \\ \quad =\left\|B(\bar{k},\bar{f})-{{g}_{\delta }}\right\|{^{2}}+\alpha \left\|L\bar{f}\right\|{^{2}}. \\ \end{array}\end{eqnarray*}$$

As the limes inferior exists, we can in particular extract a subsequence ${{({{k}^{{{m}_{j}}+1}},{{f}^{{{m}_{j}}+1}})}_{{{m}_{j}}}}$ of ${{({{k}^{m+1}},{{f}^{m+1}})}_{m}}$ such that

$$\begin{eqnarray*}&&\begin{array}{ccccccccccccccc} \mathop{{\rm lim} }\limits_{{{m}_{j}}\to \infty }\left\{ \left\|B({{k}^{{{m}_{j}}+1}},{{f}^{{{m}_{j}}+1}})-{{g}_{\delta }}\right\|{^{2}}+\alpha \left\|L{{f}^{{{m}_{j}}+1}}\right\|{^{2}} \right\} \\ \quad =\left\|B(\bar{k},\bar{f})-{{g}_{\delta }}\right\|{^{2}}+\alpha \left\|L\bar{f}\right\|{^{2}}. \\ \end{array}\end{eqnarray*}$$

Since both summands in the limit above are positive and due to (A1), we conclude that

$$\begin{eqnarray*}&&\mathop{{\rm lim} }\limits_{{{m}_{j}}\to \infty }\left\|L{{f}^{{{m}_{j}}+1}}\right\|{^{2}}=\left\|L\bar{f}\right\|{^{2}}.\end{eqnarray*}$$

Moreover, as L is a bounded and continuously invertible operator we have

$$\begin{eqnarray*}&&\mathop{{\rm lim} }\limits_{{{m}_{j}}\to \infty }\left\|{{f}^{{{m}_{j}}+1}}\right\|{^{2}}=\left\|\bar{f}\right\|{^{2}},\end{eqnarray*}$$

which in combination with the weak convergence of the subsequence gives its strong convergence ${{f}^{{{m}_{j}}+1}}\to \bar{f}$.

The second half of this proof refers to the convergence of the partial subdifferential of J with respect to f and its limit.

Since ${{f}^{m+1}}$ solves the sub-minimisation problem (13a), the optimality condition reads as $0\in {{\partial }_{f}}J({{k}^{m}},{{f}^{m+1}})$. However we are interested on the partial subderivate at the pair $({{k}^{{{m}_{j}}+1}},{{f}^{{{m}_{j}}+1}})$. Namely, with help of corollary 3.3 the subderivative (which is a unique element) $\xi _{f}^{{{m}_{j}}+1}\in {{\partial }_{f}}J({{k}^{{{m}_{j}}+1}},{{f}^{{{m}_{j}}+1}})$ is computed³ as

$$\begin{eqnarray*}&&\xi _{f}^{m+1}:=A_{{{k}^{m+1}}}^{*}({{A}_{{{k}^{m+1}}}}{{f}^{m+1}}-{{g}_{\delta }})+\alpha {{L}^{*}}L{{f}^{m+1}},\end{eqnarray*}$$

which may not be necessarily null for each cycle of the AM algorithm (13), otherwise the stoping criteria would be satisfied and nothing would be left to be proven. Therefore we shall prove that it converges towards zero.

So far we have strong convergence of both sequences ${{\left\{ {{k}^{m+1}} \right\}}_{m}}$ and ${{\left\{ {{f}^{m+1}} \right\}}_{m}}$. Additionally, the assumption A implies that both linear operators A_k and $A_{k}^{*}$ are also strongly continuous, therefore

$$\begin{eqnarray}\begin{array}{ccccccccccccccc} \mathop{{\rm lim} }\limits_{m\to \infty }\xi _{f}^{m+1} & = & \mathop{{\rm lim} }\limits_{m\to \infty }\{A_{{{k}^{m+1}}}^{*}({{A}_{{{k}^{m+1}}}}{{f}^{m+1}}-{{g}_{\delta }})+\alpha {{L}^{*}}L{{f}^{m+1}}\} \\ {} & = & A_{{\bar{k}}}^{*}({{A}_{{\bar{k}}}}\bar{f}-{{g}_{\delta }})+\alpha {{L}^{*}}L\bar{f}. \\ \end{array}\end{eqnarray} \tag{ 21 }$$

Our goal is to show that the limit given in (21) is zero. Let's suppose by contradiction that $0\notin {{\partial }_{f}}J(\bar{k},\bar{f})$. Since this set is singleton we conclude that

$$\begin{eqnarray*}&&A_{{\bar{k}}}^{*}({{A}_{{\bar{k}}}}\bar{f}-{{g}_{\delta }})+\alpha {{L}^{*}}L\bar{f}\ne 0.\end{eqnarray*}$$

This means that $\bar{f}$ does not fulfil the normal equation associated to the standard Tikhonov problem

$$\begin{eqnarray*}&&\mathop{{\rm minimise}}\limits_{f}\left\|{{A}_{{\bar{k}}}}f-{{g}_{\delta }}\right\|{^{2}}+\alpha \parallel Lf{{\parallel }^{2}},\end{eqnarray*}$$

which is a necessary condition to be a minimiser candidate to the underlying functional.

Therefore the functional $J(\bar{k},\cdot )$ for a given fixed $\bar{k}$ does not attain its minimum value at $\bar{f}$ and there is at least one element f such that $J(\bar{k},f)\lt J(\bar{k},\bar{f})$.

Moreover this functional is convex and it has a global solution, here denoted by $\tilde{f}$. By definition

$$\begin{eqnarray*}&&J(\bar{k},\tilde{f})\leqslant J(\bar{k},f)\end{eqnarray*}$$

for all $f\in V$.

In particular, since $\bar{f}$ is not a minimiser for $J(\bar{k},\cdot )$, the inequality above is strict,

$$\begin{eqnarray}&&J(\bar{k},\tilde{f})\lt J(\bar{k},\bar{f}).\end{eqnarray} \tag{ 22 }$$

On the other hand, from propostion 3.6 it also holds

$$\begin{eqnarray*}&&J(\bar{k},\bar{f})\leqslant J(\bar{k},f)\end{eqnarray*}$$

for all $f\in V$. Setting $f:=\tilde{f}$ in this inequality we get

$$\begin{eqnarray*}&&J(\bar{k},\bar{f})\leqslant J(\bar{k},\tilde{f}),\end{eqnarray*}$$

which leads to a contradiction to (22).

Therefore for $\bar{f}$ the optimality condition holds true, i.e. in the limit the source condition is fulfilled and the limit of the partial subderivative sequence is zero, i.e. $0\in {{\partial }_{f}}J(\bar{k},\bar{f})$, which completes the proof. □

Remark 3.9. One alternative proof would be assuming that the sequence ${{\{{{k}^{m+1}}\}}_{m}}$ fulfils

$$\begin{eqnarray}&&\left\|{{k}^{m+1}}-{{k}^{m}}\right\|\to 0.\end{eqnarray} \tag{ 23 }$$

More specifically, we have

$$\begin{eqnarray*}&&A_{{{k}^{m}}}^{*}({{A}_{{{k}^{m}}}}{{f}^{m+1}}-{{g}_{\delta }})+\alpha {{L}^{*}}L{{f}^{m+1}}=0\end{eqnarray*}$$

from the optimality condition, but we would like to show

$$\begin{eqnarray*}&&\mathop{{\rm lim} }\limits_{m\to \infty }\{A_{{{k}^{m+1}}}^{*}({{A}_{{{k}^{m+1}}}}{{f}^{m+1}}-{{g}_{\delta }})+\alpha {{L}^{*}}L{{f}^{m+1}}\}=0.\end{eqnarray*}$$

Subtracting the latter expression from the first one, we get

$$\begin{eqnarray*}&&(A_{{{k}^{m}}}^{*}{{A}_{{{k}^{m}}}}-A_{{{k}^{m+1}}}^{*}{{A}_{{{k}^{m+1}}}}){{f}^{m+1}}+(A_{{{k}^{m}}}^{*}-A_{{{k}^{m+1}}}^{*}){{g}_{\delta }}.\end{eqnarray*}$$

Note that by assuming the condition (23) the expression above converges to zero and the proof would be complete. Nevertheless we cannot guarantee that subsequent elements of the original sequence will be selected for the subsequence. As an alternative one can verify numerically if the sequence provided from the AM algorithm satisfies this assumption. Moreover, if we restrict the problem to the simple case that the characterising function is known, then the assumption (23) is trivial, the problem becomes the standard Tikhonov regularisation and the theory is carried on.

The forthcoming and most substantial result within this section shows that the limit $(\bar{k},\bar{f})$ of the sequence generated by the AM algorithm is a critical point (pair) of the functional J.

Theorem 3.10 Let $\{m\}$ an index set of $\mathbb{N}$ such that the sequence generated by AM algorithm ${{\{({{k}^{m+1}},{{f}^{m+1}})\}}_{m}}\to (\bar{k},\bar{f})$ and $(\xi _{k}^{m+1},\xi _{f}^{m+1})\rightharpoonup (0,0)$. Then there is subsequence converging towards to a critical point of J, i.e.

$$\begin{eqnarray*}&&(0,0)\in \partial J\left( \bar{k},\bar{f} \right).\end{eqnarray*}$$

Proof. The proposition 3.7 guarantees that ${{k}^{m+1}}\to \bar{k}$ and ${{\xi }_{{{k}^{m+1}}}}\in \partial \mathcal{R}({{k}^{m+1}})$ (or equivalently, $0\in {{\partial }_{k}}J({{k}^{m+1}},{{f}^{m+1}})$) such that $0\in {{\partial }_{k}}J(\bar{k},\bar{f})$. Likewise, proposition 3.8 guarantees that the sequence ${{f}^{m+1}}\to \bar{f}$ and ${{\xi }_{{{f}^{m+1}}}}\in \partial J({{k}^{m+1}},{{f}^{m+1}})$ such that $0\in {{\partial }_{f}}J(\bar{k},\bar{f})$. Combining this with the strong-weakly closedness property of the subderivative (see appendix) and theorem 3.2 we have

$$\begin{eqnarray*}&&(0,0)\in \partial J\left( \bar{k},\bar{f} \right)={{\partial }_{k}}J(\bar{k},\bar{f})\times {{\partial }_{f}}J(\bar{k},\bar{f})\end{eqnarray*}$$

on the limit. □