Biconjugate_Theorem

Biconjugate theorem for convex functions

1. Notations

Let $X$ be a vector space. Let $X^*$ be the vector space so that $(X, X^*)$ be a dual pair. Let $\langle \cdot, \cdot\rangle$ be the finite real-valued bi-linear map corresponds to this dual pair. Recall that, in this setup we have $X^{**} =X$. Let $w=\sigma(X, X^*)$ and $w^*=\sigma(X^*, X)$ be respectively the weak and weak* topologies on $X$ and $X^*$.

Consider $f: X\rightarrow \overline{\mathbb R}$, define its conjugate as a function $f^*: X^*\rightarrow \overline{\mathbb R}$
$$f(x^*) = \sup_{x\in X}\langle x^*, x\rangle-f(x).$$

Theorem (Fenchel-Young’s inequality) For $x\in X$ and $x^*\in X^*$, we have
$$f(x) +f^*(x^*) \geq \langle x^*, x\rangle.$$

Proof. By the definition of conjugate. $\square$

2. Biconjugate Theorem

Theorem (Biconjugate Theorem, @Zalinescu Theorem 2.3.3) If $f$ is closed proper convex function. Then $$f^{**}=f.$$

Proof. We first show that $f^{**}\leq f$. For $x\in X$ and $x^*\in X^*$ we have $$\langle x, x^*\rangle - f^*(x^*)\leq f(x).$$

The supremum of the LHS w.r.t. $x^*$ is $f^{**}(x)$, thus $f^{**}(x)\leq f(x)$.

We now show that if $f^{**}(x_0)< f(x_0)$ for some $x_0\in X$, one can derive a contradiction. Indeed, since $f$ is closed, $(x_0, f^{**}(x_0)) \notin epi f$.

Recall the Hahn-Banach separation theorem, see e.g. [@Zalinescu Theorem 1.1.5], Let $V$ be a locally convex space and $A,B \subset V$ be two nonempty convex sets. If $A$ is closed, $B$ is compact and $A \cap B =\emptyset$, then there exist $v^* \in V^* \setminus \{0\}$ so that
$$\inf v^*(B) > \sup v^*(A).$$
Apply above Hahn-Banach separation theorem for $V = X\times \mathbb R$, $A = epi f$ and $B= \{(x_0, f^{**}(x_0))\}$. There exists $(x_0^*, \alpha)\in X^*\times \mathbb R$ so that

$$\langle x_0^*, x_0\rangle + \alpha f^{**}(x_0)> \sup\{ \langle x_0^*, x\rangle + \alpha t:(x, t)\in epi f\}. \tag{1}$$

We now show contradiction by considering three cases of $\alpha$: $\alpha>0, \alpha<0, \alpha=0$.

CASE 1. $\alpha>0$. Notice that if $(x, t)\in epi f$, then $(x,t+n)\in epi f$ for all $n\in \mathbb N$. Since $\alpha>0$, the RHS of $(1)$ will be $+\infty$. Thus, LHS of $(1)$ strictly greater than $+\infty$, which is impossible.

CASE 2. $\alpha<0$. We then divide both sides of above inequality by $-\alpha$ and get

$$\begin{aligned} \langle -\alpha^{-1} x_0^*, x_0\rangle - f^{**}(x_0) > & \sup\{ \langle -\alpha^{-1}x_0^*, x\rangle - t:(x, t)\in epi f\}\\ = & \sup\{ \langle -\alpha^{-1}x_0^*, x\rangle - f(x):x\in dom f\}\\ = & f^{*}(-\alpha^{-1}x_0^*). \end{aligned}$$

Which contradicts to Fenchel Young’s inequality.

CASE 3. $\alpha=0$. In this case, $(1)$ can be rewritten

$$\langle x_0^*, x_0\rangle > \sup\{ \langle x_0^*, x\rangle: x\in dom f\}.$$

Let $h > 0$ and $y^*_0\in dom f^*$ be arbitrarily chosen. Here, the existence of $y^*$ is well defined since $f^*$ is proper. One obtains

$$\begin{aligned} f^*(y_0^*+hx_0^*) = & \sup\{\langle x, y_0^*\rangle +h \langle x, x_0^*\rangle -f(x): x\in dom f\}\\ \leq & \sup\{\langle x, y_0^*\rangle -f(x): x\in dom f\} + h \sup\{\langle x, x_0^*\rangle : x\in dom f\}\\ = & f^*(y_0^*) + h \sup\{\langle x, x_0^*\rangle : x\in dom f\} \end{aligned}$$

Combining this inequality and Fenchel-Young inequality, we obtain

$$\begin{aligned}f^*(x_0^*) \geq & \langle y_0^* +hx_0^*, x_0^*\rangle- f(y_0^*+hx_0^*)\\ \geq & \langle y_0^*, x_0^*\rangle - f^*(y_0^*) + h[\langle x_0^*, x_0\rangle - \sup_{x\in dom f} \langle x_0^*, x\rangle] \end{aligned}$$

Note that $f^*(y_0^*)$ is finite since $y^*_0 \in dom f^*$.
Therefore, letting $h\rightarrow +\infty$, we have $f^{**}(x_0)=\infty$, which is a contradiction.$\square$

Remark.

a) By definition, $f$ is closed iff $epi f$ is closed w.r.t. the weak topology $w \times topo(\mathbb R)$. If $X$ is a norm space and $f$ is proper convex, $f$ is closed iff it is weakly closed. This is because $epi f$ is a non-empty convex set, then $epi f$ is closed iff $epi f$ is weakly closed.

b) Above theorem also holds for $f: X^* \rightarrow \overline{ \mathbb R}$, here $f^*: X \rightarrow \overline{ \mathbb R}$, i.e. $f^{**}=f$

3. References

[1] Zalinescu, Constantin. Convex analysis in general vector spaces. World scientific, 2002.