The soft thresholding operator with parameter
acts element-wise on
such that for each
,
In other words, for each element in , the soft-thresholding operator brings
closer to 0 by the largest amount
, without having
change its sign. (Note:
is sometimes written as
.)
As a quick consequence of the definition, for any constant we have
.
The soft-thresholding operator appears frequently in optimization problems involving the penalty. This is because the subgradient of the
penalty involves the
function, defined as
The lemma below makes the connection between soft-thresholding and the function explicit:
Lemma: Consider the equation
where and
. The solution to this equation is
.
Proof: We consider the following cases:
Case 1: . The equation becomes
, or
. This is only valid if
, or
.
Case 2: . The equation becomes
, or
. This is only valid if
, or
.
Case 3: . This solution is only valid if
, or
.
Putting it altogether, we see that the solution is equivalent to .