The soft thresholding operator with parameter acts element-wise on such that for each ,
In other words, for each element in , the soft-thresholding operator brings closer to 0 by the largest amount , without having change its sign. (Note: is sometimes written as .)
As a quick consequence of the definition, for any constant we have .
The soft-thresholding operator appears frequently in optimization problems involving the penalty. This is because the subgradient of the penalty involves the function, defined as
The lemma below makes the connection between soft-thresholding and the function explicit:
Lemma: Consider the equation
where and . The solution to this equation is .
Proof: We consider the following cases:
Case 1: . The equation becomes , or . This is only valid if , or .
Case 2: . The equation becomes , or . This is only valid if , or .
Case 3: . This solution is only valid if , or .
Putting it altogether, we see that the solution is equivalent to .