Stein's lemma

Stein's lemma, named in honor of Charles Stein, is a theorem of probability theory that is of interest primarily because of its applications to statistical inference — in particular, to James–Stein estimation and empirical Bayes methods — and its applications to portfolio choice theory. The theorem gives a formula for the covariance of one random variable with the value of a function of another, when the two random variables are jointly normally distributed. Note that the name "Stein's lemma" is also commonly used to refer to a different result in the area of statistical hypothesis testing, which connects the error exponents in hypothesis testing with the Kullback–Leibler divergence. This result is also known as the Chernoff–Stein lemma and is not related to the lemma discussed in this article. Suppose X is a normally distributed random variable with expectation μ and variance σ. Further suppose g is a differentiable function for which the two expectations E ⁡ ( g ( X ) ( X − μ ) ) {\displaystyle \operatorname {E} (g(X)(X-\mu ))} and E ⁡ ( g ′ ( X ) ) {\displaystyle \operatorname {E} (g'(X))} both exist. (The existence of the expectation of any random variable is equivalent to the finiteness of the expectation of its absolute value.) Then In general, suppose X and Y are jointly normally distributed. Then For a general multivariate Gaussian random vector ( X 1 , . . . , X n ) ∼ N ( μ , Σ ) {\displaystyle (X_{1},...,X_{n})\sim {\mathcal {N}}(\mu ,\Sigma )} it follows that Similarly, when μ = 0 {\displaystyle \mu =0} , E ⁡ [ ∂ i g ( X ) ] = E ⁡ [ g ( X ) ( Σ − 1 X ) i ] , E ⁡ [ ∂ i ∂ j g ( X ) ] = E ⁡ [ g ( X ) ( ( Σ − 1 X ) i ( Σ − 1 X ) j − Σ i j − 1 ) ] {\displaystyle \operatorname {E} [\partial _{i}g(X)]=\operatorname {E} [g(X)(\Sigma ^{-1}X)_{i}],\quad \operatorname {E} [\partial _{i}\partial _{j}g(X)]=\operatorname {E} [g(X)((\Sigma ^{-1}X)_{i}(\Sigma ^{-1}X)_{j}-\Sigma _{ij}^{-1})]} Stein's lemma can be used to stochastically estimate gradient: ∇ E ϵ ∼ N ( 0 , I ) ⁡ ( g ( x + Σ 1 / 2 ϵ ) ) = Σ − 1 / 2 E ϵ ∼ N ( 0 , I ) ⁡ ( g ( x + Σ 1 / 2 ϵ ) ϵ ) ≈ Σ − 1 / 2 1 N ∑ i = 1 N g ( x + Σ 1 / 2 ϵ i ) ϵ i {\displaystyle \nabla \operatorname {E} _{\epsilon \sim {\mathcal {N}}(0,I)}{\bigl (}g(x+\Sigma ^{1/2}\epsilon ){\bigr )}=\Sigma ^{-1/2}\operatorname {E} _{\epsilon \sim {\mathcal {N}}(0,I)}{\bigl (}g(x+\Sigma ^{1/2}\epsilon )\epsilon {\bigr )}\approx \Sigma ^{-1/2}{\frac {1}{N}}\sum _{i=1}^{N}g(x+\Sigma ^{1/2}\epsilon _{i})\epsilon _{i}} where ϵ 1 , … , ϵ N {\displaystyle \epsilon _{1},\dots ,\epsilon _{N}} are IID samples from the standard normal distribution N ( 0 , I ) {\displaystyle {\mathcal {N}}(0,I)} . This form has applications in Stein variational gradient descent and Stein variational policy gradient.