[读书笔记][更新中] Aad van der Vaart "Asymptotic Statistics"

[hitori_bocchi]

Directional derivative & pathwise derivative for functional的东西下次再更新。

[hitori_bocchi]

半参后面的理论确实有点复杂，会涉及一些泛函的东西，我不打算写的过于理论，更多还是intuition吧

[hitori_bocchi]

不打算更directional derivative & pathwise derivative for functional和更多的半参理论的东西了，实在是太多了，够写一本书的。这篇文章本来也是一个偏实用的指南，还是不想太偏离主旨...

[hitori_bocchi]

It turns out that, for the purposes of constructing lower bound benchmarks for functional estimation, it often suffices to use one-dimensional parametric submodels. A common choice of submodel for nonparametric $\mathcal{P}$ is, for some mean-zero function $h:\mathcal{Z}\to \mathbb{R}$,

$$p_{ϵ}(z)=d\mathbb{P}(z)\{1+ϵh(z)\}$$

where $\Vert h{\Vert }_{\infty }\le M<\infty$ and $ϵ<1/M$ so that $p_{ϵ}(z)\ge 0$. Note for this submodel the score function is ${\frac{\partial }{\partial ϵ}\log p_{ϵ}(z)|}_{ϵ=0}={\frac{\partial }{\partial ϵ}\log \{1+ϵh(z)\}|}_{ϵ=0}=h(z)$. Therefore the Cramer-Rao lower bound for some $P_{ϵ}$ in the example one-dimensional submodel ${\mathcal{P}}_{ϵ}$ above is given by

$$\frac{{\psi }^{\prime }{\left(P_{ϵ}\right)}^2}{{\mathrm{var}}_{P_{ϵ}}\left\{s_{ϵ}(Z)\right\}}=\frac{{\left\{{\frac{\partial }{\partial ϵ}\psi \left(P_{ϵ}\right)|}_{ϵ=0}\right\}}^2}{{\mathbb{E}}_{P_{ϵ}}\left\{h(Z{)}^2\right\}}.$$

Comment: Why one-dimensional submodel? 详细的说明见 Michael Kosorok "Introduction to Empirical Processes and Semiparametric Inference" Chap. 18。

还需要说明的一点是为什么我们选择了$p_{ϵ}(z)=d\mathbb{P}(z)\{1+ϵh(z)\}$作为submodel（以下内容改写自Mark van der Laan的 STAT C245B Survival Analysis and Causality 的课程材料）。

We want to define a type of differentiability of $\psi :\mathcal{P}\to {\mathbb{R}}^q$, where $\psi$ is the target parameter.

We could use the definition of a directional derivative in direction $h$ :

$$d\psi (\mathbb{P})(h)={\frac{d}{dϵ}\psi (\mathbb{P}+ϵh)|}_{ϵ=0}$$

However, $\mathbb{P}+ϵh$ might not be a path through $\mathcal{P}$, and thus ill defined. We need to define a derivative along paths that are submodels of $\mathcal{P}$.

Let $\mathcal{P}$ be nonparametric. We define a class of paths such that:

$$p_{ϵ}(z)=d\mathbb{P}(z)\{1+ϵh(z)\}$$

Two key assumptions necessary for it to be a proper submodel are as follows:

• $h$ is uniformly bounded
• ${\mathbb{E}}_{P}h(z)=0$

For $ϵ\in (-\delta ,\delta )$ with $\delta =\frac{1}{\Vert h{\Vert }_{\infty }}$, this is a submodel.

To see why, first note that for the paths to be a proper density, we need:

1. $d\mathbb{P}(z)\{1+ϵh(z)\}⩾0$

Sketch proof:

Let $h(z)$ be uniformly bounded and $h(z)=\Vert h{\Vert }_{\infty }$. If $ϵ⩽|\delta |,\{1+ϵh(z)\}⩾0$. Therefore, for $ϵ$ sufficiently small and $h$ uniformly bounded, $d\mathbb{P}(z)\{1+ϵh(z)\}⩾0$.

2. $\int \{1+ϵh(z)\}d\mathbb{P}(z)=1$

Sketch proof:

Note that $\int \{1+ϵh(z)\}d\mathbb{P}(z)=\int d\mathbb{P}(z)+ϵ\int h(z)d\mathbb{P}(z)=1$ since $p$ is a proper density and $\int h(z)d\mathbb{P}(z)={\mathbb{E}}_{P}h(z)=0$ by assumption.

Now consider the score of this submodel.

$$\begin{array}{rl}{\frac{\delta }{\delta ϵ}\log \frac{d{P}_{ϵ}}{d\mathbb{P}}|}_{ϵ=0}& ={\frac{\delta }{\delta ϵ}\log \{1+ϵh(z)\}|}_{ϵ=0}\\ & ={\frac{h(z)}{1+ϵh(z)}|}_{ϵ=0}\\ & =h(z).\end{array}$$

"Since any lower bound for the submodel ${\mathcal{P}}_{ϵ}$ is also a lower bound for $\mathcal{P}$, the best and most informative is the greatest such lower bound. Can we say anything about the best such lower bound for generic functionals and/or submodels?"

2.2 Pathwise Differentiability

Recall the Cramer-Rao bound

$$\frac{{\left\{{\frac{\partial }{\partial ϵ}\psi \left(P_{ϵ}\right)|}_{ϵ=0}\right\}}^2}{{\mathbb{E}}_{P_{ϵ}}\left\{s_{ϵ}(Z{)}^2\right\}}$$

for submodel ${\mathcal{P}}_{ϵ}$ described in the previous subsection. To find the best such lower bound, we would like to optimize the above over all $P_{ϵ}$ in some submodels. It is not a priori clear how generally this can be accomplished, since different functionals $\psi$ could yield very different numerators. Therefore let us first consider what we can say about the derivative in the numerator, for a large class of pathwise differentiable functionals.

Namely, suppose the functional $\psi :\mathcal{P}\mapsto \mathbb{R}$ is smooth, as a map from distributions to the reals, in the sense that it admits a kind of distributional Taylor expansion

$$\psi (\bar{P})-\psi (P)=\int \phi (z;\bar{P})d(\bar{P}-P)(z)+R_2(\bar{P},P)$$

for distributions $\bar{P}$ and $P$, often called a von Mises expansion, where $\phi (z;P)$ is a mean-

zero, finite-variance function satisfying $\int \phi (z;P)dP(z)=0$ and $\int \phi (z;P{)}^{2}dP(z)<\infty$, and $R_2(\bar{P},P)$ is a second-order remainder term (which means it only depends on products or squares of differences between $\bar{P}$ and $P)$.

Intuitively, the von Mises expansion above is just an infinite-dimensional or distributional analog of a Taylor expansion, with $\phi (z;Q)$ acting as a usual derivative term; it describes how the functional $\psi$ changes locally when the distribution changes from $P$ to $\bar{P}$. For example, when $Z\in \{1,\dots ,k\}$ is discrete and so $\bar{P}$ and $P$ have $k$ countable components, the von Mises expansion reduces to a standard multivariate Taylor expansion with

$$R_2(\bar{P},P)=\psi \left({\bar{p}}_1,\dots ,{\bar{p}}_k\right)-\psi \left(p_1,\dots ,p_k\right)-{\sum _j\frac{\partial }{\partial t_j}\psi \left(t_1,\dots ,t_k\right)|}_{t=\bar{p}}\left({\bar{p}}_j-p_j\right).$$

[hitori_bocchi]

今天就更到这里，去看碧蓝档案3周年fes了（

[nomana]

我爱你🤟波奇酱

[hitori_bocchi]

@nomana 谢谢~

[nomana]

波奇酱好久没更了

[hitori_bocchi]

@nomana 最近科研太忙了，等放假了再更

[hashhash]

@hitori_bocchi 原来你也是档友!

[hitori_bocchi]

@hashhash 阿罗娜可爱!

[hitori_bocchi]

@hashhash 可以加贵校碧蓝档案群155199376聊天吹水