theorem，effective size，lasso

加了几个定理，解释，和算法

07-Matrix-algebra.Rmd

@@ -86,6 +86,129 @@ $$
 而对于我需要的情况，简化$m_1=0$,$m_2=0$
 
 那么就有，只需要$\mathbf{\Sigma}_{c}^{-1}=\mathbf{\Sigma}_{1}^{-1}+\mathbf{\Sigma}_{2}^{-1}$,同时$m_c=0$.
+
+
+
+## 正定阵的基础定理：
+Theorem 1 
+
+For a \(p \times p\) symmetric matrix \(\Sigma=\left(\sigma_{i j}\right),\) the following are equivalent:
+
+(a) \(\Sigma\) is nonnegative definite.
+
+(b) all its leading principal minors are nonnegative definite, that is, the \(i \times i\) matrices
+$$
+\boldsymbol{\Sigma}_{i i}=\left(\begin{array}{ccc}
+{\sigma_{11}} & {\cdots} & {\sigma_{1 i}} \\
+{\vdots} & {\ddots} & {\vdots} \\
+{\sigma_{i 1}} & {\cdots} & {\sigma_{i i}}
+\end{array}\right), i=1, \cdots, p
+$$
+are nonnegative definite.
+
+(c) all eigenvalues of \(\Sigma\) are nonnegative.
+
+(d) there exists a matrix A such that
+$$
+\boldsymbol{\Sigma}=A A^{\prime}
+$$
+
+(e) there exists a lower triangular matrix \(L\) such that
+$$
+\boldsymbol{\Sigma}=L L^{\prime}
+$$
+
+(f) there exist vectors \(\boldsymbol{u}_{1}, \cdots, \boldsymbol{u}_{p}\) in \(R^{p}\) such that \(\sigma_{i j}=\boldsymbol{u}_{i}^{\prime} \boldsymbol{u}_{j}\)
+
+
+## 对称阵的谱分解相关定理
+Theorem 4 (The Spectral Decomposition) Let \(A\) be a \(p \times p\) symmetric matrix with p pairs of eigenvalues and eigenvectors
+$$
+\left(\lambda_{1}, \mathbf{e}_{1}\right),\left(\lambda_{2}, \mathbf{e}_{2}\right), \cdots,\left(\lambda_{p}, \mathbf{e}_{p}\right)
+$$
+Then,
+
+(a) The eigenvalues \(\lambda_{1}, \ldots, \lambda_{p}\) are all real, and can be ordered from the largest
+to the smallest
+$$
+\lambda_{1} \geq \lambda_{2} \geq \ldots \geq \lambda_{p}
+$$
+
+(b) The normalized eigenvectors \(\mathbf{e}_{1}, \ldots, \mathbf{e}_{p}\) are mutually orthogonal and the matrix
+$$
+P=\left(\mathbf{e}_{1}, \mathbf{e}_{2}, \ldots, \mathbf{e}_{p}\right)
+$$
+is an orthogonal matrix, that is,
+$$
+P P^{\prime}=P^{\prime} P=I
+$$
+
+(c) The spectral decomposition of \(A\) is the expansion
+$$
+A=\lambda_{1} \mathbf{e}_{1} \mathbf{e}_{1}^{\prime}+\lambda_{2} \mathbf{e}_{2} \mathbf{e}_{2}^{\prime}+\cdots+\lambda_{p} \mathbf{e}_{p} \mathbf{e}_{p}^{\prime}=P \Lambda P^{\prime}
+$$
+where \(P\) is as above and
+$$
+\Lambda=\operatorname{diag}\left(\lambda_{1}, \cdots, \lambda_{p}\right)
+$$
+is a diagonal matrix with \(\lambda_{1}, \ldots, \lambda_{p}\) as its respective diagonal entries.
+
+(d) The matrix \(A\) is nonnegative definite, if and only if all its eigenvalues are
+nonnegative.
+
+## Covariance Structure
+
+### Compound Symmetry Covariance结构
+
+P54 of pourahmadi covariance book
+
+a sufficient condition for nonnegative definite is : $1+(p-1) \rho \geq 0$ or \(-(p-1)^{-1} \leq \rho \leq 1\)
+
+### Huynh-Feldt Structure
+
+\(\Sigma=\sigma^{2}\left(\alpha I+a \mathbf{1}_{p}^{\prime}+\mathbf{1}_{p} a^{\prime}\right)\)
+
+The $\Sigma$ is nonnegative definite provided that
+$$
+\alpha>\mathbf{1}_{p}^{\prime} a-\sqrt{p a^{\prime} a}
+$$
+
+### The One-Dependent Covariance Structure
+$$
+\Sigma=\sigma^{2}\left(\begin{array}{ccccc}
+{a} & {b} & {0} & {\cdots} & {0} \\
+{b} & {a} & {b} & {\ddots} & {\vdots} \\
+{\vdots} & {\ddots} & {\ddots} & {\ddots} & {0} \\
+{\vdots} & {} & {\ddots} & {\ddots} & {b} \\
+{0} & {\cdots} & {0} & {b} & {a}
+\end{array}\right)
+$$
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 
 
 

26-not-direct-statistical-term.Rmd

@@ -19,8 +19,12 @@
 也叫[Sample probability](https://en.wikipedia.org/wiki/Sampling_probability)
 在有限总体中抽样，一个元素的sampling probability(inclusion probability)，被一次抽样抽出来的概率。
 
+- effective size 效果大小
 
+The terms treatment effects and effect sizes are used in different ways by different people. Meta-analyses in medicine often refer to the effect size as a treatment effect, and this term is sometimes assumed to refer to odds ratios, risk ratios, or riskdifferences, which are common in meta-analyses that deal with medical interventions.Similarly, meta-analyses in the social sciences often refer to the effect size simply as an effect size and this term is sometimes assumed to refer to standardized mean differences or to correlations, which are common in social science meta-analyses. by book [Introduction to Meta-Analysis]
 
+- condition number
 
+The condition number of a nonnegative definite matrix is defined as $\lambda_1/\lambda_p$ or the ratio of its largest and smallest eigenvalues, it is infinity when the latter is zero.
 
 

27-proof-and-calculation.Rmd

@@ -0,0 +1,34 @@
+# Proof and Calculation
+
+本来会推的会证的就少，还老忘，推一个就抄一个过来就好了。虽然不保证因为很麻烦。
+
+### Coordinate Descent Algorithm for Lasso
+For a general design matrix $X$, there is no closed form for the Lasso solution and the computational details of the Lasso procedure are more involved. A fast method to solve the general Lasso regression problem is the coordinate descent algorithm which minimizes loss function over one $\beta_j$ at a time with the others kept fixed. It then cycles through all the parameters until convergence (Friedman et al.,2008). 
+
+Suppose all the values of $\beta_k$ for $k\neq j$ are held fixed at their current value $\tilde \beta_k$, so that $Q(\tilde\beta)$ can be written as
+$$
+Q(\tilde{\beta})=\frac{1}{2} \sum_{i=1}^{n}\left(Y_{i}-\sum_{k \neq j} x_{i k} \tilde{\beta}_{k}-x_{i j} \beta_{j}\right)^{2}+\lambda \sum_{k \neq j}\left|\tilde{\beta}_{k}\right|+\lambda\left|\beta_{j}\right|
+$$
+也就是写成两块，一块是fixed的$\beta_k$,$k\neq j$, 一块是当作变量的$\beta_j$.这样就是一个关于$\beta_j$的单变量函数。关于$\beta_j$ 解单变量lasso，解就能写成soft threshold的形式：
+$$
+\widehat{\beta}(\lambda)=S\left(\sum_{i=1}^{n} x_{i} y_{i}, \lambda\right)
+$$
+这时候对应的元素应该是：$r_{i}^{(j)}=Y_{i}-\sum_{k \neq j} x_{i k} \tilde{\beta}_{k}$.则解就能直接写出来：
+$$
+\tilde{\beta}_{j} \leftarrow S\left(\sum_{i=1}^{n} x_{i j} r_{i}^{(j)}, \lambda\right)
+$$
+这时候算法对所有$\beta_j$重复，$j=1,2,...p,1,2,...p,...$直到$\tilde \beta$收敛。
+一般来说，$Lasso$的解$\hat\beta$的characterization是通过KKT条件，极小化convex function。接下来是个定理，说明平方损失函数那块的梯度表示为$\beta$的函数的话，$\hat\beta$是解的充分必要条件是如果$\hat\beta\neq 0$,则梯度是$-\operatorname{sign}\left(\widehat{\beta}_{j}\right) \lambda$,如果$\hat\beta=0$,则梯度的绝对值小于$\lambda$.
+不太理解所以不写了，需要去看书。
+
+
+
+
+
+
+
+
+
+
+
+

docs/advanced-r.html

@@ -25,7 +25,7 @@
 <meta name="author" content="Jiaming Shen">
 
 
-<meta name="date" content="2020-02-18">
+<meta name="date" content="2020-02-23">
 
   <meta name="viewport" content="width=device-width, initial-scale=1">
   <meta name="apple-mobile-web-app-capable" content="yes">
@@ -152,6 +152,7 @@
 <li class="chapter" data-level="4.1.1" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#block-diagonal-matrices"><i class="fa fa-check"></i><b>4.1.1</b> Block diagonal matrices</a></li>
 </ul></li>
 <li class="chapter" data-level="4.2" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#sumSquare"><i class="fa fa-check"></i><b>4.2</b> 两个二次型相加</a></li>
+<li class="chapter" data-level="4.3" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#section-4.3"><i class="fa fa-check"></i><b>4.3</b> 正定阵的基础定理：</a></li>
 </ul></li>
 <li class="chapter" data-level="5" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html"><i class="fa fa-check"></i><b>5</b> Longitudinal data analysis</a><ul>
 <li class="chapter" data-level="5.1" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html#linear-mixed-model"><i class="fa fa-check"></i><b>5.1</b> Linear mixed model</a><ul>
@@ -315,6 +316,9 @@
 </ul></li>
 <li class="chapter" data-level="20" data-path="r-trick.html"><a href="r-trick.html"><i class="fa fa-check"></i><b>20</b> R trick</a></li>
 <li class="chapter" data-level="21" data-path="statistic-term.html"><a href="statistic-term.html"><i class="fa fa-check"></i><b>21</b> Statistic term</a></li>
+<li class="chapter" data-level="22" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html"><i class="fa fa-check"></i><b>22</b> Proof and Calculation</a><ul>
+<li class="chapter" data-level="22.0.1" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html#coordinate-descent-algorithm-for-lasso"><i class="fa fa-check"></i><b>22.0.1</b> Coordinate Descent Algorithm for Lasso</a></li>
+</ul></li>
 <li class="chapter" data-level="" data-path="references.html"><a href="references.html"><i class="fa fa-check"></i>References</a></li>
 <li class="divider"></li>
 <li><a href="https://github.com/rstudio/bookdown" target="blank">Published with bookdown</a></li>
@@ -399,20 +403,20 @@ <h3><span class="header-section-number">12.1.1</span> Ordering (integer subsetti
 df2=df[<span class="kw">sample</span>(<span class="kw">nrow</span>(df)),<span class="dv">3</span><span class="op">:</span><span class="dv">1</span>]
 df2[<span class="kw">order</span>(df2<span class="op">$</span>x),]</code></pre>
 <pre><code>##   z y x
-## 1 a 6 1
 ## 2 b 5 1
+## 1 a 6 1
 ## 4 d 3 2
 ## 3 c 4 2
 ## 5 e 2 3
 ## 6 f 1 3</code></pre>
 <pre class="sourceCode r"><code class="sourceCode r">df2[,<span class="kw">order</span>(<span class="kw">names</span>(df2))]</code></pre>
 <pre><code>##   x y z
-## 5 3 2 e
-## 1 1 6 a
-## 4 2 3 d
 ## 2 1 5 b
+## 4 2 3 d
+## 5 3 2 e
 ## 6 3 1 f
-## 3 2 4 c</code></pre>
+## 3 2 4 c
+## 1 1 6 a</code></pre>
 <pre class="sourceCode r"><code class="sourceCode r">df=<span class="kw">data.frame</span>(<span class="dt">x=</span><span class="dv">1</span><span class="op">:</span><span class="dv">3</span>,<span class="dt">y=</span><span class="dv">3</span><span class="op">:</span><span class="dv">1</span>,<span class="dt">z=</span>letters[<span class="dv">1</span><span class="op">:</span><span class="dv">3</span>])</code></pre>
 </div>
 <div id="calling-a-function-given-a-list-of-arguments" class="section level3">

docs/basicmcmc.html

@@ -25,7 +25,7 @@
 <meta name="author" content="Jiaming Shen">
 
 
-<meta name="date" content="2020-02-18">
+<meta name="date" content="2020-02-23">
 
   <meta name="viewport" content="width=device-width, initial-scale=1">
   <meta name="apple-mobile-web-app-capable" content="yes">
@@ -152,6 +152,7 @@
 <li class="chapter" data-level="4.1.1" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#block-diagonal-matrices"><i class="fa fa-check"></i><b>4.1.1</b> Block diagonal matrices</a></li>
 </ul></li>
 <li class="chapter" data-level="4.2" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#sumSquare"><i class="fa fa-check"></i><b>4.2</b> 两个二次型相加</a></li>
+<li class="chapter" data-level="4.3" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#section-4.3"><i class="fa fa-check"></i><b>4.3</b> 正定阵的基础定理：</a></li>
 </ul></li>
 <li class="chapter" data-level="5" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html"><i class="fa fa-check"></i><b>5</b> Longitudinal data analysis</a><ul>
 <li class="chapter" data-level="5.1" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html#linear-mixed-model"><i class="fa fa-check"></i><b>5.1</b> Linear mixed model</a><ul>
@@ -315,6 +316,9 @@
 </ul></li>
 <li class="chapter" data-level="20" data-path="r-trick.html"><a href="r-trick.html"><i class="fa fa-check"></i><b>20</b> R trick</a></li>
 <li class="chapter" data-level="21" data-path="statistic-term.html"><a href="statistic-term.html"><i class="fa fa-check"></i><b>21</b> Statistic term</a></li>
+<li class="chapter" data-level="22" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html"><i class="fa fa-check"></i><b>22</b> Proof and Calculation</a><ul>
+<li class="chapter" data-level="22.0.1" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html#coordinate-descent-algorithm-for-lasso"><i class="fa fa-check"></i><b>22.0.1</b> Coordinate Descent Algorithm for Lasso</a></li>
+</ul></li>
 <li class="chapter" data-level="" data-path="references.html"><a href="references.html"><i class="fa fa-check"></i>References</a></li>
 <li class="divider"></li>
 <li><a href="https://github.com/rstudio/bookdown" target="blank">Published with bookdown</a></li>

docs/bayes-variable-selection.html

@@ -25,7 +25,7 @@
 <meta name="author" content="Jiaming Shen">
 
 
-<meta name="date" content="2020-02-18">
+<meta name="date" content="2020-02-23">
 
   <meta name="viewport" content="width=device-width, initial-scale=1">
   <meta name="apple-mobile-web-app-capable" content="yes">
@@ -152,6 +152,7 @@
 <li class="chapter" data-level="4.1.1" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#block-diagonal-matrices"><i class="fa fa-check"></i><b>4.1.1</b> Block diagonal matrices</a></li>
 </ul></li>
 <li class="chapter" data-level="4.2" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#sumSquare"><i class="fa fa-check"></i><b>4.2</b> 两个二次型相加</a></li>
+<li class="chapter" data-level="4.3" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#section-4.3"><i class="fa fa-check"></i><b>4.3</b> 正定阵的基础定理：</a></li>
 </ul></li>
 <li class="chapter" data-level="5" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html"><i class="fa fa-check"></i><b>5</b> Longitudinal data analysis</a><ul>
 <li class="chapter" data-level="5.1" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html#linear-mixed-model"><i class="fa fa-check"></i><b>5.1</b> Linear mixed model</a><ul>
@@ -315,6 +316,9 @@
 </ul></li>
 <li class="chapter" data-level="20" data-path="r-trick.html"><a href="r-trick.html"><i class="fa fa-check"></i><b>20</b> R trick</a></li>
 <li class="chapter" data-level="21" data-path="statistic-term.html"><a href="statistic-term.html"><i class="fa fa-check"></i><b>21</b> Statistic term</a></li>
+<li class="chapter" data-level="22" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html"><i class="fa fa-check"></i><b>22</b> Proof and Calculation</a><ul>
+<li class="chapter" data-level="22.0.1" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html#coordinate-descent-algorithm-for-lasso"><i class="fa fa-check"></i><b>22.0.1</b> Coordinate Descent Algorithm for Lasso</a></li>
+</ul></li>
 <li class="chapter" data-level="" data-path="references.html"><a href="references.html"><i class="fa fa-check"></i>References</a></li>
 <li class="divider"></li>
 <li><a href="https://github.com/rstudio/bookdown" target="blank">Published with bookdown</a></li>

docs/e69d82e4b883e69d82e585ab.html

@@ -25,7 +25,7 @@
 <meta name="author" content="Jiaming Shen">
 
 
-<meta name="date" content="2020-02-18">
+<meta name="date" content="2020-02-23">
 
   <meta name="viewport" content="width=device-width, initial-scale=1">
   <meta name="apple-mobile-web-app-capable" content="yes">
@@ -152,6 +152,7 @@
 <li class="chapter" data-level="4.1.1" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#block-diagonal-matrices"><i class="fa fa-check"></i><b>4.1.1</b> Block diagonal matrices</a></li>
 </ul></li>
 <li class="chapter" data-level="4.2" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#sumSquare"><i class="fa fa-check"></i><b>4.2</b> 两个二次型相加</a></li>
+<li class="chapter" data-level="4.3" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#section-4.3"><i class="fa fa-check"></i><b>4.3</b> 正定阵的基础定理：</a></li>
 </ul></li>
 <li class="chapter" data-level="5" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html"><i class="fa fa-check"></i><b>5</b> Longitudinal data analysis</a><ul>
 <li class="chapter" data-level="5.1" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html#linear-mixed-model"><i class="fa fa-check"></i><b>5.1</b> Linear mixed model</a><ul>
@@ -315,6 +316,9 @@
 </ul></li>
 <li class="chapter" data-level="20" data-path="r-trick.html"><a href="r-trick.html"><i class="fa fa-check"></i><b>20</b> R trick</a></li>
 <li class="chapter" data-level="21" data-path="statistic-term.html"><a href="statistic-term.html"><i class="fa fa-check"></i><b>21</b> Statistic term</a></li>
+<li class="chapter" data-level="22" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html"><i class="fa fa-check"></i><b>22</b> Proof and Calculation</a><ul>
+<li class="chapter" data-level="22.0.1" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html#coordinate-descent-algorithm-for-lasso"><i class="fa fa-check"></i><b>22.0.1</b> Coordinate Descent Algorithm for Lasso</a></li>
+</ul></li>
 <li class="chapter" data-level="" data-path="references.html"><a href="references.html"><i class="fa fa-check"></i>References</a></li>
 <li class="divider"></li>
 <li><a href="https://github.com/rstudio/bookdown" target="blank">Published with bookdown</a></li>

docs/general-notes.html

@@ -25,7 +25,7 @@
 <meta name="author" content="Jiaming Shen">
 
 
-<meta name="date" content="2020-02-18">
+<meta name="date" content="2020-02-23">
 
   <meta name="viewport" content="width=device-width, initial-scale=1">
   <meta name="apple-mobile-web-app-capable" content="yes">
@@ -152,6 +152,7 @@
 <li class="chapter" data-level="4.1.1" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#block-diagonal-matrices"><i class="fa fa-check"></i><b>4.1.1</b> Block diagonal matrices</a></li>
 </ul></li>
 <li class="chapter" data-level="4.2" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#sumSquare"><i class="fa fa-check"></i><b>4.2</b> 两个二次型相加</a></li>
+<li class="chapter" data-level="4.3" data-path="statistician-tool-box.html"><a href="statistician-tool-box.html#section-4.3"><i class="fa fa-check"></i><b>4.3</b> 正定阵的基础定理：</a></li>
 </ul></li>
 <li class="chapter" data-level="5" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html"><i class="fa fa-check"></i><b>5</b> Longitudinal data analysis</a><ul>
 <li class="chapter" data-level="5.1" data-path="longitudinal-data-analysis.html"><a href="longitudinal-data-analysis.html#linear-mixed-model"><i class="fa fa-check"></i><b>5.1</b> Linear mixed model</a><ul>
@@ -315,6 +316,9 @@
 </ul></li>
 <li class="chapter" data-level="20" data-path="r-trick.html"><a href="r-trick.html"><i class="fa fa-check"></i><b>20</b> R trick</a></li>
 <li class="chapter" data-level="21" data-path="statistic-term.html"><a href="statistic-term.html"><i class="fa fa-check"></i><b>21</b> Statistic term</a></li>
+<li class="chapter" data-level="22" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html"><i class="fa fa-check"></i><b>22</b> Proof and Calculation</a><ul>
+<li class="chapter" data-level="22.0.1" data-path="proof-and-calculation.html"><a href="proof-and-calculation.html#coordinate-descent-algorithm-for-lasso"><i class="fa fa-check"></i><b>22.0.1</b> Coordinate Descent Algorithm for Lasso</a></li>
+</ul></li>
 <li class="chapter" data-level="" data-path="references.html"><a href="references.html"><i class="fa fa-check"></i>References</a></li>
 <li class="divider"></li>
 <li><a href="https://github.com/rstudio/bookdown" target="blank">Published with bookdown</a></li>