deploy: 2c187c7

_sources/pca_introduction.Rmd

@@ -754,34 +754,39 @@ VT.shape
 ```
 
 By the definition of the SVD, $\U$ and $\V^T$ are orthogonal matrices, so
-$\U^T$ is the inverse of $\U$ and $\U^T \U = I$.  Therefore:
+$\V$ is the inverse of $\V^T$ and $\V^T \V = I$.  Therefore:
 
 $$
 \X = \U \Sigma \V^T \implies
-\U^T \X = \Sigma \V^T
+\X \V = \U \Sigma
 $$
 
-You may recognize $\U^T \X$ as the matrix of scalar projections $\C$ above:
+You may recognize $\X \V$ as the matrix of scalar projections $\C$ above:
 
 ```{python}
-C = U.T.dot(X)
-np.allclose(np.diag(S).dot(VT), C)
+# Implement the formula above.
+C = X.dot(VT.T)
+np.allclose(C, U.dot(np.diag(S)))
 ```
 
-Because $\V^T$ is also an orthogonal matrix, it has row lengths of 1, and we
+Because $\U$ is also an orthogonal matrix, it has row lengths of 1, and we
 can get the values in $\S$ from the row lengths of $\C$:
 
 ```{python}
-S_from_C = np.sqrt(np.sum(C ** 2, axis=1))
+S_from_C = np.sqrt(np.sum(C ** 2, axis=0))
 np.allclose(S_from_C, S)
 ```
 
-Now we can reconstruct $\V^T$:
+```{python}
+S_from_C
+```
+
+Now we can reconstruct $\U$:
 
 ```{python}
 # Divide out reconstructed S values
-S_as_column = S_from_C.reshape((2, 1))
-np.allclose(C / S_as_column, VT)
+S_as_row = S_from_C.reshape((1, 2))
+np.allclose(C / S_as_row, U)
 ```
 
 The SVD is quick to compute for a small matrix like `X`, but when the larger
@@ -793,43 +798,41 @@ Here’s why that works:
 
 $$
 \U \S \V^T = \X \\
-(\U \S \V^T)(\U \S \V^T)^T = \X \X^T
+(\U \S \V^T)^T(\U \S \V^T) = \X^T \X
 $$
 
-By the matrix transpose rule and associativity of matrix multiplication:
+By the multiplication property of the matrix transpose and associativity of matrix multiplication:
 
 $$
-\U \S \V^T \V \S^T \U^T = \X \X^T
+\V \S^T \U^T \U \S \V^T = \X^T \X
 $$
 
-$\V^T$ is an orthogonal matrix, so $\V^T = \V^{-1}$ and $\V^T \V = I$. $\S$ is
-a diagonal matrix so $\S \S^T = \S^2$, where $\S^2$ is a square diagonal
+$\U$ is an orthogonal matrix, so $\U^T = \U^{-1}$ and $\U^T \U = I$. $\S$ is
+a diagonal matrix so $\S^T \S = \S^2$, where $\S^2$ is a square diagonal
 matrix shape $M$ by $M$ containing the squares of the singular values from
 $\S$:
 
 $$
-\U \S^2 \U^T = \X \X^T
+\V \S^2 \V^T = \X^T \X
 $$
 
-This last formula is the formula for the SVD of $\X \X^T$. So, we can get our
-$\U$ and $\S$ from the SVD on $\X \X^T$.
+This last formula is the formula for the SVD of $\X^T \X$. So, we can get our
+$\V^T$ and $\S$ from the SVD on $\X^T \X$.
 
 ```{python}
-# Finding principal components using SVD on X X^T
+# Finding principal components using SVD on X^T X
 unscaled_cov = X.T.dot(X)
-U_vcov, S_vcov, VT_vcov = npl.svd(unscaled_cov)
+V_vcov, S_vcov, VT_vcov = npl.svd(unscaled_cov)
 VT_vcov
 ```
 
 ```{python}
-VT_vcov
+# VT_vcov equal to VT from SVD on X.
+np.allclose(VT_vcov, VT)
 ```
 
-We know from the derivation above that `VT_vcov` is just the transpose of
-$\U$:
-
 ```{python}
-np.allclose(VT, VT_vcov.T)
+np.allclose(V_vcov.T, VT_vcov)
 ```
 
 The returned vector `S_vcov` from the SVD on $\X \X^T$ now contains the

arrays_and_images.html

@@ -557,7 +557,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7fafec1439d0&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7ffab4d1d9d0&gt;
 </pre></div>
 </div>
 <img alt="_images/8281adb405f33ed86aaddeef44ac9d32ba14a4d9abd61f026b581dea90e38c9a.png" src="_images/8281adb405f33ed86aaddeef44ac9d32ba14a4d9abd61f026b581dea90e38c9a.png" />
@@ -577,7 +577,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.colorbar.Colorbar at 0x7fafe7f47790&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.colorbar.Colorbar at 0x7ffab4637250&gt;
 </pre></div>
 </div>
 <img alt="_images/2e99d479c3f31b89649f8ccb3e62cb79305e2a8d8158da39c3c01ab215ebda9c.png" src="_images/2e99d479c3f31b89649f8ccb3e62cb79305e2a8d8158da39c3c01ab215ebda9c.png" />
@@ -593,7 +593,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.colorbar.Colorbar at 0x7fafe7f81fd0&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.colorbar.Colorbar at 0x7ffab44fa9d0&gt;
 </pre></div>
 </div>
 <img alt="_images/798ae9e4e24d3bd4caa20515273d431c46867657879075b7d10b2f5c92c38df0.png" src="_images/798ae9e4e24d3bd4caa20515273d431c46867657879075b7d10b2f5c92c38df0.png" />
@@ -621,7 +621,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.colorbar.Colorbar at 0x7fafe4dcb2d0&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.colorbar.Colorbar at 0x7ffab45bb590&gt;
 </pre></div>
 </div>
 <img alt="_images/798ae9e4e24d3bd4caa20515273d431c46867657879075b7d10b2f5c92c38df0.png" src="_images/798ae9e4e24d3bd4caa20515273d431c46867657879075b7d10b2f5c92c38df0.png" />
@@ -688,7 +688,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x7fafe4de1750&gt;]
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x7ffab44101d0&gt;]
 </pre></div>
 </div>
 <img alt="_images/6a3de8092e56ba75dfad754f07c483e623f32affa850840796726fcf08235320.png" src="_images/6a3de8092e56ba75dfad754f07c483e623f32affa850840796726fcf08235320.png" />
@@ -707,7 +707,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x7fafe4c330d0&gt;]
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x7ffab4566dd0&gt;]
 </pre></div>
 </div>
 <img alt="_images/6a3de8092e56ba75dfad754f07c483e623f32affa850840796726fcf08235320.png" src="_images/6a3de8092e56ba75dfad754f07c483e623f32affa850840796726fcf08235320.png" />
@@ -744,7 +744,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7fafe4b24b10&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7ffab4304b10&gt;
 </pre></div>
 </div>
 <img alt="_images/86d20d294a78a59e2102282be42e0df3b186d3b3e39a10533148e8bef503ba60.png" src="_images/86d20d294a78a59e2102282be42e0df3b186d3b3e39a10533148e8bef503ba60.png" />
@@ -891,7 +891,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7fafe492cb10&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7ffab410cb10&gt;
 </pre></div>
 </div>
 <img alt="_images/358e0e778a7de48cb4e04b741c7624ac2ffafc83db5fe8ff811cb10680cfdc8d.png" src="_images/358e0e778a7de48cb4e04b741c7624ac2ffafc83db5fe8ff811cb10680cfdc8d.png" />
@@ -962,7 +962,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7fafe4954b10&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7ffab4596dd0&gt;
 </pre></div>
 </div>
 <img alt="_images/ed6c6162aa68889ee82a473d35f94dc75a82c21ffa2479db3e5280499c9ea4d0.png" src="_images/ed6c6162aa68889ee82a473d35f94dc75a82c21ffa2479db3e5280499c9ea4d0.png" />
@@ -980,7 +980,7 @@ <h1>Arrays as images, images as arrays<a class="headerlink" href="#arrays-as-ima
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7fafe4824910&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.image.AxesImage at 0x7ffab41ad810&gt;
 </pre></div>
 </div>
 <img alt="_images/e456adfb126dbed7a2b12895ad90262bb205d82358f06e69b4f0bb2e7bd397e1.png" src="_images/e456adfb126dbed7a2b12895ad90262bb205d82358f06e69b4f0bb2e7bd397e1.png" />

classes_inheritance.html

@@ -1040,8 +1040,8 @@ <h2>Objects, attributes, and initialization<a class="headerlink" href="#objects-
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>I am operating on &lt;__main__.MyClass object at 0x7f13841d32d0&gt;
-I will print the &#39;information&#39; attribute of &lt;__main__.MyClass object at 0x7f13841d32d0&gt;
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>I am operating on &lt;__main__.MyClass object at 0x7f91009babd0&gt;
+I will print the &#39;information&#39; attribute of &lt;__main__.MyClass object at 0x7f91009babd0&gt;
 </pre></div>
 </div>
 <div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
@@ -1168,8 +1168,8 @@ <h2>Objects, attributes, and initialization<a class="headerlink" href="#objects-
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>I am operating on &lt;__main__.MyClass2 object at 0x7f1384399150&gt;
-I will print the &#39;information&#39; attribute of &lt;__main__.MyClass2 object at 0x7f1384399150&gt;
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>I am operating on &lt;__main__.MyClass2 object at 0x7f9100a29e10&gt;
+I will print the &#39;information&#39; attribute of &lt;__main__.MyClass2 object at 0x7f9100a29e10&gt;
 any kind of value
 </pre></div>
 </div>

cost_functions.html

@@ -718,7 +718,7 @@ <h1>Fitting models with different cost functions<a class="headerlink" href="#fit
   <th>Date:</th>             <td>Thu, 21 Mar 2024</td> <th>  Prob (F-statistic):</th> <td>6.53e-39</td>
 </tr>
 <tr>
-  <th>Time:</th>                 <td>18:01:25</td>     <th>  Log-Likelihood:    </th> <td>  118.24</td>
+  <th>Time:</th>                 <td>19:25:14</td>     <th>  Log-Likelihood:    </th> <td>  118.24</td>
 </tr>
 <tr>
   <th>No. Observations:</th>      <td>    75</td>      <th>  AIC:               </th> <td>  -230.5</td>
@@ -1933,7 +1933,7 @@ <h2>Fitting with weights<a class="headerlink" href="#fitting-with-weights" title
   <th>Date:</th>             <td>Thu, 21 Mar 2024</td> <th>  Prob (F-statistic):</th> <td>2.06e-36</td>
 </tr>
 <tr>
-  <th>Time:</th>                 <td>18:01:26</td>     <th>  Log-Likelihood:    </th> <td>  90.990</td>
+  <th>Time:</th>                 <td>19:25:15</td>     <th>  Log-Likelihood:    </th> <td>  90.990</td>
 </tr>
 <tr>
   <th>No. Observations:</th>      <td>    75</td>      <th>  AIC:               </th> <td>  -176.0</td>
@@ -2927,27 +2927,27 @@ <h2>Cross-validation<a class="headerlink" href="#cross-validation" title="Permal
     <tr>
       <th>0</th>
       <td>3.756147</td>
-      <td>-1.275072</td>
+      <td>0.793889</td>
     </tr>
     <tr>
       <th>1</th>
       <td>3.487379</td>
-      <td>-1.524645</td>
+      <td>0.639870</td>
     </tr>
     <tr>
       <th>2</th>
       <td>3.608331</td>
-      <td>-1.769073</td>
+      <td>-0.732638</td>
     </tr>
     <tr>
       <th>3</th>
       <td>3.909520</td>
-      <td>0.822342</td>
+      <td>-0.756992</td>
     </tr>
     <tr>
       <th>4</th>
       <td>3.788818</td>
-      <td>-0.203832</td>
+      <td>1.100591</td>
     </tr>
     <tr>
       <th>...</th>
@@ -2957,27 +2957,27 @@ <h2>Cross-validation<a class="headerlink" href="#cross-validation" title="Permal
     <tr>
       <th>70</th>
       <td>3.932991</td>
-      <td>0.642115</td>
+      <td>-0.404363</td>
     </tr>
     <tr>
       <th>71</th>
       <td>3.636182</td>
-      <td>0.007748</td>
+      <td>-0.528626</td>
     </tr>
     <tr>
       <th>72</th>
       <td>3.749000</td>
-      <td>0.880925</td>
+      <td>-1.433002</td>
     </tr>
     <tr>
       <th>73</th>
       <td>3.441923</td>
-      <td>-0.248896</td>
+      <td>-0.068166</td>
     </tr>
     <tr>
       <th>74</th>
       <td>3.927255</td>
-      <td>1.172609</td>
+      <td>0.012150</td>
     </tr>
   </tbody>
 </table>
@@ -2995,7 +2995,7 @@ <h2>Cross-validation<a class="headerlink" href="#cross-validation" title="Permal
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.1867700304973484
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.18607589210234252
 </pre></div>
 </div>
 </div>
@@ -3007,7 +3007,7 @@ <h2>Cross-validation<a class="headerlink" href="#cross-validation" title="Permal
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.000917495363400328
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.0016116337584062002
 </pre></div>
 </div>
 </div>

epl_modeling.html

@@ -1040,7 +1040,7 @@ <h1>Attack or defense?  An example of multiple regression<a class="headerlink" h
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;statsmodels.regression.linear_model.OLS at 0x7f9d57795a50&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;statsmodels.regression.linear_model.OLS at 0x7f9dbf0b0cd0&gt;
 </pre></div>
 </div>
 </div>
@@ -1055,7 +1055,7 @@ <h1>Attack or defense?  An example of multiple regression<a class="headerlink" h
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;statsmodels.regression.linear_model.RegressionResultsWrapper at 0x7f9d5788c850&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;statsmodels.regression.linear_model.RegressionResultsWrapper at 0x7f9dbf0cc850&gt;
 </pre></div>
 </div>
 </div>
@@ -1084,7 +1084,7 @@ <h1>Attack or defense?  An example of multiple regression<a class="headerlink" h
   <th>Date:</th>             <td>Thu, 21 Mar 2024</td> <th>  Prob (F-statistic):</th>  <td>0.00128</td>
 </tr>
 <tr>
-  <th>Time:</th>                 <td>18:01:31</td>     <th>  Log-Likelihood:    </th> <td> -92.374</td>
+  <th>Time:</th>                 <td>19:25:19</td>     <th>  Log-Likelihood:    </th> <td> -92.374</td>
 </tr>
 <tr>
   <th>No. Observations:</th>      <td>    20</td>      <th>  AIC:               </th> <td>   188.7</td>
@@ -1305,7 +1305,7 @@ <h2>Defense or attack?<a class="headerlink" href="#defense-or-attack" title="Per
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.legend.Legend at 0x7f9d555b2fd0&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.legend.Legend at 0x7f9dbcc72fd0&gt;
 </pre></div>
 </div>
 <img alt="_images/12d79b00ef96273e64fd68bf224ea586ca385c61f0f474897144a5de5f20444f.png" src="_images/12d79b00ef96273e64fd68bf224ea586ca385c61f0f474897144a5de5f20444f.png" />
@@ -1335,7 +1335,7 @@ <h2>Defense or attack?<a class="headerlink" href="#defense-or-attack" title="Per
   <th>Date:</th>             <td>Thu, 21 Mar 2024</td> <th>  Prob (F-statistic):</th> <td>0.000185</td>
 </tr>
 <tr>
-  <th>Time:</th>                 <td>18:01:31</td>     <th>  Log-Likelihood:    </th> <td> -90.322</td>
+  <th>Time:</th>                 <td>19:25:20</td>     <th>  Log-Likelihood:    </th> <td> -90.322</td>
 </tr>
 <tr>
   <th>No. Observations:</th>      <td>    20</td>      <th>  AIC:               </th> <td>   184.6</td>
@@ -1398,7 +1398,7 @@ <h2>Defense or attack?<a class="headerlink" href="#defense-or-attack" title="Per
   <th>Date:</th>             <td>Thu, 21 Mar 2024</td> <th>  Prob (F-statistic):</th>  <td>0.00787</td>
 </tr>
 <tr>
-  <th>Time:</th>                 <td>18:01:31</td>     <th>  Log-Likelihood:    </th> <td> -94.262</td>
+  <th>Time:</th>                 <td>19:25:20</td>     <th>  Log-Likelihood:    </th> <td> -94.262</td>
 </tr>
 <tr>
   <th>No. Observations:</th>      <td>    20</td>      <th>  AIC:               </th> <td>   192.5</td>
@@ -1575,7 +1575,7 @@ <h2>Defense or attack?<a class="headerlink" href="#defense-or-attack" title="Per
   <th>Date:</th>             <td>Thu, 21 Mar 2024</td> <th>  Prob (F-statistic):</th> <td>0.000270</td>
 </tr>
 <tr>
-  <th>Time:</th>                 <td>18:01:31</td>     <th>  Log-Likelihood:    </th> <td> -88.626</td>
+  <th>Time:</th>                 <td>19:25:20</td>     <th>  Log-Likelihood:    </th> <td> -88.626</td>
 </tr>
 <tr>
   <th>No. Observations:</th>      <td>    20</td>      <th>  AIC:               </th> <td>   183.3</td>
@@ -2173,7 +2173,7 @@ <h2>Defense or attack?<a class="headerlink" href="#defense-or-attack" title="Per
   <th>Date:</th>             <td>Thu, 21 Mar 2024</td> <th>  Prob (F-statistic):</th>  <td>  1.00</td> 
 </tr>
 <tr>
-  <th>Time:</th>                 <td>18:01:31</td>     <th>  Log-Likelihood:    </th> <td> -88.626</td>
+  <th>Time:</th>                 <td>19:25:20</td>     <th>  Log-Likelihood:    </th> <td> -88.626</td>
 </tr>
 <tr>
   <th>No. Observations:</th>      <td>    20</td>      <th>  AIC:               </th> <td>   181.3</td>