Built site for gh-pages

Confirm-Solutions · Jan 8, 2024 · 743da1e · 743da1e
1 parent bbbc956
commit 743da1e
Show file tree

Hide file tree

Showing 10 changed files with 103 additions and 67 deletions.
diff --git a/.nojekyll b/.nojekyll
@@ -1 +1 @@
-eb7435fb
+65694d95
diff --git a/index.html b/index.html
@@ -143,7 +143,7 @@
 
 <div class="quarto-listing quarto-listing-container-grid" id="listing-listing">
 <div class="list grid quarto-listing-cols-3">
-<div class="g-col-1" data-index="0" data-listing-date-sort="1701302400000" data-listing-file-modified-sort="1704695114679" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="7">
+<div class="g-col-1" data-index="0" data-listing-date-sort="1701302400000" data-listing-file-modified-sort="1704735082439" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="7">
 <a href="./posts/fight_the_illusion.html" class="quarto-grid-link">
 <div class="quarto-grid-item card h-100 card-left">
 <div class="listing-item-img-placeholder card-img-top" style="height: 150px;">&nbsp;</div>
@@ -166,7 +166,7 @@ <h5 class="no-anchor card-title listing-title">
 </div>
 </a>
 </div>
-<div class="g-col-1" data-index="1" data-listing-date-sort="1687651200000" data-listing-file-modified-sort="1704695114679" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="7">
+<div class="g-col-1" data-index="1" data-listing-date-sort="1687651200000" data-listing-file-modified-sort="1704735082439" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="7">
 <a href="./posts/catalog.html" class="quarto-grid-link">
 <div class="quarto-grid-item card h-100 card-left">
 <p class="card-img-top"><img src="posts/catalog_files/figure-html/cell-9-output-1.png" style="height: 150px;"  class="thumbnail-image card-img"/></p>
@@ -189,7 +189,7 @@ <h5 class="no-anchor card-title listing-title">
 </div>
 </a>
 </div>
-<div class="g-col-1" data-index="2" data-listing-date-sort="1672790400000" data-listing-file-modified-sort="1704695114659" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="24">
+<div class="g-col-1" data-index="2" data-listing-date-sort="1672790400000" data-listing-file-modified-sort="1704735082419" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="24">
 <a href="./posts/TDC2023.html" class="quarto-grid-link">
 <div class="quarto-grid-item card h-100 card-left">
 <div class="listing-item-img-placeholder card-img-top" style="height: 150px;">&nbsp;</div>

diff --git a/posts/TDC2023.html b/posts/TDC2023.html
@@ -440,9 +440,9 @@ <h4 class="anchored" data-anchor-id="tricks-that-we-found-to-improve-performance
   langid = {en}
 }
 </code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-straznickas2023" class="csl-entry quarto-appendix-citeas" role="listitem">
-Straznickas, Zygimantas, T. Ben Thompson, and Michael Sklar. 2023.
-<span>“Takeaways from the Neurips 2023 Trojan Detection
-Competition.”</span> January 4, 2023. <a href="https://confirmlabs.org/posts/TDC2023.html">https://confirmlabs.org/posts/TDC2023.html</a>.
+<div class="">Z.
+Straznickas, T. B. Thompson, and M. Sklar, <span>“Takeaways from the
+Neurips 2023 Trojan Detection Competition,”</span> Jan. 04, 2023. <a href="https://confirmlabs.org/posts/TDC2023.html">https://confirmlabs.org/posts/TDC2023.html</a></div>
 </div></div></section></div></main> <!-- /main -->
 <script id="quarto-html-after-body" type="application/javascript">
 window.document.addEventListener("DOMContentLoaded", function (event) {

diff --git a/posts/TDC2023.ipynb b/posts/TDC2023.ipynb
@@ -11,7 +11,7 @@
         "Michael Sklar  \n",
         "2023-01-04"
       ],
-      "id": "1641bdf1-d87d-4dde-b3d9-072c027d972a"
+      "id": "1a28eb12-1119-45f8-af20-c35dd2e357d8"
     },
     {
       "cell_type": "raw",
@@ -37,7 +37,7 @@
         "* Source doc: 6 ways to fight the Interpretability illusion\n",
         "----->"
       ],
-      "id": "32860964-8116-4e53-841b-0f1b458e3744"
+      "id": "3a1e5dc9-3dbb-4581-a9f5-a7043aaa3fb3"
     },
     {
       "cell_type": "markdown",
@@ -636,7 +636,7 @@
         "            not recommend extrapolating these results far beyond the\n",
         "            experimental setting."
       ],
-      "id": "5c1d97ce-e923-4d22-bb36-58537e695b76"
+      "id": "85a9fd3c-20cc-48f5-b7d4-9db8c2c2af10"
     }
   ],
   "nbformat": 4,

diff --git a/posts/catalog.html b/posts/catalog.html
diff --git a/posts/catalog.out.ipynb b/posts/catalog.out.ipynb
@@ -14,16 +14,16 @@
         "interpretability.\n",
         "\n",
         "1.  **Token-bigram and token-trigram prediction**: a dataset of n-gram\n",
-        "    statistics from [The Pile](https://pile.eleuther.ai) (Gao et\n",
-        "    al. 2020) including tables of one and two token prompts with their\n",
-        "    most likely completions. One of the simplest “tasks” for a language\n",
-        "    model is bigram completion.\n",
+        "    statistics from [The Pile](https://pile.eleuther.ai)\n",
+        "    [\\[1\\]](#ref-pile) including tables of one and two token prompts\n",
+        "    with their most likely completions. One of the simplest “tasks” for\n",
+        "    a language model is bigram completion.\n",
         "    -   for example, during training, 99.8% of the time the model sees\n",
         "        `\" telome\"`, the correct next token is `\"res\"`.\n",
         "2.  **First token deletion**: a dataset constructed by differencing the\n",
-        "    outputs of Pythia-2.8B (Biderman et al. 2023) between four and five\n",
-        "    token prompts. This method highlights tokens that are extremely\n",
-        "    predictive in context.\n",
+        "    outputs of Pythia-2.8B [\\[2\\]](#ref-biderman2023pythia) between four\n",
+        "    and five token prompts. This method highlights tokens that are\n",
+        "    extremely predictive in context.\n",
         "    -   for example, when prompted with `\", or common table\"`, the model\n",
         "        predicts `\" expression\"`\n",
         "        ([CTE](https://en.wikipedia.org/wiki/Hierarchical_and_recursive_queries_in_SQL#Common_table_expression))\n",
@@ -297,7 +297,7 @@
         "Pythia-12B is miscalibrated on 20% of the bigrams and 45% of the\n",
         "trigrams when we ask for prediction of $p \\geq 0.45$."
       ],
-      "id": "d5401b7a-e3ff-4137-9b31-46d4cb0d8f60"
+      "id": "68873125-776f-4f4c-ac88-4c1d7370e767"
     },
     {
       "cell_type": "code",
@@ -313,26 +313,28 @@
         }
       ],
       "source": [],
-      "id": "06746351-5182-4f6e-bb70-2e6e44f740b8"
+      "id": "f1fe185b-a3a6-4a30-a914-2ccc7c510dee"
     },
     {
       "cell_type": "markdown",
       "metadata": {},
       "source": [
         "Usage notes:\n",
         "\n",
-        "-   Because the byte-pair encoding tokenizer from GPT-NeoX (Black et\n",
-        "    al. 2022) was trained on The Pile, there are no single tokens in The\n",
-        "    Pile where the subsequent token is 100% predictable. However, there\n",
-        "    are many trigrams that are 100% predictable.\n",
+        "-   Because the byte-pair encoding tokenizer from GPT-NeoX\n",
+        "    [\\[3\\]](#ref-black-etal-2022-gpt) was trained on The Pile, there are\n",
+        "    no single tokens in The Pile where the subsequent token is 100%\n",
+        "    predictable. However, there are many trigrams that are 100%\n",
+        "    predictable.\n",
         "-   Some whitespace token bigrams will also tokenize as a single token.\n",
         "    For example, with the GPT-NeoX tokenizer, `\"\\n\\n\\t\\t\"` is a token,\n",
         "    `\"\\t\"` is a token *and* `\"\\n\\n\\t\\t\\t\"` is also token. It’s important\n",
         "    to be aware of this when automatically tokenizing many prompts\n",
         "    because almost all concatenated bigrams will tokenize to two tokens\n",
         "    but a few whitespace-related bigrams will tokenize to one token. We\n",
         "    have not removed these bigrams from the dataset. This white space\n",
-        "    tokenization is discussed in Appendix F of Black et al. (2022).\n",
+        "    tokenization is discussed in Appendix F of\n",
+        "    [\\[3\\]](#ref-black-etal-2022-gpt).\n",
         "\n",
         "## First token deletion\n",
         "\n",
@@ -375,7 +377,7 @@
         "The dataset is available on Huggingface:\n",
         "[pile_scan_4](https://huggingface.co/datasets/Confirm-Labs/pile_scan_4)"
       ],
-      "id": "9148c97d-cb62-4a5f-ab76-bc529291956d"
+      "id": "988663de-27b8-4b6e-a8a1-2b463bafe479"
     },
     {
       "cell_type": "code",
@@ -389,7 +391,7 @@
         }
       ],
       "source": [],
-      "id": "c22e7666-9233-44c3-941b-bafc9f11df41"
+      "id": "f6665e54-eda0-4111-969e-163e3e2b1d54"
     },
     {
       "cell_type": "markdown",
@@ -403,23 +405,25 @@
         "The code to reproduce the datasets here is available at:\n",
         "<https://github.com/Confirm-Solutions/catalog>\n",
         "\n",
-        "Biderman, Stella, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley,\n",
-        "Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, et al. 2023. “Pythia:\n",
-        "A Suite for Analyzing Large Language Models Across Training and\n",
-        "Scaling.” <https://arxiv.org/abs/2304.01373>.\n",
-        "\n",
-        "Black, Sidney, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao,\n",
-        "Laurence Golding, Horace He, et al. 2022. “GPT-NeoX-20B: An Open-Source\n",
-        "Autoregressive Language Model.” In *Proceedings of BigScience Episode\n",
-        "\\#5 – Workshop on Challenges & Perspectives in Creating Large Language\n",
-        "Models*, 95–136. virtual+Dublin: Association for Computational\n",
-        "Linguistics. <https://doi.org/10.18653/v1/2022.bigscience-1.9>.\n",
-        "\n",
-        "Gao, Leo, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe,\n",
-        "Charles Foster, Jason Phang, et al. 2020. “The Pile: An 800GB Dataset of\n",
-        "Diverse Text for Language Modeling.” *arXiv Preprint arXiv:2101.00027*."
+        "<span class=\"csl-left-margin\">\\[1\\]\n",
+        "</span><span class=\"csl-right-inline\">L. Gao *et al.*, “The Pile: An\n",
+        "800GB dataset of diverse text for language modeling,” *arXiv preprint\n",
+        "arXiv:2101.00027*, 2020.</span>\n",
+        "\n",
+        "<span class=\"csl-left-margin\">\\[2\\]\n",
+        "</span><span class=\"csl-right-inline\">S. Biderman *et al.*, “Pythia: A\n",
+        "suite for analyzing large language models across training and scaling.”\n",
+        "2023. Available: <https://arxiv.org/abs/2304.01373></span>\n",
+        "\n",
+        "<span class=\"csl-left-margin\">\\[3\\]\n",
+        "</span><span class=\"csl-right-inline\">S. Black *et al.*, “GPT-NeoX-20B:\n",
+        "An open-source autoregressive language model,” in *Proceedings of\n",
+        "BigScience episode \\#5 – workshop on challenges & perspectives in\n",
+        "creating large language models*, virtual+Dublin: Association for\n",
+        "Computational Linguistics, May 2022, pp. 95–136. doi:\n",
+        "[10.18653/v1/2022.bigscience-1.9](https://doi.org/10.18653/v1/2022.bigscience-1.9).</span>"
       ],
-      "id": "2075cf2f-1725-42a9-8390-31aef515c642"
+      "id": "168933f1-e606-49dc-9bce-2f72c20386e7"
     }
   ],
   "nbformat": 4,