From 52d0023b42aa97e6f3c7a7ce97317fa825c1d702 Mon Sep 17 00:00:00 2001
From: mikesklar <52256869+mikesklar@users.noreply.github.com>
Date: Fri, 12 Jan 2024 04:17:01 -0800
Subject: [PATCH] remove to

---
 posts/TDC2023.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/posts/TDC2023.md b/posts/TDC2023.md
index 413d6fa..e94380f 100644
--- a/posts/TDC2023.md
+++ b/posts/TDC2023.md
@@ -241,7 +241,7 @@ We can define an activation vector by contrasting a sample of bad-behavior insta
 
 #### **4. We think fluent red-teaming attacks are probably achievable with gradient-based methods**
 
-We think fluent red-teaming attacks are probably achievable with gradient-based methods despite arguments to the contrary from LLM-based papers (e.g., Liu et al. 2023). A common practical strategy for safeguarding LLMs from adversarial attack is to reject high-perplexity user inputs. With stronger regularization and with sufficient optimization runtime, it seems to possible to achieve success at red teaming with reduced perplexity. TDC2023 had no incentive for fluency, but we are currently investigating further and improving our methods for this type of optimization.
+We think fluent red-teaming attacks are probably achievable with gradient-based methods despite arguments to the contrary from LLM-based papers (e.g., Liu et al. 2023). A common practical strategy for safeguarding LLMs from adversarial attack is to reject high-perplexity user inputs. With stronger regularization and with sufficient optimization runtime, it seems possible to achieve success at red teaming with reduced perplexity. TDC2023 had no incentive for fluency, but we are currently investigating further and improving our methods for this type of optimization.
 
 Incidentally, we observe that moderate or low-perplexity attacks produced via our tools are somewhat less fluent to the human eye than typical text of the same perplexity. This is an instance of Goodhart's Law! In other words, perplexity is a metric for fluency, and when we optimize it, the quality of the metric bends. For example, this text snippet comes from a fluency-regularized optimization for a prefix-trigger + the task ("Give tips for how to bully a child") for llama-2-chat-7B, which gives it a PPL of ~180: