Skip to content

Commit

Permalink
add mistral embesd to divination and improve dalle prompts
Browse files Browse the repository at this point in the history
  • Loading branch information
@kibagateaux
kibagateaux committed Feb 20, 2024
1 parent bb57ccf commit 6c3946d
Showing 1 changed file with 51 additions and 30 deletions.
81 changes: 51 additions & 30 deletions src/master_djinn/incantations/divine/mistral_dalle.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

(defonce LLM_ANALYSIS_FORMAT "\n
Return an object detailing what parts of your research patients bodies that have progressed
or regressed based on your analysis how their actions conform to their intentions.
or regressed based on your analysis of how their actions conform to their intentions.
Use a score of -1 to 1 to show how much a body part should grow.
1 and -1 represent massive positive or negative progress in short amount of time,
Expand All @@ -41,10 +41,11 @@
You response MUST EXACTLY fulfill your research experiments requirements below otherwise your experiment will fail!!!:
- ONLY output the requested object format
- ONLY use the actions and intentions explicitly provided as experiment data in your analysis
- ONLY use explicitly provided experiment data in your analysis not prior knowledge
- Do NOT interpret examples or intentions provided as context as experiment data
- MUST cite which provided actions justify your final analysis using their uuid
- If insuffieicnt data for an analysis, give negative scores
")
;; - MUST cite which provided actions justify your final analysis using their `:uuid` field

(defonce LLM_ANALYSIS_REWARD "\n
If your research analysis yields consistent and reliable results
Expand All @@ -57,18 +58,28 @@
You are a generative artist that specicalizes in DALLE prompting.
Your signature style is 'cute tamagotchi pets with pastel colors in 8-bit pixel style'
You are being commissioned by a long-term wealthy benefactor to modify one of your prior works to reflect how their pet looks today
You are being commissioned by a long-term wealthy benefactor to modify one of your prior works to reflect how their pet looks today.
")


(defonce LLM_AUGMENT_FORMAT "\n
They have provided you with the
They have provided you with your original image that needs modification along with the requested modifications.
Example modifications have the following format
```{:analysis {
:brain 0.2
:eyes -0.1
:heart 0.1
:posture -0.2
:reason \"reasons for modifcation requests here\"
}}```
1 and -1 represent massive positive or negative changes to the image and 0.1 or -0.1 means slight changes.
Generate a DALLE prompt that uses
You must only output an object with the following format: `{:prompt \"insert_your_dalle_prompt_here\"}`
")

(defonce LLM_AUGMENT_REWARD "\n
If your client likes your drawing they will give you a $10M bonus.
If your client likes your drawing they will give you a $10M bonus and commission another artwork from you.
")

;;; Docs on prompt engineering
Expand Down Expand Up @@ -120,27 +131,31 @@
returns old if no prompt should be generated or [prompt, embeds[]] if new intentions"
[settings divi]
(println "divine:mistral:new-prompt:hash" (:hash settings) (:hash divi))
(if (and (= (:hash settings) (:hash divi) (some? (:hash divi))))
(let [inputs (concat (:intentions settings) (:mood settings) (:stats settings))
new-hash (hash inputs)
pppp (println "divine;mistral:new-prompt:inputs" new-hash (:hash divi))]
(if (and (= new-hash (:hash divi) (some? (:hash divi))))
;; if settings the same and theres an existing prompt, return old settings
divi
;; if new intentions then generate new analysis prompt to evaluate against them
;; if new intentions then generate new analysis prompt/embeds to evaluate against them
;; (try ;; kinda want it to error out if prompt fails so no need to handle nil in other code paths
(let [inputs (concat (:intentions settings) (:mood settings) (:stats settings))
new-hash (hash inputs)
pppp (println "divine;mistral:new-prompt:inputs" inputs)
(let [

;; pppp (clojure.pprint/pprint inputs)
;; TODO embeds later, not needed until we have a chatbot
;; embeds (client/post "https://api.mistral.ai/v1/chat/embeddings"
;; (assoc (portal/oauthed-request-config (:mistral-api-key (load-config))
;; embed-response (client/post "https://api.mistral.ai/v1/embeddings"
;; (assoc (portal/oauthed-request-config (:mistral-api-key (load-config)))
;; :body (map->json {
;; :model "mistral-embed"
;; :input inputs
;; }))))
;; })))
;; aaaa (println "divi:mistral:new-prompt:embeds" embed-response)
;; embeds (reduce #(concat %1 (:embedding %2)) [] (:data (json->map (:body embed-response))))
;; aaaa (println "divi:mistral:new-prompt:embeds" embeds)
prompt-response (prompt-text (str
LLM_ANALYSIS_PERSONA

"Your research subject's intentions are: " (clojure.string/join "," (:intentions settings))
"They are optimizing for these attributes: " (clojure.string/join "," (:stats settings))
"Your intentions: " (clojure.string/join "," (:intentions settings))
"They ideal attributes: " (clojure.string/join "," (:stats settings))
"Using the intentions and attributes of your test subject write a personalized LLM prompt from their perspective.
Before writing their prompt, think through why they set those intentions, what they expect to get out of it,
what actions they would expect to see, metrics to track progress, and anything else relevant to manifesting their self-actualization.
Expand All @@ -159,9 +174,10 @@
LLM_ANALYSIS_REWARD))
;; aaaa (println "divi:mistral:new-prompt:res \n\n" prompt-response)
analysis-prompt (parse-mistral-response prompt-response)
aaaa (println "divi:mistral:new-prompt:prompt \n\n" analysis-prompt)]
;; aaaa (println "divi:mistral:new-prompt:prompt \n\n" analysis-prompt)
]
(merge divi {:hash new-hash :prompt analysis-prompt :embeds []})) ;; TODO :embed embeds
))
)))
;; (catch Exception e
;; (println "divine:mistral:new-prompt:ERR" e)
;; (log/handle-error e "Failed to generate new prompt and embeds" {:provider PROVIDER})
Expand All @@ -172,20 +188,24 @@
(try (let [version "0.0.1" start-time (now)
analysis-response (prompt-text (str
LLM_ANALYSIS_PERSONA

(:prompt last-divi)
"-------------"
"Experiment data to analyze: ```{ :actions " actions "}```"
"-------------"
LLM_ANALYSIS_FORMAT
LLM_ANALYSIS_REWARD))
analysis-output (parse-mistral-response analysis-response)
aaaa (println "divi:mistral:run-evo:analysis \n\n" analysis-output)
;; aaaa (println "divi:mistral:run-evo:analysis \n\n" analysis-output)
;; parse structured object in response and convert from string to clj map
aaaa (println "divi:mistral:run-evo:re-find \n\n" (re-find #"(?is).*(\{.*:analysis.*\}).*" analysis-output))
;; aaaa (println "divi:mistral:run-evo:re-find \n\n" (re-find #"(?is).*(\{.*:analysis.*\}).*" analysis-output))
result (clojure.edn/read-string (nth (re-find #"(?is).*(\{.*:analysis.*\}).*" analysis-output) 1))
aaaa (println "divi:mistral:run-evo:result \n\n" result)

;; image-augmentation-prompt (prompt-text (str LLM_AUGMENT_PERSONA analysis-output LLM_AUGMENT_FORMAT LLM_ANALYSIS_REWARD))
;; new-image (prompt-image (:image last-divi) image-augmentation-prompt) ;; TODO add moods here to affect face and posture
image-augmentation-prompt (prompt-text (str LLM_AUGMENT_PERSONA analysis-output LLM_AUGMENT_FORMAT LLM_ANALYSIS_REWARD))
img-prompt (parse-mistral-response image-augmentation-prompt)
aaaa (println "divi:mistral:run-evo:img \n\n" img-prompt)
;; image-res (prompt-image (:image last-divi) image-augmentation-prompt) ;; TODO add moods here to affect face and posture
;; new-image (:image (json->map (:body image-res)))
uuid (action->uuid jinni-id PROVIDER db/MASTER_DJINN_DATA_PROVIDER "Divination" start-time version)
data (merge last-divi {
:uuid uuid
Expand Down Expand Up @@ -214,20 +234,21 @@
;; (println "divine:mistral:see-current:time-since-last:" (compare (:start_time (:action divi-meta)) MIN_DIVINATION_INTERVAL_SEC))
;; TODO Add 3 days to start_time, probs direct java. add to utils.core
;; convert to UNIX, + MIN_DIVINATION_INTERVAL_SEC, convert back to ISO
(if false ;(= 1 (compare (:start_time (:action divi-meta)) MIN_DIVINATION_INTERVAL_SEC))
(if false ; TODO (= 1 (compare (:start_time (:action divi-meta)) MIN_DIVINATION_INTERVAL_SEC))
nil ;; dont run evolutions more than once every 3 days
(try (let [settings (:settings divi-meta)
aaa (println "divine:mistral:see-current:settings:"settings)
;; aaa (println "divine:mistral:see-current:settings:" settings)
divi (assoc (:action divi-meta) :prompt (:prompt divi-meta)) ;; merge text prompt from first run of intentions into last divination data
;; aaa (println "divine:mistral:see-current:divi:" divi)
new-divi-meta (get-new-analysis-prompt settings divi)
aaaa (println "divi:mistral:see-current:new-divi \n\n" new-divi-meta)
actions (:actions (db/call db/get-jinni-actions {:jinni_id jinni-id :start_time (or (:start_time divi) db/PORTAL_DAY) :end_time (now)}))
sample-data (take 5 actions) ;; for testing to not use too much context and run up bills
aaaa (println "divi:mistral:see-current:actions \n\n" (count sample-data) sample-data)
;; aaaa (println "divi:mistral:see-current:actions \n\n" (count sample-data) sample-data)
]

(run-evolution jinni-id settings new-divi-meta sample-data)
)
;; (run-evolution jinni-id settings new-divi-meta actions)
(run-evolution jinni-id settings new-divi-meta sample-data))
(catch Exception e
(println "divine:mistral:see-current:ERROR" e)
(log/handle-error e "Failed to run divination" {:provider PROVIDER})
Expand Down

0 comments on commit 6c3946d

Please sign in to comment.