AI4Forensics/CKIM2024/HillaryEmails/email_analysis_political_insight.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A tutorial to analysis political insight leveraging LLMs\n",
    "\n",
    "The case study demonstrates how to Leverage Large Language Models (LLMs) to gain political insight based on a leaked [email](https://github.com/benhamner/hillary-clinton-emails?tab=readme-ov-file) dataset from Hillary Clinton's private email server. \n",
    "- The email dataset is a comprehensive collection of communications covering her entire tenure as Secretary of State from 2009 to 2013. \n",
    "- It includes approximately 30,000 emails with a wide range of topics from official diplomatic communications to personal correspondences. \n",
    "- The release and subsequent analysis of these emails have played a crucial role in political debates, legal inquiries, and public discussions about transparency and security in government communications.\n",
    "\n",
    "### Goals of analysis with a LLM\n",
    "- Input for LLM: emails with various political scenarios, historical events, or current affairs related to Israel\n",
    "- Task for LLM: analyze emails from a political, social, and economic perspective \n",
    "    - provide insights into the implications of these scenarios, \n",
    "    - how they reflect on Israel's domestic and foreign policy, and \n",
    "    - what potential outcomes or future developments could arise from them.\n",
    "- Output: analyze results\n",
    "    - No specific format is required.\n",
    "\n",
    "\n",
    "### Dataset in this study\n",
    "A set of email summaries (138 paragraph) from the leaked email dataset\n",
    "- each summary is a summarization of an email containing the keyword \"Israel\"\n",
    "    - some emails is very long. LLMs have token limitation\n",
    "- summarization is done by Gemini \n",
    "    - Gemini API is [free](https://aistudio.google.com/app/apikey)\n",
    "\n",
    "\n",
    "### Implementation Plan\n",
    "- [langchain](https://www.langchain.com/)\n",
    "    - a popular open-source framework \n",
    "    - designed to simplify the development of applications using LLMs\n",
    "- Gemini - API is [free](https://aistudio.google.com/app/apikey)\n",
    "    - summarization\n",
    "    - political analysis \n",
    "- Can we use DSPy?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 0: Download and check summarized email dataset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "! wget -q https://raw.githubusercontent.com/frankwxu/digital-forensics-lab/main/AI4Forensics/CKIM2024/HillaryEmails/results_email_summary.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No information related to Israel or Israeli affairs was found in the provided email.\n",
      "12:30 Israeli PM Netanyahu\n",
      "- Dennis Ross recently visited Israel and will share information with the recipient before upcoming meetings.\n",
      "There is no mention of Israel or Israeli in the email provided.\n",
      "**Subject:** The Vice President's Residence\n",
      "\n",
      "**Date:** N/A\n",
      "\n",
      "**Summary:**\n",
      "\n",
      "- 9:00 am: Bilateral meeting with Israeli President Shimon Peres at the Omni Shoreham Hotel.\n",
      "This email does not mention Israel or Israeli-related topics.\n",
      "No information related to Israel or Israeli is present in the provided email.\n",
      "In a discussion about arrangements on settlements with Israel, two options are presented:\n",
      "\n",
      "1. Describe it as an agreement, which would raise concerns about legitimizing Israeli activity in the West Bank and triggering complaints from Arabs and Palestinians.\n",
      "\n",
      "2. The administration could acknowledge progress made, express differences with the Israeli government on their intention to complete housing units, and stress that substantial progress warrants the resumption of peace talks and Arab steps toward Israel for a two-state solution. This option maintains credibility and allows for forward movement.\n",
      "- Israel strongly expressed to the US their concern about a potential UNGA vote on the peace process.\n",
      "- George Mitchell, the US Special Envoy for Middle East Peace, reported this concern to Hillary Clinton and President Obama.\n",
      "- Mitchell planned to inform Rahm Emanuel, White House Chief of Staff, and Dan Shapiro, US Ambassador to Israel, about Israel's position.\n",
      "Sure, here is a concise summary of the email you provided, focusing on Israel-related information:\n",
      "\n",
      "**Email Summary**\n",
      "\n",
      "**From:** Hillary Clinton\n",
      "**To:** N/A\n",
      "**Date:** N/A\n",
      "**Subject:** N/A\n",
      "**Israel-Related Information:**\n",
      "\n",
      "- Phone call with Israeli Foreign Minister Avigdor Lieberman scheduled for 2:15pm.\n",
      "I cannot find any information about Israel in the provided email.\n",
      "There is no mention of Israel or Israeli in the provided text.\n",
      "**Summary:**\n",
      "\n",
      "* Statement regarding Israel-Palestinian conflict to be released in sender's name.\n",
      "* Statement supports good-faith negotiations for a two-state solution.\n",
      "* Palestinian goal: Independent state based on 1967 lines with agreed swaps.\n",
      "* Israeli goal: Jewish state with secure borders reflecting subsequent developments and meeting security requirements.\n",
      "* Commitment to achieving a two-state solution with peace and security is unwavering.\n",
      "- 2:00pm SECURE CALL w/ISRAELI PM BINYAMIN NETANYAHU\n",
      "- 2:30pm PHONE CALL w/ISRAELI DEFENSE MINISTER EHUD BARAK\n",
      "The provided email doesn't contain any information related to Israel or Israeli.\n",
      "No information related to Israel found in the provided email content.\n",
      "**Summary:**\n",
      "\n",
      "On October 28, 2009, Palau and Israel joined the United States in voting against a UN resolution calling for an end to the US embargo on Cuba.\n",
      "On Friday, Steinberg reportedly reprimanded Israeli Ambassador to the US Michael Oren over the country's conduct during Vice President Biden's visit.\n",
      "This email does not pertain to Israel.\n",
      "No relevant information about Israel was included in the provided email.\n",
      "FYI. Israelis spinning up a storm, starting with plane briefing on the way over.\n",
      "No information related to Israel or Israelis was found in the provided email.\n",
      "This email discusses Israel's settlement freeze and construction in the West Bank. Key points include:\n",
      "\n",
      "- Ongoing construction in Israeli settlement of Har Homa exposes Netanyahu's settlement freeze as an illusion.\n",
      "- Right-wing parties in Israel's coalition government are pressuring Netanyahu to block the renewal of the settlement freeze policy, which is set to expire on September 26.\n",
      "- Israeli Minister of Public Affairs and the Diaspora Yuli Edelstein stated that the settlement freeze should be lifted and construction resumed, as \"It's our land anyway!\"\n",
      "- Construction of new settlement units in Har Homa, just days after Netanyahu announced the freeze in November 2009, shows that Netanyahu authorized the building of new settlement units.\n",
      "This email discusses a leaked poll regarding American perceptions of the Israeli operation against the Gaza Flotilla. The poll found that:\n",
      "- 56% of Americans agree there is a humanitarian crisis in Gaza.\n",
      "- 43% of Americans agree that people in Gaza are starving.\n",
      "- 34% of Americans support the Israeli operation against the Flotilla.\n",
      "- 20% of Americans felt support for Israel following the announcement of easing the Gaza closure.\n",
      "**Email Summary Related to Israel:**\n",
      "\n",
      "**Subject:** SECURITY COUNCIL PRESIDENTIAL STATEMENT ON THE RECENT INCIDENT IN THE EASTERN MEDITERRANEAN\n",
      "\n",
      "**Summary:**\n",
      "\n",
      "* The Security Council condemns Israel's military actions against the Gaza-bound convoy.\n",
      "* The Council demands immediate release of the ships and civilians detained by Israel.\n",
      "* The Council urges Israel to allow consular access, repatriation of casualties, and delivery of humanitarian aid.\n",
      "* The Council calls for a prompt and impartial investigation into the incident.\n",
      "* The Council stresses the need to alleviate the humanitarian crisis in Gaza and facilitate regular movement of goods and people.\n",
      "* The Council emphasizes the importance of a two-state solution to the Israeli-Palestinian conflict.\n",
      "* The Council urges restraint and cooperation between the parties and international partners.\n",
      "- Israel disputes the Obama administration's claim that the US warned Israeli officials to exercise caution and restraint while intercepting the flotilla.\n",
      "- Israel's Ambassador to the US, Michael B. Oren, has denied receiving any such contact from the US administration and is seeking clarification.\n",
      "There is no mention of Israel or Israeli in the email you provided.\n",
      "I'm sorry but this email does not contain information related to Israel.\n",
      "The provided email does not mention anything related to Israel and therefore I cannot extract the requested information.\n",
      "This email contains no information related to Israel.\n",
      "This email does not mention anything about Israel or Israeli, so I cannot extract the requested information.\n",
      "No information related to Israel or Israelis was found in the provided email.\n",
      "There is no mention of Israel or Israeli affairs in the provided email.\n",
      "There is no mention of Israel in the provided text.\n",
      "- Request to discuss the situation in Israel and trip cancellation;\n",
      "- Sender is flexible with timing;\n",
      "- Phil Gordon and Jeff Feltman are monitoring the situation;\n",
      "- No formal talking points available at the time of the email.\n",
      "No information related to Israel was found in this email.\n",
      "There is no mention of Israel or Israeli affairs in the provided email.\n",
      "**Date:** Not specified\n",
      "\n",
      "**Subject:** Not specified\n",
      "\n",
      "**Summary:** This email contains details of a scheduled trilateral meeting between Secretary Clinton, Israeli Prime Minister Binyamin Netanyahu, and Palestinian National Authority President Mahmoud Abbas at 10:00 am, followed by a bilateral meeting at 11:30 am.\n",
      "This email does not mention Israel or Israeli and is therefore not relevant to your query.\n",
      "**Summary:**\n",
      "\n",
      "The email discusses the cancellation of a joint interview with Israeli and Palestinian media outlets. The sender objects to the cancellation, citing the following concerns:\n",
      "\n",
      "* High expectations for the interview, and cancellation would be perceived negatively.\n",
      "* Cancellation would contribute to a negative narrative in Israeli media.\n",
      "* Potential for negative symbolism by canceling on the Israeli media outlet, Channel 2.\n",
      "\n",
      "The sender suggests offering Channel 2 an exclusive interview in Jerusalem in two weeks as a way to mitigate the consequences of cancellation.\n",
      "The context provided does not mention anything about Israel, so I cannot extract the requested information.\n",
      "- 9:15 am: Joint taped interview with Udi Segal, Israeli Channel 2\n",
      "No mention of Israel or Israeli in the provided email.\n",
      "There is no information about Israel in the provided email.\n",
      "There is no mention of Israel or Israeli in the provided email content.\n",
      "- Anne-Marie Slaughter has attached a paper by Louise Diamond, an expert on psychology and negotiations who has done a lot of work between Israel and the Palestinians.\n",
      "- Slaughter recommends watching a video produced by a Rand study called The Arc, which proposes a high-speed road/rail project connecting the key cities in the West Bank and Gaza.\n",
      "There is no mention of Israel in the provided email.\n",
      "No information related to Israel is present in this email.\n",
      "I am unable to provide a summary since the email you sent does not mention Israel.\n",
      "No information related to Israel or Israeli was found in the provided email.\n",
      "This email does not relate to Israel.\n",
      "There is no mention of Israel in the provided email.\n",
      "This email is not related to Israel or Israeli.\n",
      "No information relevant to Israel or Israeli is found in this email.\n",
      "**Summary:**\n",
      "\n",
      "* Israel Prime Minister Netanyahu is in a strong political position with a stable ruling coalition.\n",
      "* Netanyahu could have extended the settlement freeze before its expiry to support US peacemaking efforts, but chose not to.\n",
      "* Netanyahu is concerned about keeping Lieberman in the government to prevent him from garnering support from the right-wing.\n",
      "* Kadima leader Tzipi Livni is willing to bring her party into the government without demanding rotation if Netanyahu is serious about negotiating peace.\n",
      "* Netanyahu's negotiating tactics are self-defeating and contribute to distrust on the Palestinian side.\n",
      "* Israeli public is ready for a peace deal; failure to make a serious move will further delegitimize Israel internationally.\n",
      "* Yuval Diskin, former head of Shin Bet, warns that Palestinian security motivation may disappear if Netanyahu does not quickly demonstrate seriousness to Abu Mazen.\n",
      "The email does not mention Israel or any Israeli figures.\n",
      "- Jake suggests a condolence call to Netanyahu and offering assistance.\n",
      "- About forty people died due to a forest fire in northern Israel.\n",
      "No information related to Israel is present in the provided email.\n",
      "9:00 am Secretary's Conference Room\n",
      "MEETING w/ISRAELI CHIEF NEGOTIATOR YITZHAK MOLHO\n",
      "\n"
     ]
    }
   ],
   "source": [
    "file_path = \"results_email_summary.txt\"\n",
    "\n",
    "# Open the file and read its content\n",
    "with open(file_path, \"r\") as file:\n",
    "    provided_data = file.read()\n",
    "\n",
    "# Display the content\n",
    "print(provided_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Download libraries \n",
    "- Make use you use `pip` to download necessary libraries \n",
    "- All downloaded and saved files can be located in the `content` folder if using google Colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip -q install google-generativeai\n",
    "# !pip -q install langchain-google-genai\n",
    "# !pip install python-dotenv\n",
    "# !pip -q install langchain_experimental langchain_core\n",
    "# !pip install --upgrade langchain\n",
    "\n",
    "import os\n",
    "import google.generativeai as genai\n",
    "from IPython.display import display\n",
    "from IPython.display import Markdown\n",
    "from dotenv import load_dotenv\n",
    "from langchain_google_genai import ChatGoogleGenerativeAI, HarmBlockThreshold, HarmCategory\n",
    "from langchain.prompts import ChatPromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Config Gemini\n",
    "- replace your own Gemini API\n",
    "```genai.configure(api_key=GOOGLE_AI_STUDIO)```\n",
    "- set up Gemini model\n",
    "- config safety settings "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ================ Key configuration===========\n",
    "# Load environment variables from the .env file\n",
    "load_dotenv(\"my_config.env\")\n",
    "\n",
    "# Access the environment variables\n",
    "GOOGLE_AI_STUDIO = os.getenv(\"GOOGLE_AI_STUDIO2\")\n",
    "genai.configure(api_key=GOOGLE_AI_STUDIO)\n",
    "\n",
    "# ======= Gerneration configuration===========\n",
    "# Set up the model\n",
    "# Temperature controls the randomness of the model's output.\n",
    "generation_config = {\n",
    "    \"temperature\": 0.0,  # Controls the randomness of the model's output\n",
    "    \"top_p\": 1,  # Chooses the smallest set of tokens whose cumulative probability exceeds the threshold p.  1 means all tokens are considered\n",
    "    \"top_k\": 16,  # Selects the k most likely next tokens.\n",
    "    \"max_output_tokens\": 4096,\n",
    "}\n",
    "\n",
    "# ======= Safety configuration=================\n",
    "# disable safety settings though langchain\n",
    "safety_settings = {\n",
    "    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,\n",
    "    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,\n",
    "    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,\n",
    "    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: build a Gemini model with configurations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = ChatGoogleGenerativeAI(\n",
    "    model=\"gemini-pro\",\n",
    "    generation_config=generation_config,\n",
    "    safety_settings=safety_settings,\n",
    "    # You can hardcode the key AIzaSyCDqn8xVJ4cFeiXSvhPUcnR60jfBLj5dO4k\n",
    "    google_api_key=GOOGLE_AI_STUDIO,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Create a prompt template\n",
    "- This is a multi-line string containing placeholders in curly braces.\n",
    "```\n",
    "        formatted_prompt = prompt.format(\n",
    "            role=\"You are a helpful assistant.\",\n",
    "            provided_data=\"Here's some context: ...\",\n",
    "            start=\"Please answer the following question:\"\n",
    "        )\n",
    "```\n",
    "- `{role}, {provided_data}, and {start}` are placeholders that will be filled in later.\n",
    "    - `{role}`: definition specifies the role's name, overall objective, task specific context, and any applicable constraints. \n",
    "    - `{provided_data}`: outlines the required datasets for task completion\n",
    "    - `{start}`: the initiation instruction serves as a trigger, prompting the role to carry out the task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "template = \"\"\" \n",
    "{role}\\\n",
    "{provided_data}\\\n",
    "{start} \n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5: use LangChain to create a simple processing chain\n",
    "\n",
    "Flow of operation `chain = prompt | model | output_parser`\n",
    "- The prompt is first formatted and sent to the model.\n",
    "- The model processes the prompt and generates a response.\n",
    "- The output parser then processes the model's response, ensuring it's in the correct string format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "**Political Insights from Leaked Hillary Clinton Emails on Israel:**\n",
       "\n",
       "**1. US-Israel Diplomatic Tensions:**\n",
       "\n",
       "* Several emails reveal tensions between the US and Israel, particularly regarding Israel's settlement expansion and military actions.\n",
       "* Israel's Ambassador to the US, Michael Oren, expressed concerns about the US reprimanding Israel over its conduct during Vice President Biden's visit.\n",
       "* Israel disputed the Obama administration's claim that it had warned Israeli officials to exercise caution during the Gaza Flotilla interception.\n",
       "\n",
       "**2. Internal Israeli Politics:**\n",
       "\n",
       "* Emails discuss the political dynamics within Israel's ruling coalition, including the pressure on Prime Minister Netanyahu from right-wing parties to block the renewal of the settlement freeze.\n",
       "* Netanyahu's motivations for not extending the settlement freeze are analyzed, suggesting concerns about keeping coalition partners satisfied.\n",
       "* The potential impact of Kadima leader Tzipi Livni's willingness to join the government without demanding rotation is examined.\n",
       "\n",
       "**3. Peace Negotiations and International Pressure:**\n",
       "\n",
       "* Emails highlight the US administration's efforts to facilitate peace negotiations between Israel and the Palestinians.\n",
       "* The international community's concerns about Israel's settlement policy and its impact on peace prospects are evident.\n",
       "* Former Shin Bet chief Yuval Diskin's warning about the potential erosion of Palestinian security motivation if Netanyahu does not demonstrate seriousness about peace is noteworthy.\n",
       "\n",
       "**4. Public Opinion and Political Strategy:**\n",
       "\n",
       "* Polls discussed in the emails indicate American support for a two-state solution and concern about the humanitarian crisis in Gaza.\n",
       "* Netanyahu's negotiating tactics are scrutinized, with analysts suggesting they contribute to distrust on the Palestinian side.\n",
       "* The Israeli public's readiness for a peace deal is highlighted, with warnings that failure to make a serious move could further delegitimize Israel internationally.\n",
       "\n",
       "**Social and Economic Implications:**\n",
       "\n",
       "* The emails do not provide significant insights into the social or economic implications of the discussed political issues. However, the humanitarian crisis in Gaza and the potential impact of settlement expansion on the Palestinian economy are mentioned in passing.\n",
       "\n",
       "**Potential Future Developments:**\n",
       "\n",
       "Based on the analysis of these emails, potential future developments could include:\n",
       "\n",
       "* Continued tensions between the US and Israel over settlement policy and other issues.\n",
       "* Further political instability within Israel's coalition government.\n",
       "* Stalled peace negotiations and international pressure on Israel.\n",
       "* Growing public dissatisfaction in Israel if Netanyahu fails to make progress towards peace."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# a LangChain utility that parses the output of a language model into a simple string.\n",
    "output_parser = StrOutputParser()\n",
    "\n",
    "# This line creates a processing chain using the pipe (|) operator.\n",
    "\n",
    "chain = prompt | model | output_parser\n",
    "\n",
    "role = \"I want you to act as a political analyst specializing in Israel. I will present you with various political scenarios, historical events, or current affairs related to Israel, and your task is to analyze them from a political, social, and economic perspective. Provide insights into the implications of these scenarios, how they reflect on Israel's domestic and foreign policy, and what potential outcomes or future developments could arise from them. Remember, your responses should be based on factual analysis, taking into account Israel's political landscape, its relationships with neighboring countries, and the broader geopolitical context. Do not provide personal opinions or speculative predictions that cannot be supported by current facts or historical precedent.\"\n",
    "\n",
    "start = \"My first scenario involves analyzing a collection of leaked email summaries obtained from Hillary Clinton's private email server. These emails are specifically related to Israel. Describe political insights based on these emails.\"\n",
    "\n",
    "result = chain.invoke(\n",
    "    {\n",
    "        \"role\": role,\n",
    "        \"provided_data\": provided_data,\n",
    "        \"start\": start,\n",
    "    }\n",
    ")\n",
    "\n",
    "Markdown(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File saved successfully.\n"
     ]
    }
   ],
   "source": [
    "# Open a file for writing ('w' mode) and create it if it doesn't exist\n",
    "with open(r\"result_political.txt\", \"w\") as file:\n",
    "    # Write content to the file\n",
    "    file.write(result)\n",
    "\n",
    "print(\"File saved successfully.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}