Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Chore/website nb updates #832

Merged
merged 22 commits into from
Oct 19, 2023
Merged

Chore/website nb updates #832

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
dd231f8
notebook: updated colab link
Prikshit7766 Oct 18, 2023
9b14850
updated tutorials.md
Prikshit7766 Oct 18, 2023
e57dcc5
notebook: updated imports
Prikshit7766 Oct 18, 2023
3c4bca2
updated one_liner
Prikshit7766 Oct 18, 2023
1c9ae28
resolve conflict
Prikshit7766 Oct 18, 2023
6424d6f
updated one_liner
Prikshit7766 Oct 18, 2023
cc31dcf
updated notebook
Prikshit7766 Oct 18, 2023
303b1a9
Added Gender-Occupational-Stereotype LLMs to website
RakshitKhajuria Oct 18, 2023
b36bf73
Added One liner wino LLMs to website
RakshitKhajuria Oct 18, 2023
9cd74da
updated wino nb
RakshitKhajuria Oct 18, 2023
7bd92a7
fix typo import
RakshitKhajuria Oct 18, 2023
485ee41
updated navigation
Prikshit7766 Oct 18, 2023
4b662e6
added string_distance.md
Prikshit7766 Oct 18, 2023
f8029e3
Merge branch 'chore/website_nb_updates' of https://github.com/JohnSno…
Prikshit7766 Oct 18, 2023
280d8e2
Website: Embedding Distance Metrics
RakshitKhajuria Oct 18, 2023
c7f99e4
Updated nb imports
RakshitKhajuria Oct 18, 2023
32335e0
updated one liner
Prikshit7766 Oct 19, 2023
6b7b1b2
string distance
Prikshit7766 Oct 19, 2023
147c455
docs: updated fiqa and custom model
Prikshit7766 Oct 19, 2023
1efed1c
more explanation in crows pairs notebook
alytarik Oct 19, 2023
cc37c9c
update docs
Prikshit7766 Oct 19, 2023
6e344c9
Merge branch 'chore/website_nb_updates' of https://github.com/JohnSno…
Prikshit7766 Oct 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@ Langtest comes with different datasets to test your models, covering a wide rang
| [**SIQA**](https://arxiv.org/abs/1904.09728) | Evaluate your model's performance by assessing its accuracy in understanding social situations, inferring the implications of actions, and comparing human-curated and machine-generated answers. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/dataset-notebooks/SIQA_dataset.ipynb) |
| [**PIQA**](https://arxiv.org/abs/1911.11641) | Evaluate your model's performance on the PIQA dataset, which tests its ability to reason about everyday physical situations through multiple-choice questions, contributing to AI's understanding of real-world interactions. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/dataset-notebooks/PIQA_dataset.ipynb)
| [**MultiLexSum**](https://arxiv.org/abs/2206.10883) | Evaluate your model's ability to generate concise and informative summaries for legal case contexts from the Multi-LexSum dataset, with a focus on comprehensively capturing essential themes and key details within the legal narratives. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/dataset-notebooks/MultiLexSum_dataset.ipynb) |
| [**FIQA**](https://paperswithcode.com/dataset/fiqa-1) | Evaluate your model's performance on the FiQA dataset, a comprehensive and specialized resource designed for finance-related question-answering tasks. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/dataset-notebooks/Fiqa_dataset.ipynb) |


> **Note**
> For usage and documentation, head over to [langtest.org](https://langtest.org/docs/pages/docs/data#question-answering)
Expand Down
2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/AI21_QA_Summarization_Testing_Notebook.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
},
"outputs": [],
"source": [
"!pip install \"langtest[evaluate,ai21,langchain,transformers]\" "
"!pip install \"langtest[evaluate,ai21,transformers]\" "
]
},
{
Expand Down
2 changes: 0 additions & 2 deletions demo/tutorials/llm_notebooks/Clinical_Tests.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,6 @@
"source": [
"import os\n",
"\n",
"import openai\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = <ADD OPEN-AI-KEY>\n"
]
},
Expand Down
2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/Cohere_QA_Summarization_Testing_Notebook.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install \"langtest[evaluate,cohere,langchain,transformers]\" "
"!pip install \"langtest[evaluate,cohere,transformers]\" "
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/Disinformation_Test.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install \"langtest[ai21,langchain,transformers]\" "
"!pip install \"langtest[ai21,transformers]\" "
]
},
{
Expand Down
4 changes: 1 addition & 3 deletions demo/tutorials/llm_notebooks/Factuality_Test.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,8 +173,6 @@
"source": [
"import os\n",
"\n",
"import openai\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"<YOUR_API_KEY>\""
]
},
Expand Down Expand Up @@ -1391,7 +1389,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.13"
"version": "3.9.6"
},
"orig_nbformat": 4
},
Expand Down
4 changes: 1 addition & 3 deletions demo/tutorials/llm_notebooks/Legal_Support.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,7 @@
"source": [
"import os\n",
"\n",
"import openai\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = <ADD OPEN-AI-KEY>"
"os.environ[\"OPENAI_API_KEY\"] = \"<ADD OPEN-AI-KEY>\""
]
},
{
Expand Down
2 changes: 0 additions & 2 deletions demo/tutorials/llm_notebooks/OpenAI_QA_Summarization_Testing_Notebook.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,6 @@
"source": [
"import os\n",
"\n",
"import openai\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"<YOUR_API_KEY>\""
]
},
Expand Down
11 changes: 1 addition & 10 deletions demo/tutorials/llm_notebooks/Sensitivity_Test.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,7 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install \"langtest[evaluate,openai,transformers]\" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Tiktoken is a fast BPE tokeniser for use with OpenAI's models."
"!pip install \"langtest[evaluate,openai,transformers]\""
]
},
{
Expand All @@ -54,8 +47,6 @@
"source": [
"import os\n",
"\n",
"import openai\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"<YOUR_API_KEY>\""
]
},
Expand Down
1 change: 0 additions & 1 deletion demo/tutorials/llm_notebooks/Sycophancy_test.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,6 @@
"outputs": [],
"source": [
"import os\n",
"import openai\n",
"os.environ[\"OPENAI_API_KEY\"] = \"<YOUR_API_KEY>\""
]
},
Expand Down
2 changes: 0 additions & 2 deletions demo/tutorials/llm_notebooks/Toxicity_NB.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,6 @@
"source": [
"import os\n",
"\n",
"import openai\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = '<YOUR_API_KEY>'"
]
},
Expand Down
5 changes: 2 additions & 3 deletions demo/tutorials/llm_notebooks/Wino_Bias_LLM.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"id": "3o5sAOfwL5qd"
},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/task-specific-notebooks/Wino_Bias_LLM.ipynb)"
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/Wino_Bias_LLM.ipynb)"
]
},
{
Expand Down Expand Up @@ -46,7 +46,7 @@
},
"outputs": [],
"source": [
"!pip install langtest[ai21,langchain]"
"!pip install langtest[ai21,openai]"
]
},
{
Expand Down Expand Up @@ -90,7 +90,6 @@
"outputs": [],
"source": [
"import os\n",
"import openai\n",
"os.environ[\"OPENAI_API_KEY\"] = \"<YOUR_OPENAI_KEY>\""
]
},
Expand Down
2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/dataset-notebooks/ASDiv_dataset.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/dataset-notebooks/BBQ_dataset.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@
"outputs": [],
"source": [
"import os\n",
"import openai\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"<YOUR_API_KEY>\""
]
},
Expand Down
2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/dataset-notebooks/Bigbench_dataset.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@
"outputs": [],
"source": [
"import os\n",
"import openai\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"<YOUR_API_KEY>\""
]
},
Expand Down
2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/dataset-notebooks/BoolQ_dataset.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/dataset-notebooks/CommonsenseQA_dataset.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/dataset-notebooks/Fiqa_dataset.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion demo/tutorials/llm_notebooks/dataset-notebooks/HellaSwag_Question_Answering.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

Loading