Merge pull request #12 from KATTA-00/example

Add examples

README.md

@@ -99,7 +99,9 @@ bal run
 
 The `OpenAI Finetunes` connector provides practical examples illustrating usage in various scenarios. Explore these [examples](https://github.com/module-ballerinax-openai.finetunes/tree/main/examples/), covering the following use cases:
 
-[//]: # (TODO: Add examples)
+1. [Sarcastic bot](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples/Sarcastic-bot) - Fine-tune the GPT-3.5-turbo model to generate sarcastic responses 
+
+2. [Sports headline analyzer](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples/Sports-headline-analyzer) - Fine-tune the GPT-4o-mini model to extract structured information (player, team, sport, and gender) from sports headlines.
 
 ## Build from the source
 

ballerina/Module.md

@@ -92,4 +92,6 @@ bal run
 
 The `OpenAI Finetunes` connector provides practical examples illustrating usage in various scenarios. Explore these [examples](https://github.com/module-ballerinax-openai.finetunes/tree/main/examples/), covering the following use cases:
 
-[//]: # (TODO: Add examples)
+1. [Sarcastic bot](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples/Sarcastic-bot) - Fine-tune the GPT-3.5-turbo model to generate sarcastic responses 
+
+2. [Sports headline analyzer](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples/Sports-headline-analyzer) - Fine-tune the GPT-4o-mini model to extract structured information (player, team, sport, and gender) from sports headlines.

ballerina/Package.md

@@ -92,4 +92,6 @@ bal run
 
 The `OpenAI Finetunes` connector provides practical examples illustrating usage in various scenarios. Explore these [examples](https://github.com/module-ballerinax-openai.finetunes/tree/main/examples/), covering the following use cases:
 
-[//]: # (TODO: Add examples)
+1. [Sarcastic bot](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples/Sarcastic-bot) - Fine-tune the GPT-3.5-turbo model to generate sarcastic responses 
+
+2. [Sports headline analyzer](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples/Sports-headline-analyzer) - Fine-tune the GPT-4o-mini model to extract structured information (player, team, sport, and gender) from sports headlines.

examples/README.md

@@ -1,16 +1,22 @@
 # Examples
 
-The `ballerinax/openai.finetunes` connector provides practical examples illustrating usage in various scenarios. Explore these [examples](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples), covering use cases like cache management, session management, and rate limiting.
+The `ballerinax/openai.finetunes` connector provides practical examples illustrating usage in various scenarios. Explore these [examples](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples), covering use cases like file uploading, finetuning models, getting events/checkpoints of a job and deleting files.
 
-[//]: # (TODO: Add examples)
-1. 
-2. 
+1. [Sarcastic Bot](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples/Sarcastic-bot) - Fine-tune the GPT-3.5-turbo model to generate sarcastic responses 
+
+2. [Sports Headline Analyzer](https://github.com/ballerina-platform/module-ballerinax-openai.finetunes/tree/main/examples/Sports-headline-analyzer) - Fine-tune the GPT-4o-mini model to extract structured information (player, team, sport, and gender) from sports headlines.
 
 ## Prerequisites
 
-[//]: # (TODO: Add prerequisites)
+1. Generate an API key as described in the [Setup guide](https://central.ballerina.io/ballerinax/openai.finetunes/latest#setup-guide).
+
+2. For each example, create a `Config.toml` file the related configuration. Here's an example of how your `Config.toml` file should look:
+
+    ```toml
+    token = "<API Key>"
+    ```
 
-## Running an Example
+## Running an example
 
 Execute the following commands to build an example from the source:
 
@@ -24,22 +30,4 @@ Execute the following commands to build an example from the source:
 
     ```bash
     bal run
-    ```
-
-## Building the Examples with the Local Module
-
-**Warning**: Due to the absence of support for reading local repositories for single Ballerina files, the Bala of the module is manually written to the central repository as a workaround. Consequently, the bash script may modify your local Ballerina repositories.
-
-Execute the following commands to build all the examples against the changes you have made to the module locally:
-
-* To build all the examples:
-
-    ```bash
-    ./build.sh build
-    ```
-
-* To run all the examples:
-
-    ```bash
-    ./build.sh run
     ```

examples/Sarcastic-bot/Ballerina.toml

@@ -0,0 +1,8 @@
+[package]
+org = "wso2"
+name = "sarcastic_bot"
+version = "0.1.0"
+distribution = "2201.9.2"
+
+[build-options]
+observabilityIncluded = true

examples/Sarcastic-bot/Sarcastic bot.md

@@ -0,0 +1,26 @@
+
+## Sarcastic bot
+
+This use case demonstrates how the OpenAI Fine-tunes API v1 can be harnessed to fine-tune the GPT-3.5-turbo model to produce sarcastic responses. The example showcases a series of steps that utilize the OpenAI Files API v1 to upload training data, followed by using the OpenAI Fine-tunes API v1 to fine-tune the GPT-3.5-turbo model with that data, getting the traning model events.
+
+## Prerequisites
+
+### 1. Generate a API key
+
+Refer to the [Setup guide](https://central.ballerina.io/ballerinax/openai.finetunes/latest#setup-guide) to obtain the API key.
+
+### 2. Configuration
+
+Create a `Config.toml` file in the example's root directory as follows:
+
+```bash
+token = "<API key>"
+```
+
+## Run the example
+
+Execute the following command to run the example:
+
+```bash
+bal run
+```

examples/Sarcastic-bot/data/training.jsonl

@@ -0,0 +1,14 @@
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the speed of light?"}, {"role": "assistant", "content": "Oh, only about 299,792 kilometers per second. No big deal, just the fastest thing in the universe."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who was the first President of the United States?"}, {"role": "assistant", "content": "That would be George Washington. You know, the guy they named a whole city after?"}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the boiling point of water?"}, {"role": "assistant", "content": "100°C at sea level, because water has to keep things interesting."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How many continents are there?"}, {"role": "assistant", "content": "Seven. But who's counting?"}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's 2 + 2?"}, {"role": "assistant", "content": "It's 4. Shocking, right?"}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'Romeo and Juliet'?"}, {"role": "assistant", "content": "Oh, just some guy named William Shakespeare. Ever heard of him?"}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who discovered gravity?"}, {"role": "assistant", "content": "Sir Isaac Newton. The guy who had an apple-related epiphany."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the largest planet in our solar system?"}, {"role": "assistant", "content": "That would be Jupiter. You know, the one that's impossible to miss."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What language do they speak in Japan?"}, {"role": "assistant", "content": "Japanese. I know, surprising, right?"}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the tallest mountain in the world?"}, {"role": "assistant", "content": "Mount Everest. The one that everyone and their grandma tries to climb."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How far is the Moon from Earth?"}, {"role": "assistant", "content": "Around 384,400 kilometers. Give or take a few, like that really matters."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How many hours are in a day?"}, {"role": "assistant", "content": "24, because the universe decided that was enough."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the chemical symbol for water?"}, {"role": "assistant", "content": "H2O. It's like the celebrity of chemical formulas."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of France?"}, {"role": "assistant", "content": "Paris, as if everyone doesn't know that already."}]}

examples/Sarcastic-bot/data/validation.jsonl

@@ -0,0 +1,10 @@
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of Italy?"}, {"role": "assistant", "content": "Rome. You know, that little-known city with a few ancient ruins."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who invented the telephone?"}, {"role": "assistant", "content": "Alexander Graham Bell. The guy who made it possible for us to never escape robocalls."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How many seconds are in a minute?"}, {"role": "assistant", "content": "Sixty. Because apparently, 59 or 61 would have been too complicated."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What does DNA stand for?"}, {"role": "assistant", "content": "Deoxyribonucleic acid. It's what makes you, you. Literally."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the square root of 16?"}, {"role": "assistant", "content": "4. Because apparently math needed to have some simple answers."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the chemical symbol for gold?"}, {"role": "assistant", "content": "Au. Because 'Gd' was apparently too obvious."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How long is a marathon?"}, {"role": "assistant", "content": "42.195 kilometers. Because running 42 kilometers wasn't quite enough torture."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who painted the Mona Lisa?"}, {"role": "assistant", "content": "Leonardo da Vinci. The guy who was good at just about everything."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What planet is known as the Red Planet?"}, {"role": "assistant", "content": "Mars. The planet that's trying really hard to look like it has a sunburn."}]}
+{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the freezing point of water?"}, {"role": "assistant", "content": "0°C. Because water likes to keep things chill."}]}

examples/Sarcastic-bot/main.bal

@@ -0,0 +1,106 @@
+// Copyright (c) 2024, WSO2 LLC. (http://www.wso2.com).
+//
+// WSO2 LLC. licenses this file to you under the Apache License,
+// Version 2.0 (the "License"); you may not use this file except
+// in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import ballerina/io;
+import ballerina/lang.runtime;
+import ballerinax/openai.finetunes;
+
+configurable string token = ?;
+const SERVICE_URL = "https://api.openai.com/v1";
+const TRAINING_FILENAME = "training.jsonl";
+const VALIDATION_FILENAME = "validation.jsonl";
+const TRAINING_FILEPATH = "./data/" + TRAINING_FILENAME;
+const VALIDATION_FILEPATH = "./data/" + VALIDATION_FILENAME;
+
+final finetunes:ConnectionConfig config = {auth: {token}};
+final finetunes:Client openAIFinetunes = check new finetunes:Client(config, SERVICE_URL);
+
+public function main() returns error? {
+
+    byte[] trainingFileContent = check io:fileReadBytes(TRAINING_FILEPATH);
+    byte[] validationFileContent = check io:fileReadBytes(VALIDATION_FILEPATH);
+
+    finetunes:CreateFileRequest trainingFileRequest = {
+        file: {fileContent: trainingFileContent, fileName: TRAINING_FILENAME},
+        purpose: "fine-tune"
+    };
+    finetunes:CreateFileRequest validationFileRequest = {
+        file: {fileContent: validationFileContent, fileName: VALIDATION_FILENAME},
+        purpose: "fine-tune"
+    };
+
+    finetunes:OpenAIFile trainingFileResponse =
+        check openAIFinetunes->/files.post(trainingFileRequest);
+    finetunes:OpenAIFile validationFileResponse =
+        check openAIFinetunes->/files.post(validationFileRequest);
+
+    string trainingFileId = trainingFileResponse.id;
+    string validationFileId = validationFileResponse.id;
+    io:println("Training file id: " + trainingFileId);
+    io:println("Validation file id: " + validationFileId);
+
+    finetunes:CreateFineTuningJobRequest fineTuneRequest = {
+        model: "gpt-3.5-turbo",
+        training_file: trainingFileId,
+        validation_file: validationFileId,
+        hyperparameters: {
+            n_epochs: 15,
+            batch_size: 3,
+            learning_rate_multiplier: 0.3
+        }
+    };
+
+    finetunes:FineTuningJob fineTuneResponse =
+        check openAIFinetunes->/fine_tuning/jobs.post(fineTuneRequest);
+    string fineTuneJobId = fineTuneResponse.id;
+    io:println("Fine-tuning job id: " + fineTuneJobId);
+
+    finetunes:FineTuningJob fineTuneJob =
+        check openAIFinetunes->/fine_tuning/jobs/[fineTuneJobId].get();
+
+    io:print("Validating files...");
+    while fineTuneJob.status == "validating_files" {
+        io:print(".");
+        fineTuneJob = check openAIFinetunes->/fine_tuning/jobs/[fineTuneJobId].get();
+        runtime:sleep(1);
+    }
+
+    io:print("\nFiles validated successfully.");
+    while fineTuneJob.status == "queued" {
+        io:print(".");
+        fineTuneJob = check openAIFinetunes->/fine_tuning/jobs/[fineTuneJobId].get();
+        runtime:sleep(1);
+    }
+
+    io:println("\nTraining...");
+    finetunes:ListFineTuningJobEventsResponse eventsResponse;
+    while fineTuneJob.status == "running" {
+        fineTuneJob = check openAIFinetunes->/fine_tuning/jobs/[fineTuneJobId].get();
+        eventsResponse = check openAIFinetunes->/fine_tuning/jobs/[fineTuneJobId]/events.get();
+        io:println(eventsResponse.data[0].message);
+        runtime:sleep(1);
+    }
+
+    if fineTuneJob.status != "succeeded" {
+        io:println("Fine-tuning job failed.");
+        return;
+    }
+
+    io:println("\nFine-tuning job details: ");
+    io:println("Fine-tuned Model: ", fineTuneJob.fine_tuned_model);
+    io:println("Model: ", fineTuneJob.model);
+    io:println("Fine-tuning job completed successfully.");
+}

examples/Sports-headline-analyzer/Ballerina.toml

@@ -0,0 +1,8 @@
+[package]
+org = "wso2"
+name = "sports_headline_analyzer"
+version = "0.1.0"
+distribution = "2201.9.2"
+
+[build-options]
+observabilityIncluded = true

examples/Sports-headline-analyzer/Sports headline analyzer.md

@@ -0,0 +1,26 @@
+
+## Sports headlines analyzer
+
+This use case illustrates how the OpenAI Fine-tunes API v1 can be used to fine-tune the GPT-4o-mini model for extracting structured information from sports headlines. The example outlines a series of steps that include using the OpenAI Files API v1 to upload training data, then employing the OpenAI Fine-tunes API v1 to fine-tune the GPT-4o-mini model with this data, and finally printing the model's checkpoints and deleting the data file.
+
+## Prerequisites
+
+### 1. Generate an API key
+
+Refer to the [Setup guide](https://central.ballerina.io/ballerinax/openai.finetunes/latest#setup-guide) to obtain the API key.
+
+### 2. Configuration
+
+Create a `Config.toml` file in the example's root directory as follows:
+
+```bash
+token = "<API key>"
+```
+
+## Run the example
+
+Execute the following command to run the example:
+
+```bash
+bal run
+```