Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENG_1431_inference_example_in_the_ft-notebook_is_applying_templating_twice #4

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

ENG_1431_inference_example_in_the_ft-notebook_is_applying_templating_twice #4

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
599ccdc
prompt example for inference
Nov 14, 2024
4542e1e
only notebook
Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions cmd_gen_finetuning.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,15 @@
"Note that the inputs passed to model are in the [TRL convertsational format](https://huggingface.co/docs/trl/en/sft_trainer#dataset-format-support) as the Hugging Face [chat template requires them to be](https://huggingface.co/docs/transformers/main/en/chat_templating#how-do-i-use-chat-templates). During training TRL will [automatically convert the instruction format to the conversational format](https://github.com/huggingface/trl/blob/main/trl/extras/dataset_formatting.py). However, you have to do this yourself when applying chat templates manually for inference."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"prompt_example = '\\n\\nYour task is to analyze the current conversation context and generate a list of actions to start new business processes that we call flows, to extract slots, or respond to small talk and knowledge requests.\\n\\nThese are the flows that can be started, with their description and slots:\\n\\nbook_excursion: search and book an excursion\\n slot: trip_destination\\n slot: excursion_name (use the official name as stated by the assistant)\\n \\nsearch_hotel: search for hotels\\n slot: trip_destination\\n slot: hotel_price_range\\n slot: hotel_start_date (check in date for the hotel. Do not fill this slot unless user explicitly specifies it in the conversation.)\\n slot: hotel_end_date (check out date for the hotel. Do not fill this slot unless user explicitly specifies it in the conversation.)\\n slot: hotel_search_results_readable\\n slot: hotel_name (Name of the hotel which the user has selected. Must be an exact element of the slot \"hotel_search_results_readable\". If the user wants an option that isn\\'t listed, do not fill the slot with that value.)\\n \\nsearch_rental_car: search for rental cars\\n slot: trip_destination\\n slot: car_rental_start_date\\n slot: car_rental_end_date\\n slot: car_rental_search_results_readable\\n slot: car_rental_selection (Car rental option which the user has selected. Must be an exact element of the slot car_rental_search_results_readable. If the user wants an option that isn\\'t listed, do not fill the slot with that value.)\\n \\nchange_flight: change an existing booking to a new flight\\n slot: flight_search_start_date (start date for flight search. Refer to the current time available and always output this in the ISO format.)\\n slot: flight_search_end_date (end date for flight search. Refer to the current time available and always output this in the ISO format.)\\n slot: selected_flight_id (the selected id number from the search results presented to the user.)\\n \\nlist_bookings: list flight bookings - booked flights\\n \\ngoodbye: say goodbye to user\\n \\nlist_skills: tell the user what i can do\\n \\nout_of_scope: trigger this if the user asks for something we don\\'t have a flow for or an existing command does not apply for it. Conversation repair flow for off-topic interactions that won\\'t disrupt the main conversation. should not respond to greetings or anything else for which there is a flow defined or an existing command is apt for it.\\n \\nwelcome: greet the user and ask how you can help\\n \\n\\n===\\nHere is what happened previously in the conversation:\\n\\nUSER: I\\'d like to book an excursion\\n\\n===\\n\\nYou are currently not in any flow and so there are no active slots.\\nThis means you can only set a slot if you first start a flow that requires that slot.\\n\\nIf you start a flow, first start the flow and then optionally fill that flow\\'s slots with information the user provided in their message.\\n\\nThe user just said \"\"\"I\\'d like to book an excursion\"\"\".\\n\\n===\\nBased on this information generate a list of actions you want to take. Your job is to start flows and to fill slots where appropriate. Any logic of what happens afterwards is handled by the flow engine. These are your available actions:\\n* Slot setting, described by \"SetSlot(slot_name, slot_value)\". An example would be \"SetSlot(recipient, Freddy)\"\\n* Starting another flow, described by \"StartFlow(flow_name)\". An example would be \"StartFlow(transfer_money)\"\\n* Cancelling the current flow, described by \"CancelFlow()\"\\n* Clarifying which flow should be started. An example would be Clarify(list_contacts, add_contact, remove_contact) if the user just wrote \"contacts\" and there are multiple potential candidates. It also works with a single flow name to confirm you understood correctly, as in Clarify(transfer_money).\\n* Intercepting and handle user messages with the intent to bypass the current step in the flow, described by \"SkipQuestion()\". Examples of user skip phrases are: \"Go to the next question\", \"Ask me something else\".\\n* Responding to knowledge-oriented user messages, described by \"SearchAndReply()\"\\n* Responding to a casual, non-task-oriented user message, described by \"ChitChat()\".\\n* Handing off to a human, in case the user seems frustrated or explicitly asks to speak to one, described by \"HumanHandoff()\".\\n\\n===\\nWrite out the actions you want to take, one per line, in the order they should take place.\\nDo not fill slots with abstract values or placeholders.\\nOnly use information provided by the user.\\nOnly start a flow if it\\'s completely clear what the user wants. Imagine you were a person reading this message. If it\\'s not 100% clear, clarify the next step.\\nDon\\'t be overly confident. Take a conservative approach and clarify before proceeding.\\nIf the user asks for two things which seem contradictory, clarify before starting a flow.\\nIf it\\'s not clear whether the user wants to skip the step or to cancel the flow, cancel the flow.\\nStrictly adhere to the provided action types listed above.\\nFocus on the last message and take it one step at a time.\\nUse the previous conversation steps only to aid understanding.\\n\\nCurrent time is \"2024-04-27 18:13:47.329404 +0000\" and it is a Saturday today!\\n\\nYour action list:'"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -409,8 +418,7 @@
"streamer = TextStreamer(tokenizer) # stream model outputs as they are generated\n",
"\n",
"# the content to include in the input prompt\n",
"# by default, a value from the validation dataset as example\n",
"content = eval_dataset[\"text\"][0]\n",
"content = prompt_example\n",
"\n",
"# apply prompt template and tokenize\n",
"input_ids = tokenizer.apply_chat_template(\n",
Expand Down Expand Up @@ -472,7 +480,7 @@
"uri": "us-docker.pkg.dev/deeplearning-platform-release/gcr.io/:m125"
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand Down