Cook Along chat bot

[rosebeats]

First Check

• I used the GitHub search to find a similar requests and didn't find it.
• Checked the tasks tagged issues and verified my feature is not covered

Please provide a concise description of the problem that would be addressed by this feature.

Cooking can be a hectic process and I typically find it difficult or annoying when working on recipes to keep track of what I need to do. Ideally, I would proceed step by step, reading each instruction, executing that instruction, and moving on. In practice, I have trouble with that. It doesn't matter how the instructions are formatted, often times I still end up jumping around to different instructions and losing track of what exactly I needed to do, or I've finished some steps and need to make sure I didn't accidentally miss something. When cooking with another person, I can ask them questions about the recipe while I'm in the middle of cooking. That still requires someone to actually read through the recipe though and isn't a solution for people cooking alone.

Please provide a concise description of the feature that would resolve your issue.

With modern LLMs, instead of asking someone I'm cooking with questions about the recipe, I can ask a generative model. This feature would add:

• New API endpoints for
  • text to speech
  • speech to text
  • a conversational AI for asking questions which incorporates the details of the recipe
• Frontend
  • display a conversation with an AI assistant in recipe cooking mode
  • speech input for the conversation UI
  • playback of the response from the assistant
  • button to read out an instruction with text to speech (not strictly needed as part of this feature, but would be a nice addition if we implement an API for text to speech)

Please consider and list out some caveats or tradeoffs made in your design decision

Some implementation notes:

• the wyoming protocol implements a standard way of communicating with services needed for chat assistants. This is the same protocol Home Assistant uses. This lets us easily connect to a text to speech and speech to text service external to mealie over tcp, unix, or stdio
• OpenAI is currently one of the leaders in large language models and mealie already integrates with OpenAI for other services, so this seems like a sensible service to integrate with for the text assistant API. Other models like Ollama can also be set up to take API calls in the same way the OpenAI's models do if users want to run a local model.

Additional Information

• If this is accepted I'm willing to submit a PR to provide this feature
• If this is accepted I'm willing to help maintain this feature
• I'm willing to sponsor/pay a developer to do this work

[CloCkWeRX]

A simpler version of this would be a "play recipe steps" mode, which read each step and presented a "next" or "repeat" button, for use on a tablet.

https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API covers the read/parse simple speech parts.

[CloCkWeRX]

HTML mockup; Receipe -> Cook Mode.

Depicts play this step, pause currently playing, and repeat/start over

While this implies a bit of state management - step done, current step, what's playing, and potentially some "auto scroll to focus"; this is the basic version of this.

When in cook mode though; it may be better to have a "Really big play/pause/next" section of the UI; as your hands might be covered in something and you are using a knuckle or similar to minimise contamination risk

[CloCkWeRX]

#4637 initial PR describing the original idea