How to set up OpenAI using Google Gemini API

[y8s]

Is this even possible?
https://ai.google.dev/gemini-api/docs/openai

For example if I plop this in to my compose.yml it fails miserably.

      - OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/
      - OPENAI_API_KEY=FakeAPIKey_$DLKFsfJSd;lkfjSDfuLKSJDfKASJDGFUAQWYf*EWF
      - OPENAI_MODEL=gemini-1.5-flash

mealie-omni-1 | openai.BadRequestError: Error code: 400 - [{'error': {'code': 400, 'message': 'Invalid JSON payload received. Unknown name "response_format": Cannot find field.', 'status': 'INVALID_ARGUMENT', 'details': [{'@type': 'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations': [{'description': 'Invalid JSON payload received. Unknown name "response_format": Cannot find field.'}]}]}}]

(Also for a second I thought the "upload an image" was an image of FOOD which would have been amazing if AI could create a recipe from a picture of the food dish.)

Thanks!

[michael-genson]

Seems like they don't support the full API spec yet (which makes sense since it's still in beta).
FYI: https://platform.openai.com/docs/guides/structured-outputs#function-calling-vs-response-format

We haven't switched to the new structured outputs yet (it didn't exist when we implemented the integration).

[y8s]

Ah ok. I was just looking at free alternatives to OpenAI. I also run oobabooga but not regularly enough to warrant pointing to it.

[KiahConnor]

OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
OPENAI_API_KEY=Key generated here https://ai.google.dev/gemini-api/docs/api-key
OPENAI_MODEL=gemini-1.5-flash

This seemed to work intermittently (which is an improvement!), but mainly you just get "error code 500: The model is overloaded. Please try again later". Might have more success with a different model or paid access.

[y8s]

the openai suffix was key!

but I really want the reverse image recipe gen!

[Borriborri]

so do I understand correctly that you guys are using google gemini and not the openai parser? since this is free?

[y8s]

@Borriborri yes, using gemini for free.

[nickmaleao]

I was able to use gemini to import a recipe from an image, but the smart ingredient parsing is failing.

File "/app/mealie/services/parser_services/openai/parser.py", line 95, in _parse
raise Exception("Failed to parse OpenAI response") from e
Exception: Failed to parse OpenAI response

But if I try to use the debug tool it works, it's only failing in the menu when editing a recipe and trying to parse the ingredients using the openai option.

[nickmaleao]

I opened the issue 4829 and it was closed attributing the problem to the gemini ai. Meanwhile Google has launched a new model Gemini 2.0 Flash and with that model the ingredient parsing inside the recipe menu is working.

The gemini free tier has some limits, but here aren't prices available yet for that specific model, in this reddit post a user mentioned that the experimental models aren't chargeable, but I still have statistics usage and quotas in the api usage reports. I have hundreds of recipes to parse and will monitor the api usage.