Handle Callable-like types in get_request_model (eg streaming_callback) + Fix component output serialization

[mpangrazzi]

This should add to handle_unsupported_types the Callable type support (which will be converted to dict, since it's not serializable by default by Pydantic).

Probably not the neatest way, but should make deployment and docs work (ref #38 and #40)

[mpangrazzi]

Updates:

• Callable-like types parameters like streaming_callback will be skipped for now when creating the pipeline schema after deploying a pipeline. This will make /docs and /redoc endpoints work. We probably need to add on Haystack the support of having a generator as output (so we can wrap it in a FastAPI's StreamingResponse).
• convert_component_output now is serializing recursively the component output to handle OpenAI responses, which contains nested Pydantic models. See here for an example test.

I've tested with this sample pipeline:

components:
  llm:
    init_parameters:
      api_base_url: null
      api_key:
        env_vars:
        - OPENAI_API_KEY
        strict: true
        type: env_var
      generation_kwargs: {}
      model: gpt-4o-mini
      organization: null
      streaming_callback: null
      system_prompt: null
    type: haystack.components.generators.openai.OpenAIGenerator
  prompt_builder:
    init_parameters:
      required_variables: null
      template: "\nGiven the following information, answer the question.\n\nContext:\n\
        {% for document in documents %}\n    {{ document.content }}\n{% endfor %}\n\
        \nQuestion: {{question}}\nAnswer:\n"
      variables: null
    type: haystack.components.builders.prompt_builder.PromptBuilder
  retriever:
    init_parameters:
      document_store:
        init_parameters:
          bm25_algorithm: BM25L
          bm25_parameters: {}
          bm25_tokenization_regex: (?u)\b\w\w+\b
          embedding_similarity_function: dot_product
          index: 69eb366f-b05a-4811-befe-ed9d49c59a23
        type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
      filter_policy: replace
      filters: null
      return_embedding: false
      scale_score: false
      top_k: 10
    type: haystack.components.retrievers.in_memory.embedding_retriever.InMemoryEmbeddingRetriever
  text_embedder:
    init_parameters:
      batch_size: 32
      config_kwargs: null
      device:
        device: mps
        type: single
      model: sentence-transformers/all-MiniLM-L6-v2
      model_kwargs: null
      normalize_embeddings: false
      precision: float32
      prefix: ''
      progress_bar: true
      suffix: ''
      token:
        env_vars:
        - HF_API_TOKEN
        - HF_TOKEN
        strict: false
        type: env_var
      tokenizer_kwargs: null
      truncate_dim: null
      trust_remote_code: false
    type: haystack.components.embedders.sentence_transformers_text_embedder.SentenceTransformersTextEmbedder
connections:
- receiver: retriever.query_embedding
  sender: text_embedder.embedding
- receiver: prompt_builder.documents
  sender: retriever.documents
- receiver: llm.prompt
  sender: prompt_builder.prompt
max_runs_per_component: 100
metadata: {}

And both deployment and calls works correctly now. Tested with:

curl -X 'POST' \
  'http://localhost:1416/pipeline' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "llm": {
    "generation_kwargs": {}
  },
  "prompt_builder": {
    "question": "How is the light written in equations?",
    "template_variables": {}
  },
  "retriever": {
    "filters": {},
    "top_k": 0,
    "scale_score": true,
    "return_embedding": true
  },
  "text_embedder": {
      "text": "The speed of light in a vacuum is 299,792,458 meters per second (approximately 186,282 miles per second). This is a universal constant, typically denoted as '\''c'\'' in physics equations."
  }
}'

[silvanocerza]

Nice! 🚀

[Yuyang105]

I recently started experimenting with Haystack as a framework for RAG (Retrieval-Augmented Generation). During local testing, everything worked perfectly. However, when attempting to deploy a pipeline with a generator component using hayhooks deploy, I encountered an error related to the OpenAIGenerator.

Here is the pipeline file I'm using:

pipelines/test.yml   # same as @mpangrazzi  sample pipeline

And here is the error message I received:

(venv) (base) ➜  rag-biz git:(main) ✗ hayhooks deploy pipelines/test.yml
Error deploying pipeline: Couldn't deserialize component 'llm' of class 'OpenAIGenerator' with the following data: {'init_parameters': {'api_base_url': None, 'api_key': EnvVarSecret(_env_vars=('OPENAI_API_KEY',), _strict=True, _type=<SecretType.ENV_VAR: 'env_var'>), 'generation_kwargs': {}, 'model': 'gpt-4o-mini', 'organization': None, 'streaming_callback': None, 'system_prompt': None}, 'type': 'haystack.components.generators.openai.OpenAIGenerator'}. Possible reasons include malformed serialized data, mismatch between the serialized component and the loaded one (due to a breaking change, see https://github.com/deepset-ai/haystack/releases), etc.

Error deploying pipeline: Couldn't deserialize component 'llm' of class 'OpenAIGenerator' with the following data:
{
    'init_parameters': {
        'api_base_url': None,
        'api_key': EnvVarSecret(_env_vars=('OPENAI_API_KEY',), _strict=True, _type=<SecretType.ENV_VAR: 'env_var'>),
        'generation_kwargs': {},
        'model': 'gpt-4o-mini',
        'organization': None,
        'streaming_callback': None,
        'system_prompt': None
    },
    'type': 'haystack.components.generators.openai.OpenAIGenerator'
}.
Possible reasons include malformed serialized data, mismatch between the serialized component and the loaded one (due to a breaking change, see https://github.com/deepset-ai/haystack/releases), etc.

Steps to Reproduce

Install dependencies with the following versions:
haystack-ai==2.8.1rc2
hayhooks==0.0.19
FastAPI, Starlette, and other related packages (see detailed environment below).
Use hayhooks deploy pipelines/test.yml to deploy a pipeline containing an OpenAIGenerator component.
Encounter the error above.

Environment

Python version: 3.12.4
OS: macOS
Key libraries and versions:

(venv) (base) ➜  rag-biz git:(main) ✗ pip list
Package               Version
--------------------- -----------
annotated-types       0.7.0
anyio                 4.7.0
backoff               2.2.1
certifi               2024.12.14
charset-normalizer    3.4.1
click                 8.1.8
distro                1.9.0
fastapi               0.115.6
h11                   0.14.0
hayhooks              0.0.19
haystack-ai           2.8.1rc2
haystack-experimental 0.4.0
httpcore              1.0.7
httpx                 0.28.1
idna                  3.10
Jinja2                3.1.5
jiter                 0.8.2
lazy_imports          0.4.0
loguru                0.7.3
MarkupSafe            3.0.2
monotonic             1.6
more-itertools        10.5.0
networkx              3.4.2
numpy                 2.2.1
openai                1.58.1
pandas                2.2.3
pip                   24.3.1
posthog               3.7.4
pydantic              2.10.4
pydantic_core         2.27.2
python-dateutil       2.9.0.post0
python-multipart      0.0.20
pytz                  2024.2
PyYAML                6.0.2
requests              2.32.3
six                   1.17.0
sniffio               1.3.1
starlette             0.41.3
tenacity              9.0.0
tqdm                  4.67.1
typing_extensions     4.12.2
tzdata                2024.2
urllib3               2.3.0
uvicorn               0.34.0

Would it be possible to confirm if this is a version mismatch or if there’s something wrong with how the generator component or its parameters are defined?

Any help or guidance would be much appreciated!

[mpangrazzi]

Hi @Yuyang105! I've tried to reproduce your error, but I didn't get it. What I've done:

# setup a new virtual env
python -m venv .venv

# activate it
. .venv/bin/activate 

# install hayhooks (will come with haystack-ai 2.8.0)
pip install hayhooks

# to use your version 
pip install --no-deps haystack-ai==2.8.1rc2

then run hayhooks with:

hayhooks run

and in another shell instance

hayhooks deploy {pipeline_file}

And pipeline is then correctly deployed.
Note that I've tested with the above sample pipeline. If you have another one, please share it! :)