Accessing Stream Chunks (Streamed generation)

[backslash112]

• I'm submitting a ...
  [x] question about how to use this project

• Summary
  I'm encountering two problems when working with the streaming example:

1. When running the code from examples/streaming2.ts with stream: true, I get an error: Missing required fields: answerInPoints. What's causing this error and how can I resolve it?
2. After setting stream: true, how can I access the result chunks? Are there methods similar to for await (const chunk of result) or completion.data.on() that I can use to process the incoming stream? (Similar to How to use stream: true? openai/openai-node#18)

Any guidance on resolving these issues would be greatly appreciated. Thank you!

[dosco]

sorry was planning to fix this earlier was in the middle of our big migration to a monorepo. looking into this now.

[dosco]

fix in latest release

[taieb-tk]

I would like to reopen this issue. Number two, i dont understand on how to do it with .chat.

I can see that it is suppose to return a Readable Stream, and i have set the stream to true. But i cannot get it to work.

Any example or ideas @dosco ?

[dosco]

@taieb-tk have you looked at the streaming1.ts and streaming2.ts examples? stream: true enables streaming with the underlying llm provider to speed up things the final fields are not streamed out.

[taieb-tk]

@dosco
Yes i did, i could not get it to work, probably a skill issue from my side.
I tried to just use the

`const ai = new ax.AxAIOpenAI({
apiKey: apiKey as string,
});

    ai.setOptions({ debug: true })


    const response = await ai.chat({
        chatPrompt: conversationHistory,

        config: {
            stream: true
        },
        ...(tools?.length && { functions: normalizeFunctions(tools) }),
    });

`

Not sure what to do with the response in the next step... Could you possibly help me? :)

[taieb-tk]

Bump any help would be appriciated :)

[dosco]

The bug is in the line below which is wrong. Also typescript should catch this it's even in the api docs. https://axllm.dev/apidocs/classes/axai/

 config: {
            stream: true
        },

It should be

ai.setOptions({ debug: true })


const response = await ai.chat({
    chatPrompt: conversationHistory,
    ...(tools?.length && { functions: normalizeFunctions(tools) }),
}, {
   stream: true
});

[taieb-tk]

The response is suppose to be asyncgenerator? Any examples on how to catch that stream?

[taieb-tk]

Ahh ok! I read a bit fast, i will try that! thanks!! :)

[dosco]

use an async for loop like in the streaming examples Vikram Rangnekar

…

On Mon, Oct 14, 2024 at 12:56 PM taieb-tk ***@***.***> wrote: The response is suppose to be asyncgenerator? Any examples on how to catch that stream? — Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGLF277TMKA4E4RFB64GRDZ3QOWRAVCNFSM6AAAAABKDK5QRWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJSGA4TINZSGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[taieb-tk]

Your answer above solved my problem! Really appriciate the help. Looking forward to continue testing this 👍