Can this framework used in a production scenario with multiple users?

[MohammedShokr]

I am asking about the scalability of this framework can it be used in production with too many users sending streams and the same time?

[makaveli10]

yes but with batching support which is not upstream as of now.

[yatoooon]

Are there any standards for servers such as what gpu, cpu, and memory can accommodate the number of clients?

[makaveli10]

Not yet but whisper at inference doesnt take up a lot of memory so, with batching supported a single 4090 for whisper-small should support around 25-30 clients.