What is the default max_tokens for ChatOpenAI? How can I have it be calculated automatically?

[ADTC]

It seems that setting max_tokens=-1 only works in OpenAI (non-chat), but in ChatOpenAI, it generates the error:

openai.error.InvalidRequestError: -1 is less than the minimum of 1 - 'max_tokens'

I also noticed that the default value of max_tokens in ChatOpenAI is actually None. So what really happens? Is there some kind of computation? Is there a fixed value?

For context, I'm using gpt-3.5-turbo-16k to maximize the number of total completion tokens to 16,384, and I'm using VectorstoreIndexCreator to generate a final prompt from a PDF file, so I don't know the final prompt in advance, and can't use tiktoken to do the counting myself.

Based on my testing, it looks like leaving max_tokens in ChatOpenAI as None may mean the same thing as setting max_tokens=-1 in OpenAI. That is, there's no limit to the number of tokens generated.

But I'm not able to ascertain that from the code. Could you please confirm it?

[ShantanuNair]

@ADTC Yes that is the case. None in ChatOpenAI is equivalent to -1 for OpenAI which is equivalent to as-many-as-posssible, at least the last time I trawled through the relevant code. Things do change rather rapidly!