Query : What is the intention behind the specified max_new_tokens/max_tokens in the video understanding notebooks ?

[PHOENIXFURY007]

04_video_understanding.ipynb
max_new_tokens=2048

04_video_understanding_vllm.ipynb
max_tokens=4096

Does the token size here refer to the total context window? Or does it refer to the context window for the chat reply?
It was mentioned in the introduction that the Long multimodal context window consists of 64K tokens. So can you elaborate on it a bit? Thanks

[xffxff]

04_video_understanding.ipynb max_new_tokens=2048

04_video_understanding_vllm.ipynb max_tokens=4096

Does the token size here refer to the total context window? Or does it refer to the context window for the chat reply? It was mentioned in the introduction that the Long multimodal context window consists of 64K tokens. So can you elaborate on it a bit? Thanks

max_tokens and max_new_tokens do not refer to the total context window size. Instead, they specify the maximum number of tokens that the model can generate in a single output sequence. These values must be less than the total context window size of 64k tokens. You can set them to larger values if you have enough GPU memory available.