LLaVA-Video-72B-Qwen2

[HulkZh]

I tested lmms-lab/LLaVA-Video-72B-Qwen2 on videoMME using a 4-card GPU with 96GB video memory and encountered oom. I only loaded 20 weight files before encountering oom, but we did not encounter oom with the same configuration in the code of other warehouses. This is the run command:

accelerate launch --num_processes=4 -m lmms_eval --model llava_vid --model_args pretrained=/root/autodl-tmp/models/LLaVA-Video-72B-Qwen2,conv_template=qwen_1_5,max_frames_num=64,mm_spatial_pool_mode=a verage --tasks videomme_w_subtitle --batch_size 1 --log_samples --log_samples_suffix llava_vid --output_path ./logs/

[kcz358]

You need to pass in device_map=auto into model_args to shard the model and switch to num_processes to 1. For faster inference, you can utilize srt api and setup a server of llavavid