--gradient_accumulation_steps

[wxtnull77]

Hello, I would like to ask you about this step in README, The default batch size is 512. When GPU memory is insufficient, you can proceed with training by adjusting the value of --gradient_accumulation_steps.how to do it specifically？

[rentainhe]

I have the same question :(

[HughSeven]

my GPU reported that the memory was full, and then I looked at the code
There is a line in the train function:
args.train_ batch_ size = args.train_ batch_ size // args.gradient_ accumulation_ steps
so I use ''python train.py -- XX -- XX -- XX -- args.gradient during training_ accumulation_ steps 3''
Then I can run