Confusion about the running time of dp train stage when running dpgen

[LlIVEAir]

Nowadays I have completed the previous work before runnning dpgen to generate a dp potential. However, using the example from: https://bohrium.dp.tech/notebooks/3313403083, I found the overall traning time around 600s for each disp of 10s (The logfile for 575248.err, sbatch .sh for 575248.sh and input.json for 575248.json).
Then I modified the relative files when using dpgen. The initial data has 7501 frames with 282 atoms for each frame. The work has bee conducted on the HPC slurm with 128 cpu cores where gpu was unaccessible. The attached files can be found as machine.json, param.json and train.log. I was wondering if the runnning time was reasonable. Since there are expected to be 4 potential models to train, iteration 4+1=5 rounds, and each round each model will take 600×500×4/60/60≈80h. It seems costly and time-consuming based on the condition.
Addtionally, what could I take to reduce the cost?
upload.zip