--devices 2 failed for test case

[jiaboli007]

I can run the prediction of prot.fasta (the sample input in the package) with default setting:

boltz predict prot.fasta

The run finished succesfully.

However, if I run the same job with option --devices 2 (trying to use two GPUs on my machine), it failed. See the log file of the error message,
boltz.log

I am running on Ubuntu 22.04.4 LTS, with pytorch version '2.5.1+cu124'. I have installed boltz from source and updated to the latest version (0.2.1)

[eyal-converge]

Got a similar error with --device=2 (using 2xA100 40GB)
debian11
pytorch==2.5
cuda==12.4

[jwohlwend]

This happens because you are trying to use multiple devices eventhough you only have a single input, so one GPU has nothing to do and fails

[eyal-converge]

@jwohlwend
I assumed --device can turn on model parallelism or something

[jwohlwend]

We do not support model parallelism, devices should be used to run many predictions in parallel. I've added an error message.

[eyal-converge]

@jwohlwend
Can you give an example when to use --device flag ?

[jwohlwend]

Say you have a directory containing multiple yaml files:

my_directory
     input1.yaml
     input2.yaml

When you do boltz predict my_directory it will run input1.yaml and then input2.yaml. But if you have multiple GPU, say 2 in this case, you can add the flag --devices 2 in which case input1.yaml and input2.yaml will be predicted in parallel, one on each device.

[jiaboli007]

Thank. I tested the current version (0.3) with the sample input ligand.fasta on my machine, it finished without failure. Many thanks for making constant improvements.