error for assigning config file by snakemake option

[vicfabienne]

Hey,
I have another issue regarding the use of the config file.
I'm running different tests for testing different parameters on and for different sample sets. Therefore it would be great if I could just create different config files and running vpipe using snakemakes --configfile option. I thought that should be possible since it is a basic snakemake functionality.
However, after some tests I figured that using this option will always throw an error. Even if I assign the path to the default path/to/vpipe/workdir/vipipe.config with the --configfile. So really the same file which is definitely used when running vpipe without specifying the config-file. I get YAML format errors, that's why I tested multiple times if it really is the same file I assign there. I really can't think about any reason for this anymore.
Do you have an idea what could be the cause of it? Or am I indeed just misunderstood some snakemake logic?

Those are the errors I get:

yaml.parser.ParserError: expected '<document start>', but found '<scalar>'
  in "vpipe.config", line 2, column 1
[...]
During handling of the above exception, another exception occurred:
[...]
snakemake.exceptions.WorkflowError: Config file is not valid JSON or YAML. In case of YAML, make sure to not mix whitespace and tab indentation.```

[DrYak]

Hi!

Upgrading the configuration from the old bespoke format based around Python classes, into something that actually use modern configuration format is on our todo list.

[DrYak]

Hi again,
just to keep you informed that the branch rubicon is currently supporting configuration files (--configfile) from the command line.
This branch is experimental, but it passes end-to-end tests with both HIV and SARS-CoV-2 data.

Current limitations are that Snakemake can only use YAML or JSON configuration format as per the error message you got.
There is currenlty no support for configuration files that rely on the INI-like format of Python configparser.
We're currently testing a possible modification of Sankemake itself to support multiple formats, including the current YAML and JSON as well as the older Python configparser, thanks to module anyconfig.

In the meantime, if you want to experiment with this branch, you can use the above module's command line interface anyconfig_cli -O yaml -o config.yaml -I ini vpipe.config to convert a vpipe.config into a config.yaml - this might require some manual fixing.

[DrYak]

Hi,

the latest modifications have been pulled into master as part of #102 :

• We now support modern JSON / YAML config format as supported by Snakemake.
• V-pipe can also import old INI-style configratuion by Python config parse.
• --configfile and --config options now work.
  • note that as Snakemake PR 1111 isn't merged yet, INI-style is only supported in vpipe.config file. Not with --configfile option.
  • note that there is a bug (see Snakemake PR 1126) causing trouble when mergeing multiple files specified after --configfile (and/or providing multiple keys after --config): sections are overwritten instead of correctly deep-merged.