cli(tcl): add ability to pass argument to tcl script from cli

[anonkey]

Pass all argument after the tcl script name to tcl execution in argv, argument number in argc and tcl script name in argv0 like in tclsh or wish :
https://wiki.tcl-lang.org/page/argv

[jalcim]

for
yosys -c file.tcl arg1 arg2 ...

[anonkey]

Possible to just have a comment to know if there a problem with this or if i just need to wait please ?
i need this for work and wish to know if i should maintain durably this in a fork or if i have a chance to have this PR merged

[nakengelhardt]

I think this would break passing arguments to yosys itself. For example a common use case is yosys -c script.tcl design.v, to run the same script on different designs just by changing the command line argument. I don't see a way to reconcile the two...

[anonkey]

I think this would break passing arguments to yosys itself. For example a common use case is yosys -c script.tcl design.v, to run the same script on different designs just by changing the command line argument. I don't see a way to reconcile the two...

Adding a getopt_long and using -- to pass down arguments to tcl ?

[jalcim]

I'm sure all people understand cannot pass cmd line argument is a major problem.

For that, i didn't understand why this verilog file isn't a tcl argument ?
Isn't at yosys to manage that, its the purpose of the tcl.

yosys -c script.tcl design.v

[clairexen]

I'm sure all people understand cannot pass cmd line argument is a major problem.

Tbh I don't understand how this is a "major problem". You can just use something like the following to pass whatever you want through the environment instead:

MY_TCL_ARG="foo bar" yosys -c script.tcl

[nakengelhardt]

Adding a getopt_long and using -- to pass down arguments to tcl ?

That seems reasonable, can you update the PR?

[anonkey]

Adding a getopt_long and using -- to pass down arguments to tcl ?

That seems reasonable, can you update the PR?

Yes, i'll do this asap

[anonkey]

Was tired no need to use getopt long, update done
EDIT: Seem to have an issue for handling -- in getopt at least on ubuntu, have the problem in ci's docker i'm investigating
EDIT2: okay it's fixed was because different implementation of getopt between mac os and linux, i've done a cross platform solution

[anonkey]

Can i do someting else, to get it accepted ?

[nakengelhardt]

Thanks, at first glance it looks good, it's just that we need to find some time to review/try it out now!

[anonkey]

Okay thanks

[jix]

I gave the current PR a try and it's not quite implementing what we had in mind with -- and also has different behavior depending on the getopt implementation. The different getopt implementations might make it hard to actually implement that behavior in a portable way.

The idea was that you can use yosys -p some_script.tcl source.v formal.v -- tcl_option_1 tcl_option_2 with source.v and formal.v being passed to the frontend before the tcl script runs which would receive tcl_option_1 and tcl_option_2 in argv.

When I tried making a small change to your existing PR to implement this, I realized that on linux glibc's getopt actually reorders argv in such a way that it ends up corresponding to yosys -p some_script.tcl -- source.v formal.v tcl_option_1 tcl_option_2 leaving no way to implement that behavior. For getopt implementations that don't implement the GNU extension that allows mixing positional arguments and options by reordering argv it would be possible to implement the behavior we intended, and while you can force glibc to behave that way, that would break mixing positional arguments and options, which we do want to keep supporting.

I'm not sure what's the best way to solve this, one way would be to switch the argument handling to something that doesn't depend on the platform's getopt at all. This would also need to support mixing positional arguments and options to make sure current yosys invocations on linux still work and ideally would support this on all platforms. This means the included fallback getopt implementation doesn't suffice.

Another way that would work, but where I'm not sure if it is something we would want to add that way (@nakengelhardt ?), would be to use a separator for input files and tcl arguments that is not handled by getopt at all, i.e. something that doesn't start with -, I'm thinking of something like ++. For getopt this would be just another positional argument and it wouldn't be reordered and it would be a relatively simple change on top of the existing PR. If you happen to have an input file in the current directory that's named ++ you could still pass it as ./++.

[anonkey]

Ok found why here : https://refspecs.linuxbase.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/libutil-getopt-3.html

I updated my changes to have well working on linux/macos/windows with passing down to frontend arguments before the '--' and to tcl arguments after the '--'
(for windows testing i've commented the #ifdef since i don't have a windows ready for test but should be the same)

3 solutions are easy to use :
use the env but not the best IMHO
use the + at the opstring start (what i've pushed currently)
or since there is a getopt implementation for windows use why not using it everytime to ensure cross platform identical behavior

[jix]

Using + at the opstring start or using the current windows fallback everywhere is a backwards incompatible change for the command line parsing on Linux. As I wrote above, it would be OK to allow mixing options and positional args on non-Linux platforms but not ok to suddenly forbid mixing on Linux as that would break existing uses. More concretely yosys test.v -c test.tcl currently works on Linux and should remain working, being equivalent to yosys -c test.tcl test.v. Even better if that also starts working on other platforms.

[anonkey]

So what about using a - at start of opstring :

RETURN_IN_ORDER  The order of arguments is not altered, and all arguments are processed. Non-option arguments (operands) are handled as if they were the argument to an option with the value 1 ('\001'). This ordering is selected by setting the first character of optstring to '-';

and adding a rule '\001' to handle arguments like now on linux (and maybe at same time code this behavior in the windows version, for mac os it's harder to implement)

[anonkey]

I made an update to use getopt implementation on all platform and fit to RETURN_IN_ORDER behavior.
So positional arguments and options mixing works, on all platform as you asked.

Hope this will be ok for you, it's a pain to have to use a fork in our CI and our project.

[jix]

Testing revealed an issue with a lone "-" as argument (used to read from stdin, should be treated as positional argument) and build failures for the emscripten build, additionally there was another small change needed to make sure existing uses of -- with no TCL script still work. I pushed fixes for all that directly to your branch.

[anonkey]

Thanks a lot, can i do anything else ?

[jix]

I think this is ready now, but given that this affects code used for every yosys invocation and we're about to make a new release, I want to wait a few days and merge it after the release is made to give us a chance to spot any issues I might have missed before it makes it into the release after that.

[anonkey]

Ok thanks a lot for your help