We have the same problem as @db4 reported above as "problem 2)".

Here is our advanced scenario:
We have a project "myPrj" which needs three tools to build: toolA, toolB and toolC (all available as conan packages).

• myPrj/1.0.0 - tool_requires() ->
  • toolA/1.0.0
  • toolB/1.0.0
  • toolC/1.0.0

All the tools have a dependency to an interpreter tool (available as conan package).

• toolA/1.0.0 - requires() -> interpreter/1.0.0
• toolB/1.0.0 - requires() -> interpreter/1.0.0
• toolC/1.0.0 - requires() -> interpreter/2.0.0 (yes, it is the same tool with the same name, just a different version)

When building myPrj the tools have to be called via the interpreter like:

• interpreter.exe toolA.bin
• interpreter.exe toolB.bin
• interpreter.exe toolC.bin

Note that is important that each tool is executed with the correct version of the interpreter.

Because of the above mentioned problem, the environment variables specified in the package_info() method of the interpreter, which points to the interpreter executeable in the conan cache after intall, are not visible in myPrj (not generated into runenv or buildenv scripts).

Any suggestions how to tackle this with conan?

At the moment I assume conan has a bug (tested with 2.016 and 2.12.1) here not providing the runenv and buildenv information of transitive dependencies on requires with trait "build=True".

Our workaround at the moment is to set the trait "build=False" on the requires from myPrj to the tools.
But this workaround is incomplete as conan seems not to allow different versions of the same package (in our case: interpreter) in the graph. This seems only be possible with build requires.

Hi @SzBosch

Sorry @db4 for not responding this back then.

There is an issue in the definition of the graph. If Conan propagates the interpreter run requirements into myprj it would need to raise a version conflict exception, because it is not possible to simultaneously have 2 versions of the same executable in the path, as that would be an undefined behavior scenario.

I don't know about the tools or exact requirements, but the rules of requires also apply to tool_requires.

That means that if some package like myprj needs to call interpreter or has any direct usage or reference of interpreter, then interpreter must be direct tool_requires from myprj to interpreter package.

This is aligned with the default requires behavior, by default now transitive dependencies are hidden, that means that trying to do a #include "transitivedep/header.h" will fail, because that transitive header is not visible. If the package has direct #include to transitivedep, it must have a requires("transitivedep/version") explicitly.

There is now an example in the docs about the scenario for wanting to use different versions of the same tool-requires in the same package, if you haven't, please check it https://docs.conan.io/2/examples/graph/tool_requires/different_versions.html. Still, it is not expected that they can be added automatically to the environments, but the consumer needs to differentiate explicitly between the different versions (unless they are called interpreter1.0.exe and interpreter2.0.exe), like using the dependency package folder path explicitly for example.

I have also added a explicit test in #17714 for this scenario, and to show how the dependencies package folders paths can be used to call the executable, instead of relying on different executable name.
The test contains the 2 cases:

• test_require_different_versions_transitive_noconflict if there is no runtime conflict of the transitive dependency, it is possible to make it work
• test_require_different_versions_transitive if there are runtime conflicts, the way is to hide their propagation with run=False, and let the consumers downstream explicitly instantiate what they want

Hi @memsharded, thank you very much for your answer.

I think it will take some more time to dig into it but let me provide some more details to you what we are trying to achieve:

Yes, myPrj needs to call technically "interpreter.exe" directly, but logically, we would like to encapsulate/hide the interpreter e.g. behind toolA in a way, that myPrj does not need not know about this. myPrj shall just call an abstracted e.g. "toolA".

There are (at least) 2 ways of doing this:

1. During the conan package creation of e.g. toolA the correct interpreter in the desired version can be copied into the package-binary/payload of toolA like having interpreter.exe and toolA.bin contained inside.
   Furthermore a cmd/sh script can be generated like toolA.cmd/.sh with having the call "interpreter.exe toolA.bin" inside. In myPrj then just toolA.cmd/.sh can be called, without knowing there is an underlying interpreter.

This approach would solve both issues at the same time: Encapsulating the interpreter from downstream myPry and having multiple versions of the same interpreter used.

But unfortunately these interpreters are very huge in size to that if multiple tools, like toolA and toolB in the example above are using the same interpreter version, the disk space consumption and traffic (esp. in CI/CD environments) is unnecessarily high.

Therefore we need an alternative.

2. What I wanted to try instead: In the recipe of the interpreter in the package_info() an versioned environment variable is set like:

self.runenv_info.define("INTERPRETER_1_0_0", os.path.join(self.package_folder, 'bin', 'interpreter.exe')

During the build and conan package creation of e.g. toolA a toolA.cmd/.sh script is generated and packaged to "bin" folder" containing the invocation like:

${INTERPRETER_1_0_0} toolA.bin

This is should possible, because conan knows about the dependency and it's version.
Also here there is a package_info() with e.g.:

self.runenv_info.define("TOOL_A", os.path.join(self.package_folder, 'bin')

(Alternative: Instead of a cmd/sh script, the whole invocation command might be able to encode on an environment variable completely; have not tried this)

Now in myPrj with a build_ or tool_requires() (here independent of tait "run") to toolA/1.0.0 there should be a conanbuildenv.sh generated on "conan install" command containing the definition of the environment variables TOOL_A and also INTERPRETER_1_0_0.
The invocation would then be:

source conanbuildenv.sh
${TOOL_A}/toolA.sh

This approached failes if the requires() to toolA/1.0.0 has trail "build=True" (like tool_requries has) because the runenv_info of the transitive dependency to interpreter/1.0.0 lead not to the generation of the environment variable INTERPRETER_1_0_0 in the context of myPrj.
This means the script ${TOOL_A}/toolA.sh cannot run.

But when I change the trait to "build=False" then the environment variable INTERPRETER_1_0_0 appear in the generated conanrunenv.sh script, so an invocation is then possible with:

source conanrunenv.sh
${TOOL_A}/toolA.sh

With this the goal is achieved that myPrj does not to know anything about the interpreter.
But unfortunately this is the wrong type of require, as the tool is a build tool only and furthermore this type of require does not allow different version of the same package in the graph, which we also need (build_requires() or tool_requires() with trailt "run=False, which seems to be the same, but both have trait "build=True").

I still do not understand why the propagation of transitive dependencies is different between a requires() with trait "build=True" and "build=False", but as I said maybe I need to dig more into your answer first.

I had a look into your test: 366070f

...
class Pkg(ConanFile):
           name = "project"
           version = "1.0"
           def build_requirements(self):
               self.tool_requires("wrappera/1.0")
               self.tool_requires("wrapperb/1.0")
           def build(self):
               ext = "bat" if platform.system() == "Windows" else "sh"
               self.run(f'mygcc.{ext}')
...

Here "project" has to know about "mygcc". Exactly this I try to avoid.

To come closer to my scenario can you please make a further test with something like:

...
class Pkg(ConanFile):
           name = "project"
           version = "1.0"
           def build_requirements(self):
               self.tool_requires("wrappera/1.0")
               self.tool_requires("wrapperb/1.0")
           def build(self):
               ext = "bat" if platform.system() == "Windows" else "sh"
               self.run(f'wrappera.{ext}')  # mygcc will be called under the hood
               self.run(f'wrapperb.{ext}')  # mygcc will be called under the hood
...

The difficulty here is that wrappera/b need to know after "conan install" the location of "gcc" in the local cache.

Note that we do not use build() funciton in "project", we just use "conan install" and the generated env scripts to call the build tools.

I have added a new test in #17714

The most important part is that this is not a C++ package management thing. The problem at and is:

• A build process depends on 2 different executables, wrappera.exe and wrapperb.exe
• Each of those executables depends in turn, exclusively at runtime on 2 different versions of mygcc.exe, versions 1.0 and 2.0
• But there is really no build time information, no link time information that allows some executable to locate other executable.
• The only possible information is defining the PATH environment variable.
• But if the build process defines one environment, with one PATH variable, by definition of the process it is not possible to run 2 different versions of the mygcc.exe executable.
• It really doesn't matter if the build process tries to call mygcc.exe directly or it is being called by wrapper.exe indirectly. The environment defined is just one, so only one executable can be located.

There are different possible strategies that could be used to address this system problem:

• By vendoring the mygcc executable together with the wrapperx executable. That way, when the executable wrapper runs, it knows that it can find the executable just besides itself. This is the approach I have implemented in the test
• The vendoring approach does a copy of the mygcc.exe at build/package time. Another alternative would be to use the finalize() method to establish that connection at package install time, which might be worth if the mygcc.exe was huge and doing extra copies would be a huge cost
• Another approach could be controlling the environment in the consumer process or recipe. In the same way the test before used the full path via path_a = self.dependencies.build["gcc/1.0"].cpp_info.bindir to explicitly control which mygcc.exe is being executed, it is possible in the recipe to create different environments scripts for the different dependencies and activate them explicitly before launching the respective wrapper. But this sounded a bit more convoluted, so I experimented with the vendoring approach.

In summary, propagating transitive dependencies for executable applications with different versions is mostly an unresolved problem on the OS level, this cannot be solved automatically for relocatable executables, this cannot be implemented by any tooling. Conan can implement different strategies to address this problem by isolating the transitive executable, via vendoring, finalize() or by custom activation/activation of different environments for the different executations of wrappers.