From 9350bad060daf202484d457a0a3121e40e5b913d Mon Sep 17 00:00:00 2001 From: druoso Date: Mon, 1 Aug 2022 16:12:02 -0400 Subject: [PATCH 1/3] Incorporate P2577R2 This adds a new annex where the conventions are documented. There are still more details to be added, including the format of the metadata file itself --- ...discovery-in-prebuilt-library-releases.tex | 118 ++++++++++++++++++ buildprocess.tex | 14 +++ ...cpp_modules_ecosystem_technical_report.tex | 4 + 3 files changed, 136 insertions(+) create mode 100644 appendix/discovery-in-prebuilt-library-releases.tex diff --git a/appendix/discovery-in-prebuilt-library-releases.tex b/appendix/discovery-in-prebuilt-library-releases.tex new file mode 100644 index 00000000..6aa28599 --- /dev/null +++ b/appendix/discovery-in-prebuilt-library-releases.tex @@ -0,0 +1,118 @@ +\infannex{discovery-prebuilt}{C++ Modules Discovery in Prebuilt + Library Releases} + +\rSec1[discovery-prebuilt.linker-arguments]{``Linker Arguments'' as a + mechanism to discover \Cpp{} module metadata files} + +\pnum This section describes a framework for the discovery of module +metadata files on environments where the linker arguments for the +library can be resolved early enough. This is not a requirement for +all implementations, as some build systems need to defer the +resolution of linker arguments to a step that is later than the +translation of the module interfaces. + +\pnum While almost the entirety of the behavior of build systems and +package managers are implementation-defined, there is once concept +that is common to most of them: At some point, the build system and +the package manager, when consuming a library as a prebuilt artifact, +need to communicate enough to identify what are the linker arguments +that a consuming build needs in order to successfully use that +library. + +\pnum While the way in which those arguments are discovered is fully +implementation-defined, while the format and semantics of those +arguments are also fully implementation-defined, the concept that when +you consume a prebuilt library you will need to use additional linker +arguments is a point of convergence. + +\pnum It is also a point of convergence that those arguments will be +translatable to files on disk, through implementation-defined +translations. This is a requirement for C++ libraries being +distributed as prebuilt artifacts today. + +\pnum While the format and arguments of the linker are specified in an +implementation-defined way, the current reality is that build systems, +package managers, static analysis tools, etc that want to interact +with different linkers consistently need to re-implement the parsing +of the arguments and trying to emulate the behavior of the linker. + +\pnum This report recommends that a toolchain should provide a +mechanism to translate linker arguments, even if just a fragment of a +complete linker invocation, into the input files that are going to be +used by the linker. + +\pnum Third-party implementations of that translation can be provided +in case vendors do not provide it directly, but there is room for +better interoperability if a tool that performs that translation +exists one way or another. + +\pnum The convention estalishes that metadata files will be +deterministically named alongside the files that are used as inputs to +the linker, such that the build system can parse those to create a +complete understanding of all modules relevant to this project. + +\pnum The specific conversions from the paths to the input to the +linkers to the paths of the metadata files are implementation +defined. Different sections of this report will describe the +convention for specific environments. + +\rSec2[discovery-prebuilt.header-only]{Header-only Libraries} + +\pnum Header-only libraries have been a common practice in the C++ +ecosystem. It makes a specific trade-off on how to use specific C++ +language constructs in order to avoid the complexities of the lack of +convergence on the package management for various different +environments. + +\pnum It also allows a library to avoid ABI-compatibility questions, +and therefore support virtually any standards-conforming toolchain +without having to provide a prebuilt artifact to any of them. + +\pnum It is often the case that package managers will still provide a +release of those header-only libraries within their ecosystem, such +that they can be addressed for dependency management as well as to +manage required compiler arguments, such as include directories and +compile definitions, even when linker arguments are not required. + +\pnum As we transition to C++ modules, it is reasonable to presume +that the same approach can be translated, we can call them +“importable-unit-only module libraries”. As with header-only +libraries, there are specific constraints on how the code has to be +written, but, in principle, any library that could be implemented as a +header-only library could also be implemented as an +“importable-unit-only module library”. + +\pnum However, C++ modules create additional requirements for the +parsing of the consumers of a library. Therefore even if a library +doesn’t require linker arguments, it needs to be able to specify how +to parse those interface units coherently. In practice, that means +there is distinctively more metadata that needs to be provided with an +“importable-unit-only module library” than the comparable “header-only +library”. + +\pnum Specifically, this will mean much more importance to the work +done by the maintainers of the package management metadata for those +libraries in the various package management systems. It is fair to say +that it will not be practical, beyond illustrative cases, to consume +C++ module libraries in the absence of some amount of package +management infrastructure. + +\pnum Solving the consumption of “importable-unit-only module +libraries” in an interoperable way will require a wider convergence on +package management in general. Therefore this report recommends that +package managers following this convention produce library artifacts, +even if empty, and include linker arguments for importable-unit-only +module libraries. + +\pnum This will allow the linker arguments to be a consistent point of +convergence for all cases, which is enough ground to stand a +convention on how to discover C++ modules in pre-built library +releases. Future work for the convergence in the area of package +management may make this recommendation unnecessary. + +\rSec2[discovery-prebuilt.conventions]{Conventions for translating + linker input files to module metadata files} + +\pnum TODO: This section should document the different conventions in +place, there was an example in the original paper, so we need a new +paper where that is discussed in more details. diff --git a/buildprocess.tex b/buildprocess.tex index 51e300cd..e4a560b8 100644 --- a/buildprocess.tex +++ b/buildprocess.tex @@ -19,6 +19,20 @@ case that a build system will be able to consider only modules declared inside the same build context. +\pnum External importable units will be described in metadata files +made available to the build system. While this report recommends a +specific format for that metadata file, the mechanism by which the set +of metadata files to be considered is identified will be +implementation specific. + +\pnum Annex \ref{discovery-prebuilt.linker-arguments} specifies a +convention for environments where the build system and the package +manager can resolve linker argument fragments early enough such that +it can be used to identify the set of module metadata files that need +to be considered. + +\pnum TODO: Format of the metadata file. + \pnum TODO: \href{https://github.com/cplusplus/modules-ecosystem-tr/issues/5}{modules-ecosystem-tr\#5} \rSec2[build.steps.importable-headers]{Identify Importable Headers} diff --git a/iso_cpp_modules_ecosystem_technical_report.tex b/iso_cpp_modules_ecosystem_technical_report.tex index 86444799..8033f564 100644 --- a/iso_cpp_modules_ecosystem_technical_report.tex +++ b/iso_cpp_modules_ecosystem_technical_report.tex @@ -96,6 +96,10 @@ \include{languagesemantics} \include{distribution} +\appendix + +\include{appendix/discovery-in-prebuilt-library-releases} + %%-------------------------------------------------- %% back matter \backmatter From 070351d3a0a715ba277fe642ed15dc6e53a8c1bd Mon Sep 17 00:00:00 2001 From: druoso Date: Mon, 1 Aug 2022 16:12:22 -0400 Subject: [PATCH 2/3] ignore emacs backup files --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 57e3e070..0613c8de 100644 --- a/.gitignore +++ b/.gitignore @@ -20,3 +20,4 @@ iso_cpp_modules_ecosystem_technical_report.pdf tools/sections *.synctex.gz *.synctex* +*~ From f716fecda4e4af079ccd06fd9875481b3ad0f5cb Mon Sep 17 00:00:00 2001 From: Daniel Ruoso Date: Fri, 5 Aug 2022 13:25:27 -0400 Subject: [PATCH 3/3] fix typo --- appendix/discovery-in-prebuilt-library-releases.tex | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appendix/discovery-in-prebuilt-library-releases.tex b/appendix/discovery-in-prebuilt-library-releases.tex index 6aa28599..74e922a9 100644 --- a/appendix/discovery-in-prebuilt-library-releases.tex +++ b/appendix/discovery-in-prebuilt-library-releases.tex @@ -12,7 +12,7 @@ translation of the module interfaces. \pnum While almost the entirety of the behavior of build systems and -package managers are implementation-defined, there is once concept +package managers are implementation-defined, there is one concept that is common to most of them: At some point, the build system and the package manager, when consuming a library as a prebuilt artifact, need to communicate enough to identify what are the linker arguments