Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identify tooling providers that we need to request input from #29

Open
ruoso opened this issue Nov 16, 2023 · 26 comments
Open

Identify tooling providers that we need to request input from #29

ruoso opened this issue Nov 16, 2023 · 26 comments

Comments

@ruoso
Copy link
Collaborator

ruoso commented Nov 16, 2023

For the first milestone of the Modules Roadmap, we want to be able to have a project that builds modules locally in the project and is able to import std.

The first step of this process will be to collect input from a wide range of tooling providers in order to identify requirements in the ecosystem, for that we need to identify who we need to reach out to in order to get the requirements.

Compilers:

  • msvc
  • edg
  • clang
  • gcc
  • Circle
  • Zig

Standard Libraries:

  • libstdc++
  • libc++
  • Microsoft STL

Build Systems:

  • CMake
  • build2
  • buck
  • buck2
  • bazel
  • hmake
  • meson
  • xmake
  • MSBuild

Packaging Systems:

  • RedHat
  • Canonical
  • Debian
  • vcpkg
  • Conan
  • tipi.build
  • Anaconda
  • Spack
  • pkg-config
  • pkgconf
  • build2
  • Mamba
  • pixi
  • FreeBSD Ports
  • NetBSD pkgsrc
  • OpenBSD ports
  • MacPorts
  • Gentoo Linux Portage
  • Homebrew
  • Fink

Static Analysis Tools:

  • clang-based
  • gcc -fanalyze
  • Coverity
  • Fortify
  • Github Advanced Security (codeql)
  • Sonar
  • PVS Studio
  • ReSharper C++ (for Microsoft Visual Studio)

Documentation Systems:

  • doxygen
  • hdoc
  • standardese
  • doxypress
  • Sphinx

IDEs:

  • CLion
  • Visual Studio
  • vscode
  • vim
  • emacs
  • Eclipse
  • Qt Creator
  • Compiler Explorer
  • Xcode

The list above will be edited as the discussion progresses, please comment with suggestions to change this list.

@ruoso ruoso converted this from a draft issue Nov 16, 2023
@mordante
Copy link
Contributor

Maybe list Standard Libraries projects too. They are separate projects.

@daminetreg
Copy link

Hi @ruoso, we are working actively on this at tipi.build to support modules well and have done some breakthrough, I would definitely see us on the list of packaging systems.

@mathstuf
Copy link
Collaborator

Build systems: meson and xmake

Packaging: Anaconda, Spack

Static analysis: GCC's -fanalyze

IDEs: Eclipse, Qt Creator

@bretbrownjr
Copy link
Contributor

bretbrownjr commented Nov 19, 2023

Compiler Explorer would be an IDE on this list, I think.

Circle and zig would be compilers.

Possibly pkg-config and pkgconf would count as "vendors"... maybe as "other".

There are a lot more package managers of course: homebrew, Arch/AIR, Gentoo, PYPI, etc.

What are the plans to use this list? Possibly the list would be different depending on what it would be used for.

@mathstuf
Copy link
Collaborator

Another analyzer: PVS Studio

@ruoso
Copy link
Collaborator Author

ruoso commented Nov 20, 2023

What are the plans to use this list? Possibly the list would be different depending on what it would be used for.

My plan is to create an issue for each one of the items in the list and get in touch with the vendors/implementers for each one of them to get their input on what would be needed in the ecosystem for it to be possible for them to support a project that does import std.

@Klaim
Copy link

Klaim commented Nov 23, 2023

Note that build2 should also be listed in the "packaging system", it provides both build-system and packaging manager.

@Klaim
Copy link

Klaim commented Nov 23, 2023

About Anaconda: I'm part of the Mamba team which implements Conda package management but in C++ (Conda was originally in Python but recently adopted Mamba's library) so maybe that's relevant. The original author of Mamba also created pixi which also support Conda packages.
Note that I am not sure if this ecosystem of libraries is relevant in the context of C++ modules, as these packages can hold not only C++-made stuffs.

@jwakely
Copy link
Member

jwakely commented Nov 23, 2023

Packaging Systems:

It's not clear to me what this section means. Does it mean actual packaging systems, or vendors of packages built by such systems?

* RedHat

Red Hat isn't a packaging system. RPM (as used by Red Hat, SUSE, and Mandriva descendants) is a packaging format, DNF and Zypper are package management systems.

* Canonical

* Debian

Canonical isn't a packaging system. Deb (as used by Ubuntu, Debian and descendants) is a packaging format, APT is a package management system.

If the purpose of this section is to list vendors who might need to be contacted to discuss support for packaging CMIs, there are also source-based systems like BSD Ports, Arch, Gentoo, Nix/NixOS, ...

Documentation Systems:

* doxygen

Also https://hdoc.io/ and https://github.com/standardese/standardese and https://github.com/copperspice/doxypress/

Would a Debuggers section make sense? I'm not sure if debuggers need any particular support for modules.

@grafikrobot
Copy link

Would a Debuggers section make sense? I'm not sure if debuggers need any particular support for modules.

That does raise an interesting aspect. Packages can come with source. And debuggers do benefit from knowing where the source is. So, yes, I think adding debuggers to the list is useful. Irrespective of needing to know about modules or not.

@bretbrownjr
Copy link
Contributor

In a brainstorming pass like this, all sorts of input are constructive. I'd rather people be more focused on connecting ideas than making sure they are naming a packaging system, a package manager, or a package repository.

Part of the reason I feel that way is that I expect the feedback process will be a bit fluid anyway. In my experience in engineer coordination exercises, the communication mechanisms can vary widely depending on the community and how it works. To be effective, one set of stakeholders might work best in email threads, another in text chat channels, another in issues on repos, and a third might require direct customer interaction with a relevant vendor. So it could look like discussing an organization that sells software sometimes and specific source repo in others.

That's to say I like the flexibility of the mechanism that @ruoso proposes. We can retitle, merge, split, or close issues as needed. And if attempts to engage particular organizations don't result in desired feedback, we'll have some record of what happened at least.

That being said, if people have a hard time making sense of our issues, that's good feedback, and something can probably be tweaked in that case. Specifics about what was confusing would help in that case.

@ChuanqiXu9
Copy link

Would a Debuggers section make sense? I'm not sure if debuggers need any particular support for modules.

As far as I know, as long as the generated dwarf information is correct, there won't be an issue in debuggers. Even if there are problems, it should be the job of the compiler to emit correct DWARF information. I feel the concept of modules is transparent to debuggers.

@ChuanqiXu9
Copy link

For the first milestone of the Modules Roadmap, we want to be able to have a project that builds modules locally in the project and is able to import std.

I feel it is already the case. e.g., Boris can built a relative complex project with libc++'s std module now: https://github.com/boris-kolpackov/async_simple/tree/CXX20Modules-build2.

The first milestone in my mind is that, a newbie can write and compile a hello world example by using std module without understanding what is modules and build systems. Just like we don't need to understand what is standard library and how is that get included and linked when we wrote our first C++ hello world example.

(There might not be gap. I just want to make things more clear)

@mordante
Copy link
Contributor

For the first milestone of the Modules Roadmap, we want to be able to have a project that builds modules locally in the project and is able to import std.

I feel it is already the case. e.g., Boris can built a relative complex project with libc++'s std module now: https://github.com/boris-kolpackov/async_simple/tree/CXX20Modules-build2.

Boris points out the current solution is kind of a hack. Build2 ships files copied from libc++, so for every LLVM release build2 needs to be updated to the latest state of libc++. I really like to have a portable method for (Standard) libraries and implement that in libc++.

@ChuanqiXu9
Copy link

For the first milestone of the Modules Roadmap, we want to be able to have a project that builds modules locally in the project and is able to import std.

I feel it is already the case. e.g., Boris can built a relative complex project with libc++'s std module now: https://github.com/boris-kolpackov/async_simple/tree/CXX20Modules-build2.

Boris points out the current solution is kind of a hack. Build2 ships files copied from libc++, so for every LLVM release build2 needs to be updated to the latest state of libc++. I really like to have a portable method for (Standard) libraries and implement that in libc++.

Yeah, understood. I just want to say that currently we're already able to import std in codes. For example, I believe the above sample can be compiled with CMake as the example you give in the libc++'s documents.

It is almost a wording issue for me. I think currently it is already able to do that. And what we want to do here is to make it easier to get accessed and lower the bar to import std for users.

@ChuanqiXu9
Copy link

FWIW, I created an issue to support std module in compiler-explorer: compiler-explorer/compiler-explorer#5404

@RedBeard0531
Copy link

Would a Debuggers section make sense? I'm not sure if debuggers need any particular support for modules.

As far as I know, as long as the generated dwarf information is correct, there won't be an issue in debuggers. Even if there are problems, it should be the job of the compiler to emit correct DWARF information. I feel the concept of modules is transparent to debuggers.

One issue that will come up is how to refer to module linkage entities inside the debugger. Modules introduce a new form of namespacing separate from C++ namespaces. It is now possible to have more than one ::a variable in a program without violating ODR. But the language doesn't provide any way to disambiguate them because there are no contexts in C++ sources where they can both be named. However, I think a debugger will often want to provide access to both of them.

@ruoso
Copy link
Collaborator Author

ruoso commented Nov 28, 2023

One issue that will come up is how to refer to module linkage entities inside the debugger.

Those do get a different mangling, so the debugger should be able to tell them apart. We do need to consider how we represent the module linkage in the demangled representation of the symbol, but that's not something that has needed convergence before, as it's just meant for human consumers.

@ruoso ruoso moved this from Todo to In Progress in C++ Modules Roadmap Nov 28, 2023
@ruoso
Copy link
Collaborator Author

ruoso commented Nov 28, 2023

Thanks everyone for the suggestions and feedback. I think this is probably a good starting point for this project. I am now moving to #30 where I will formulate a questionnaire to serve as jumping point for the conversation with each of the tooling vendors.

@ChuanqiXu9
Copy link

ChuanqiXu9 commented Nov 29, 2023

I just got that if we forget ccache? ccache can't work with modules without refactoring. I filed an issue here: ccache/ccache#1252

And also distributed build tools like distcc (there may be more).

@mathstuf
Copy link
Collaborator

sccache exists too and would need support.

@olgaark
Copy link

olgaark commented Nov 29, 2023

The Build section is missing MSBuild. It already supports STL modules with /std:c++latest though.

@RandomDSdevel
Copy link

RandomDSdevel commented Nov 30, 2023

For the first milestone of the Modules Roadmap, we want to be able to have a project that builds modules locally in the project and is able to import std.

The first step of this process will be to collect input from a wide range of tooling providers in order to identify requirements in the ecosystem, for that we need to identify who we need to reach out to in order to get the requirements.

     I'd also like to suggest some additional candidates for some of these lists, if I may. I'll take this one section at a time for each one I have potential candidates for. (I'll only focus on relatively prominent/popular tools.)

⁝ (Some sections snipped…)

Build Systems:

  • CMake
  • build2
  • buck
  • buck2
  • bazel
  • hmake
  • meson
  • xmake

  • GNU make
  • Non-GNU make implementations from other Unix distributions and distribution families:
    • BSD make:
      • FreeBSD make
      • OpenBSD make
      • NetBSD make
  • GNU autotools
  • Ninja
  • Apple's llbuild, used by Xcode (for all languages it supports, not just Swift)
  • Anything else that might be relevant from the Wikipedia article 'List of build automation software' (though this includes tools for more programming languages than just C++, of course.)

Packaging Systems:

  • RedHat
  • Canonical
  • Debian
  • vcpkg
  • Conan
  • tipi.build
  • Anaconda
  • Spack
  • pkg-config
  • pkgconf
  • build2
  • Mamba
  • pixi

 (    (I won't touch on the naming debate that's started for this section; @bretbrownjr's response on that subject is fine for now.)

⁝ (Section snipped…)

Documentation Systems:

  • doxygen
  • hdoc
  • standardese
  • doxypress

  • Sphinx (Can also work with Doxygen)

IDEs:

  • CLion
  • Visual Studio
  • vscode
  • vim
  • emacs
  • Eclipse
  • Qt Creator
  • Compiler Explorer

The list above will be edited as the discussion progresses, please comment with suggestions to change this list.

  • JetBrains extensions for other IDEs:
    • ReSharper C++ (for Microsoft Visual Studio)
  • Xcode

@mathstuf
Copy link
Collaborator

mathstuf commented Nov 30, 2023

make, ninja

These are not build systems (well, make can be one, but I don't think it will survive C++ modules in any meaningful way), but build tools (or "executors"). ninja is explicitly not one and doesn't want to be one; I think we can leave these off of the list at least.

There are ninja implementations which may qualify; particularly shake.

@ruoso
Copy link
Collaborator Author

ruoso commented Nov 30, 2023

I think I agree with @mathstuf on the take on make and ninja. Make is a programming language and pretty much everyone writing a makefile is creating a build system of sorts using that language.

From the point of view of gathering input, I don't know if the maintainers of those would have much opinion on what C++ Modules require.

@jm4R
Copy link

jm4R commented Dec 1, 2023

KDevelop IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

No branches or pull requests