Build system: support for application subfolders

[mguetschow]

Contribution description

Currently, it is not possible to:

1. organize code in subfolders without using RIOT modules (both in an application and in other modules)
2. define source files manually for an application in it's Makefile

Using RIOT modules for subfolders has two drawbacks which I found annoying enough to look into this:

• it is cumbersome, since each subfolder needs to contain a Makefile (see also How to use subdirectories in project workspace #13007 (comment))
• RIOT module names need to be unique across the whole codebase to avoid aliasing during compilation

This PR addresses both issues by:

• passing SRC, SRCXX, ASMSRC and ASSMSRC down to the recursive call to make for the application module, following the approach used for BLOB
• automatically creating the folder structure necessary for compilation of source files in subfolders, if they are manually added to SRC and friends (the automatic recognition of source files is still restricted to the application/module folder)

Additionally, it adds a Makefile warning to ensure users are not using the same subfolder both as a RIOT module and manually in SRC (inspired by a chat with @miri64 about this topic).

Testing procedure

The second commit adds an example that show-cases both the traditional RIOT module approach and the newly supported SRC variable approach in examples/subfolders.

[maribu]

I'm personally not fully convinced that this solves a real problem. As you pointed out, the build system does support external modules. Those do require a one-liner Makefile, but they don't need to list all the source by hand. This is less clutter when subfolders contain more than one C file. (But if a subfolder only contains one C file, creating the folder doesn't really help organizing the code, does it?)

More generally: The buildsystem is a pain in the ass to maintain as it is and few maintainers like to actually maintain it, most maintainers/developers much rather maintain/write C code than Makefiles. I'm personally reluctant on adding features that makes niche use cases a tiny bit more convenient. And again: I personally don't think that this indeed is an improvement over using external modules, anyway.

[Teufelchen1]

@mguetschow so far two maintainer have voted against this and there has been little interest. Do you want convert it into a draft and discuss / find a common ground?

[plmorange]

@maribu only for applications, this feature can be beneficial. It allows for easy organization of folders and subfolders without having to create a module each time. It's also better for people discovering RIOT, as it seems easier to create one file and add SRC in it than to create a Makefile for each folder in an application.

[mguetschow]

I've come across the need for subfolder SRC for external packages as well, whose folder structure is not under RIOT's control.

[maribu]

Let's have this.

Requests for this feature have been popping up in the Matrix chat and in the forum a few times. I think it is fair to say that this proves me wrong that just using modules for code organization is sufficient.

The code changes look sane to me and I trust your testing.

[mguetschow]

We actually had a side-discussion with @plmorange over at mguetschow#1. They proposed a test for the BLOB in subfolder case and I was wondering if the application showcasing the subfolder functionality should actually rather go into tests/build_system instead of examples. WDYT?

[maribu]

Hmm, given how often this question popped up recently, I'd personally favor the example to increase visibility. But I think both are fine.

[mguetschow]

@plmorange I've now reworded the documentation to mainly link to the example. Also I found that there was already a test for testing blob files in subdirectories with tests/build_system/blob (this PR does not add support for it, it just followed the example of BLOB to pass down source files to recursive make invocations).

@maribu After checking the CI generated documentation, this should be ready for merge.

[riot-ci]

Murdock results

✔️ PASSED

518f75f docs: link to subfolders example in documentation

Success	Failures	Total	Runtime
10233	0	10233	16m:40s

Artifacts

• Documentation preview

[mguetschow]

Thank you all!