RFC: Amended Require Syntax

[bradsharp]

This RFC proposes a new syntax for require by string that resolves the concerns with the current iteration. Rather than using paths relative to the file, they will be relative to a directory as defined by the RFC.

Rendered.

[Dekkonot]

Presumably this means calling require without a prefixed string, but it should be more clear.

[Dekkonot]

I cannot support this RFC.

I support this approach in theory but as-is I've personally written a non-trivial amount of Luau that relies upon the existing semantics (most particularly, relative requires) and the work required to update that code is not insignificant. I assume others have also written large amounts of Luau code that assumed that the RFCs already ratified were in fact going to be maintained and not just freely replaced. Additionally, multiple tools have been written that assume that the existing semantics are what we were actually going to stick with.

If you'll allow me to speak plainly: it undermines the stability of the entire language if a critical part of the language like this is allowed to change in a breaking way due to RFCs like this. I don't believe any RFC that actively breaks compatibility with no path to clear migration should be seriously considered, even if there's a compelling reason like this.

Are you expecting tools like Lune and Luau LSP to instantly update to follow the new semantics, or is fragmenting the ecosystem an acceptable loss here? And what of users who cannot just drop what they're doing to update their code but would like to not use outdated tooling?

---

The only immediate advantage I can find for this vs other approaches is that this RFC doesn't impact the structure of things at all, which is not even an advantage. It just hides that behind the nebulous concept of a 'library'.

[ffrostfall]

I think that this is an enormous breaking change to deal with a problem that is localized to relative requires in purely folder/init.luau.

This would break all code using relative requires to settle that issue.

While relative require isn't really ideal, and likely shouldn't have existed from the start, this is too big to just let slide.

Plus, this means on Roblox, any code wanting to go through their client folder would need to do src/client, or a .luaurc would need to be defined there. This isn't ideal, you shouldn't need .luaurc everywhere to get acceptable file paths.

This would be better if tooling supported this at the moment, it doesn't, and as such this could render require-by-string unusable while tooling catches up.

[gaymeowing]

Why isn't the new syntax required to be explicit, like in the choice chosen in the last accepted require-by-string RFC?

[gaymeowing]

Gonna post what I've said in ROSS here, but this seems like much more of a roblox issue, and not something luau should be dealing with at all.
Because having a tool for roblox to "fix" things in games, such as converting existing games to not have nested module scripts, removing usages of depricated methods with their new equivalents, replacing special meshes with MeshParts, ect.
Already has multiple uses vs a luaufix tool that has only one use; for converting current string requires to the format proposed here.

[dphblox]

Could the rationale for moving away from relative paths be explained? While I don't use relative paths in my work, I don't think I understand what the problem is, and why it's so important that it would warrant removing relative paths language-wide.

Internally, we also have a desire to keep Luau and the community in-sync with require syntax, as divergent require styles between Roblox and external code is one of the issues we're actively butting up against in our efforts to merge with community forks. I would hope that drastic changes to path resolution would not cause fragmentation as this could trip our efforts up.

[dphblox]

Talked with a few people about this. As best I can determine, the issue with require paths is to do with script nesting behaviour.

Specifically, imagining this nesting of modules:

• 📜 frobulate
• 📜 myModule
  • 📜 frobulate

Represented by these files on disk:

• 📄 frobulate.luau
• 📁 myModule
  • 📄 frobulate.luau
  • 📄 init.luau

The problem this RFC grapples with is this: inside of /myModule/init.luau, what does the relative path ./frobulate.luau resolve to?

Does it:

• resolve to /myModule/frobulate.luau?
  • This makes sense from the perspective of the files on disk, but not from the module system.
  • Caveat: this makes ./ only usable for scripts with nesting.
• resolve to /frobulate.luau?
  • This makes sense from the perspective of the module system, but not from the files on disk.

This is what I think the core tension is here.

[dphblox]

And to throw my personal opinion on here:

I wouldn't instinctively choose one of these perspectives over the other, nor do I think we should. Ultimately, neither one wins: even with embedded use cases like the various game engines now using Luau (even beyond Roblox, let's not forget!), there will likely always be a need to sync modules into a file system structure. I don't think the choice of module representation should change the way the language resolves paths, because this would disrupt sync.

I think it is right and good that this is being explicitly cleaned up. I don't think this should be set in stone, because it's a very important detail to get right.

Onto the discrepancy. As I understand it, there are two common path resolution schemes (could be wrong).

Path resolution on the web:

• ../foo refers to /parent/foo
• ./foo refers to /parent/self/foo
• foo also refers to /parent/self/foo

Path resolution for filesystems:

• ../foo refers to /foo
• ./foo refers to /parent/foo
• foo also refers to /parent/foo

As best I can discern, these are different because the web has a similar notion of automatically accessing nested index.html files, meaning relative paths are always resolved starting from inside the directory, whereas filesystems don't have such a notion so they operate one level up.

In a nested module system, you would want the ability to refer to either siblings or direct descendants. This is incompatible with filesystem resolution, which only easily supports siblings, but perfectly congruent with web resolution.

As a result of this tension between module perspective and filesystem perspective, this RFC takes the opinion that the only way out is to disallow relative requires, which I can sympathise with. This ensures a completely consistent and intuitive worldview from both the filesystem and module perspectives.

However, I do think there is value in being able to refer to siblings and children. I would suggest we follow the example of the web here, and treat init.luau files like how the web treats index.html files.

Concretely, this means for some Luau module at /parent/self in the module system:

• ../foo refers to /parent/foo
• ./foo refers to /parent/self/foo
• foo also refers to /parent/self/foo

Applying this logic to the earlier example:

Inside of /myModule/init.luau, what does the relative path ./frobulate.luau resolve to?

In the module system, we are currently in /myModule. Using web resolution rules:

• ./frobulate maps to /myModule/frobulate
• frobulate maps to /myModule/frobulate
• ../frobulate maps to /frobulate

[vrn-sn]

@dphblox Could you clarify what the base path is (and the overall file tree) in these examples? I'm pretty sure that the behavior is somewhat dependent on whether the base path is a file or a directory in both of these contexts.

Path resolution on the web:

../foo refers to /parent/foo
./foo refers to /parent/self/foo
foo also refers to /parent/self/foo

Path resolution for filesystems:

../foo refers to /foo
./foo refers to /parent/foo
foo also refers to /parent/foo

[dphblox]

The base path is /parent/self/. On the filesystem, this is treated literally, but on the web, this is treated as /parent/self/index.html/, hence the difference.

---

I think I could be clearer in general about what I propose (which it has been brought to my attention bears some resemblance to #76, but maybe might be slightly different? I haven't had time to investigate).

TLDR: the target is the directory, the source lives one level down.

Let's start with a simplified file system, where everything is init.luau or a directory.

- 📁 myModule
    - 📄 init.luau
    - 📁 foo
        - 📄 init.luau
- 📁 foo
    - 📄 init.luau

This maps to the following module structure:

- 📜 myModule
    - 📜 foo
- 📜 foo

In this simplified model of the file system, it is the directory that represents the module. The init.luau is just a pragmatic concession because directories can't hold data. So, if we want to refer to a module on the file system, we must end at a directory, at least in this simplified file system.

Notably, this doesn't prescribe that you must start from a directory. You can start from wherever you like. But on the file system, the directory is the target. That's reflected in the require-by-string syntax:

require "/myModule"
require "/myModule/foo"
require "/foo"

We all agree on this as best I can tell. So far, so good.

Now, let's evolve the filesystem model, because it sucks to make everything a directory just to house a init.luau file. So we will introduce some shorthand.

This longhand:

- 📁 helloWorld
    - 📄 init.luau

Becomes this shorthand:

- 📄 helloWorld.luau

This is just a convenient syntax sugar to mean "this is a directory called helloWorld with an init.lua inside".

To reinforce what I mean, we can translate this longhand:

- 📁 myModule
    - 📄 init.luau
    - 📁 foo
        - 📄 init.luau
- 📁 foo
    - 📄 init.luau

Into this shorthand:

- 📁 myModule
    - 📄 init.luau
    - 📄 foo.luau
- 📄 foo.luau

Because foo.luau actually represents foo/init.luau, we still require it the same way.

require "/myModule"
require "/myModule/foo"
require "/foo"

Again, so far, so good.

Now, suppose we are inside of the myModule/init.luau file, and would like to require the nested foo using a relative path.

- 📁 myModule 
    - 📄 init.luau <- you are here
    - 📄 foo.luau <- you want to require this
- 📄 foo.luau

If we expand to the longhand, we can see our target:

- 📁 myModule 
    - 📄 init.luau <- you are here
    - 📁 foo <- you want to require this
        - 📄 init.luau
- 📁 foo
    - 📄 init.luau

And we require it like so:

require "./foo"
require "foo"

Similarly, you can target the outer foo:

- 📁 myModule 
    - 📄 init.luau <- you are here
    - 📄 foo.luau 
- 📄 foo.luau <- you want to require this

- 📁 myModule 
    - 📄 init.luau <- you are here
    - 📁 foo 
        - 📄 init.luau
- 📁 foo <- you want to require this
    - 📄 init.luau

require "../foo"

Now for a possible complication point. Suppose you're starting from the outer foo.luau and want to index the inner foo.luau.

- 📁 myModule 
    - 📄 init.luau
    - 📄 foo.luau <- you want to require this
- 📄 foo.luau  <- you are here

If we consider the longhand, notice how we drop down into the init.luau file for one but not the other. This ensures that every file, whether shorthand or longhand, has a consistent view of the world.

- 📁 myModule 
    - 📄 init.luau 
    - 📁 foo  <- you want to require this
        - 📄 init.luau
- 📁 foo
    - 📄 init.luau <- you are here

So, to require that directory, we must do:

require "../myModule/foo"

So, we have a complete first-principles answer for how to handle modules on the filesystem:

• Modules are targeted by directory.
• Source code lives one level down.
• Therefore, sibling modules are accessed via ../.
• Therefore, nested modules are accessed via ./ or with no prefix.

This provides easy traversal up, sideways and down, without introducing new complications on top such as library definitions or contextual aliases.

Notably, this has already been adopted in other places. To name a few examples:

• Web browsers already interpret URLs in HTML files using these rules.
  • Historically, if you access a web page like roblox.com/home, the browser would look for a roblox.com/home/index.html file.
  • In that way, HTML files could be targeted by directory.
  • The HTML source code still lives one level down.
  • Therefore, sibling web pages are accessed via ../.
  • Therefore, nested web pages are accessed via ./.
• The Rust programming language has a module system at scale, which functions identically.
  • When a statement like mod foo is written, Rust looks for a foo directory containing a mod.rs file. As shorthand, a foo.rs file can be provided, but it behaves identically to a foo directory containing a mod.rs file.
  • So, just like in my proposed system, Rust modules are targeted by directory.
  • Inside of a mod.rs file, further mod statements can be written to include nested modules.
  • This implies the mod.rs file lives one level down.
  • Sibling modules are accessed via super::, analogous to ../.
  • Nested modules are accessed with no prefix.

So, given the internal consistency of this system, plus its relation to familiar Web paradigms and the succesful-at-scale Rust module system, I feel reasonably confident in saying this is a viable solution. Plus, this solution should also cleanly map to a proper Rust-like modules system, should we ever choose to explore that path, because the traversal will be identical.

Again, TLDR: the target is the directory, the source lives one level down.

[Dekkonot]

@dphblox That does solve this issue but it is in fact essentially the same suggestion as #76. However your explanation is better.

I am still very much opposed to this RFC. I'm also not a big fan of your changes because they are both breaking changes and involves changing the prefixes used. Those were just reconfirmed in the amended syntax RFC, so it'd be really frustrating if they changed.

[dphblox]

@dphblox That does solve this issue but it is in fact essentially the same suggestion as #76. However your explanation is better.

I've been reading through. It's the same with one key difference - in the construction there, the source code doesn't live one level down, so they have to introduce a special alias to access nested modules. That could also work in theory - either approach works - but I do prefer my approach for simplicity since it does not introduce special contextual aliases.

[AxisAngles]

As a person who just learned about the ./ and ../ syntax, my immediate conclusion is that:

1. It's weird that the standard is to call the source file init.luau. Shouldn't it be the name of the module, or like, source.luau
2. If I translate my project from Roblox into my computer's filesystem, all my requires should still work.

The only logical conclusion for me is that:

1. ./ starts at the parent folder if the file's name is init.luau, and starts at the file itself otherwise.
2. ../ goes to the parent

It also seems to me that ./ is superfluous.

I don't think I really added anything, but this is just what my expectation is as an end user of Luau.

[deviaze]

One the strongest things about Luau is its backwards compatibility promise. Luau does not break backwards compatibility with (oftenly considered) unfortunate Lua design choices, does not add a keyword to the language if there is a single user who uses it as an identifier, etc.

There was an assumption that once relative-to-file string requires were implemented into Luau CLI, they would not be removed, similar to how we don't remove global variables and getfenv/setfenv.

This amendment would break compatibility with Luau Language Server, external consumers of Luau, and standalone runtimes that may or may not have anything to do with Roblox compatibility but want to maintain compatibility with mainline Luau.

The vast majority of Lune (and similar) projects, including scripts, programs such as Discord bots, webservers, etc., in the fledgeling Luau OSS ecosystem rely on relative-to-file requires as the default.

We use Luau because we really like the language and want to spread its usage as a competent general purpose programming language in its own right.

This amendment, as proposed, will further diverge Luau string require semantics from ones implemented in Luau Language Server, Lune, etc. and make it more difficult for those tools, and users of those tools, to remain, become, and/or maintain compatible with Luau require-by-string semantics.