Make workspace member inclusion/exclusion more useful, consistent, and intuitive

[weihanglo]

Problem

As of now, Cargo supports glob syntax for workspace.member to include crates as workspace members. However, for workspace.exclude it is implemented by a plain starts_with comparing with manifest path of certain crates. This confuses people as they need to figure out which rules overrides the other. It is also inconsistent with package.include and package.exclude, as they choose gitignore syntax.

For workspace.member, it proactively checks the existence of Cargo.toml to include a member, and throws an errors if Cargo.toml doesn't exist, which makes globbing **/* often invalid.

Similar issues have been raised from time to time:

• Functionality to glob all crates in workspace when mixed with other content #4593
• Workspace exclude doesn't work if nested under a member path #6745
• cargo should support glob syntax on workspace excludes #6009
• For member under [workspace], Glob pattern doesn’t work for paths whose parent is excluded #11362

Proposed Solution

There are several possible ways to "fix" the situation (to some extent).

• The first is implementing glob support for workspace.exclude. Add globbing to workspace.exclude list #11374 tries to do that. This is a relatively low effort approach but might confuse people still, and it won't fix Workspace exclude doesn't work if nested under a member path #6745.
• The other way is using gitignore syntax, as ehuss mentioned earlier cargo should support glob syntax on workspace excludes #6009 (comment). This would be a breaking change and might need a transition period and provide migration steps.
• In For member under [workspace], Glob pattern doesn’t work for paths whose parent is excluded #11362 (comment), we talked about using a single workspace.member and drop workspace.exclude. However, the re-include is not possible due to the limit of gitignore itself¹.
• We might want to avoid starts_with and instead check the existence of Cargo.toml to exclude a package.

There might be other possible solution I am not aware of here.

Notes

I would keep this as the main issue and close some. If this is inappropriate, please say it out load, and we can reopen the original issues.

Footnotes

1. An optional prefix "!" which negates the pattern; any matching file excluded by a previous pattern will become included again. It is not possible to re-include a file if a parent directory of that file is excluded.
   See https://git-scm.com/docs/gitignore#_pattern_format ↩

[Rustin170506]

I would like to find a solution to this issue.
@rustbot claim