How should I handle generating for "remote" Protobuf definitions?

[doriable]

A common workflow is to pull Protobuf definitions from external repositories, either as a Git submodule or part of some automated workflow, and then generate code based on a combination of local and "remote" Protobuf definitions. This is currently difficult to do with Buf, especially if dependencies are involved (for example, see #1103). How should these use cases be handled?

[doriable]

If the external Protobuf definitions do not have any dependencies, the most simple solution is to use the --config flag and target the external Protobuf definitions:

Example:

$ buf generate https://github.com/<external_owner>/<external_repo>.git#subdir=proto --config '{"version": "v1"}'

If the external Protobuf definitions do have dependencies, then we'll need to use a workaround involving Buf Workspaces.

In your buf.work.yaml, add your external Protobuf definition target and their dependencies. As an example, the let's say you're adding external Protobuf definitions that have protoc-gen-validate and googleapis as dependencies:

version: v1
directories:
  - <external_repo_dir>/proto # your git sub module
  - vendor/protoc-gen-validate
  - vendor/googleapis

Then you can use buf export to pull the most updated versions from the BSR into the directories defined in your workspace:

buf export buf.build/envoyproxy/protoc-gen-validate -o vendor/protoc-gen-validate
buf export buf.build/googleapis/googleapis -o vendor/googleapis