Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for generating go module deps #205

Merged
merged 1 commit into from
Apr 29, 2024
Merged

Conversation

cpuguy83
Copy link
Member

@cpuguy83 cpuguy83 commented Apr 10, 2024

This adds a new Generate field on Source which is a list of
generator configs.
Today the only type of generator we support is for go modules but we'll
likely need to add support for other things, e.g. Rust cargo deps.

Multiple geneator configs are allowed in the case of a source with
multiple go modules defined in it.

Closes #180

@cpuguy83 cpuguy83 added this to the v0.3.0 milestone Apr 12, 2024
@cpuguy83 cpuguy83 marked this pull request as ready for review April 16, 2024 18:37
@cpuguy83 cpuguy83 requested a review from a team as a code owner April 16, 2024 18:37
@cpuguy83 cpuguy83 force-pushed the gomods branch 2 times, most recently from 0c63b4d to 4ff4445 Compare April 16, 2024 18:41
@cpuguy83 cpuguy83 changed the title Add gomod source type Add support for generating go module deps Apr 16, 2024
@cpuguy83
Copy link
Member Author

I've updated this with a slightly different implementation that is more extensible and easier to discover.
Also marked this ready for review.

generator_gomod.go Outdated Show resolved Hide resolved

opts := append(opts, ProgressGroup("Fetch go module dependencies for source: "+key))
hasGomod = true
withMods, err := WithModuleDeps(worker, sOpt, &src, opts...)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit confused on how this works, so this whole PR would allow for any specified source to have in this case a gomod generator. So for a scenario that we have at least 2 sources that have gomod gen. When we retrieve these gomods do these get merged into a single directory and state for all the gomods for the different sources? or each gomod would have it's own state? The other relevant section for this question is when preparing the sources in the rpm template: frontend/rpm/template.go

	prepareGomods := sync.OnceFunc(func() {
		if !w.Spec.HasGomods() {
			return
		}
		fmt.Fprintf(b, "mkdir -p \"%%{_builddir}/%s\"\n", gomodsName)
		fmt.Fprintf(b, "tar -C \"%%{_builddir}/%s\" -xzf \"%%{_sourcedir}/%s.tar.gz\"\n", gomodsName, gomodsName)
	})

the gomodsName is a constant so it doesn't make sense to do this more than once if the gomod modules are merged into one for all sources.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we retrieve these gomods do these get merged into a single directory and state for all the gomods for the different sources?

Yes, that is what this function is doing.
The modules are stored in the state referenced by the deps variable, which is getting overwritten below (L90) with deps = deps.With(withMods), where withMods is an llb.StateOption that modifies the state to include the deps for an individual source.

The trick there is in the withGomod function where it is using return value of llb.Run(...).AddMount(gomodCachedir, in).
Here the return value of AddMount is the the content of the mount point after the Run has executed.

@cpuguy83
Copy link
Member Author

Updated this, also made sure to patch any go mod sources before fetching deps in case go.mod or go.sum are being patched.

@cpuguy83 cpuguy83 force-pushed the gomods branch 5 times, most recently from a6eebd0 to 1f0aa3c Compare April 19, 2024 17:23
This adds a new `Generate` field on `Source` which is a list of
generator configs.
Today the only type of generator we support is for go modules but we'll
likely need to add support for other things, e.g. Rust cargo deps.

Multiple geneator configs are allowed in the case of a source with
multiple go modules defined in it.

Signed-off-by: Brian Goff <[email protected]>
Copy link
Contributor

@adamperlin adamperlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good! It would be interesting to see how this could possibly be slotted into an interface as described in #213

@cpuguy83 cpuguy83 merged commit 7c7f4e8 into Azure:main Apr 29, 2024
10 checks passed
@cpuguy83 cpuguy83 deleted the gomods branch April 29, 2024 22:45
@cpuguy83 cpuguy83 mentioned this pull request May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[REQ] Built-in Support for golang sources
3 participants