diff --git a/docs/plugins/debugging-go-plugins.md b/docs/plugins/debugging-go-plugins.md index 0d0d8acfdc0..07372ef2bf4 100644 --- a/docs/plugins/debugging-go-plugins.md +++ b/docs/plugins/debugging-go-plugins.md @@ -9,30 +9,30 @@ description: Debugging guide for Go plugins date: "2024-10-11" --- -Plugins are native go code compiled to a binary shared object file. The code may depend on CGO and require libraries like libc provided by the runtime environment. The following are some debugging steps for diagnosing issues arising from using plugins. +Plugins are native Go code compiled to a binary shared object file. The code may depend on `cgo` and require libraries like `libc` provided by the runtime environment. The following are some debugging steps for diagnosing issues arising from using plugins. ## Warnings -The [plugin package - Warnings](https://pkg.go.dev/plugin#hdr-Warnings) section outlines several requirements which can't be ignored. The most important restriction is the following: +The [Plugin package - Warnings](https://pkg.go.dev/plugin#hdr-Warnings) section in the Go documentation outlines several requirements which can't be ignored when working with plugins. The most important restriction is the following: > Runtime crashes are likely to occur unless all parts of the program (the application and all its plugins) are compiled using exactly the same version of the toolchain, the same build tags, and the same values of certain flags and environment variables. -We provide a Plugin Compiler docker image, which should be used to build plugins compatible with the official gateway releases and their architectures. It provides the cross compilation toolchain, Go version used to build the release, and ensure compatible flags are used when compiling plugins, like `-trimpath`, `CC`, `CGO_ENABLED`, `GOOS`, `GOARCH`. +We provide the *Tyk Plugin Compiler* docker image, which we strongly recommend is used to build plugins compatible with the official Gateway releases. This tool provides the cross compilation toolchain, Go version used to build the release, and ensures that compatible flags are used when compiling plugins, like `-trimpath`, `CC`, `CGO_ENABLED`, `GOOS`, `GOARCH`. -The plugin compiler also works around known Go issues. +The *Plugin Compiler* also works around known Go issues such as: - https://github.com/golang/go/issues/19004 - https://www.reddit.com/r/golang/comments/qxghjv/plugin_already_loaded_when_a_plugin_is_loaded/ -The argument plugin_id ensures the same plugin can be rebuilt. The plugin compiler does this by replacing the plugin go.mod module path. +Supplying the argument `build_id` to the *Plugin Compiler* ensures the same plugin can be rebuilt. The *Plugin Compiler* does this by replacing the plugin `go.mod` module path. -Continue with [Go Plugin Compiler](https://tyk.io/docs/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler/). +Continue with [Tyk Plugin Compiler](https://tyk.io/docs/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler/). ### Examples -When working with Go plugins, it's easy to miss the restriction that the plugin at the very least requires to be built with the same Go version, and the same flags, notably `-trimpath`, which is part of the Gateway official release. +When working with Go plugins, it's easy to miss the restriction that the plugin at the very least must be built with the same Go version, and the same flags (notably `-trimpath`) as the Tyk Gateway on which it is to be used. -If you miss an argument like forgetting `-trimpath` for the plugin build, you'll get a load error like the one below. Usually when the error hints at a standard library package, the build flags between the binaries don't match. For example, if gateway is compiled with `-race`, the plugin needs to be compiled with the flag as well to be compatible. +If you miss an argument (for example `-trimpath`) when building the plugin, the Gateway will report an error when your API attempts to load the plugin, for example: ``` task: [test] cd tyk-release-5.3.6 && go build -tags=goplugin -trimpath . @@ -41,53 +41,55 @@ task: [test] ./tyk-release-5.3.6/tyk plugin load -f plugins/testplugin.so -s Aut tyk: error: unexpected error: plugin.Open("plugins/testplugin"): plugin was built with a different version of package internal/goarch, try --help ``` -Other error messages may occur, depending on what triggered the issue. For example, if you omitted `-race` in the plugin but the gateway was built with `-race`, the error reported is: +Usually when the error hints at a standard library package, the build flags between the Gateway and plugin binaries don't match. + +Other error messages may be reported, depending on what triggered the issue. For example, if you omitted `-race` in the plugin but the gateway was built with `-race`, the following error will be reported: ``` plugin was built with a different version of package runtime/internal/sys, try --help ``` -Stricly speaking: +Strictly speaking: -- build flags like `-trimpath`, `-race` need to match -- go toolchain / build env needs to be exactly the same -- cross compilation means using the same `CC` value for the build (CGO) -- matching `CGO_ENABLED=1`, `GOOS`, `GOARCH` with runtime +- Build flags like `-trimpath`, `-race` need to match. +- Go toolchain / build env needs to be exactly the same. +- For cross compilation you must use the same `CC` value for the build (CGO). +- `CGO_ENABLED=1`, `GOOS`, `GOARCH` must match with runtime. -When something is off, proofing these can be done with `go version -m tyk` and `go version -m plugin.so` for the plugin, inspecting and comparing the output of `build` tokens usually yields the difference that caused the compatibility issue. +When something is off, you can check what is different by using the `go version -m` command for the Gateway (`go version -m tyk`) and plugin (`go version -m plugin.so`). Inspecting and comparing the output of `build` tokens usually yields the difference that caused the compatibility issue. ## Plugin compatibility issues This is a short list of cases when dependencies may be causing problems. -- A gateway dependency does not have a go.mod and plugin wants to use it, -- A gateway dependency has a shared dependency, same version must be used, -- A plugin wants to use a different dependency version +- A Gateway dependency does not have a `go.mod` and the plugin wants to use it. +- Gateway and plugin have a shared dependency: the same version must be used by the plugin. +- A plugin wants to use a different dependency version. The cases need to be expanded, but the process for each is: -Case 1: +**Case 1:** -- Plugin uses gateway as a dependency but wants to use A -- A does not have a go.mod, so a pseudo version is generated on both ends of the build -- Expect: build success, error when loading plugin due to a version mismatch +- Plugin uses Gateway as a dependency but wants to use *A* +- *A* does not have a `go.mod`, so a pseudo version is generated on both ends of the build +- Result: build success, error when loading plugin due to a version mismatch -Fix: update to remove dependency A, or use a version with go.mod; +Fix: update to remove dependency *A*, or use a version with `go.mod` -Case 2: +**Case 2:** -- Plugin uses gateway as a dependency and wants to use a shared dependency -- As the dependency has go.mod, the version matches -- Dependency is promoted to direct in go.mod -- Expect: user has to keep dependency in sync with gateway +- Plugin uses Gateway as a dependency and wants to use a shared dependency +- As the dependency has `go.mod`, the version matches +- Dependency is promoted to *direct* in `go.mod` +- Expect: you have to keep the dependency in sync with Gateway -Case 3: +**Case 3:** -- Plugin uses gateway as a dependency but wants to use a different version of a dependency +- Plugin uses Gateway as a dependency but wants to use a different version of a shared dependency - It's likely using a major release with `/v4` or similar works like a charm (new package) - Expectation: If it's just a different version of the same package, loading the plugin will fail -It's definitely recommended that all dependencies would follow go package metaversion, however the reality is most gateway dependencies follow a basic v1 semver which doesn't break import paths for every release. +We recommend that all dependencies should follow Go package metaversion, however the reality is most Gateway dependencies follow a basic v1 semver which doesn't break import paths for every release. ## List plugin symbols @@ -112,9 +114,9 @@ Sometimes it's useful to list symbols from a plugin. For example, we can list th 0000000001310c40 T testplugin.MakeOutboundCall.deferwrap1 ``` -The command prints other symbols that are part of the binary. In the worst case, a build compatibility issue may cause a crash in the gateway due to an unrecoverable error and this can be used to further debug the binaries produced. +This command prints other symbols that are part of the binary. In the worst case, a build compatibility issue may cause a crash in the Gateway due to an unrecoverable error and this can be used to further debug the binaries produced. -A very basic check to ensure gateway/plugin compatibility is using the built in `go version -m `: +A very basic check to ensure Gateway/plugin compatibility is using the built in `go version -m `: ``` [output truncated] @@ -133,4 +135,4 @@ A very basic check to ensure gateway/plugin compatibility is using the built in build vcs.modified=false ``` -It's typical that these options should match between the gateway binary and the plugin, and you can use the command for both binaries. +These options should match between the Gateway binary and the plugin. You can use the command for both binaries and then compare the outputs. diff --git a/docs/plugins/go-development-flow.md b/docs/plugins/go-development-flow.md index ec45ac6a57d..9893ce7059c 100644 --- a/docs/plugins/go-development-flow.md +++ b/docs/plugins/go-development-flow.md @@ -9,105 +9,136 @@ description: Development flow working with Go Plugins date: "2024-10-11" --- -For effectively developing go plugins, familiarize yourself with the following: +We recommend that you familiarize yourself with the following official Go documentation to help you work effectively with Go plugins: - [The official plugin package documentation - Warnings](https://pkg.go.dev/plugin) - [Tutorial: Getting started with multi-module workspaces](https://go.dev/doc/tutorial/workspaces) -Plugins need to be compiled to native shared object code, which are then loaded by Gateway. For best results, knowing the restrictions of plugins is recommended. For maximum compatibility, Go workspaces are recommended, as shown below. +{{< note success >}} +**Note** + +Plugins are currently supported only on Linux, FreeBSD, and macOS, making them unsuitable for applications intended to be portable. +{{< /note >}} + +Plugins need to be compiled to native shared object code, which can then be loaded by Tyk Gateway. It's important to understand the need for plugins to be compiled using exactly the same environment and [build flags]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-development-flow#build-flags" >}}) as the Gateway. To simplify this and minimise the risk of compatibility problems, we recommend the use of [Go workspaces](https://go.dev/blog/get-familiar-with-workspaces), to provide a consistent environment. ## Setting up your environment To develop plugins, you'll need: -- Go based on the gateway version (from go.mod) -- Git to check out gateway source code -- A folder with plugins to build +- Go (matching the version used in the Gateway, which you can determine using `go.mod`). +- Git to check out Tyk Gateway source code. +- A folder with the code that you want to build into plugins. -Set up a workspace, which, at the end, is going to look like: +We recommend that you set up a *Go workspace*, which, at the end, is going to contain: -- `/tyk-release-5.3.6` - the checkout +- `/tyk-release-x.y.z` - the Tyk Gateway source code - `/plugins` - the plugins -- `/go.work` - the go workspace file -- `/go.work.sum` - workspace package checksums +- `/go.work` - the *Go workspace* file +- `/go.work.sum` - *Go workspace* package checksums -Using the workspace ensures build compatibility, matching plugin restrictions. +Using the *Go workspace* ensures build compatibility between the plugins and Gateway. -### 1. Checking out tyk +### 1. Checking out Tyk Gateway source code ``` -# git clone --branch release-5.3.6 https://github.com/TykTechnologies/tyk.git tyk-release-5.3.6 || true +git clone --branch release-5.3.6 https://github.com/TykTechnologies/tyk.git tyk-release-5.3.6 || true ``` -The example checkout uses a particular `release-5.3.6` branch, to match a release. With newer `git` version, you may pass `--branch v5.3.6` and it would use the tag. In case you want to use the tag it's also possible to navigate into the folder and issue `git checkout tags/v5.3.6`. +This example uses a particular `release-5.3.6` branch, to match Tyk Gateway release 5.3.6. With newer `git` versions, you may pass `--branch v5.3.6` and it would use the tag. In case you want to use the tag it's also possible to navigate into the folder and issue `git checkout tags/v5.3.6`. -### 2. Preparing the workspace - plugins +### 2. Preparing the Go workspace -The plugin workspace can be very simple. Generally you would: +Your Go workspace can be very simple: -1. create a .go file with code for your plugin -2. create a go.mod for the plugin -3. ensure the correct go version is in use -4. add gateway dependency with `go get`, using the commit hash +1. Create a `.go` file containing the code for your plugin. +2. Create a `go.mod` file for the plugin. +3. Ensure the correct Go version is in use. -For implementation examples, see [CustomGoPlugin.go](https://github.com/TykTechnologies/custom-go-plugin/blob/master/go/src/CustomGoPlugin.go). We'll be using this as the source for our plugin as shown: +As an example, we can use the [CustomGoPlugin.go](https://github.com/TykTechnologies/custom-go-plugin/blob/master/go/src/CustomGoPlugin.go) sample as the source for our plugin as shown: ``` -# mkdir -p plugins -# rm -f go.mod go.sum -# go mod init testplugin -go: creating new go.mod: module testplugin -# go mod edit -go "1.22.6" -# wget -q https://raw.githubusercontent.com/TykTechnologies/custom-go-plugin/refs/heads/master/go/src/CustomGoPlugin.go +mkdir -p plugins +cd plugins +go mod init testplugin +go mod edit -go $(go mod edit -json go.mod | jq -r .Go) +wget -q https://raw.githubusercontent.com/TykTechnologies/custom-go-plugin/refs/heads/master/go/src/CustomGoPlugin.go +cd - ``` -The following snippets provide you with a way to: +The following snippet provides you with a way to get the exact Go version used by Gateway from it's [go.mod](https://github.com/TykTechnologies/tyk/blob/release-5.3.6/go.mod#L3) file: -- `go mod edit -json go.mod | jq -r .Go` - get the go version from the gateway [go.mod](https://github.com/TykTechnologies/tyk/blob/release-5.3.6/go.mod#L3) file -- `git rev-parse HEAD` - get the commit hash so the exact commit can be used with `go get` +- `go mod edit -json go.mod | jq -r .Go` (e.g. `1.22.7`) -This should be used to ensure the matching between gateway and the plugin. The commit is used to `go get` the dependency in later steps. +This should be used to ensure the version matches between gateway and the plugin. -The internal `workspace:plugins` step ensures a few things: +To summarize what was done: -1. create a plugin, create go.mod, -2. set go.mod go version to what's set in gateway, -3. have some code to compile in the folder +1. We created a plugins folder and initialzed a `go` project using `go mod` command. +2. Set the Go version of `go.mod` to match the one set in the Gateway. +3. Initialzied the project with sample plugin `go` code. -At this point, we don't have a workspace yet. In order to share the gateway dependency across go modules we'll create a Go workspace next. +At this point, we don't have a *Go workspace* but we will create one next so that we can effectively share the Gateway dependency across Go modules. ### 3. Creating the Go workspace +From the folder that contains the Gateway checkout and the plugins folder, you will update `go.mod` with the Gateway commit hash that corresponds to the checkout and then run `go mod tidy`, creating the `go.work` workspace file. + +Follow these commands: + ``` -# go work init ./tyk-release-5.3.6 -# go work use ./plugins -# cd plugins && go get github.com/TykTechnologies/tyk@c808608b9a3c44b2ef0e060f8d3f3d2269582a1c +go work init ./tyk-release-5.3.6 +go work use ./plugins +commit_hash=$(cd tyk-release-5.3.6 && git rev-parse HEAD) +cd plugins && go get github.com/TykTechnologies/tyk@${commit_hash} && go mod tidy && cd - ``` -These are the final steps on how to create the workspace. The last step is used to update go.mod with the gateway commit corresponding to the checkout. After this step you're able to use `go mod tidy` in the plugins folder. +The following snippet provides you to get the commit hash exactly, so it can be used with `go get`. + +- `git rev-parse HEAD` -### 4. Testing out the plugin +The Go workspace file (`go.work`) should look like this: ``` -# cd tyk-release-5.3.6 && go build -tags=goplugin -trimpath -race . -# cd plugins && go build -trimpath -race -buildmode=plugin . +go 1.22.7 + +use ( + ./plugins + ./tyk-release-5.3.6 +) ``` -The above few steps build gateway, and the plugin. +### 4. Building and validating the plugin + +Now that your *Go workspace* is ready, you can build your plugin as follows: ``` -# ./tyk-release-5.3.6/tyk plugin load -f plugins/testplugin.so -s AuthCheck -time="Oct 01 21:29:24" level=info msg="--- Go custom plugin init success! ---- " -[file=plugins/testplugin.so, symbol=AuthCheck] loaded ok, got 0x7fdcd650f980 +cd tyk-release-5.3.6 && go build -tags=goplugin -trimpath . && cd - +cd plugins && go build -trimpath -buildmode=plugin . && cd - ``` -We can use the built gateway binary to test plugin loading without invoking the plugin symbol like a request would. However, as demonstrated, the plugins `init` function is invoked, printing to the log. +These steps build both the Gateway and the plugin. + +You can use the Gateway binary that you just built to test that your new plugin loads into the Gateway without having to configure and then make a request to an API using this command: + +``` +./tyk-release-5.3.6/tyk plugin load -f plugins/testplugin.so -s AuthCheck +``` + +You should see an output similar to: + +``` +time="Oct 14 13:39:55" level=info msg="--- Go custom plugin init success! ---- " +[file=plugins/testplugin.so, symbol=AuthCheck] loaded ok, got 0x76e1aeb52140 +``` + +The log shows that the plugin has correctly loaded into the Gateway and that its `init` function has been successfully invoked. ### 5. Summary -We've put together an end-to-end build environment for both the gateway and the plugin. However, runtime environments have additional restrictions which the plugin developer must pay attention to. +In the preceding steps we have put together an end-to-end build environment for both the Gateway and the plugin. Bear in mind that runtime environments may have additional restrictions beyond Go version and build flags to which the plugin developer must pay attention. -Compatibility in general is a big concern around plugins; since the plugins are tightly coupled to gateway, they need to be built with some consideration to the restrictions around them. +Compatibility in general is a big concern when working with Go plugins: as the plugins are tightly coupled to the Gateway, consideration must always be made for the build restrictions enforced by environment and configuration options. Continue with [Loading Go Plugins into Tyk](https://tyk.io/docs/product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins/). @@ -115,15 +146,18 @@ Continue with [Loading Go Plugins into Tyk](https://tyk.io/docs/product-stack/ty ### Plugin compiler -We provide a plugin build environment to manage compatibility restrictions for plugins. The plugin compiler ensures compatibility between the system architecture and the go version for your target environment. +We provide a plugin build environment to manage compatibility restrictions for plugins. + +- Ensures compatibility of the system architecture and an official Tyk release +- Provides a cross-compilation environment to build plugins for your target architecture -The plugin compiler also provides cross-compilation support. +It's recommended to use the Plugin Compiler to build plugins for an official release. Continue with [Go Plugin Compiler](https://tyk.io/docs/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler/). -### Build flag restrictions +### Build flags -It's a requirement that build flags for gateway match build flags for the plugin. One such flag that need to be included in builds is `-trimpath`: +It's a requirement that build flags for plugins match the build flags for the Gateway. One such flag that must be included in builds is `-trimpath`: > `-trimpath` - remove all file system paths from the resulting executable. > @@ -131,4 +165,4 @@ It's a requirement that build flags for gateway match build flags for the plugin The Gateway uses `-trimpath` to clear local build environment details from the binary, and it must in turn be used for the plugin build as well. The use of the flag increases compatibility for plugins. -For more detailed restrictions, please see [Debugging Go Plugins](debugging-go-plugins.md). +For more detailed restrictions, please see [Debugging Go Plugins]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/debugging-go-plugins" >}}).