Support for running tests using bazel

[rajneesh2k10]

Bazel Ref: https://bazel.build/versions/6.3.0/run/build

Thank you for building this adapter.

My project uses Bazel for building and running tests in the project. I am looking at neotest-golang adapter if it can provide me the ability to specify the test runner commands for test execution.
Skimming through the neotest-golang plugin code, I came across code that attempts to find the test runner executable. However, in the configuration (Readme), I didn't see any option to provide a custom executable name.

I am wondering if it's possible to use a different runner out-of-the-box? Or, will I need to extend this plugin? Or, is the plugin not build for extending to new runners?

If I can extend the plugin for new runners, any pointers will be helpful.

Thank you!

[fredrikaverpil]

Hi @rajneesh2k10, I haven't used Bazel myself so I am not aware of how it works - but I'd be happy to have support in neotest-golang for it. There's no out-of-the-box or formal support for extending the adapter with test runners, so we would have to extend the adapter a little bit.

Just like you have noticed, I'm actually using gotestsum (and not pure go test) as the test runner myself, but I haven't fully documented this yet, kind of waiting for something like your inquiry to occur (so to better be able to factor in how "runners" should work and be configured). There's an open PR which outlines how it works currently: #150

I like your idea of hooking into the neotest-golang code with your custom test runner executable with potentially custom logic for the test command/args etc. Such efforts could perhaps also cater for the request/question in #186 which talks about extending functionality of how to provide custom arguments to go test (going beyond "static" logic" defined on neovim startup).

Some pointers on getting started

I would personally like to make test runners a "first class citizen" in the codebase. Meaning; refactoring a little bit so that if-statements are not scattered around in the code like; if runner == "go" then ... else .... But first we can just raw-dog something up just to see what hurdles we encounter in the current codebase.

I would suggest adding new "elseif"-clauses where you can currently see how I am checking whether you intend to run go or goteststum as test runner and add your own one in.
I would recommend you focus on just running a single test (runspect/test.lua) and then modify the code in cmd.lua.

When it comes to test command output, I currently have one unified way of expecting go test JSON output which is then processed in process.lua. This is by far the most complex part of the adapter. It would be great if this part did not have to be modified. Do you think you can achieve go test JSON output from your custom test runner command?

I would also recommend you enable debug logging, so that you can more easily inspect the test commands produced etc.

Finally, we need to write the final code so it's testable. Since I'm not using bazel myself, I want to make sure I'm not going to break this functionality for someone without noticing. But that can come later once we have a proof of concept.

Let me know if you need any additional help or if you get stuck - and I'll start thinking of how we can restructure the code a bit so to make test runners more of a "first class citizen" feature. 😄

EDIT: hmmm... it looks like Bazel indeed does not output the same way go test -json does. This is really unfortunate, as then this will not be a trivial change.

[fredrikaverpil]

Great input. We could break up these arguments which are normally passed to go test, so that you receive them not as just a table but as a structured key/value table. Then you can much more easily extract what parts of it you need and insert it into your custom test command.

[fredrikaverpil]

I just added a commit to #189 where I now pass in a cmd_data table (which I would like to make more strictly typed). I really whipped this up very quickly and haven't had time to test it out really, but I think this is going in a really nice direction.

opts = {
  runner = "bazel",
  runners = {
    bazel = {
      cmd = function(cmd_data)
        local bazel_cmd = {}
        local output_filepath = nil
        local absolute_folder_path = vim.fn.fnamemodify(cmd_data.position.path, ":h")
        if cmd_data.position.type == "test" and cmd_data.regexp ~= nil then
          bazel_cmd = {"bazel", "test", absolute_folder_path .. ":test_target", "--test_filter=" .. cmd_data.regexp}
        elseif cmd_data.position.type == "test" then
          bazel_cmd = {"bazel", "test", absolute_folder_path .. ":test_target"}
        else
          -- more cases...
        end
        return bazel_cmd, output_filepath
      end,
    },
  },
}

What is this test_target and is it something the adapter could provide in cmd_data?

To make this more clear, these are examples of the current potential values of cmd_data:

-- running a single test
{
  package_name = "github.com/fredrikaverpil/neotest-golang",
  position = {
    id = "/Users/fredrik/code/public/neotest-golang/tests/go/positions_test.go::TestSubTestTableTestInlineStruct",
    name = "TestSubTestTableTestInlineStruct",
    path = "/Users/fredrik/code/public/neotest-golang/tests/go/positions_test.go",
    range = { 99, 0, 119, 1 },
    type = "test"
  },
  regexp = "^TestSubTestTableTestInlineStruct$"
}

-- running all tests in a file
{
  package_name = "github.com/fredrikaverpil/neotest-golang",
  position = {
    id = "/Users/fredrik/code/public/neotest-golang/tests/go/positions_test.go",
    name = "positions_test.go",
    path = "/Users/fredrik/code/public/neotest-golang/tests/go/positions_test.go",
    range = { 0, 0, 180, 0 },
    type = "file"
  },
  regexp = "^(^TestMain$|^TestTopLevel$|^TestTopLevelWithSubTest$|^TestTableTestStruct$|^TestSubTestTableTestStruct$|^TestTableTestInlineStruct$|^TestSubTestTableTestInlineStruct$|^TestTableTestInlineStructLoop$|^TestSubTestTableTestInlineStructLoop$|^TestTableTestMap$)$"
}

-- running the top-level folder (same as where the go.mod resides)
{
  package_name = "./...",
  position = {
    id = "/Users/fredrik/code/public/neotest-golang/tests/go",
    name = "go",
    path = "/Users/fredrik/code/public/neotest-golang/tests/go",
    type = "dir"
  }
}

-- running a folder (which is not the one containing the go.mod)
{
  package_name = "github.com/fredrikaverpil/neotest-golang/testify",
  position = {
    id = "/Users/fredrik/code/public/neotest-golang/tests/go/testify",
    name = "testify",
    path = "/Users/fredrik/code/public/neotest-golang/tests/go/testify",
    type = "dir"
  }
}

I could also add here the full output of go list ./..., in case that's useful.

[fredrikaverpil]

Headsup that I modified the above examples as I added a few more refactorings.

[rajneesh2k10]

Thank you for making it so quick.

What is this test_target and is it something the adapter could provide in cmd_data?

Bazel creates a build script file that contains the recipe for building a module, running all tests in a module, etc. You can imagine this script file as "build.xml" file in java that contains ANT scripts for various build and test targets. With Bazel, every go module will have a BUILD.bazel that will contain a list of targets (for building the module, running tests, etc.). When you invoke a test, you specify the test target definition in the BUILD.bazel file. Though you can define the test and build targets with any name, by convention we define it as go_default_library for building the module and go_default_test for running all the tests in the module. You can have more targets.

So, we cannot hardcode the the target names in the adapter code. We can make it configurable with a "conventionally accepted" names but it won't be fool proof. But, it will still offer a good control to the users. Ideal would be - to read the BUILD.bazel file and interpret which target is running the whole test suite (let's say) and then run that with the appropriate filter for different subcommands (like running a single test, or file, etc.).

As we dig more, we find extending the adapter to Bazel is not as simple as we thought - and deserves a good amount of code dedicated to itself. :)

[fredrikaverpil]

So, we cannot hardcode the the target names in the adapter code.

I'm pretty sure that you have to bring your own logic to support bazel, and use that to fully populate the test command. The adapter can only supply information to you (writing the custom test runner logic) such as test name, go package, filepaths etc to use.

Anyway, maybe in the case of bazel, you may want to prefer using https://github.com/sluongng/neotest-bazel but I'll continue ironing out the ability to more easily extend neotest-golang with additional test runners.

[fredrikaverpil]

@rajneesh2k10 I just noticed there's a new neotest-bazel project: https://github.com/sluongng/neotest-bazel 😄

Regardless, I'll continue to refactor neotest-golang so to support custom runners which would like to provide their own test command and output parsing.

[rajneesh2k10]

Wow! I would love to give that a spin too - given it has support for Go (and other languages planned for future).
Thanks for sharing!