feat: add jobs option (#861)

* feat: add recursive 'actions' option * fix: inherit glob and root from action options * fix: implement merging named actions and groups * chore: add unit tests * chore: add unit tests * chore: rename actions to jobs * fix: rename to run jobs * chore: fix integrity test * fix: rename actions to jobs * fix: add integrity tests and fix some odd merging * docs: add docs about jobs * fix: replace panics with readable errors
evilmartians · Dec 19, 2024 · 73d28b7 · 73d28b7
1 parent 5b620aa
commit 73d28b7
Show file tree

Hide file tree

Showing 39 changed files with 1,435 additions and 428 deletions.
diff --git a/.golangci.yml b/.golangci.yml
@@ -18,6 +18,13 @@ linters-settings:
     rules:
       - name: unused-parameter
         disabled: true
+  unused:
+    field-writes-are-uses: false
+    post-statements-are-reads: true
+    exported-fields-are-used: false
+    parameters-are-used: true
+    local-variables-are-used: false
+    generated-is-used: false
 
 issues:
   exclude:

diff --git a/docs/mdbook/SUMMARY.md b/docs/mdbook/SUMMARY.md
@@ -50,6 +50,28 @@
     - [`exclude_tags`](./configuration/exclude_tags.md)
     - [`skip`](./configuration/skip.md)
     - [`only`](./configuration/only.md)
+    - [`jobs`](./configuration/jobs.md)
+      - [`name`](./configuration/name.md)
+      - [`run`](./configuration/run.md)
+      - [`script`](./configuration/script.md)
+      - [`runner`](./configuration/runner.md)
+      - [`group`](./configuration/group.md)
+        - [`parallel`](./configuration/parallel.md)
+        - [`piped`](./configuration/piped.md)
+        - [`jobs`](./configuration/jobs.md)
+      - [`skip`](./configuration/skip.md)
+      - [`only`](./configuration/only.md)
+      - [`tags`](./configuration/tags.md)
+      - [`glob`](./configuration/glob.md)
+      - [`files`](./configuration/files.md)
+      - [`file_types`](./configuration/file_types.md)
+      - [`env`](./configuration/env.md)
+      - [`root`](./configuration/root.md)
+      - [`exclude`](./configuration/exclude.md)
+      - [`fail_text`](./configuration/fail_text.md)
+      - [`stage_fixed`](./configuration/stage_fixed.md)
+      - [`interactive`](./configuration/interactive.md)
+      - [`use_stdin`](./configuration/use_stdin.md)
     - [`commands`](./configuration/Commands.md)
       - [`run`](./configuration/run.md)
       - [`skip`](./configuration/skip.md)

diff --git a/docs/mdbook/configuration/Scripts.md b/docs/mdbook/configuration/Scripts.md
@@ -16,7 +16,20 @@ To add a script for a `pre-commit` hook:
          runner: bash
    ```
 
-**Example**
+### Script options
+
+- [`runner`](./runner.md)
+- [`skip`](./skip.md)
+- [`only`](./only.md)
+- [`tags`](./tags.md)
+- [`env`](./env.md)
+- [`fail_text`](./fail_text.md)
+- [`stage_fixed`](./stage_fixed.md)
+- [`interactive`](./interactive.md)
+- [`use_stdin`](./use_stdin.md)
+- [`priority`](./priority.md)
+
+### Example
 
 Let's create a bash script to check commit templates `.lefthook/commit-msg/template_checker`:
 
@@ -43,16 +56,3 @@ commit-msg:
 ```
 
 When you try to commit `git commit -m "bad commit text"` script `template_checker` will be executed. Since commit text doesn't match the described pattern the commit process will be interrupted.
-
-### Script options
-
-- [`runner`](./runner.md)
-- [`skip`](./skip.md)
-- [`only`](./only.md)
-- [`tags`](./tags.md)
-- [`env`](./env.md)
-- [`fail_text`](./fail_text.md)
-- [`stage_fixed`](./stage_fixed.md)
-- [`interactive`](./interactive.md)
-- [`use_stdin`](./use_stdin.md)
-- [`priority`](./priority.md)
diff --git a/docs/mdbook/configuration/group.md b/docs/mdbook/configuration/group.md
@@ -0,0 +1,30 @@
+## `group`
+
+Specifies a group of jobs and option to run them with.
+
+- [`parallel`](./parallel.md)
+- [`piped`](./piped.md)
+- [`jobs`](./jobs.md)
+
+### Example
+
+```yml
+# lefthook.yml
+
+pre-commit:
+  jobs:
+    - group:
+        parallel: true
+        jobs:
+          - run: echo hello from a group
+```
+
+> **Note:** To make a group mergeable with settings defined in local config or extends you have to specify the name of the job group belongs to:
+> ```yml
+> pre-commit:
+>   jobs:
+>     - name: a name of a group
+>       group:
+>         jobs:
+>           - run: echo from a group job
+> ```
diff --git a/docs/mdbook/configuration/jobs.md b/docs/mdbook/configuration/jobs.md
@@ -0,0 +1,72 @@
+## `jobs`
+
+Job can either be a command or a script. Configuring `jobs` is more flexible than configuring `commands` and `scripts`, although all options are supported now.
+
+```yml
+# lefthook.yml
+
+pre-commit:
+  jobs:
+    - run: yarn lint
+    - run: yarn test
+```
+
+This is how jobs configuration differ from commands and scripts:
+
+- Jobs have optional names. Lefthook merges named jobs across [extends](./extends.md) and [local configs](../examples/lefthook-local.md). Unnamed jobs get appended in the definition order.
+- Jobs can have groups of other jobs. For groups you can specify [`parallel`](./parallel.md) or [`piped`](./piped.md) flow for a bunch of jobs. Also [`glob`](./glob.md) and [`root`](./root.md) parameters of a group apply to all its jobs (even nested).
+
+### Job options
+
+- [`name`](./name.md)
+- [`run`](./run.md)
+- [`script`](./script.md)
+- [`runner`](./runner.md)
+- [`group`](./group.md)
+  - [`parallel`](./parallel.md)
+  - [`piped`](./piped.md)
+  - [`jobs`](./jobs.md)
+- [`skip`](./skip.md)
+- [`only`](./only.md)
+- [`tags`](./tags.md)
+- [`glob`](./glob.md)
+- [`files`](./files.md)
+- [`file_types`](./file_types.md)
+- [`env`](./env.md)
+- [`root`](./root.md)
+- [`exclude`](./exclude.md)
+- [`fail_text`](./fail_text.md)
+- [`stage_fixed`](./stage_fixed.md)
+- [`interactive`](./interactive.md)
+- [`use_stdin`](./use_stdin.md)
+
+### Example
+
+> **Note:** Currently only `root` and `glob` options are applied to group jobs. Other options must be set for each job separately. If you find this inconvenient, please submit a [feature request](https://github.com/evilmartians/lefthook/issues/new?assignees=&labels=feature+request&projects=&template=feature_request.md).
+
+A simple configuration with one piped group which executes in parallel with other jobs.
+
+```yml
+# lefthook.yml
+
+pre-commit:
+  parallel: true
+  jobs:
+    - name: migrate
+      root: backend/
+      glob: "db/migrations/*"
+      group:
+        piped: true
+        jobs:
+          - run: bundle install
+          - run: rails db:migrate
+    - run: yarn lint --fix {staged_files}
+      root: frontend/
+      stage_fixed: true
+    - run: bundle exec rubocop
+      root: backend/
+    - run: golangci-lint
+      root: proxy/
+    - script: verify.sh
+      runner: bash
+```
diff --git a/docs/mdbook/configuration/name.md b/docs/mdbook/configuration/name.md
@@ -0,0 +1,14 @@
+## `name`
+
+Name of a job. Will be printed in summary. If specified, the jobs can be merged with a jobs of the same name in a [local config](../examples/lefthook-local.md) or [extends](./extends.md).
+
+### Example
+
+```yml
+# lefthook.yml
+
+pre-commit:
+  jobs:
+    - name: lint and fix
+      run: yarn run eslint --fix {staged_files}
+```
diff --git a/docs/mdbook/configuration/script.md b/docs/mdbook/configuration/script.md
@@ -0,0 +1,20 @@
+## `script`
+
+Name of a script to execute. The rules are the same as for [`scripts`](./Scripts.md)
+
+### Example
+
+```yml
+# lefthook.yml
+
+pre-commit:
+  jobs:
+    - script: linter.sh
+      runner: bash
+```
+
+```bash
+# .lefthook/pre-commit/linter.sh
+
+echo "Everything is OK"
+```
diff --git a/docs/mdbook/examples/lefthook-local.md b/docs/mdbook/examples/lefthook-local.md
@@ -11,20 +11,9 @@
 
 pre-commit:
   commands:
-    lint-frontend:
-      run: yarn lint
-      glob: ".{ts,tsx}"
-    lint-backend:
+    lint:
       run: bundle exec rubocop {staged_files}
       glob: "*.rb"
-    test-frontend:
-      run: yarn test
-      glob: "*.tsx"
-    test-backend:
-      run: bundle exec rspec
-      glob: "spec/*"
-    check-typos:
-      run: typos {staged_files}
     check-links:
       run: lychee {staged_files}
 ```
@@ -35,10 +24,8 @@ pre-commit:
 pre-commit:
   parallel: true # run all commands concurrently
   commands:
-    lint-backend:
+    lint:
       run: docker-compose run backend {cmd} # wrap the original command with docker-compose
-    test-backend:
-      run: docker-compose run backend {cmd}
     check-links:
       skip: true # skip checking links
 
@@ -59,23 +46,12 @@ post-merge:
 pre-commit:
   parallel: true
   commands:
-    lint-frontend:
-      run: yarn lint
-      glob: "*.{ts,tsx}"
-    lint-backend:
+    lint:
       run: docker-compose run backend bundle exec rubocop {staged_files}
       glob: "*.rb"
-    test-frontend:
-      run: yarn test
-      glob: "*.tsx"
-    test-backend:
-      run: docker-compose run backend bundle exec rspec
-      glob: "spec/*"
     check-links:
       run: lychee {staged_files}
       skip: true
-    check-typos:
-      run: typos {staged_files}
 
 post-merge:
   files: "git diff-tree -r --name-only --no-commit-id ORIG_HEAD HEAD"

diff --git a/internal/config/command.go b/internal/config/command.go
@@ -2,12 +2,9 @@ package config
 
 import (
 	"errors"
-
-	"github.com/evilmartians/lefthook/internal/git"
-	"github.com/evilmartians/lefthook/internal/system"
 )
 
-var errFilesIncompatible = errors.New("One of your runners contains incompatible file types")
+var ErrFilesIncompatible = errors.New("One of your runners contains incompatible file types")
 
 type Command struct {
 	Run   string `json:"run"             mapstructure:"run"   toml:"run"             yaml:"run"`
@@ -31,19 +28,6 @@ type Command struct {
 	StageFixed  bool   `json:"stage_fixed,omitempty" koanf:"stage_fixed"        mapstructure:"stage_fixed"   toml:"stage_fixed,omitempty" yaml:"stage_fixed,omitempty"`
 }
 
-func (c Command) Validate() error {
-	if !isRunnerFilesCompatible(c.Run) {
-		return errFilesIncompatible
-	}
-
-	return nil
-}
-
-func (c Command) DoSkip(state func() git.State) bool {
-	skipChecker := NewSkipChecker(system.Cmd)
-	return skipChecker.check(state, c.Skip, c.Only)
-}
-
 func (c Command) ExecutionPriority() int {
 	return c.Priority
 }
diff --git a/internal/config/config.go b/internal/config/config.go
@@ -11,8 +11,6 @@ import (
 	"github.com/mitchellh/mapstructure"
 	toml "github.com/pelletier/go-toml/v2"
 	"gopkg.in/yaml.v3"
-
-	"github.com/evilmartians/lefthook/internal/version"
 )
 
 type DumpFormat int
@@ -46,10 +44,6 @@ type Config struct {
 	Hooks map[string]*Hook `mapstructure:"-"`
 }
 
-func (c *Config) Validate() error {
-	return version.CheckCovered(c.MinVersion)
-}
-
 func (c *Config) Md5() (checksum string, err error) {
 	configBytes := new(bytes.Buffer)
 

diff --git a/internal/config/files.go b/internal/config/files.go
@@ -9,9 +9,6 @@ const (
 	SubPushFiles   string = "{push_files}"
 )
 
-func isRunnerFilesCompatible(runner string) bool {
-	if strings.Contains(runner, SubStagedFiles) && strings.Contains(runner, SubPushFiles) {
-		return false
-	}
-	return true
+func IsRunFilesCompatible(run string) bool {
+	return !(strings.Contains(run, SubStagedFiles) && strings.Contains(run, SubPushFiles))
 }
diff --git a/internal/config/hook.go b/internal/config/hook.go
@@ -1,38 +1,28 @@
 package config
 
 import (
-	"errors"
-
 	"github.com/evilmartians/lefthook/internal/git"
 	"github.com/evilmartians/lefthook/internal/system"
 )
 
 const CMD = "{cmd}"
 
-var errPipedAndParallelSet = errors.New("conflicting options 'piped' and 'parallel' are set to 'true', remove one of this option from hook group")
-
 type Hook struct {
-	Commands map[string]*Command `json:"commands,omitempty" mapstructure:"-" toml:"commands,omitempty" yaml:",omitempty"`
-	Scripts  map[string]*Script  `json:"scripts,omitempty"  mapstructure:"-" toml:"scripts,omitempty"  yaml:",omitempty"`
-
-	Files       string      `json:"files,omitempty"        mapstructure:"files"    toml:"files,omitempty"       yaml:",omitempty"`
 	Parallel    bool        `json:"parallel,omitempty"     mapstructure:"parallel" toml:"parallel,omitempty"    yaml:",omitempty"`
 	Piped       bool        `json:"piped,omitempty"        mapstructure:"piped"    toml:"piped,omitempty"       yaml:",omitempty"`
 	Follow      bool        `json:"follow,omitempty"       mapstructure:"follow"   toml:"follow,omitempty"      yaml:",omitempty"`
+	Files       string      `json:"files,omitempty"        mapstructure:"files"    toml:"files,omitempty"       yaml:",omitempty"`
 	ExcludeTags []string    `json:"exclude_tags,omitempty" koanf:"exclude_tags"    mapstructure:"exclude_tags"  toml:"exclude_tags,omitempty" yaml:"exclude_tags,omitempty"`
 	Skip        interface{} `json:"skip,omitempty"         mapstructure:"skip"     toml:"skip,omitempty,inline" yaml:",omitempty"`
 	Only        interface{} `json:"only,omitempty"         mapstructure:"only"     toml:"only,omitempty,inline" yaml:",omitempty"`
-}
 
-func (h *Hook) Validate() error {
-	if h.Parallel && h.Piped {
-		return errPipedAndParallelSet
-	}
+	Jobs []*Job `json:"jobs,omitempty" mapstructure:"jobs" toml:"jobs,omitempty" yaml:",omitempty"`
 
-	return nil
+	Commands map[string]*Command `json:"commands,omitempty" mapstructure:"-" toml:"commands,omitempty" yaml:",omitempty"`
+	Scripts  map[string]*Script  `json:"scripts,omitempty"  mapstructure:"-" toml:"scripts,omitempty"  yaml:",omitempty"`
 }
 
 func (h *Hook) DoSkip(state func() git.State) bool {
 	skipChecker := NewSkipChecker(system.Cmd)
-	return skipChecker.check(state, h.Skip, h.Only)
+	return skipChecker.Check(state, h.Skip, h.Only)
 }