Skip to content

Commit

Permalink
refactor: reorganize code structure and improve filtering logic
Browse files Browse the repository at this point in the history 
- Move config and validation logic to separate pkg/ directory
- Add dedicated MessageFilter type with proper error handling
- Refactor command initialization using factory methods
- Improve configuration and status reporting
- Simplify consumer implementation
- Add proper error handling for filter compilation

chore: test
  • Loading branch information
@marianozunino
marianozunino committed Nov 28, 2024
1 parent 5eb1003 commit 7240cf3
Show file tree
Hide file tree
Showing 27 changed files with 951 additions and 635 deletions.
5 changes: 5 additions & 0 deletions .github/workflows/releaser.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ jobs:
-
name: Set up Go
uses: actions/setup-go@v5
-
name: Install UPX
uses: crazy-max/ghaction-upx@v3
with:
install-only: true
-
name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
Expand Down
32 changes: 12 additions & 20 deletions .goreleaser.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,54 +1,46 @@
# This is an example .goreleaser.yml file with some sensible defaults.
# Make sure to check the documentation at https://goreleaser.com

# The lines below are called `modelines`. See `:help modeline`
# Feel free to remove those if you don't want/need to use them.
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj

version: 2

upx:
- enabled: true
goos: [linux, darwin]
compress: best
lzma: true

before:
hooks:
# You may remove this if you don't use go modules.
- go mod tidy
# you may remove this if you don't need go generate
- go generate ./...

builds:
- env:
- CGO_ENABLED=0
goos:
- linux
- darwin
ldflags: "-s -w -X github.com/marianozunino/goq/cmd.Version={{.Version}}"
ldflags: >-
-s -w
-X github.com/marianozunino/goq/cmd.Version={{.Version}}
-extldflags '-static'
flags:
- -trimpath

release:
github:
owner: marianozunino
name: goq
name_template: "{{.ProjectName}} v{{.Version}}"

# You can disable this pipe in order to not upload any artifacts to
# GitHub.
# Defaults to false.
disable: false

archives:
- format: tar.gz
# this name template makes the OS and Arch compatible with the results of `uname`.
name_template: >-
{{ .ProjectName }}_
{{- title .Os }}_
{{- if eq .Arch "amd64" }}x86_64
{{- else if eq .Arch "386" }}i386
{{- else }}{{ .Arch }}{{ end }}
{{- if .Arm }}v{{ .Arm }}{{ end }}
# use zip for windows archives
format_overrides:
- goos: windows
format: zip

changelog:
sort: asc
filters:
Expand Down
108 changes: 6 additions & 102 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
\/_____/ \/_____/ \/___/_/
```

GOQ is a command-line tool for working with RabbitMQ queues, allowing you to dump, monitor, and manage messages efficiently.

## Installation
To install GOQ, run:

Expand All @@ -17,45 +19,10 @@ go install github.com/marianozunino/goq@latest

Or download the binary from the [GitHub releases](https://github.com/marianozunino/goq/releases)

## Usage
```
Usage:
goq [command]
Available Commands:
configure Generate a sample configuration file for goq.
dump Dump messages from a RabbitMQ queue to a file.
monitor Monitor RabbitMQ messages using routing keys and a temporary queue.
update Update the goq tool to the latest available version.
version Display the current version of the goq tool.
Additional Commands:
completion Generate the autocompletion script for the specified shell
help Help about any command
## Documentation
For detailed usage instructions, configuration options, and examples, please refer to the [GOQ Documentation](docs/goq.md).

Flags:
-s, --amqps Use AMQPS instead of AMQP
--config string config file (default "/home/forbi/.config/goq/goq.yaml")
-e, --exchange string RabbitMQ exchange name
-x, --exclude-patterns strings Exclude messages containing these patterns
-m, --file-mode string File mode (append or overwrite, only valid for file writer) (default "overwrite")
-f, --full-message Print full message
-h, --help help for goq
-i, --include-patterns strings Include messages containing these patterns
-j, --json-filter string JSON filter expression
-z, --max-message-size int Maximum message size in bytes (default -1)
-o, --output string Output file name (required when writer is 'file')
-p, --pretty-print Pretty print JSON messages
-R, --regex-filter string Regex pattern to filter messages
-k, --skip-tls-verify Skip TLS certificate verification (insecure)
-u, --url string RabbitMQ URL (e.g., localhost:5672)
-v, --virtualhost string RabbitMQ virtual host
-w, --writer string Output writer type (console or file) (default "file")
Use "goq [command] --help" for more information about a command.
```

### Example Commands
## Quick Start
1. **Dump messages from a local RabbitMQ server**:
```bash
goq dump -u localhost:5672 -q my_queue
Expand All @@ -66,74 +33,11 @@ Use "goq [command] --help" for more information about a command.
goq monitor -u localhost:5672 -r routing_key1,routing_key2,routing_key_pattern.#
```

3. **Use AMQPS with a specific virtual host**:
```bash
goq dump -u rabbitmq.example.com:5671 -q important_queue -s -v my_vhost
```

4. **Automatically acknowledge messages and stop after consuming**:
```bash
goq dump -u localhost:5672 -q my_queue -a -c
```

5. **Append to an existing output file**:
```bash
goq dump -u localhost:5672 -q my_queue -m append -o existing_output.txt
```
6. **Print the full message including headers, exchange, and timestamp:**
```bash
goq dump -u localhost:5672 -q my_queue -f
```

## Configuration
GOQ can be configured using a YAML file. By default, it looks for a configuration file at `$XDG_CONFIG_HOME/goq/goq.yaml`.

### Example Configuration File
```yaml
# Configuration for rabbitmq-dumper

# Use AMQPS instead of AMQP
amqps: true

# Skip TLS certificate verification (insecure)
skip-tls-verify: true

# RabbitMQ server URL
url: "127.0.0.1:5672"

# RabbitMQ exchange name, leave empty if not used
exchange: "some_exchange"

# RabbitMQ virtual host, leave empty if not used
virtualhost: "some-virtual-host"

# Output file name (relative to the current working directory)
output: "messages.txt"

# File mode (append or overwrite)
file-mode: "overwrite"

# Pretty print JSON messages
pretty-print: true

# Print full message including headers, exchange, timestamp, and body
full-message: false
```
When using a config file, the flags will override the values in the config file. It's useful to combine the config file with command-line flags for more flexibility.
## How Does GOQ Work?
1. **Connection**: Establishes a connection to the specified RabbitMQ server using either AMQP or AMQPS.
2. **Queue Binding**: Binds to the specified queue and exchange (if provided).
3. **Message Consumption**: Begins consuming messages from the queue.
4. **File Writing**: Writes consumed messages to the specified output file.
5. **Acknowledgment**: Acknowledges messages based on the auto-ack setting.
6. **Monitoring**: Displays the progress of consumed messages.
7. **Termination**: Stops consuming messages based on user input (CTRL+C) or the stop-after-consume flag.
GOQ can be configured using a YAML file. By default, it looks for a configuration file at `$XDG_CONFIG_HOME/goq/goq.yaml`. For configuration options and examples, see the [documentation](docs/goq.md#configuration).

## Contributing
Contributions to GOQ are welcome! Please feel free to submit pull requests, create issues for bugs and feature requests, or contribute to the documentation.

## License
GOQ is released under the MIT License. See the LICENSE file for more details.
26 changes: 12 additions & 14 deletions cmd/configure.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,24 +34,22 @@ import (
"github.com/spf13/cobra"
)

var configureCmd = &cobra.Command{
Use: "configure",
Short: "Generate a sample configuration file for goq.",
Long: `Generate a sample configuration file for goq in the default location.
//go:embed example_config.yml
var exampleConfig string

func NewConfigureCommand() *cobra.Command {
return &cobra.Command{
Use: "configure",
Short: "Generate a sample configuration file for goq.",
Long: `Generate a sample configuration file for goq in the default location.
This file provides default settings for RabbitMQ connections and message handling, which can be customized for your environment.
The configuration file simplifies using the tool by predefining connection parameters, output settings, and other preferences.`,
Example: `goq configure`,
GroupID: "available-commands",
Run: runConfigure,
}

func init() {
rootCmd.AddCommand(configureCmd)
Example: `goq configure`,
GroupID: "available-commands",
Run: runConfigure,
}
}

//go:embed example_config.yml
var exampleConfig string

func runConfigure(cmd *cobra.Command, args []string) {
configPath, err := xdg.ConfigFile("goq/goq.yaml")
if err != nil {
Expand Down
69 changes: 13 additions & 56 deletions cmd/dump.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,73 +19,30 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/

package cmd

import (
"fmt"

app "github.com/marianozunino/goq/internal"
"github.com/marianozunino/goq/internal/config"
"github.com/marianozunino/goq/pkg/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)

// NewDumpCmd creates the `dump` command.
func NewDumpCmd() *cobra.Command {
var queue string
var autoAck bool
var stopAfterConsume bool

dumpCmd := &cobra.Command{
Use: "dump",
Short: "Dump messages from a RabbitMQ queue to a file.",
Long: `Dump messages from the specified RabbitMQ queue to a file.
This command provides options for automatically acknowledging messages, controlling when consumption stops, and configuring file output behavior.
Messages can be captured from an AMQP or AMQPS RabbitMQ server, with flexible TLS and virtual host settings.`,
Example: `goq dump -q my_queue -o output.txt -a -c`,
GroupID: "available-commands",
PreRunE: validateFlags,
cmd := &cobra.Command{
Use: "dump",
Short: "Dump messages from a RabbitMQ queue",
Long: "Dump messages from a specified RabbitMQ queue with flexible filtering and output options.",
Example: "goq dump -q my_queue -o output.txt",
RunE: func(cmd *cobra.Command, args []string) error {
cfg := config.New(
config.WithRabbitMQURL(fmt.Sprintf("%s://%s/%s", getProtocol(), viper.GetString("url"), viper.GetString("virtualhost"))),
config.WithExchange(viper.GetString("exchange")),

config.WithOutputFile(viper.GetString("output")),
config.WithFileMode(viper.GetString("file-mode")),

config.WithWriter(viper.GetString("writer")),
config.WithOutputFile(viper.GetString("output")),
config.WithFileMode(viper.GetString("file-mode")),

config.WithUseAMQPS(viper.GetBool("amqps")),
config.WithVirtualHost(viper.GetString("virtualhost")),
config.WithSkipTLSVerification(viper.GetBool("skip-tls-verify")),
config.WithPrettyPrint(viper.GetBool("pretty-print")),
config.WithQueue(queue),
config.WithStopAfterConsume(stopAfterConsume),
config.WithAutoAck(autoAck),
config.WithFullMessage(viper.GetBool("full-message")),

config.WithIncludePatterns(viper.GetStringSlice("include-patterns")),
config.WithExcludePatterns(viper.GetStringSlice("exclude-patterns")),
config.WithMaxMessageSize(viper.GetInt("max-message-size")),
config.WithRegexFilter(viper.GetString("regex-filter")),
)

return app.Dump(cfg)
return app.Dump(config.CreateCommonConfig(cmd))
},
}

dumpCmd.Flags().SortFlags = false
dumpCmd.Flags().StringVarP(&queue, "queue", "q", "", "RabbitMQ queue name")
dumpCmd.Flags().BoolVarP(&autoAck, "auto-ack", "a", false, "Auto ack messages")
dumpCmd.Flags().BoolVarP(&stopAfterConsume, "stop-after-consume", "c", false, "Stop after consuming messages")

dumpCmd.MarkFlagRequired("queue")

return dumpCmd
}
cmd.Flags().StringP("queue", "q", "", "RabbitMQ queue name (required)")
cmd.Flags().BoolP("auto-ack", "a", false, "Automatically acknowledge messages")
cmd.Flags().BoolP("stop-after-consume", "c", false, "Stop after consuming messages")
cmd.MarkFlagRequired("queue")

func init() {
rootCmd.AddCommand(NewDumpCmd())
return cmd
}
Loading

0 comments on commit 7240cf3

Please sign in to comment.