Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic utility command ctk tail, for tailing a database table and optionally following the tail #330

Merged
merged 3 commits into from
Dec 18, 2024

Conversation

amotl
Copy link
Member

@amotl amotl commented Dec 16, 2024

About

Implement tail -f for sys.jobs_log, to support ad hoc tracing of SQL statements processed by CrateDB.

Synopsis

export CRATEDB_SQLALCHEMY_URL=crate://crate@localhost:4200/
ctk tail -n 3 --follow --format=log-pretty sys.jobs_log
crash -c "SELECT * FROM sys.summits LIMIT 3;"

The implementation may be versatile enough to also work well on arbitrary tables.

ctk tail -n 3 sys.summits

Documentation

References

@amotl amotl force-pushed the tail branch 2 times, most recently from 3a5cfbc to 397c8a5 Compare December 16, 2024 22:16
@amotl amotl requested review from seut and WalBeh December 16, 2024 22:21
@amotl amotl marked this pull request as ready for review December 16, 2024 22:21
Copy link
Member

@seut seut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great stuff! Spotted a cli arg issue..

@click.option("--format", "format_", type=str, required=False, help="Select output format. Default: log / jsonl")
@click.option("--follow", "-f", is_flag=True, required=False, help="Follow new records added, by polling the table")
@click.option("--verbose", is_flag=True, required=False, help="Turn on logging")
@click.option("--debug", is_flag=True, required=False, help="Turn on logging with debug level")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the --interval option is not defined and passed through?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice spot. Thanks!

Copy link
Member Author

@amotl amotl Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed per 9a60d62. Also added 476d4f2.

It is mostly sufficient and still provides enough interactivity.
By reducing the frequency, it will incur less spam on sys.jobs_log
itself.
Copy link
Member

@seut seut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines +20 to +29
@click.option(
"--lines", "-n", type=int, required=False, default=10, help="Displays n last lines of the input. Default: 10"
)
@click.option("--format", "format_", type=str, required=False, help="Select output format. Default: log / jsonl")
@click.option("--follow", "-f", is_flag=True, required=False, help="Follow new records added, by polling the table")
@click.option(
"--interval", "-i", type=float, required=False, help="When following the tail, poll each N seconds. Default: 0.5"
)
@click.option("--verbose", is_flag=True, required=False, help="Turn on logging")
@click.option("--debug", is_flag=True, required=False, help="Turn on logging with debug level")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason not to use click default option settings for all default values instead of later on inside the code?

Copy link
Member Author

@amotl amotl Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the defaults would be applied to click, they wouldn't apply when using ctk's Python APIs in your own programs, that's why I decided to put them there, also to avoid redundancy declaring them on both sides.

@amotl amotl merged commit ed593da into main Dec 18, 2024
17 checks passed
@amotl amotl deleted the tail branch December 18, 2024 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants