Streamline usage, installation and configuration of `ots-git-gpg-wrapper.sh` #121

nobodyinperson · 2021-09-28T07:05:02Z

These changes improve the ots-git-gpg-wrapper.sh script by making it configurable:

via git config opentimestamps.{enable,debug,flags} options
via environment variables OPENTIMESTAMPS*

This makes it possible to do the following:

temporarily disabling timestamping for a commit: OPENTIMESTAMPS=false git commit -m "Message"
per-repository setting for OpenTimestamps: git config opentimestamps.enable false
global default to OpenTimestamps: git config --global opentimestamps.enable true
debugging of the OpenTimestamps process: OPENTIMESTAMPS_GIT_GPG_WRAPPER_DEBUG=true OPENTIMESTAMPS_GIT_GPG_WRAPPER_FLAGS=-vvvvvvv git log --show-signature
Disabling Bitcoin node checking upon verification with git config --global opentimestamps.flags '--no-bitcoin'

The backwards-compatible default is still to just use OpenTimestamps.

Furthermore, the ots-git-gpg-wrapper now uses shutil.which to find the gpg binary as the default value for --gpg-program, so this option does not need to be set from the ots-git-gpg-wrapper.sh-wrapper anymore.

As proposed in #67, the ots-git-gpg-wrapper.sh is now installed as a script and doesn't need to be manually downloaded by the users, which renders #68 obsolete. Concerning Windows users (if this solution doesn't work with Git Bash), we might add and install a separate .bat-file later.

The documentation doc/git-integration.md was updated to reflect the changes.

closes #64 #68 #74

- via `git config opentimestamps.enable` option - via environment variables OPENTIMESTAMPS*

This is more straight-forward than to hard-code /usr/bin/gpg in ots-git-gpg-wrapper and then using `which` in the ots-git-gpg-wrapper.sh to find gpg.

petertodd · 2021-09-30T00:00:40Z

doc/git-integration.md

+
+```bash
+# just specify ots-git-gpg-wrapper.sh and let `git` find it itself
+git config --global gpg.program ots-git-gpg-wrapper.sh


By "find it itself" doesn't git just use $PATH as normal?

Exactly. It is possible to only specify ots-git-gpg-wrapper.sh without a full path if it is in the PATH, which is a convenience as on different machines it can be installed in different places (e.g. ~/.local/bin or /usr/local/bin or /usr/bin/, ...). This is a strong argument for putting it into a standard executable location.

petertodd · 2021-09-30T00:02:50Z

doc/git-integration.md

+
+```bash
+# Disable OpenTimestamps for the current repository:
+git config opentimestamps.enable false


Would it be more useful to disable opentimestamps selectively for just log viewing?

Also, can you set gpg.program on a per-repo basis?

Also, can you set gpg.program on a per-repo basis?

Yes, you can. Without --global, git config operates only per-repo.

Would it be more useful to disable opentimestamps selectively for just log viewing?

Sure we can add something like git config opentimestamps.on_log and git config opentimestamps.on_commit or git config commit.opentimestamps etc. for more fine-grained control. But AFAIK it is not so simple or elegant to find the current git operation from within the gpg wrapper. We could find the parent git process' command-line and parse it. Doing this in a shell wrapper is not nice, however. Another argument to making ots-git-gpg-wrapper.sh a Python script (or finding a way to have ots-git-gpg-wrapper do this already.)

Would it be more useful to disable opentimestamps selectively for just log viewing?

In 31c1bde I implemented picking for which git commands the OpenTimestamps wrapper should be active by setting git config opentimestamps.only-for 'commit,show' (in this case the wrapper will then only do anything different during git commit and git show).

But AFAIK it is not so simple or elegant to find the current git operation from within the gpg wrapper. We could find the parent git process' command-line and parse it. Doing this in a shell wrapper is not nice, however.

Finding out which git command is currently active works in the above way, except if one passes options directly to git (like git --no-pager log). To handle this, the wrapper would need to fully parse the git command-line, involving some guesswork. In my current implementation, the wrapper detects this and skips the check, effectively acting as if the user wanted to use OpenTimestamps for all commands.

petertodd · 2021-09-30T00:05:26Z

setup.py

@@ -101,4 +101,7 @@
            'ots-git-gpg-wrapper = otsclient.git_gpg_wrapper:main',
        ],
    },
+
+    # Install the ots-git-gpg-wrapper.sh-script
+    scripts = ["ots-git-gpg-wrapper.sh"],


One issue with installing the script is it'll clutter up tab-completion with another program starting with ots. Is there a better place to put this? Under /usr/lib/ or something?

See #121 (comment)

Not putting it into PATH forces one to hard-code an absolute path with git config gpg.program. For people synchronizing their .gitconfig for example this is annoying as the wrapper can be installed in different places on different machines.

If only the namespace cluttering is the problem with putting the ots-git-gpg-wrapper.sh into a location in PATH, we could just rename it to something like opentimestamps-git-gpg-wrapper.sh, or gpg-opentimestamps-wrapper.sh. This will work around tab-completition with shells that get winy if there are multiple possible commands (instead of just completing up to the common stem like zsh does).

petertodd · 2021-09-30T00:12:44Z

otsclient/git_gpg_wrapper.py

@@ -30,7 +31,7 @@
 def main():
    parser = otsclient.args.make_common_options_arg_parser()

-    parser.add_argument("-g", "--gpg-program", action="store", default="/usr/bin/gpg",
+    parser.add_argument("-g", "--gpg-program", action="store", default=shutil.which("gpg") or "/usr/bin/gpg",


Suggested change

parser.add_argument("-g", "--gpg-program", action="store", default=shutil.which("gpg") or "/usr/bin/gpg",

parser.add_argument("-g", "--gpg-program", action="store", default="gpg",

Looks like Popen already searches in $PATH for us, so if we're going to use $PATH we don't need to figure it out ourselves.

Okay. But then it was also unnecessary to provide

--gpg-program "`which gpg`"

in ots-git-gpg-wrapper.sh.

A nice side-effect of specifying the shutil.which()-found path is that the help page will show exactly which gpg program will be used. This wouldn't be the case if we just let Popen find it.

nobodyinperson · 2021-10-15T12:39:01Z

In 31c1bde I introduced the git config sections opentimestamps.flags (flags for the ots-git-gpg-wrapper) and opentimestamps.debug (for showing debug output in ots-git-gpg-wrapper.sh). This enables setting git config opentimestamps.flags '--no-bitcoin' for example to get rid of the error message that there is no local Bitcoin node running upon verification in git log --show-signature, instead showing the instructions how to verify the commit manually. This will then be enough for closing #123.

nobodyinperson · 2021-10-26T21:15:35Z

I noticed that git -C ... seems to make problems as it adds a -C\n between gpgsig and -----BEGIN PGP SIGNATURE-----, which breaks git's --show-signature...

Probably we do indeed need to parse git's flags with getopt or something like that...

Options directly passed to git (like `git -c config.bla` or `git --git-dir=...`) caused the first argument to show up before the PGP signature, breaking git {log,show} --show-signature and thus also OTS verification. This was especially annyoing as `git annex sync` or `datalad save` pass options like this to git when committing. With quite some `strace`ing and debugging I was finally able to track down this bug to this missing `-q` to that one grep in the OTS shell wrapper 🤦 It might be a good idea to ditch the shell wrapper and use a proper Python for this. Wouldn't have happened with Python...

nobodyinperson · 2023-03-06T22:50:17Z

I've been working with this PR's changes for over a year now on a daily basis and feel quite confident that it adds value to this package. Do you need something else to merge it?

petertodd · 2023-03-08T14:54:08Z

Thanks for the heads up! I didn't notice all the new stuff you've done. If I don't get back to you in a week or so with a response, bother me again. :)

petertodd · 2023-03-15T15:42:15Z

ots-git-gpg-wrapper.sh

You need to change the #!/bin/sh to #!/bin/bash, as you're using bash-specific commands; newer linux distros like Debian don't put bash in /bin/sh any more.

petertodd · 2023-03-15T15:44:45Z

In addition to my comment, let's go ahead and rename ots-git-gpg-wrapper.sh to opentimestamps-git-gpg-wrapper to both avoid any command completion issues, and because it being a shell script is an implementation detail.

nobodyinperson added 3 commits September 28, 2021 08:40

Make ots-git-gpg-wrapper.sh configurable

741b0b1

- via `git config opentimestamps.enable` option - via environment variables OPENTIMESTAMPS*

Let ots-git-gpg-wrapper find gpg, not the wrapper

50d4858

This is more straight-forward than to hard-code /usr/bin/gpg in ots-git-gpg-wrapper and then using `which` in the ots-git-gpg-wrapper.sh to find gpg.

Install ots-git-gpg-wrapper.sh next to ots-git-gpg-wrapper

116221f

nobodyinperson force-pushed the configure-ots-git-gpg-wrapper-from-outside branch from 966482b to 0cc4cd9 Compare September 28, 2021 08:05

Update git integration docs

ff6ef6e

nobodyinperson force-pushed the configure-ots-git-gpg-wrapper-from-outside branch from 0cc4cd9 to ff6ef6e Compare September 28, 2021 08:10

nobodyinperson changed the title ~~Make ots-git-gpg-wrapper.sh configurable~~ Streamline usage and installation and configuration of ots-git-gpg-wrapper.sh Sep 28, 2021

nobodyinperson changed the title ~~Streamline usage and installation and configuration of ots-git-gpg-wrapper.sh~~ Streamline usage, installation and configuration of ots-git-gpg-wrapper.sh Sep 28, 2021

petertodd reviewed Sep 30, 2021

View reviewed changes

nobodyinperson mentioned this pull request Oct 13, 2021

Manually verifying timestamped signed git commits #123

Closed

Introduce git config opentimestamps.{flags,debug}

31c1bde

nobodyinperson marked this pull request as draft October 26, 2021 21:10

nobodyinperson marked this pull request as ready for review May 23, 2022 05:32

petertodd reviewed Mar 15, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Streamline usage, installation and configuration of `ots-git-gpg-wrapper.sh` #121

Streamline usage, installation and configuration of `ots-git-gpg-wrapper.sh` #121

nobodyinperson commented Sep 28, 2021 •

edited

Loading

petertodd Sep 30, 2021

nobodyinperson Sep 30, 2021

petertodd Sep 30, 2021

nobodyinperson Sep 30, 2021

nobodyinperson Oct 15, 2021

petertodd Sep 30, 2021

nobodyinperson Sep 30, 2021

nobodyinperson Oct 15, 2021

petertodd Sep 30, 2021

nobodyinperson Sep 30, 2021 •

edited

Loading

nobodyinperson Oct 1, 2021

nobodyinperson commented Oct 15, 2021

nobodyinperson commented Oct 26, 2021

nobodyinperson commented Mar 6, 2023

petertodd commented Mar 8, 2023

petertodd Mar 15, 2023

petertodd commented Mar 15, 2023

	parser.add_argument("-g", "--gpg-program", action="store", default=shutil.which("gpg") or "/usr/bin/gpg",
	parser.add_argument("-g", "--gpg-program", action="store", default="gpg",

Streamline usage, installation and configuration of ots-git-gpg-wrapper.sh #121

Are you sure you want to change the base?

Streamline usage, installation and configuration of ots-git-gpg-wrapper.sh #121

Conversation

nobodyinperson commented Sep 28, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nobodyinperson Sep 30, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nobodyinperson commented Oct 15, 2021

nobodyinperson commented Oct 26, 2021

nobodyinperson commented Mar 6, 2023

petertodd commented Mar 8, 2023

Choose a reason for hiding this comment

petertodd commented Mar 15, 2023

Streamline usage, installation and configuration of `ots-git-gpg-wrapper.sh` #121

Streamline usage, installation and configuration of `ots-git-gpg-wrapper.sh` #121

nobodyinperson commented Sep 28, 2021 •

edited

Loading

nobodyinperson Sep 30, 2021 •

edited

Loading