My org-babel based emacs configuration

Last update: call_last_modified()

This is my emacs configuration file that is loaded with org-babel-load-file in the Emacs init file. The intent is to have as much of my Emacs configuration in here as possible. The system works as a literal programming system where with a tangle the elisp code that actually makes up my configuration is extracted automatically and loaded.

This document is irregularly published and lives in different locations:

• config sources are available on github
• published on my writefreely instance qua.name

Files for configuration

Next to this main configuration file (mrb.org), two other files are involved in the configuration:

1. early-init.el emacs’ early initialization file;
2. init.el the main init file, which is used to generate mrb.el from mrb.org.

Ideally these files should be produced by org-babel as well, just like the main configuration, but that generates a dependency because these two files need to be available from the beginning.

Small snippet to have last modified timestamp inserted when exporting this file (see line above which has the call to last_modified)

git log -1 --pretty="format:%ci" ~/.emacs.d/mrb.org

Preparing for lift-off

The first thing I want to take care of is to make customization possible and stored in a place to my liking. I want to load this first so anything in the configuration I define explicitly overrides it.

(setq custom-file (expand-file-name "custom.el" user-emacs-directory))
(load custom-file)

The config file mrb.org gets opened a lot because I tend to fiddle with this config at least once a day. So, it warrants it’s own keybinding, but it will have to wait until Key bindings before we can use the bind-key package.

(defun mrb/open-config ()
  (interactive)
  (find-file config-file))

Also, it helps to have a config-reload option sometimes.

(defun mrb/reload-config ()
  "Reload init file, which will effectively reload everything"
  (interactive)
  (load-file (expand-file-name "init.el" user-emacs-directory)))

Most of the time the editing process starts at the command line. Because I use emacs in server mode, this needs a little script. The script does two things:

1. check if we already have the emacs server, if not, start it;
2. treat input from stdin a little bit special.

While we are here, define a snippet which should go in shell scripts. We will repeat this per language at some point.

[ Org-mode generated this file from a code block, changes will be overwritten ]

And edit the file obviously.

# <<tangle-header>>
# Wrapper for emacs-client usage to handle specific cases:
# - no filename given
# - stdin editing
# - force tty
# - adjustment to systemd invocation
# FIXME: why does which not work here?
BASE=/home/mrb/.guix-profile/bin/
ME=$(basename $0)
EC=$BASE/emacsclient
EM=$BASE/emacs
SNAME=server
SFILE=$XDG_RUNTIME_DIR/emacs/$SNAME

# Default argument needs our socket in any case
ARGS="--socket-name=$SNAME --suppress-output"

# Set to false when done
debug() { false $@; }

# Do we already have an emacs?
# I could let emacsclient do this automatically, but it
# is hard-coded to exec emacs --daemon (without socket name)

# This goes along an emacs.service file obviously
if ( ! systemctl -q --user is-active emacs );then
    debug "Emacs systemd service not active...";

    # We might have a socket though
    if [ -S $SFILE ];then
        # The systemd starting has always been a little problematic
        # with the environment, so I might have a server running another
        # way.
        # TODO  why not /just/ checking for the socket?
        #
        debug "..but server socket is there, using it..."
    else
        debug "...and we have no socket either."
        notify-send "Starting Emacs server, this might take a while....";
        $EM --bg-daemon=$SFILE
        #systemctl --user start emacs
    fi
fi

# Assert that the socket file exists
[ ! -S $SFILE ] || debug "Socket file not found???\n"

# Handle special 'stdin' case
if [ $# -ge 1 ] && [ "$1" == "-" ]; then
    TMPSTDIN="$(mktemp /tmp/emacsstdinXXX)";
    cat >"$TMPSTDIN";
    # Recurse or call with flags below
    $ME $TMPSTDIN;
    rm $TMPSTDIN;
    exit 0;
fi

# We're basically expecting X windows, but if not, we good
# force tty: call this like 'DISPLAY="" edit ....' see my aliases
if [ $DISPLAY ]; then
    debug "X-display value seen, using GUI"
    case $ME in
      "edit")
          ARGS=$ARGS' --no-wait'
          ;;
      "edit-wait")
          ARGS=$ARGS' '
          ;;
      "edit-frame")
          ARGS=$ARGS' --no-wait --create-frame'
          ;;
      "edit-wait-frame")
          ARGS=$ARGS' --create-frame'
          ;;
      *)
          echo "Wrong calling name..."
          ;;
    esac
else
    # tty always waits, as it is in the terminal, create-frame has no meaning
    debug "Starting tty emacs"
    ARGS=$ARGS' -tty'
fi

# Go with the original arguments
exec $EC ${ARGS} "$@"

The above script is the result of trying to get the server based setup of Emacs a little better. The thoughts leading to the above script are included below.

Emacs can run as a server meaning that its process is always active and a specific client program called emacsclient can be used to connect to it to do the actual user interaction and let me edit and use the lisp interpreter.

It’s far from trivial, for me at least, on how to properly configure this, assuming you have some special wishes. I’ve certainly spent my time on trying to get it right.

Things that I’m sure of I want

Here’s what I know for sure on what I want:

1. I want to have at least one server process for ‘default’ editing so emacs does not have to start a process for every file; the goal here is to have fast opening of files to edit. This is critical. I’m assuming that for this a server-like-setup is needed.
2. Given that I tweak emacs a lot, it must be very easy to either disable the server temporarily or have it restart easily so I can test new changes without shutting me out of emacs completely.
3. The previous item also implies that I want to run multiple independent emacs-en next to each other, regardless if they are run as a daemon or not.
4. Emacs must work on a non X-windows screen, but extra effort to get that is OK.
5. Be able to use ‘-’ as placeholder for stdin to be able to pipe information directly into editor
6. allow specific handlers, like mailto:

Things I see potential for, but not sure of

Having a server/servers opens a couple of options which I may like, but not sure on how to do or even if I understand them properly

• given that the socket can be a TCP socket, does that mean I can use emacsclient -some-arg <the IP of the socket> on one machine to use emacs on another machine? How well would that work? This would be awesome to have a small computer which doesn’t need an emacs config to use my emacs on another machine, but I suspect I misunderstand how this works
• having multiple emacs servers could help with specific scenario’s for mu4e for example. I’d love to have the mu-server in its own process and just use that from other emacs servers Similarly for long running language servers and or other processes which shouldn’t be affected by me clawing words into my keyboard.

Package Management

Package handling, do this early so emacs knows where to find things. Package handling basically means two things:

1. getting and installing packages, by default handled by package.el;
2. configuring packages, just elisp, but I use the use-package macro typically.

As I want to have version controlled package installation that is easily reproducible, straight.el seem a logical choice, so let’s start by bootstrapping that.

(defvar bootstrap-version)
(setq straight-repository-branch "develop") ; Need this for new org-contrib location
(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
    (with-current-buffer
        (url-retrieve-synchronously
         "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
         'silent 'inhibit-cookies)
      (goto-char (point-max))
      (eval-print-last-sexp)))
  (load bootstrap-file nil 'nomessage))

Now that we have straight we will use it from the use-package macro.

(setq use-package-always-ensure nil   ; Make sure this is nil, so we do not use package.el
      use-package-verbose 'debug      ; TODO  use a debug var for all of config?
      )
;; From this point on we should be able to use `use-package
(use-package straight
  :custom
  (straight-host-usernames '((github . "mrvdb"))) ; TODO Move to personal information?
  (straight-use-package-by-default t)
  ;; Make sure packages do not pull in internal org, we pregister org from straight.el
  (straight-register-package 'org)
  (straight-register-package 'org-contrib))

;; FROM THIS POINT use-package should work as intended, i.e. using straight.

;; Need to install dependencies of use-package manually, why??
(use-package diminish)

I want to delegate garbage collection to someone who knows more than I do and made a package for it. The idea is still similar to what I had for years, i.e. large threshold on an idle-timer to do garbage collection. The package just adds some convenience, so I dont have to think about it other than making the threshold large on startup.

(use-package gcmh
  :config
  (setq gcmh-verbose 1)
  (setq gcmh-high-cons-threshold (* 16 1024 1024 1024))
  (gcmh-mode 1))

Sometimes it is comfortable to use GUIX packages, for example when autobuilding fails. My usecase was pdf-tools failing frequently, so using the builtin from GUIX was an easy way out.

This is the only thing needed, after installing an emacs package with GUIX, it is available for require and can be treated by straight as a built-in type package.

(add-to-list 'load-path "/home/mrb/.guix-profile/share/emacs/site-lisp")
(guix-emacs-autoload-packages)

As emacs lives inside the systemd environment it is not automatically the case that my shell environment is inherited. In many cases, this is not a problem, but for those cases that expect the same environment, I’m making explicit that the enviroment needs to be from the shell. We should do this a soon as use-package is useable.

(use-package exec-path-from-shell
  :demand t
  :config
  (exec-path-from-shell-initialize))

This makes sure that things like TeX and git and other programs run in the same context as they would in my shell. Especially for guix-installed programs this is important.

Personal information

I’d like to arrange 3 things related to personal information:

1. register the proper identification to use;
2. Make sure the proper authentication information is stored;
3. Store this information privately.

So, identification, authorization and encryption.

Seems like a good idea to start configuration with some details about me. The idea being that anything personal goes here and nowhere else. For one, this means my name only appears in this section in this document. Most of the variables I just made up, but some are used in other places too.

#+COMMENT TODO read this from a file, as I publish this file and there’s no need for casual readers to know this.

(setq mrb/maildir                                    "~/Maildir/"
      mrb/default-email-address                      "[email protected]"
      mrb/default-signature-file (concat mrb/maildir ".signature")
      mrb/private-addresses '(                       "[email protected]"
                                                     "[email protected]")
      mrb/private-signature-file (concat mrb/maildir ".signature-private")
      mrb/default-private-address (car mrb/private-addresses)

      mrb/bcc-address                                "[email protected]"
      user-full-name                                 "Marcel van der Boom"
      user-mail-address                              mrb/default-email-address
      user-domain                                    "hsdev.com"
      user-organisation                              "HS-Development BV"
      user-gpg-key                                  "77DDA1B68D04792A8F85D855235E5C8CF5E8DFFB"
      )

Authorization

Several things I use within Emacs need authorization, such as tramp, jabber, erc etc. The authorization can come from several sources; ideally as few as possible. Many packages in Emacs have support for a .netrc like mechanism, others want to use the key ring in GNOME. The variables auth-sources defines the sources available.

I want to use systems which are desktop independent, so things like the gnome key ring are out because they depend on the gnome environment being present, which I can not guarantee, nor want to related to authentication. The situation which I want to prevent is that if gnome is broken, I can’t authenticate to services I need.

I have a gpg-agent daemon configured which manages gpg and ssh keys, protected by a hardware key. Let’s make the system as simple as we can for now and just store passwords in the gpg protected store only, i.e. the password-store program.

;; Use only password-store
(use-package password-store)
(use-package auth-source-pass
  :straight (:type built-in)
  :init
  (auth-source-pass-enable)
  :after password-store
  :custom
  ;; Make sure it's the only mechanism
  (auth-sources '(password-store))
  (auth-source-gpg-encrypt-to (list user-mail-address)))

;; I like the pass interface, so install that too
(use-package pass
  :after password-store)

Encrypting information

I need a way to store some sensitive information without that being published, should I decide some day to push this config somewhere.

When needed, the gpg key is used to encrypt information.

;; Use my email-address for encryption
(setq-default epa-file-encrypt-to user-mail-address)
;; Make sure we always use this
(setq-default epa-file-select-keys nil)

For org-mode, there is a way to encrypt sections separately. See Encrypting information in org-mode for the details on the settings for this.

Next to inline content in org that needs encryption, there is also content that needs encrypting which is more suitable to store in a separate file for several reasons.

Global Generic settings

Section contains global settings which are either generic or I haven’t figured out how to classify it under a better section.

Let’s begin with the setup of some fundamentals like if we want tabs or spaces, how to select stuff and what system facilities to use.

I’m of the spaces-no-tabs religion:

(setq-default indent-tabs-mode nil)     ; do not insert tabs when indenting by default
(setq tab-width 4)                      ; 4 spaces by default

Finally, I want to prevent producing files which have trailing spaces, because they serve no purpose at all. Removing these spaces is less trivial than it seems and I am not pretending to know what the best strategy is. For a while now, the ws-butler package has done a great job for me.

(use-package ws-butler
  :diminish
  :init
  (ws-butler-global-mode)             ; Enable by default
  ;; List exemptions here
  (add-to-list 'ws-butler-global-exempt-modes 'magit-mode)
  (add-to-list 'ws-butler-global-exempt-modes 'message-mode) ; might interfere with format-flowed?
  (add-to-list 'ws-butler-global-exempt-modes 'mu4e-compose-mode))

(setq
 delete-by-moving-to-trash t          ; move files to the trash instead of rm
 select-enable-clipboard t            ; use the clipboard in addition to kill-ring

 display-warning-minimum-level 'error
 large-file-warning-threshold nil
 find-file-use-truenames nil
 find-file-compare-truenames t

 minibuffer-max-depth nil
 minibuffer-confirm-incomplete t
 complex-buffers-menu-p t
 next-line-add-newlines nil
 kill-whole-line t

 auto-window-vscroll nil)

;; Map backspace to DEL and delete to C-d
(if window-system  (normal-erase-is-backspace-mode t))

;; Let marks be set when shift arrowing, everybody does this
(setq shift-select-mode t)
(delete-selection-mode 1)

;; Only require to type 'y' or 'n' instead of 'yes' or 'no' when prompted
(defalias 'yes-or-no-p 'y-or-n-p)

;; Use auto revert mode globally
;; This is safe because emacs tracks if the file is saved in the editing buffer
;; and if so, it will not revert to the saved file.
(use-package autorevert
  :diminish auto-revert-mode

  :config
  (setq auto-revert-interval 5)               ; Perhaps too often, let's see
  (setq global-auto-revert-non-file-buffers t)  ; things like dired buffers


  (global-auto-revert-mode t))

;; Should this be here?
;; Try to have urls and mailto links clickable everywhere
(setq goto-address-url-mouse-face 'default)
(define-global-minor-mode global-goto-address-mode
  goto-address-mode
  (lambda ()
    (goto-address-mode 1)))
(global-goto-address-mode t)

;; Shorten urls for mode specific syntax
(defun mrb/shortenurl-at-point ()
  (interactive)
  (let ((url (thing-at-point 'url))
        (bounds (bounds-of-thing-at-point 'url)))
    (kill-region (car bounds) (cdr bounds))

    ; Leave url as is, unless mode has specific link syntax
    (insert (format (cond ((eq major-mode 'org-mode)      "[[%1s][%2s]]")
                          ((eq major-mode 'markdown-mode) "[[%2s][%1s]]")
                          (t "%1s"))
                    url
                    (truncate-string-to-width url 40 nil nil "…")))))
(bind-key "C-c s" 'mrb/shortenurl-at-point)

Make life easier if we have sudo, so we can just edit the files and be done with them if possible

(use-package sudo-save)

Using pdf-tools for PDF handling. This is a lot better than docview. I’m a bit annoyed that they are not one package, they are very similar.

(use-package pdf-tools
  :straight (:type built-in)            ; i.e GUIX in this case
  :demand t                             ; prevent configuring while opening pdf files
  :after (fullframe nano-modeline)
  :magic ("%PDF" . pdf-view-mode)
  :hook ((pdf-view-mode . (lambda () (cua-mode 0)))
         (pdf-view-mode . nano-modeline-pdf-mode))
  :commands (pdf-info-getannots mrb/pdf-extract-info)
  :bind ( :map pdf-view-mode-map
          ("h"   . 'pdf-annot-add-highlight-markup-annotation)
          ("t"   . 'pdf-annot-add-text-annotation)
          ("D"   . 'pdf-annot-delete)
          ("C-s" . 'isearch-forward)
          ("s-c". 'pdf-view-kill-ring-save)
          ("m"   . 'mrb/mailfile)
          ("q"   . 'kill-current-buffer)
          ([remap pdf-misc-print-document] . 'mrb/pdf-misc-print-pages)
          :map pdf-annot-edit-contents-minor-mode-map
          ("<return>"   . 'pdf-annot-edit-contents-commit)
          ("<S-return>" .  'newline))

  :config
  (setq pdf-info-epdfinfo-program "/home/mrb/.guix-profile/bin/epdfinfo")
  (pdf-tools-install) ; autobuild should not happen now, so enable the question if it does
  (require 'pdf-annot)

  ;; Some settings from http://pragmaticemacs.com/emacs/even-more-pdf-tools-tweaks/
  (fullframe pdf-view-mode kill-this-buffer)
  (setq-default pdf-view-display-size 'fit-page) ;scale to fit page by default
  (setq pdf-annot-activate-created-annotations t ; automatically annotate highlights
        pdf-view-resize-factor 1.1              ; more fine-grained zooming
        pdf-view-midnight-colors '("#DCDCCC" . "#383838")) ; Not sure what this is

  (add-to-list 'revert-without-query ".+\\.pdf") ; pdf is always a reflection of disk file
  ;; FIXME does this work out ok with annotations?

  ;; NOTE: assume lp as printer program!
  (setq pdf-misc-print-program-executable "/usr/bin/lp")
  (defun mrb/pdf-misc-print-pages(filename pages &optional interactive-p)
    "Wrapper for `pdf-misc-print-document` to add page selection support"
    (interactive (list (pdf-view-buffer-file-name)
                       (read-string "Page range (empty for all pages): "
                                    (number-to-string (pdf-view-current-page)))
                       t) pdf-view-mode)
    (let ((pdf-misc-print-program-args
           (if (not (string-blank-p pages))
               (cons (concat "-P " pages) pdf-misc-print-program-args)
             pdf-misc-print-program-args)))
      (pdf-misc-print-document filename))))

Integrate this with orgmode as well. org-pdftools provides a link type, so we can link into pdf documents. pdf-tools-org provides utility functions to help export pdf annotations into org mode and vice versa

(use-package org-pdftools               ; this brings in org-noter as well
  :after (org pdf-tools)
  :hook (org-mode . org-pdftools-setup-link))

(use-package pdf-tools-org
  :demand t  ; when should this be loaded? goes on pdftools load now
  :after (org pdf-tools)
  :straight (:host github :repo "machc/pdf-tools-org"))

I tend to compile Emacs with ImageMagick support, let’s make sure we use it too.

(when (fboundp 'imagemagick-register-types)
  (imagemagick-register-types))

Quick install of package that exports qrencode-region function. I use this mainly to send links or package tracking codes to my mobile.

(use-package qrencode)

Internationalization and multi-language features

If anything multi-language should work, UTF-8 encoding is a must, so let’s make sure we try to use that everywhere

(use-package mule
  :straight (:type built-in)
  :init
  (setq default-input-method 'TeX)
  (prefer-coding-system 'utf-8)
  :config
  (set-terminal-coding-system 'utf-8)
  (set-keyboard-coding-system 'utf-8))

For conveniently editing accented characters like ‘é’ and ‘è’ there are quite a few options to reach that result. I have dead characters configured as an option in the operating system, but that is far from ideal, especially when programming. As I hardly need those characters outside of emacs, i can leave that option as needed and optimize Emacs to my needs.

The fallback is C-x 8 C-h which gives specific shortcuts for special characters which are available. For the exotic characters that will do just fine. For the more common characters the C-x 8 prefix is to complex.

After evaluating some systems, the TeX input method suits me the best. I’d like to enable that globally by default, which needs two things:

1. enable multi-language editing by default (input-method is only active in a multi-language environment)
2. set the default input-method to tex

There is however a problem, the TeX input method assumes the first character / string produced will always be the choice I need, without allowing selecting an alternative. This turns out to be due to quail-define-package which determines the way the completions work. The problem is the DETERMINISTIC argument of the function, that is set to ‘t’. (8th argument). While I am at it, I also changed the FORGET-LAST-SELECTION (7th argument) to nil, so the last selection is remembered.

For this to work properly we have to define a whole new input-method based on a copy of latin-ltx.el

;; No input method will be active by default, so for each mode where
;; it needs to be active we need to activate it (by a hook for example)
(defun mrb/set-default-input-method()
  (interactive)
  (activate-input-method "TeX")

  ;; Define a few omissions which I use regularly
  (let ((quail-current-package (assoc "TeX" quail-package-alist)))
    (quail-define-rules ((append . t))
                        ("\\bitcoin" ?฿)
                        ("\\cmd" ?⌘)
                        ("\\shift" ?⇧)
                        ("\\alt" ?⎇)
                        ("\\option" ?⌥)
                        ("\\return" ?⏎)
                        ("\\tab" ?↹)
                        ("\\backspace" ?⌫)
                        ("\\delete" ?⌦)
                        ("\\plusminus" ?±)
                        ("\\_1" ?₁)
                        ("\\_2" ?₂))))


;; Set the default language environment and make sure the default
;; input method is activated
(add-hook 'set-language-environment-hook 'mrb/set-default-input-method)
;; And now we can set it
(set-language-environment "UTF-8")
;; Make sure our timestamps within emacs are independent of system locale
;; This is mostly for orgmode I guess.
(setq system-time-locale "nl_NL.UTF-8")


;; Activate it for all text modes
(add-hook 'text-mode-hook 'mrb/set-default-input-method)

For even more esoteric characters we have to do some extra work. No font provides all Unicode characters. While I previously had a manual solution, I’m now relying on the package uni-code-fonts to do the proper mapping. The first run will be very slow, but the information gets cached.

The problem with applying this function is that we need to be ‘done’ with our visual initialization or otherwise they’ll do nothing (at least that I can see). So, let’s group our corrections in a function and call that when we are done with our visual (X) init.

(use-package unicode-fonts
  :demand t
  )

(defun mrb/unicode-font-corrections()
  (interactive)
  ;; TODO this fails with recent emacs-28, finds '...' in unicode-fonts pcache file??
  ;; TODO it works with emacs-29 but cache state is not save, so it runs on startup always
  (unicode-fonts-setup)
  ;; Add a line like the following  for each char displaying improperly

  ;; mu4e uses Ⓟ,Ⓛ and Ⓣ, let's correct all letters
  (set-fontset-font t '(?Ⓐ . ?Ⓩ) "Symbola") ; noto line-height too high
)

So, when characters do not show properly, the steps to take now are:

1. Find a font which has the char
2. Map the character(-range) to that font
3. Optional: define a convenient way to type the character

To make my life a bit easier, using the all the icons package for using special symbols.

(use-package all-the-icons
  :if (display-graphic-p))              ; TODO  when in daemon mode, this does not get loaded

The above in itself does nothing. It just makes using the icons easier and consistent. Other packages. With the command M-x all-the-icons-install-fonts the latest version of the proper fonts will be installed automaticaly, so run that once in a while.

Visual

Many settings have to do with how the screen looks and behaves in a visual way. Things like screen layout, colors, highlighting etc. fall into this category.

Let’s set up some basics first, background dark, some default frame and cursor properties:

(setq mrb/cursor-type 'box)
(setq mrb/cursor-color "DarkOrange")  ; in fat orange color


(setq-default frame-background-mode 'dark)
(set-mouse-color "white")

;; Only show cursor in the active window.
(setq-default cursor-in-non-selected-windows nil)

;;Default frame properties frame position, color, etc
(setq default-frame-alist
      `((cursor-type . ,mrb/cursor-type )
        (height . 60) (width . 100)
        (cursor-color . ,mrb/cursor-color)
        (internal-border-width . 24)
        (mouse-color . "white")))

Mode-line and status info

One thing that always bothered me is the repeated mode-line for every buffer with information that only needs to be displayed once. Like the new-mail indicator or time-display for example. This section explores a configuration to scratch that itch.

Local information, that is per buffer or per mode, should be displayed in the mode-line at the bottom of each visible buffer. There will be exceptions to this for special buffers where this does not make sense. This will be handled by the nano-modeline package.

Global information, typically at least what global-mode-string holds, needs to be displayed only once (per frame). This will be handled by tab-bar-mode at the top of each frame, under the menu-bar.

The format of the mode-line is:

[ status name (primary) secondary ]

where:

status

gives status of buffer (RO/RW/**/– etc.)

name

name or filename to identify the contents of the buffer

primary

buffer specific primary information, left aligned

secondary

buffer specific secondary information, right aligned

Both primary and secondary are buffer and/or mode-specific and some are configured by the nano-modeline package. Per mode configuration of the mode-line contents is possible in the nano-modeline package.

The format of the status-bar is:

[ primary secondary ]

where:

primary

global or mode specific information, no buffer specific information, left aligned

secondary

global or mode specific information, no buffer specific information, right aligned

One of the elements in the status bar will be a list of buffers which need attention for some reason (irc, compilation etc.). For this the tracking package will be used, which is part of circe, but has its own install recipe. I only use it for irc right now.

(use-package tracking
  :custom
  (tracking-max-mode-line-entries 1)
  (tracking-shorten-buffer-names-p nil)

  :config
  (tracking-mode 1))

(use-package nano-modeline
  :after tracking

  :config
  ;; Take control of the mode-line, mostly leaving out what goes into the tab-bar
  ;; - tracking is global, so move to tab bar
  ;; - misc info has globabl mode string, moved to tab bar, but may have more?? TODO

  (setq-default mode-line-format
        (list
         "%e" mode-line-front-space
         '(:propertize ("" mode-line-mule-info mode-line-client mode-line-modified mode-line-remote)
          display (min-width (5.0)))
         mode-line-frame-identification
         mode-line-buffer-identification
         "   "
         mode-line-position
         '(vc-mode vc-mode)
         "  "
         mode-line-modes
         ;mode-line-misc-info
         mode-line-end-spaces))

  :custom
  ;; tab-bar customization
    (tab-bar-format '(mrb/tab-bar-format-primary
                    tab-bar-format-align-right
                    mrb/tab-bar-format-secondary))
    (tab-bar-position t "Position tab-bar under the menu-bar")
    (tab-bar-mode t "Enable tab-bar-mode unconditionally")
    (global-tab-line-mode nil "Make sure tab-line-mode is disabled")

  (nano-modeline-position 'nano-modeline-footer "Mode line goes at the bottom")

  :config
  (nano-modeline-text-mode t)            ; Make this the default

  :custom-face
  ;; TODO I like :custom-face, but not the scattering of all the colors, name them?
  (nano-modeline-active ((t ( :height 1.1 :))))
  (nano-modeline-status ((t ( :background "#bf616a"))))

  (tab-bar   ((t ( :background "#4c566a"
                   :height 1.1
                   :box (:line-width 1 :color "dim gray" :style flat-button)))))

  :init
  ; Helper functions for the tab-bar-format
  (defun mrb/tab-bar-format-primary () "") ;not used yet
  (defun mrb/tab-bar-format-secondary ()
    (concat
     (propertize " " 'display `(raise 0.2))
     (format-mode-line tracking-mode-line-buffers) ; buffers that need attention
     (format-mode-line global-mode-string))))

Theme

I’m using the Nord color palette. This palette defines 16 colors.

There is a base16 theme which uses exactly these colors and there are varieties which use the 16 colors and some derivatives of it.

Although I think the 16 color system is a bit simplistic and, for emacs at least, I will need to customize more later on, the main lure of this system is that I can use the same set for many of my programs I use (emacs, xterm, i3, dunst etc.) which sounds attractive. So, I’m starting with the base16-nord theme and see where this leaves me.

(setq custom--inhibit-theme-enable nil)  ; This was needed for Emacs 27, but cant recall why
(use-package base16-theme
:custom
  ;; Do config here and finally load the theme
(base16-theme-distinct-fringe-background nil)
(base16-theme-highlight-mode-line 'contrast)
(base16-theme-256-color-source "terminal")

:config
  (load-theme 'base16-nord t)

  ;; Define our adjustments to faces
  ;; This should be the only place where we adjust our faces
  ;; For reference, these are the theme colors

  ;; :base00 "#2e3440"
  ;; :base01 "#3b4252"
  ;; :base02 "#434c5e"
  ;; :base03 "#4c566a"
  ;; :base04 "#d8dee9"
  ;; :base05 "#e5e9f0"
  ;; :base06 "#eceff4"
  ;; :base07 "#8fbcbb"
  ;; :base08 "#88c0d0"
  ;; :base09 "#81a1c1"
  ;; :base0A "#5e81ac"
  ;; :base0B "#bf616a"
  ;; :base0C "#d08770"
  ;; :base0D "#ebcb8b"
  ;; :base0E "#a3be8c"
  ;; :base0F "#b48ead"

  ;; TODO  some of these adjustments come in too early and the package redefines it on loading
  ;; the only solution for this is to move the visual adjustments in the use-package clauses of
  ;; the package itself.
  (setq base16-adjustments
        `(
          ;; Explicitly define the outline colors
          (outline-1 :foreground "#5e81ac")
          (outline-2 :foreground "#a3be8c")
          (outline-3 :foreground "#b48ead")
          (outline-4 :foreground "#81a1c1")
          (outline-5 :foreground "#ebcb8b")
          (outline-6 :foreground "#8fbcbb")
          (outline-7 :foreground "#d08770")
          (outline-8 :foreground "#bf616a")

          ;; Adjustment for non-package faces
          ;; Comments should not be that dark as base03
          (font-lock-comment-face         :foreground "#777777")
          (font-lock-doc-face             :inherit font-lock-comment-face)

          (button                         :foreground "#d08770" :weight semi-bold)

          (show-paren-match               :foreground "#eceff4" :background "#81a1c1"
                                          :inherit nil :weight normal)
          (show-paren-mismatch            :foreground "#2e3440" :background "#bf616a"
                                          :inherit nil :weight normal)

          (region                         :foreground unspecified :background "#3b4252"
                                          :inherit nil :weight normal)
          (fringe                         :foreground "#a3be8c")

          (hl-line                        :foreground unspecified :background "#5e81ac" )

          (line-number-current-line       :background "#4c566a")

          (lazy-highlight                 :foreground "#434c5e" :background "#5e81ac")
          (isearch                        :foreground "#434c5e" :background ,mrb/cursor-color)

          ;; Mode-line faces
          (header-line                    :foreground "#4c566a" :background "#ebcb8b")


          ;; Man pages inside emacs
          (Man-overstrike                 :foreground "#bf616a" :weight bold)
          (Man-underline                  :foreground "#a3be8c" :weight bold)
          ))

  ;; Apply our adjustment using the theme function
  (base16-theme-set-faces 'base16-nord base16-nord-theme-colors base16-adjustments))

Miscellaneous other visual settings follow.

;; no splash screen
(setq inhibit-startup-screen  t)
(setq inhibit-startup-message t)
(setq initial-scratch-message nil)

;; check speed consequences of this
(setq column-number-mode t)

(use-package mic-paren
  :custom
  (paren-highlight-at-point nil)

  :config
  (paren-activate))

;; Font locking has always been the major performance hog for me, so here's the current
;; state of variables trying to minimize typing delays. The 3 position indication in
;; comments is the  experience playing with it:
;;   ? unknown effect
;;   ! needed can't change
;;   + minor effect, but noticeable
;;  ++ major effect, very noticeable
;; +++ very high effect on performance (mostly subjective, but supported by some light profiling)
(setq font-lock-support-mode 'jit-lock-mode)
(setq jit-lock-defer-time 0)   ;   ?  defer only when input pending
(setq jit-lock-stealth-time 10) ;  ++ stealth fontification after 5 seconds, nil is slower


;; Make underlining nicer
(setq  underline-minimum-offset 3)

;; Show color of '#RRBBGG texts
(use-package rainbow-mode
  :diminish)

;; Give commands the option to display full-screen
(use-package fullframe)

Lines

The most important element of an editor is probably how lines of text are displayed. This warrants its own section.

The problem is that ‘line’ needs clarification because there can be a difference between a logical line and a visual line. It’s sometimes important that things stay on one logical line but are displayed over multiple lines. This may be different per mode.

I recently reversed my stance on this and am now enabling visual line mode everywhere (meaning one logical line can be smeared out over multiple display lines. This also enables word-wrap which breaks lines visually between words instead of at random characters.

To help me see the difference between visual lines and logical lines I let emacs display indicators in the fringe and define a function to help me unfill existing paragraphs.

;; Set some defaults
(setq-default word-wrap t               ; why would i want to have no wrapping by word?
              fill-column 100
              truncate-lines nil)

(setq sentence-end-double-space nil)    ; A period ends a sentence, period. relevant for fill

(use-package visual-fill-column
  :commands (turn-on-visual-fill-column-mode))

  ;; Similar to mail messages, use vertical bar for wrapped paragraphs
  (setq visual-line-fringe-indicators
      '(vertical-bar nil))

;; For all text modes use visual-line-mode
(add-hook 'text-mode-hook 'visual-line-mode)

;; From:https://www.emacswiki.org/emacs/UnfillParagraph
(defun unfill-paragraph (&optional region)
  "Takes a multi-line paragraph and makes it into a single line of text."
  (interactive (progn (barf-if-buffer-read-only) '(t)))
  (let ((fill-column (point-max))
        ;; This would override `fill-column' if it's an integer.
        (emacs-lisp-docstring-fill-column t))
    (fill-paragraph nil region)))

 ;; Similar to M-q for fill, define M-Q for unfill
(bind-key "M-Q" 'unfill-paragraph)

Line-numbering is off by default, it tends to slow things down, but if I want it on, I almost always want them relative

(use-package display-line-numbers
  :straight (:type built-in)
  :custom
  (display-line-numbers-type 'relative)
  (display-line-numbers-mode -1))

Client dependent settings

Because most of my usage involves having a long lasting emacs daemon, some settings only come into scope once a client connects to that daemon. Most of these settings have to do with appearance and are related to having X available. Anyways, some settings need to be moved to the event when a client visits the server, so we can still apply these settings transparently.

Note that if this code is evaluated any call to emacsclient (be that from external or, more importantly Magit) will try to run this code and magit will fail if there’s an error in the next section. Take extra care here.

(defun mrb/run-client-settings(&optional args)
  (interactive)

  (tool-bar-mode -1)   ;; No tool-bar
  (scroll-bar-mode -1) ;; No scroll-bar
  (menu-bar-mode -1)   ;; No menu-bar
  (tooltip-mode -1)    ;; No tooltips
  (setq fringe-mode '(14 . 14)) ;; Fringe, left and right for the continuation characters
  (set-fringe-mode fringe-mode)
  (setq indicate-buffer-boundaries 'left)

  (mrb/unicode-font-corrections))


;; Run our client settings after we make a frame This is probably
;; overkill to do on every frame, but it covers the situation where
;; `server-visit-hook` won't work because we started emacs without a
;; file. This config covers all situations.
(add-hook 'server-after-make-frame-hook 'mrb/run-client-settings)
;; And we need to run those settings *now* too, when we are not in server mode
(mrb/run-client-settings)

Context dependent visualization

Next to the information registered in the theme which controls the overall look and feel, there is additional visualization which depend on context dependent factors, such as:

• the user spells a word wrong
• the syntax of a certain phrase is wrong grammatically, depending on what language the user is writing in
• the status of a certain region is changing, dependent on certain rules
• a part of the text could be rendered as non-text, images or formulas for example

Flycheck

Flycheck is a syntax checking package which seems to have better support than flymake, which is built-in. I’ve no configuration for flymake specifically, but some packages enable it automatically (elpy for example). Where applicable, I gather the necessary disable actions here first.

Now we can start configuring flycheck

(use-package flycheck-pos-tip)

(use-package flycheck
  ;; Use popup to show flycheck message
  :after flycheck-pos-tip
  :config
  (with-eval-after-load 'flycheck
    (flycheck-pos-tip-mode)))

Currently actively configured are:

javascript

eslint with a config file

Flyspell is similar to flycheck but for text languages. I’m setting some of the flyspell faces to the flycheck faces so they are consistent

(use-package flyspell
  :straight (:type built-in)
  :custom-face
  (flyspell-incorrect ((t (:inherit flycheck-error))))
  (flyspell-duplicate ((t (:foreground "#2e3440" :inherit flycheck-warning))))
  :custom
  (flyspell-mark-duplications-flag nil "Flyspell Duplicates as warning")
  (flyspell-issue-message-flag nil))

Still on the wish-list:

• activate flyspell automatically for all text-modes?
• language detection (English and Dutch mainly)
• check for language pythons, which was troublesome before
• evaluate usage in org source blocks (many checks do not apply)

Math rendering

Mathematics requires some specific rendering. In simple cases, using the proper unicode symbols may suffice, but for anything other than the most basis equations or formulas, some help is needed.  $\LaTeX$ is the defacto standard for this and there’s plenty of support for that in emacs.

For $\TeX$ and $\LaTeX$ fragments I use the texfrag package. This allows to insert math fragments inside documents.

Example:

$$eix = cos x + i sinx$$

(use-package texfrag
  :disabled t
  :diminish

  :custom
  (texfrag-prefix "��")

  :config
  (texfrag-global-mode 1))

Highlighting tags and keywords

In many modes, but especially in orgmode, the use of tags and keywords is abundant. In the set of packages that Nicolas P. Rougier has made for emacs, there’s also a gem which generates SVG labels from keyword matches.

The use of that package is simple, just create a list of keyword matches in the svg-tag-tags variable and match those up with a command to create a tag.

I have at least the following usecases in mind:

1. Render my orgmode keywords as tags;
2. Use in mu4e to render the tags
3. perhaps some TODO keywords in code, I use that quite a bit.

The first is the easiest, as the format of org-todo-keyword-faces is nigh perfect to transform into a list that can be used by svg-tag-mode.

;; Trickiest here is to know when org-todo-keyword-faces has been filled
(use-package svg-tag-mode
  :after (org base16-theme)
  :commands svg-tag-mode                ; Possibly redundant?
  :hook (org-mode . svg-tag-mode)

  :config
  ;; Readymade svg-tag-tags generator from org keywords
  (defun mrb/gen-svg-tags (kf)
    "Generate a `svg-tag-tags element from a `org-todo-keyword-faces element"
    (let ((k (concat "\\(" (car kf) "\\) ")) ; pass keyword, not the space
          (f (cdr kf)))
      `(,k . ((lambda (tag) (svg-tag-make tag :face ',f :inverse t :margin 0 ))))))

  ;; This gets initialized the first time svg-tag-mode is called, so then it must have
  ;; the proper keywords in org-todo-keyword-faces
  (setq svg-tag-tags (mapcar 'mrb/gen-svg-tags org-todo-keyword-faces))

  ;; See: https://github.com/rougier/svg-tag-mode/issues/27
  ;; Org agenda does not use font-lock, so needs separate overlay to render
  (defun mrb/org-agenda-show-svg ()
  (let* ((case-fold-search nil)
         (keywords (mapcar #'svg-tag--build-keywords svg-tag--active-tags))
         (keyword (car keywords)))
    (while keyword
      (save-excursion
        (while (re-search-forward (nth 0 keyword) nil t)
          (overlay-put (make-overlay
                        (match-beginning 0) (match-end 0))
                       'display  (nth 3 (eval (nth 2 keyword)))) ))
      (pop keywords)
      (setq keyword (car keywords)))))
  (add-hook 'org-agenda-finalize-hook #'mrb/org-agenda-show-svg))

Buffers and files

How do I deal with all those buffers and files?

For starters, make sure that they have unique buffer names so I don’t get confused:

(setq uniquify-buffer-name-style 'forward)

For every file-based buffer, I want auto-save to be on, but not in the same location as the file, as that clutters up everything. For that, I add to the list of file-name transforms to have (local) files auto saved in a designated folder)

(setq auto-save-default t
      mrb/auto-save-folder (concat user-emacs-directory "auto-save-list/"))

(add-to-list 'auto-save-file-name-transforms
             (list "\\(.+/\\)*\\(.*?\\)" (expand-file-name "\\2" mrb/auto-save-folder))
             t)

Auto save helps to automatically recover to the latest save point, but I need a bit more. This is done through versioned backup file. The recovery process is then manual though.

(setq create-lockfiles nil           ; just clutters for me, no real use?
      backup-by-copying t            ; otherwise symlinks galore

      backup-directory-alist '(
                               ("." . "/tmp/emacs/"))

      delete-old-versions t
      kept-new-versions 3
      kept-old-versions 2
      version-control t)

The auto save helps for the minor disasters, my backups help for the major disasters. What else is needed is a ‘normal save’ but automatically when I forget to do this.

What I am aiming for here is to not have to think about explicitly saving for certain files. Typically when typing stuff in org-mode I just want the stuff saved that I have types so far. For some files, each save is committed if I think the content warrants this (for example if I think going back to an earlier version is a likely event).

(defconst mrb/idle-timer 15
  "Time emacs needs to be idle to trigger the save idle timer")

;; This function does the actual useful thing
(defun mrb/save-timer-callback()
  "Callback function that runs when emacs is idle for
`mrb/idle-timer' seconds. Typically we save files here"
  (org-save-all-org-buffers))

;; Activate the timer
;; The # means: evaluate first ( I keep forgetting this stuff )
(run-with-idle-timer mrb/idle-timer 'always #'mrb/save-timer-callback)

Also, save the location in files between sessions.

;; Minibuffer prompt is a prompt, don't enter it as text.
(setq minibuffer-prompt-properties '(read-only t point-entered minibuffer-avoid-prompt face minibuffer-prompt))
;; Don't echo keystrokes in the minibuffer, prefix will be hinted on after 2 seconds or so.
(setq echo-keystrokes 0)

;; Save places in buffers between sessions
(use-package saveplace
  :straight (:type built-in)
  :init
  (setq-default save-place-mode t))

Bind renaming the visiting file to a new location to C-x-w now that we have rename-visited-file

(bind-key "C-x C-w" 'rename-visited-file)
(bind-key "C-S-s" 'write-file)

For file management itself, dired is the defacto standard, but I do not use it much really. To help me out with keys and commands, I’m adding the casual-dired package.

(use-package dired
  :straight (:type built-in)
  ;; auto-revert global is only for file buffers, so explicit enable is needed
  :hook (dired-mode-hook . auto-revert-mode)
  :config
  (use-package casual-dired
    :ensure t
    :bind (:map dired-mode-map ("C-o" . 'casual-dired-tmenu))))

Modes

Customization setting for specific modes. Most of the modes I use have a separate section, so this section is only for other modes.

To determine the default major mode; the mode that is started with before all the magic starts is determined by buffer-file-name. If we have it, the normal routine can be followed. If there is no filename yet, the buffer-name is used to determine which mode is needed.

By looking at the code this may have a side-effect, because the buffer-file-name is given a value. Let’s try this and see if it gives any issues.

(setq-default major-mode
              (lambda ()
                (if buffer-file-name
                    (fundamental-mode)
                  (let ((buffer-file-name (buffer-name)))
                    (set-auto-mode)))))

When emacs does not determine the right mode, I sometimes want a mode-string somewhere in the file. Typically that string gets inserted on a line which is commented out in the proper syntax for that mode.

(defun mrb/buffer-majormode (&optional buffer-or-name)
  "Returns the major-mode of the specified buffer, or
    the current buffer if no buffer is specified"
  (buffer-local-value 'major-mode
                      (if buffer-or-name
                          (get-buffer buffer-or-name)
                        (current-buffer))))

(defun mrb/insert-mode-string()
  "Inserts a mode string for the current mode at beginning of the current buffer"
  (interactive)
  (let ((m (symbol-name (mrb/buffer-majormode))))
    (save-excursion
      (goto-char (point-min))
      (insert "-*- mode:"
              (substring m 0 (string-match "-mode" m))
              " -*-")
      (comment-region (point-min) (point)))))

First bring in a bunch of modes that are used regularly, but have no other config than a map to some extensions. :requires is set to nil for those packages which aret built in.

(use-package adoc-mode             :mode ("\\.asciidoc\\'" "\\.adoc\\'"))
(use-package apache-mode           :mode "\\.htaccess\\'")
(use-package conf-mode )
(use-package i3wm-config-mode)
(use-package css-mode   :mode ("\\.css\\'" "\\.mss\\'"))
(use-package csv-mode              :mode "\\.csv\\'")
(use-package diff-mode  :mode "\\.patch\\'")
(use-package gnuplot)
(use-package gnuplot-mode          :mode "\\.gp\\'")
(use-package js2-mode              :mode "\\.js\\'")
(use-package lua-mode              :mode "\\.lua\\'")
(use-package php-mode              :mode "\\.php\\'")
(use-package sass-mode             :mode "\\.sass\\'")
(use-package markdown-mode         :mode "\\.md\\'")
(use-package udev-mode             :mode "\\.rules\\'")
(use-package dockerfile-mode       :mode "Dockerfile")

Now, do the other mode related packages which require a bit of configuration.

(use-package eimp
  :hook (image-mode . eimp-mode))

(use-package rainbow-mode
  :hook (conf-mode css-mode))

;; Open scratch buffer by default in the mode we are in at the moment
;; with C-u prefix a mode will be asked to use
(use-package scratch)
;; Don't throw away scratch buffer
(use-package persistent-scratch
  :config
  (persistent-scratch-setup-default))

(use-package nxml-mode
  :straight (:type built-in)
  :hook ((nxml-mode . (lambda () (set-input-method nil)))
         (nxml-mode . turn-off-auto-fill))
  :custom
  (nxml-heading-element-name-regexp "\\|.*")
  (nxml-section-element-name-regexp "\\|file\\|.+"))

Org-mode

Orgmode configuration is probably the largest part of my Emacs configuration, because most of the time I spent in Emacs, when not coding, is spent in org-mode.

Initialization of Orgmode

I loosely follow the GTD method for organizing things and I want a quick shortcut to start my main file.

(defun mrb/gtd()
  "Start my GTD system"
  (interactive)
  (find-file org-default-notes-file))

We do not have to load the main orgmode location, because we already did that on the main initialization to get org-babel started.

Also make sure we never load org from internal, this can happen when functions were defined in the included org version and not anymore in newer versions. We want an error, not a silent load of the older function.

(require 'org-agenda)

(use-package org
  :straight (org
             :includes (org-agenda org-capture org-crypt org-datetree org-protocol))
  :hook ((org-mode . turn-on-visual-line-mode)
         (org-mode . turn-on-visual-fill-column-mode)
         (org-mode . (lambda () (setq tab-with 8)))
         (org-indent-mode . (lambda () (diminish 'org-indent-mode)))
         (org-agenda-mode . (lambda () (hl-line-mode 1))))

  :init
   <<orgmode-keywords>>
  <<gtd-starter>>

  :mode ("\\.txt\\'" "\\.org\'")
  :bind (("C-c g"   . 'mrb/gtd)
         ("C-c a"   . 'org-agenda)
         ("C-c b"   . 'org-switchb)
         ("C-s-s"   . 'org-save-all-org-buffers)
         ("C-c l"   . 'org-store-link)        ; Is this not bound to C-c l by default
         :map org-mode-map
         ("s-."     . 'org-todo)
         ("s->"     . 'mrb/org-cancel-todo)
         ("C-s-."   . 'mrb/force-org-todo)
         ("M-p"     . 'org-set-property)
         :map org-agenda-mode-map
         ("s-."   . 'org-agenda-todo)
         ("s->"   . 'mrb/org-agenda-cancel-todo)
         ("C-s-." . 'mrb/force-org-agenda-todo)
         ("C-."   . 'org-agenda-schedule)
         ("M-p"   . 'org-set-property)
         ("C-x m" . 'mrb/construct-mail))

  :custom-face
  ; Orgmode
  (org-headline-done              ((t (:foreground "#4c566a"))))
  (org-date                       ((t (:foreground "#8fbcbb" :underline nil))))
  (org-verbatim                   ((t (:foreground "#8fbcbb"))))
  (org-list-dt                    ((t (:foreground "#b48ead"))))
  (org-drawer                     ((t (:height 0.8 :inherit font-lock-comment-face))))
  (org-code                       ((t (:foreground "dodger blue" :weight bold))))
  (org-block                      ((t (:background "#313745"))))
  (org-block-begin-line           ((t (:foreground "dark gray"))))

  ;; User defined faces for org-mode
  (mrb/org-todo-keyword-TODO     ((t ( :foreground "#88c0d0" :weight bold))))
  (mrb/org-todo-keyword-DONE     ((t ( :foreground "#a3be8c" :weight bold))))
  (mrb/org-todo-keyword-WAITING  ((t ( :foreground "#bf616a" :weight bold))))
  (mrb/org-todo-keyword-CANCELLED((t ( :foreground "#3b4252" :weight bold))))
  (mrb/org-todo-keyword-BUY      ((t ( :foreground "#d08770" :weight bold))))
  (mrb/org-todo-keyword-HOWTO    ((t ( :foreground "#5e81ac" :weight bold))))
  (mrb/org-todo-keyword-INFO     ((t ( :foreground "#ebcb8b" :weight bold))))
  (mrb/org-todo-keyword-COLLECT  ((t ( :foreground "#a3be8c" :weight bold))))
  (mrb/org-todo-keyword-SOLVE    ((t ( :foreground "#8fbcbb" :weight bold))))
  (mrb/org-todo-keyword-READ     ((t ( :foreground "#b48ead" :weight bold))))
  (mrb/org-todo-keyword-READING  ((t ( :foreground "#bf616a" :weight bold))))
  (mrb/org-todo-keyword-PLAN     ((t ( :foreground "#d8dee9" :weight bold))))


  :config
  <<orgmode-generalconfig>>
  ;; Category icons I have configured elsewhere.
  <<org-agenda-visuals>>

  <<orgmode-itemactions>>

  <<orgmode-scheduling>>

  ;; Allow for archiving and refiling in a date organized tree
  (use-package org-datetree )
  (use-package org-protocol
    :config
    ;; If nothing is specified, create a TODO item
    (setq org-protocol-default-template-key "t")))

Gather generic config variables for orgmode in one section

(setq org-directory "~/dat/org/"
      org-metadir (concat org-directory "_orgmeta/")

      org-use-fast-todo-selection t
      org-use-fast-tag-selection t
      org-use-speed-commands t
      org-fast-tag-selection-single-key 'expert
      org-enforce-todo-dependencies t       ; we do want task dependencies
      org-enforce-todo-checkbox-dependencies nil ; but relax checkbox constraints for it

      ;; We do not do priorities
      org-enable-priority-commands nil

      ;; Agenda settings
      org-agenda-inhibit-startup t
      org-agenda-dim-blocked-tasks nil
      org-agenda-use-tag-inheritance nil
      org-agenda-files (concat org-directory ".agenda_files")
      org-agenda-include-diary t
      org-agenda-start-with-log-mode t
      org-agenda-todo-ignore-scheduled "future"
      org-agenda-ignore-properties '(effort stats appt category)
      org-agenda-todo-ignore-scheduled 'future
      org-agenda-log-mode-items '(closed clock state)
      org-agenda-skip-deadline-prewarning-if-scheduled t
      org-agenda-text-search-extra-files (cons 'agenda-archives (directory-files
                                                                 (expand-file-name (concat org-metadir)) t ".+\\.org"))
      ;; Habits
      org-habit-show-habits-only-for-today nil

      ;; Pressing enter on a link should activate it
      org-return-follows-link t

      ;; S-left and friends should work in context
      ;; TODO if 'always, the whole command does not work, even if using the other binding
      org-support-shift-select t


      ;; Auto detect blank line need, this is the default, but I explicitly set this
      ;; because it needs to be this or my capturing breaks due to org-capture popup
      org-blank-before-new-entry '((heading . auto) (plain-list-item . auto))
      org-export-htmlize-output-type 'css

      ;; Use auto-mode-alist for all file links/types
      org-file-apps '((auto-mode . emacs))

      org-fontify-done-headline t
      org-goto-interface 'outline-path-completion
      ;; non nil is just direct children, what an ODD name!!!!
      org-hierarchical-todo-statistics nil
      org-provide-todo-statistics t

      org-log-into-drawer t
      org-log-redeadline 'note
      org-log-reschedule 'time

      org-modules '(org-info org-habit
                             org-inlinetask org-irc
                             org-toc org-mac-iCal org-mouse
                             org-tempo)
      org-remember-default-headline ""
      org-special-ctrl-a/e t
      org-stuck-projects '("-inactive/TODO" ("TODO" "WAITING") nil "")
      org-track-ordered-property-with-tag nil
      org-startup-indented t

      org-archive-location (concat org-metadir "archive.org::datetree/")
      org-default-notes-file (concat org-directory "GTD.org")
      diary-file (concat org-metadir "DIARY"))

Capturing information

I guess 90% of the information I keep in the main orgmode files starts life originally as a captured item. I use it for:

1. TODO items;
2. BUY items;
3. Journal entries;
4. Logbook entries;
5. Weblinks;
6. Toots.

(use-package org-capture
  :diminish

  :custom
  ;; Do not do the automatic bookmarking when capturing
  (org-capture-bookmark nil)
  (bookmark-set-fringe-mark nil)

  ;; Special config for handling tags while capturing
  <<org-capture-taghandling>>

  :config
  ;; Configuration of all the capture templates
  <<org-capture-templates>>
  ;; and some shortcuts to run them
  <<org-capture-runners>>

  ;; Not a :custom, we're making this up as we go.
  (setq ocpf-frame-parameters
        '((name . "*capture*")
          (width . 115) (height . 15)
          (tool-bar-lines . 0) (menu-bar-lines . 0)
          (internal-border-width . 5)))

  (defun ocpf--funcall (func &rest args)
    "Call FUNC in our capture frame only."
    (if (equal (cdr (assoc 'name ocpf-frame-parameters))
               (frame-parameter nil 'name))
        (funcall func args)))

  (defun ocpf--delete-frame (&rest args)
    "Close capture frame"
    (ocpf--funcall 'delete-frame))

  (defun ocpf--delete-other-windows (&rest args)
    "Make sure we have the capture frame to ourselves"
    (ocpf--funcall 'delete-other-windows))

  (defun ocpf--org-capture (orig-fun &optional goto keys)
    "Create a new frame and run org-capture."
    (interactive)
    (let ((frame-window-system
           (cond ((eq system-type 'darwin) 'ns)
                 ((eq system-type 'gnu/linux) 'x)
                 ((eq system-type 'windows-nt) 'w32)))
          (after-make-frame-functions
           #'(lambda (frame)
               (progn
                 (select-frame frame)
                 (funcall orig-fun goto keys)))))
      (make-frame
       `((window-system . ,frame-window-system)
         ,@ocpf-frame-parameters))
      (setq header-line-format nil)     ; this works, is it the right way to do this?
      ))

  ;; All capture frames should have one window
  (add-hook 'org-capture-mode-hook 'delete-other-windows)

  ;; Wrap org-capture in our org-capture
  (advice-add 'org-capture :around #'ocpf--org-capture)

  ;; Proper clean up
  (advice-add 'org-capture-finalize :after #'ocpf--delete-frame))

Here are the templates used by org-capture. The todo template is the most used, it is the same as the link template, but does not include a reference to the current context, which is, in most cases, just annoying.

(setq
 org-capture-templates
 `(("b" "Buy"
    entry (function mrb/capture-location) "* BUY %?\n%(mrb/prop-or-nothing :annotation)\n" :prepend t :empty-lines 1)
   ("l" "Link"
    entry (function mrb/capture-location) "* TODO %?\n%(mrb/prop-or-nothing :annotation)\n" :prepend t :empty-lines 1)
   ("t" "Todo"
    entry (function mrb/capture-location) "* TODO %?\n" :prepend t :empty-lines 1)
   ("g" "Generic Log Entry"
    item  (here) "- %u %?")
   ("z" "Ziggo Log Entry"
    item  (file+olp
           "/home/mrb/dat/org/GTD.org"
           "Partners" "Leveranciers" "Ziggo" "Logbook" ,(format-time-string "%Y"))
    "- %u %?")))

(defun mrb/prop-or-nothing(prop)
  "Insert `[:type]: [:property]` when :property has a value, otherwise nothing.
Meant to be used inside org-capture-templates

Example: `%(mrb/prop-or-nothing :annotation)`"

  (let ((prop-value (plist-get org-store-link-plist prop))
        (type (plist-get org-store-link-plist :type)))
    (if (equal prop-value "")
        ""
      (concat (when type (concat type ": ")) prop-value))))

(defun mrb/capture-location()
  "This function is meant to be used inside org-capture-templates
to find a file and location where a captured ITEM should
be stored."

  ;; Open journal file without creating a journal entry This has the
  ;; side effect that it creates the file and moves TODO items over
  ;; on first call and leaves the cursor at the end of the file.
  (org-journal-new-entry 1)

  ;; Find the id in this file and go there for the capture
  (setq loc (org-find-entry-with-id "new-todo-receiver"))
  (when loc
    (goto-char loc)))

Define functions for each piece of information captured, so they can be easily bound to keys.

:init
(defun mrb/capture-todo ()
  "Capture a TODO item"
  (interactive)
  (org-capture nil "t"))

(defun mrb/capture-buy ()
  "Capture a BUY item"
  (interactive)
  (org-capture nil "b"))

(defun mrb/capture-link ()
  "Capture a TODO item, but link to source when we can"
  (interactive)
  (org-capture nil "l"))

These capture functions are called from shell scripts in the operating system and have a shortcut key assigned to them. The scripts are produced directly from this document, in a similar way as the main edit script was produced in Preparing for lift-off

edit-noframe --eval '(mrb/capture-todo)'

edit-noframe --eval '(mrb/capture-buy)'

By default C-c C-c ends the capture, but is normally the shortcut to enter tags, so I define a shortcut to define tags while capturing.

:bind ( :map org-capture-mode-map
        ("C-c C-t" . mrb/add-tags-in-capture))

:init
(defun mrb/add-tags-in-capture()
  (interactive)
  "Insert tags in a capture window without losing the point"
  (save-excursion
    (org-back-to-heading)
    (org-set-tags-command)))

Workflow

Orgmode used a couple of thing which enable you to steer the workflow for items. Item states are the most prominent ones. Org-mode uses keyword definitions to denote states on items. I keep an Orgmode configuration file (org-config.org) file which contains the description of the workflow in a format suitable to include directly into orgmode files. The configuration of emacs itself is limited to dressing up this configuration with things less suitable to go into that config file. The configuration here and the org config file should be kept in sync.

Adapt the colors of the states I use a bit:

;; Define face specs for our keywords, so they can be used in the
;; theme adjustment like standard faces
(defface mrb/org-todo-keyword-TODO      nil "")
(defface mrb/org-todo-keyword-DONE      nil "")
(defface mrb/org-todo-keyword-WAITING   nil "")
(defface mrb/org-todo-keyword-CANCELLED nil "")
(defface mrb/org-todo-keyword-BUY       nil "")
(defface mrb/org-todo-keyword-HOWTO     nil "")
(defface mrb/org-todo-keyword-INFO      nil "")
(defface mrb/org-todo-keyword-COLLECT   nil "")
(defface mrb/org-todo-keyword-SOLVE     nil "")
(defface mrb/org-todo-keyword-READ      nil "")
(defface mrb/org-todo-keyword-READING   nil "")
(defface mrb/org-todo-keyword-PLAN      nil "")


(setq org-todo-keyword-faces `(
  ("TODO"      . mrb/org-todo-keyword-TODO)
  ("DONE"      . mrb/org-todo-keyword-DONE)
  ("WAITING"   . mrb/org-todo-keyword-WAITING)
  ("CANCELLED" . mrb/org-todo-keyword-CANCELLED)
  ("BUY"       . mrb/org-todo-keyword-BUY)
  ("HOWTO"     . mrb/org-todo-keyword-HOWTO)
  ("INFO"      . mrb/org-todo-keyword-INFO)
  ("COLLECT"   . mrb/org-todo-keyword-COLLECT)
  ("SOLVE"     . mrb/org-todo-keyword-SOLVE)
  ("READ"      . mrb/org-todo-keyword-READ)
  ("READING"   . mrb/org-todo-keyword-READING)
  ("PLAN"      . mrb/org-todo-keyword-PLAN)))

Make sure we keep a clean tag slate when changing tag state. This means that when I move to an active state, remove inactive tags; if something is DONE, remove tags from it and automatically adding a ‘buy’ tag when a BUY item is created. Note: capturing does not honor this, i.e. when creating a new item.

(setq org-todo-state-tags-triggers
      '(('todo ("inactive"))          ; remove inactive tags if moved to any active state
        ('done ("inactive") ("fork")) ; remove tags from any inactive state
        ("BUY"  ("buy" . t))))        ; add buy tag when this is a buying action

To keep the TODO list clean we immediately archive the completed entry in the archive. The archiving only occurs when an item enters the ‘DONE’ state and the item is not marked as a habit.

I’m not sure if this works out in practice without having a confirmation (because we archive the whole sub-tree), so for now, I’m building in the confirmation.

;; I need a modified version of org-is-habit, which takes inheritance
;; in to account
(defun mrb/org-is-habit-p (&optional pom)
  "org-is-habit-p taking property inheritance into account"
  (equalp "habit" (org-with-point-at (or pom (point))
                    (org-entry-get-with-inheritance "STYLE"))))

(defun mrb/archive-done-item()
  "Determine if the item went to the DONE/CANCELLED state
   if so, ask to archive it, but skip habits which have
   their own logic."
  (when (not (mrb/org-is-habit-p))
    ;; No habit, so we have a candidate
    (progn
      ;; Try to use a dialog box to ask for confirmation
      (setq use-dialog-box-override t)

      ;; When a note is going to be added, postpone that Otherwise just
      ;; run the archiving question
      ;; TODO  org-add-note runs through post-command-hook,
      ;;        which is kinda weird, how to i get it to run
      ;;        before the archiving question?
      (when (equal org-state "DONE")
        (org-archive-subtree-default-with-confirmation)))))


;; Run archive for the item that changed state
(add-hook 'org-after-todo-state-change-hook
          'mrb/archive-done-item t)

In addition to the above I’m experimenting with org-edna (Extensible Dependencies ‘N’ Actions) which has a bit more flexibility. Notably the blocking of tasks is what I want to peruse a bit, because the builtin system hasn’t worked for me. I may still use the builtin way to block items, but perhaps under the dynamic control of org-edna instead of manually specifying order properties etc.

For now, a basic config and enabling of org-edna

(use-package org-edna
  :diminish
  :after org
  :config
  (org-edna-mode))

Marking items as DONE

Marking work as completed should be a smooth process to stop getting in the way of doing the actual work. A shortcut is defined to mark items done in the standard way and have an additional shortcut to mark it done should it be blocked.

When an item changes to the DONE state, a question is asked if the item should be archived, to which the normal answer should be ‘Yes’ to keep the active file as clean as possible.

;; Normal flow, but ignore blocking
(defun mrb/force-org-todo()
  (interactive)
  ;; Disable blocking temporarily
  (let ((org-inhibit-blocking t))
    (org-todo)))

;; Normal flow, but ignore blocking, agenda scope
(defun mrb/force-org-agenda-todo()
  (interactive)
  ;; Disable blocking temporally
  (let ((org-inhibit-blocking t))
    (org-agenda-todo)))

;; Break flow, cancel directly
(defun mrb/org-cancel-todo ()
  (interactive)
  (org-todo "CANCELLED"))

;; Break flow, cancel directly, agenda scope
(defun mrb/org-agenda-cancel-todo ()
  (interactive)
  (org-agenda-todo "CANCELLED"))

The above may be influenced, or even made redundant, by the org-edna package configuration.

Registering creation time of todo items

Over time it gets a bit messy in my orgmode files. I can not remember when something was created and thus, by judging the time I didn’t do anything with the item, decide if it is still important or not.

So, to help with that I created a little glue to make sure each actionable item gets a CREATED property with the date in it on which that item was created. I use the contributed org-expiry for that and adjust it a bit.

I want the property to be name ‘CREATED’ (I don’t remember what the org-expiry default name is, but it is different) and the timestamps inserted must not be active, otherwise they’ll appear all over the place in the agenda.

(use-package org-contrib)
(use-package org-expiry
  :custom
  (org-expiry-created-property-name "CREATED")
  (org-expiry-inactive-timestamps  t)

  :config
  <<org-expiry-createdtimestampfunction>>

  )

So, to create the timestamp I need a little helper function which actually inserts it, using org-expiry. There is some additional cursor munging to make sure it is used comfortably during editing. To actually make this active we advice the operations that insert headers: org-insert-todo-heading and org-capture when we are actually capturing a TODO item.

(defun mrb/insert-created-timestamp(&rest args)
  "Insert a CREATED property using org-expiry.el for TODO entries"
  (org-expiry-insert-created)
  (org-back-to-heading)
  (org-end-of-line))

;; for insert todo heading, always add the CREATED property
(advice-add 'org-insert-todo-heading
            :after #'mrb/insert-created-timestamp)

;; for capturing, only when state is in the TODO keywords list active in buffer
(advice-add 'org-capture
            :after (lambda (&rest r)
                     (when (member (org-get-todo-state) org-todo-keywords-1)
                       (mrb/insert-created-timestamp))))

Related to the above, with some regularity I want to record timestamps, for example for documenting longer during tasks and recording incremental progress or information on them. Orgmode provides the binding ’C-c !’ for this which inserts an inactive date-stamp, optionally including the time as well.

Lastly, there’s a mechanism in emacs which can automatically insert time-stamps, based on a placeholder in the file and a format variable. I sometimes use that, so I add it to my before save hook. Typically, the format string is file specific and will be held in a file local variable.

(add-hook 'before-save-hook 'time-stamp)

Scheduling items

Orgmode has a number of provisions to schedule items, either explicitly by setting the SCHEDULE property, inferring a deadline by setting the DEADLINE property, thus scheduling the task in an interval before the deadline expires.

The main key for scheduling will be C-. (Control key with dot). A todo is marked to the next state with M-. so this makes sense, at least to me. In plain org and in org-agenda mode this key is used most often, but I expect this to be useful in other modes as well. I will try to u use the same keybinding in those modes as well.

I had schedule-for-today functions earlier, but those really just save me from pressing return in a normal org-schedule function, so their added value was minimal and I have since deleted them.

Visual settings

Having an attractive screen to look at becomes more important if you use the system all day long. Attractive is rather subjective here. For me it mainly consists of functional things. Anyways, this section groups settings for the visual characteristics of orgmode.

I want to hide the leading stars in the out-liner, and do it exactly in the background color. This is redundant actually in my case, as it is also specified in the org config file that I include. Or rather, it is redundant there, because I want it always to be the case.

(setq org-hide-leading-stars t)

For the collapsed items in the outline orgmode uses the variable org-ellipsis to determine what character-sequence should be used to show that the item can be expanded. The variable can contain a string, which will then be used instead of the standard 3 dots, or a face which will then be used to render the standard 3 dots.

(setq org-ellipsis "…")

There are a couple of ways within org to emphasize text inline for bold, italics, underlined etc. These are set in the text by enclosing regions with delimiters. I do not want to see these delimiters, but rather render the text. The org-appear package makes this even fancier by showing the characters when inside the marked area, so editing is a lot easier.

(use-package org-appear
  :after org
  :hook (org-mode . org-appear-mode)
  :custom
  (org-hide-emphasis-markers t)    ; Is this used by org-appear mode?
  )

Similar to inline emphasis is the rewriting with pretty entity characters (like ‘δ’ for example). These characters can be added to the text by adding a ‘' before a symbol name (‘delta’ in the example). I make an exception for the sub- and superscript characters. This happens a lot in variable names etc. and I a big annoyance if those get rendered to subscript all the time.

(setq org-pretty-entities 1)
(setq org-pretty-entities-include-sub-superscripts nil)

Related to that is the display of links. I want them to be explicit most of the time to avoid confusion, but the ‘fancy’ display is easier at first:

(setq org-descriptive-links t)

For most of the source blocks I want Emacs to render those blocks in their native mode. This had a serious performance problem in the past, but I think it has been solved recently.

(setq org-src-fontify-natively t)

For the headings at each level a * is normally used. As we’re in unicode world now, we can do a bit better.

(use-package org-bullets
  :hook (org-mode . (lambda () (org-bullets-mode 1))))

The item lists can be made a whole lot more attractive by attaching some icons based on the category an items belongs to. The category assignment itself is done by setting the CATEGORY property explicitly on the item or on the file.

(setq org-agenda-category-icon-alist
      `(
        ("Afspraak"      ,(concat org-directory "images/stock_new-meeting.png") nil nil :ascent center)
        ("Blogging"      ,(concat org-directory "images/edit.png") nil nil :ascent center)
        ("Car"           ,(concat org-directory "images/car.png") nil nil :ascent center)
        ("Cobra"         ,(concat org-directory "images/car.png") nil nil :ascent center)
        ("DVD"           ,(concat org-directory "images/media-cdrom.png") nil nil :ascent center)
        ("Emacs"         ,(concat org-directory "images/emacs.png") nil nil :ascent center)
        ("Finance"       ,(concat org-directory "images/finance.png") nil nil :ascent center)
        ("Habitat"       ,(concat org-directory "images/house.png") nil nil :ascent center)
        ("Habit"         ,(concat org-directory "images/stock_task-recurring.png") nil nil :ascent center)
        ("Hobbies"       ,(concat org-directory "images/hobbies.png") nil nil :ascent center)
        ("Partners"      ,(concat org-directory "images/partners.png") nil nil :ascent center)
        ("Personal"      ,(concat org-directory "images/personal.png") nil nil :ascent center)
        ("Task"          ,(concat org-directory "images/stock_todo.png") nil nil :ascent center)
        ("Org"           ,(concat org-directory "images/org-mode-unicorn.png") nil nil :ascent center)
        ("Systeem"       ,(concat org-directory "images/systeembeheer.png") nil nil :ascent center)
        ("Wordpress"     ,(concat org-directory "images/wordpress.png") nil nil :ascent center)
        ))

Showing items in the agenda views reacts to a number of settings. In my setup I want blocked tasks hidden, that is the reason for blocking. Hide tasks which are DONE already and a deadline is coming up, no use showing those; the same goes for tasks which are DONE and are scheduled. In short, anything that does not need my attention needs to be hidden.

(setq
    org-agenda-dim-blocked-tasks t
    org-agenda-skip-deadline-if-done t
    org-agenda-skip-scheduled-if-done t
    org-agenda-skip-archived-trees nil
)

After experimenting with automatic aligning of tags I’ve come to the conclusion I want to have it manual. For now, I set the tags to align flush right with column 110 but ideally I’d want something more dynamic (in the sense that visual-line mode is also dynamic)

(setq org-tags-column -95)
(defun mrb/org-align-all-tags ()
  "Wrap org-align-tags to be interactive and apply to all"
  (interactive)
  (org-align-tags t))
(bind-key "M-\"" 'mrb/org-align-all-tags)

Agenda customization

Settings which are just applicable for the org-mode agenda view.

;; Show properties in agenda view
(use-package org-agenda-property
  :custom
  (org-agenda-property-list '("LOCATION" "Responsible")))

;; When done rendering, go to the top
(add-hook 'org-agenda-finalize-hook #'(lambda () (goto-char (point-min))) 100)

A snippet which resized the agenda to its window whenever its rebuilt by orgmode The ‘auto value of the org-agenda-tags-column variable right aligns the tags in the agenda view

(setq org-agenda-tags-column 'auto)
(defadvice org-agenda-redo (after fit-windows-for-agenda activate)
  "Fit the Org Agenda to its buffer."
  (fit-window-to-buffer)
  (goto-char (point-min)))

Calendaring

Traditionally somewhat related to mail, although mostly through orgmode these days, I want a simple setup to pull in some calendaring information. I run a radicale calendar server somewhere, but having the same info in orgmode makes sense for me.

I have used org-caldav but even in just a read only setup it does not work properly. I’m using ical2orgpy which is a python script that takes .ics input and creates an org-mode file.

I combine ical2orgpy with a cron job which pulls in the calendars I am interested into. Adding the produced org-mode files to the org-agenda-files variable is enough to get the information into the org-mode agenda.

# <<tangle-header>>
# manually test this script with `env -i /bin/sh`
#
# Mostly assume nothing is implied, like paths etc.

BASEDIR=/home/mrb/dat/org/_calendars
ICAL2ORGPY="/home/mrb/.local/bin/ical2orgpy"

# wget uses .netrc to gett auth info
WGET="/usr/bin/wget -O wget.log -O - --quiet"

# Base URI for calendar access, perhaps also put in the datafile?
CALBASE="https://calendars.hsdev.com/mrb"

# Just go over each one manually
# TODO make this nice with url . file pairs in a loop
cd $BASEDIR

# Read destination org file and the calendar id from a file .calendars
while IFS=' ' read -r orgfile calendar_id
do
    $WGET "$CALBASE/$calendar_id" 2>/dev/null | $ICAL2ORGPY - $orgfile
done < .calendars

Babel / Literate programming

Specific settings for babel and literate programming within org-mode

(setq org-babel-load-languages
      '((awk        . t)
        (calc       . t)
        (css        . t)
        (ditaa      . t)
        (emacs-lisp . t)
        (gnuplot    . t)
        (haskell    . t)
        (js         . t)
        (lisp       . t)
        (org        . t)
        (plantuml   . t)
        (python     . t)
        (scheme     . t)
        (shell      . t)
        (sql        . t)))

;; Activate Babel languages
(org-babel-do-load-languages 'org-babel-load-languages org-babel-load-languages)

The elements of org babel are blocks of source code. Some of the modes for sources have some helper programs.

Plantuml transforms diagram specifications with a mini programming language into diagrams. It has support for many types of diagram. Ditaa transforms ascii diagrams into graphics.

(use-package plantuml-mode
  :after org    ; strictly not needed, but i use it mainly from org
  :init
  (setq plantuml-jar-path "/usr/share/java/plantuml/plantuml.jar")
  (setq org-plantuml-jar-path plantuml-jar-path)
  (setq plantuml-default-exec-mode 'jar))

(setq org-ditaa-jar-path    "/usr/share/java/ditaa/ditaa-0.11.jar")

Refiling

A big part of organizing information and task is shuffling things around. The ‘thing’ to throw around is a heading and ‘refiling’ is the term org-mode uses for throwing.

When filing, or capturing we want the items at the bottom of what we are filing it into. The main reason for this is that a large part of the sections that contain items are ordered. Should we file the item at the top, in many cases that would mean it is the most imminent thing to do, which is not the case. For some files, we do want the items on top though

(setq org-reverse-note-order '(("workshop.org" . t)
                               (".*" . nil))    ; File for others at the bottom of an entry
      org-refile-allow-creating-parent-nodes 'confirm
      org-refile-use-outline-path 'file
      org-outline-path-complete-in-steps nil
      org-refile-use-cache t)

The base list of files I want to be able to refile to are:

1. The active file list, i.e. everything in org-agenda-files, regardless of these files are currently opened or not;
2. All open orgmode based files which, surprisingly, I need to generate myself?

  (setq org-refile-targets '((org-agenda-files :maxlevel . 10)))

;; Borrows from https://yiming.dev/blog/2018/03/02/my-org-refile-workflow/
  (defun mrb/org-opened-buffers ()
    "Return the list of org files opened in emacs"

    (defun buffer-mode (buf)
      (buffer-local-value 'major-mode (get-buffer buf)))

    (delq nil
          (mapcar (lambda (x)
                    (if (and (buffer-file-name x) ;-buffer has a file?
                             (provided-mode-derived-p
                              (buffer-mode x) 'org-mode))
                        (buffer-file-name x)))
                  (buffer-list))))

  ;; Adjust refile target to include all opened org files
  ;; Is it bad that we have duplicates here?
  (add-to-list 'org-refile-targets
               '(mrb/org-opened-buffers :maxlevel . 10))

The type of headers to refile to is, in the default orgmode config, basically everything. I limit it above to a maximum depth of 10, but then still. By using org-refile-target-verify-function we can fine-tune the decision whether to use a header as target or not. The following conditions have been implemented:

1. The header must not be a DONE item;
2. The header needs to have children.

The latter is the most important one and will prevent creating ‘gathering’ items to tasks themselves.

(defun mrb/has-DONE-keyword()
  "Return t when the heading at point has a `DONE' like keyword"
  (member (nth 2 (org-heading-components)) org-done-keywords))

(defun mrb/verify-refile-target()
  "Decide if a target header can be used as a refile target
Conditions to return t:
- header must not have one of the DONE keywords
- it must be a parent of something already"
  ;; interactive during testing
  (interactive)

  (and
   ; exclude done keyword headers
   (not (mrb/has-DONE-keyword))
   ; must have a child
   (save-excursion (org-goto-first-child))))


(setq org-refile-target-verify-function 'mrb/verify-refile-target)

Exporting to other formats

Orgmode can export to a variety of formats, I mainly use LaTeX (PDF) and HTML as destination format.

(use-package htmlize)                   ; for html export source highlighting
;; {% raw %}
(setq org-export-latex-hyperref-format "\\ref{%s}:{%s}"
      ;; old system
      org-export-latex-title-command " "
      ;; new system > 8.0
      org-latex-title-command " "

      org-export-docbook-xsl-fo-proc-command "fop %i %o"
      org-export-docbook-xslt-proc-command "xsltproc --output %o %s %i"
      org-export-htmlize-output-type 'css
      org-org-htmlized-css-url "orgmode.css"

      org-latex-listings 'engraved

      org-export-copy-to-kill-ring 'if-interactive

      org-export-htmlized-org-css-url "orgmode.css"
      org-export-latex-classes
      '(
        ("article" "\\documentclass[11pt,a4paper,twoside]{article}"
         ("\\section{%s}" . "\\section*{%s}")
         ("\\subsection{%s}" . "\\subsection*{%s}")
         ("\\subsubsection{%s}" . "\\subsubsection*{%s}")
         ("\\paragraph{%s}" . "\\paragraph*{%s}")
         ("\\subparagraph{%s}" . "\\subparagraph*{%s}"))
        ("report" "\\documentclass[11pt]{report}"
         ("\\part{%s}" . "\\part*{%s}")
         ("\\chapter{%s}" . "\\chapter*{%s}")
         ("\\section{%s}" . "\\section*{%s}")
         ("\\subsection{%s}" . "\\subsection*{%s}")
         ("\\subsubsection{%s}" . "\\subsubsection*{%s}"))
        ("book" "\\documentclass[11pt]{book}"
         ("\\part{%s}" . "\\part*{%s}")
         ("\\chapter{%s}" . "\\chapter*{%s}")
         ("\\section{%s}" . "\\section*{%s}")
         ("\\subsection{%s}" . "\\subsection*{%s}")
         ("\\subsubsection{%s}" . "\\subsubsection*{%s}"))
        ("beamer" "\\documentclass{beamer}" org-beamer-sectioning))

      ;; No tags, todo keywords or time-stamp
      org-export-with-tags nil
      org-export-with-todo-keywords nil
      org-export-time-stamp-file nil
      org-export-backends '(ascii html icalendar latex md odt org texinfo)
      )
;; {% endraw %}

Journaling

While the standard capture method is useful I found myself not using it very much. Not sure why. It turns out that org-journal is a much better fit for my workflow, especially using the carry-over functionality.

(use-package org-journal
  :after (org)
  :commands (org-journal-new-entry)
  :hook ((org-journal-mode . turn-on-visual-line-mode)
         (org-journal-mode . turn-on-visual-fill-column-mode))
  :bind (("C-c j"   . org-journal-new-entry)
         ("C-s-j"   . mrb/open-current-journal))

  :config
  (defun mrb/open-current-journal()
    (interactive)
    (let* ((org-journal-file (org-journal--get-entry-path))
           (buf-exists (find-buffer-visiting org-journal-file)))
      (if buf-exists
          (switch-to-buffer buf-exists) ;; or use org-journal-open-current-journal-file?
        (org-journal-new-entry 1))))


  :custom
  ;; The expand-file-name is needed, which is odd, because for single
  ;; files this is not needed.
  (org-journal-dir (expand-file-name (concat org-directory "journal/")))

  ;; Bring our config to every journal file
  (org-journal-file-header (concat "#+SETUPFILE: " user-emacs-directory "org-config.org"))

  ;; Match the journal files (TODO  make this independent of earlier assignments)
  (org-agenda-file-regexp "^[0-9]+\\.org")
  (org-journal-file-format  "%Y%m%d.org")

  ;; Put day on top of file, uses `format-time-string`
  (org-journal-date-format "%A, %-e-%m-%Y")

  ;; Put day on top of file, uses `org-journal-date-format`
  (org-journal-date-prefix "#+TITLE: ")
  ;; New entries go at the bottom, make sure we are at top level
  (org-journal-time-format "[%R] ")
  (org-journal-time-prefix "* ")

  ;; Carry over TODO items and items explicitly marked
  (org-journal-carryover-items "+carryover|+TODO=\"TODO\"")
  ;; and for this to work, we need agenda integration
  (org-journal-enable-agenda-integration t)
  ;; Remove empty journals after carryover
  (org-journal-carryover-delete-empty-journal 'always)

  ;; I want to have encryption, but how do TODO items bubble up then in the agenda
  (org-journal-enable-encryption nil)
  (org-crypt-disable-auto-save t)
  (org-journal-enable-cache t)
  (org-element-use-cache t))

Particularly in journaling, but also in org-mode in general, I want to be able to quickly insert screenshots. Rather, images in general but 90% of my use-case is really screenshots.

There’s a package org-attach-screenshot which matches my use-case 99% so let’s use that and worry about extending it to images later on.

(use-package org-attach-screenshot
  :bind (("C-c i" . org-attach-screenshot)))

The 1% I was referring to above is that the original package exclusively supports org-mode. I patched it, which was trivial to support org-mode and all its derived modes. (org-journal in my case)

Publishing

There’s plenty ways to publish orgmode content on the web. I use a couple of them sparingly. For most of them I don’t need a permanent configuration. For the ones that I do need a config, there’s this section.

Writefreely

Writefreely is a Write.as derivative and published as open source. This allows me to just kick out a quick orgmode file with a #+TITLE: and a #+DATE: header and throw it on a blog like site rather quickly.

;; Still needed:
;; - post as draft by default, this prevents automatic posting by accident for federated collections
;; - save augment -> publish / update automatically, probably a custom hook on minor mode?
;; - wf: have autorefresh GET parameter which monitors updates and
;;       refreshes automatically
;; - set local save dir for the files
;; Make sure the variables of THIS file are loaded before THIS file is loaded
(use-package writefreely
  :after org
  :custom
  (writefreely-instance-url "https://qua.name")
  (writefreely-instance-api-endpoint "https://qua.name/api")
  (writefreely-maybe-publish-created-date t)
  (writefreely-auth-token (password-store-get "Qua.name/accesstoken")))

Hugo

I use hugo to sporadically publish some content to my blog. Not sure if this is what I want, I post too little to spend much time on it. But here is a small config for the main posts:

(use-package easy-hugo
:init
(setq easy-hugo-basedir "~/dat/blogs/mrblog/" ;all my subblogs live below this
      easy-hugo-postdir "content/post/main"
      easy-hugo-url "https://mrblog.nl"
      easy-hugo-default-ext ".org"
      easy-hugo-org-header t))

Encrypting information in org-mode

I use the encrypt tag for encrypting sections in org-mode (and sometimes my journal). The sections get encrypted and decrypted automatically on saving and opening. This uses the EasyPG library to get to my GPG key.

(use-package org-crypt
  :custom
  (org-crypt-tag-matcher "encrypt")
  (org-crypt-key user-gpg-key)

  :config
  (org-crypt-use-before-save-magic))

We do not want to inherit this tag automatically, as its behavior is already subsection inclusive. When you encrypt a section, everything below it is considered content of that section and gets encrypted. I also add the value “crypt” as that is the org default, so it won’t be inherited by mistake.

(add-to-list 'org-tags-exclude-from-inheritance '"encrypt")
(add-to-list 'org-tags-exclude-from-inheritance '"crypt")

Old configuration

Below is what was contained in the old configuration. I will slowly migrate this into more literal sections

;; Bit of a leftover from reorganizing bits, do this later
(add-to-list 'org-tags-exclude-from-inheritance '"sell")

(defun mrb/is-project-p ()
  "This function returns true if the entry is considered a project.
    A project is defined to be:
    - having a TODO keyword itself (why was this again?);
    - having at least one todo entry, regardless of their state."
  (let ((has-todokeyword)
        (has-subtask)
        (subtree-end (save-excursion (org-end-of-subtree t)))
        (is-a-task (member (nth 2 (org-heading-components)) org-todo-keywords-1)))
    (save-excursion
      (forward-line 1)
      (while (and (not has-subtask)
                  (< (point) subtree-end)
                  (re-search-forward "^\*+ " subtree-end t))
        (when (member (org-get-todo-state) org-todo-keywords-1)
          (setq has-subtask t))))
    ;; both subtasks and a keyword on the container need to be present.
    (and is-a-task has-subtask)))

;; TODO  testing for tag presence should be easier than a re-search forward
;; TODO  are we not searching for all 'incomplete' type keywords here?,
;;        there must be an org function for that
(defun mrb/skip-non-stuck-projects ()
  "Skip trees that are not stuck projects"
  (let* ((subtree-end (save-excursion (org-end-of-subtree t)))
         (has-next (save-excursion
                     (forward-line 1)
                     (and (< (point) subtree-end)
                          (re-search-forward "^*+ \\(TODO\\|BUY\\|WAITING\\)" subtree-end t)))))
    (if (and (mrb/is-project-p) (not has-next))
        nil ; a stuck project, has subtasks but no next task
      subtree-end)))

(defun mrb/skip-non-projects ()
  "Skip trees that are not projects"
  (let* ((subtree-end (save-excursion (org-end-of-subtree t))))
    (if (mrb/is-project-p)
        nil
      subtree-end)))

(defun mrb/skip-projects ()
  "Skip trees that are projects"
  (let* ((subtree-end (save-excursion (org-end-of-subtree t))))
    (if (mrb/is-project-p)
        subtree-end
      nil)))

;; Remove empty property drawers
(defun mrb/org-remove-empty-propert-drawers ()
  "*Remove all empty property drawers in current file."
  (interactive)
  (unless (eq major-mode 'org-mode)
    (error "You need to turn on Org mode for this function."))
  (save-excursion
    (goto-char (point-min))
    (while (re-search-forward ":PROPERTIES:" nil t)
      (save-excursion
        (org-remove-empty-drawer-at "PROPERTIES" (match-beginning 0))))))

(defun mrb/org-remove-redundant-tags ()
  "Remove redundant tags of headlines in current buffer.

 A tag is considered redundant if it is local to a headline and
 inherited by a parent headline."
  (interactive)
  (when (eq major-mode 'org-mode)
    (save-excursion
      (org-map-entries
       '(lambda ()
          (let ((alltags (split-string (or (org-entry-get (point) "ALLTAGS") "") ":"))
                local inherited tag)
            (dolist (tag alltags)
              (if (get-text-property 0 'inherited tag)
                  (push tag inherited) (push tag local)))
            (dolist (tag local)
              (if (member tag inherited) (org-toggle-tag tag 'off)))))
       t nil))))


(defvar org-agenda-group-by-property nil
  "Set this in org-mode agenda views to group tasks by property")

(defun mrb/org-group-bucket-items (prop items)
  (let ((buckets ()))
    (dolist (item items)
      (let* ((marker (get-text-property 0 'org-marker item))
             (pvalue (org-entry-get marker prop t))
             (cell (assoc pvalue buckets)))
        (if cell
            (setcdr cell (cons item (rest cell)))
          (setq buckets (cons (cons pvalue (list item))
                              buckets)))))
    (setq buckets (mapcar (lambda (bucket)
                            (cons (first bucket)
                                  (reverse (rest bucket))))
                          buckets))
    (sort buckets (lambda (i1 i2)
                    (string< (first i1) (first i2))))))

(defadvice org-agenda-finalize-entries (around org-group-agenda-finalize
                                               (list &optional nosort))
  "Prepare bucketed agenda entry lists"
  (if org-agenda-group-by-property
      ;; bucketed, handle appropriately
      (let ((text ""))
        (dolist (bucket (mrb/org-group-bucket-items
                         org-agenda-group-by-property
                         list))
          (let ((header (concat "Property "
                                org-agenda-group-by-property
                                " is "
                                (or (first bucket) "<nil>") ":\n")))
            (add-text-properties 0 (1- (length header))
                                 (list 'face 'org-agenda-structure)
                                 header)
            (setq text
                  (concat text header
                          ;; recursively process
                          (let ((org-agenda-group-by-property nil))
                            (org-agenda-finalize-entries
                             (rest bucket) nosort))
                          "\n\n"))))
        (setq ad-return-value text))
    ad-do-it))
(ad-activate 'org-agenda-finalize-entries)


(defvar mrb/org-my-archive-expiry-days 365
  "The number of days after which a completed task should be auto-archived.
 This can be 0 for immediate, or a floating point value.")

(defun mrb/org-my-archive-done-tasks ()
  (interactive)
  (save-excursion
    (goto-char (point-min))
    (let ((done-regexp
           (concat "\\* \\(" (regexp-opt org-done-keywords) "\\) "))
          (state-regexp
           (concat "- State \"\\(" (regexp-opt org-done-keywords)
                   "\\)\"\\s-*\\[\\([^]\n]+\\)\\]")))
      (while (re-search-forward done-regexp nil t)
        (let ((end (save-excursion
                     (outline-next-heading)
                     (point)))
              begin)
          (goto-char (line-beginning-position))
          (setq begin (point))
          (when (re-search-forward state-regexp end t)
            (let* ((time-string (match-string 2))
                   (when-closed (org-parse-time-string time-string)))
              (if (>= (time-to-number-of-days
                       (time-subtract (current-time)
                                      (apply #'encode-time when-closed)))
                      mrb/org-my-archive-expiry-days)
                  (org-archive-subtree)))))))))

(defalias 'archive-done-tasks 'mrb/org-my-archive-done-tasks)

Key and mouseu bindings

Keyboard bindings are the primary way to interact for me. I have been struggling with consistent keyboard shortcuts and how to integrate them with the other systems on my machine which capture shortcut keys. At this time the following applications capture shortcut keys:

1. the awesome window manager captures keys;
2. xbindkeys provides a number of key bindings for application dependent operations;
3. emacs (and obviously all other applications, but those are largely irrelevant).
4. the X-windows server has the kbd extension which has some keyboard related things to configure.
5. The linux kernel provides key mapping, so I have to look at that place too (xmodmap)

Because I am daft, here is the notation for the modifiers:

C -

control

s -

super, meaning the (left) windows key in my configuration

M -

meta, meaning the (left) alt key in my configuration

S -

shift

To help me out with this when writing about key bindings, the lisp function key-description can help out, with a little bit of glue around it:

(defun mrb/insert-key-description ()
  "Insert a pretty printed representation of a key sequence"
  (interactive)
  (insert (key-description (read-key-sequence "Type a key sequence:"))))

I like the explicit notation where the name of the key is spelled out better, and I’ll move all configured keybindings to that eventually.

The right alt and the right <menu> key should be the same as the left alt and the super key, but I haven’t gotten around to configuring that yet.

Furthermore, because I still can’t remember keys, after pressing a prefix key like C-c the package which-keys can show me a formatted menu with the combinations of keys that can follow it.

; which keys shows menu with completions of prefix-key
(use-package which-key
  :diminish
  :custom
  (which-key-idle-delay 1.8)
  (which-key-show-operator-state-maps t)

  :config
  (which-key-mode))

For binding keys there are many ways it seems, with all different syntaxes and uses. I’ve tried to do everything with the bind-key package, because that is part of use-package so we get that for free.

Bind-key can define both global keys as map-based key settings and accepts all kinds of key specifications, including strings.

First, unsetting the keys I don’t want.

Let’s begin with killing some bindings which are in my way, notably the standard right mouse click behavior. This is because I want it to behave in org-mode, which apparently sets this. I should probably find out a better way for this.

(unbind-key "<mouse-3>")

;; Make `C-x C-m' and `C-x RET' be different (since I tend
;; to type the latter by accident sometimes.)
(unbind-key "C-x RET")

Setting keys

The standard open-line-splits the line, which is useful, but not what I want, so define a version which can open a line above or below the current line without changing the cursor position.

(defun mrb/openline (arg)
  (interactive "P")
  (save-excursion
    (end-of-line (if arg nil 0))        ; with prefix, open line below, else above
    (open-line 1)))

;; should prefix be for above or below open line?
(bind-key "C-o" 'mrb/openline)

Not sure how to bind it, C-o seems kinda busy.

Key bindings

Global

I am running the emacs daemon and sometime when I quit emacs, I want it to quit too. This sounds a bit counter-intuitive, but as long as my emacs config is moving and I am not proficient enough in making sure I can apply the changed settings reliably from within emacs, restarting emacs is just easier. This saves me from having to kill the emacs daemon from the terminal.

(bind-key "C-x C-q" 'save-buffers-kill-emacs)

Probably the most important key is M-x (as set by default). That key gives access to other commands within emacs, so it better be effective. If I wasn’t already used to it, I’d certainly not consider M-x as a first candidate. The main objection I have is that the two keys are close to each-other, making it hard to press in a typing flow.

Set of bindings on which I would like to have cut, copy and paste and friends:

;; this kinda sucks now, because the rest of the OS does not do this
;; SOLUTION: learn to work with standard emacs keybinding and adjust the OS  ?
(bind-keys ("s-z" . undo)
           ("s-x" . clipboard-kill-region)
           ("s-c" . clipboard-kill-ring-save)
           ("s-v" . yank)
           ("s-a" . mark-whole-buffer))

Some operations on buffers:

;; Buffer handling shortcuts
(bind-keys ("s-n" . (lambda () (interactive) (switch-to-buffer (generate-new-buffer "Untitled"))))
           ("s-s" . save-buffer)
           ("s-k" . kill-buffer))

And the rest, for now uncategorized:

  ;; Open my emacs config; I wanted something with '~'
(bind-key "C-~" 'mrb/open-config)

;; Font scaling, like in firefox
(bind-key "C-+" 'text-scale-increase)
(bind-key "C--" 'text-scale-decrease)

;; Line handling functions
(bind-key "s-`" 'toggle-truncate-lines)
;; Most of the time I want return to be newline and indent
;; Every mode can augment this at will obviously (org-mode does, for example)
(bind-key "RET" 'newline-and-indent)

;; Comment code lines, command reacts based on the major mode.
(bind-key "s-/" 'comment-dwim)

;; Keypad delete
(bind-key [(kp-delete)] 'delete-char)

What should have been in emacs

Sometimes there are logical gaps in emacs’ keybinding. I put them here

(bind-keys :map help-map
          ("A" . describe-face))

Special keys

For some special keys I have defined some commands. Special keys are those keys that may not be on every keyboard, within reason. I consider the function keys also as special, although they do not fit the previous definition.

;; Menu key does M-x, if we have it.
;;(bind-key (kbd "<apps>") 'execute-extended-command)
(bind-key "<f1>" 'help-command)
(bind-key "<f2>" 'save-buffer)
(bind-key "<f4>" 'find-file)

Resizing and switching windows and frames

Treading cautiously here as ideally frames sizing is the responsibility of the window manager.

;; Moving back and forth in windows For now, I'm using the Fn Key +
;; Arrows, seems consistent with the other window movements
(bind-key [XF86AudioNext] 'next-multiframe-window)
(bind-key [XF86AudioPrev] 'previous-multiframe-window)

;; Alt-Cmd left-right arrows browse through buffers within the same frame
(bind-key "<M-s-left>"  'previous-buffer)
(bind-key "<M-s-right>" 'next-buffer)

;; These would conflict with awesome bindings, perhaps we should change those bindings
;; (bind-key "<C-S-left>"  'buf-move-left)
;; (bind-key "<C-S-right>" 'buf-move-right)

;; Awesome uses Super+Arrows to move between its 'frames'
;; Emacs   uses Shift-Super+Arrows to move between its windows
(bind-key "<S-s-right>" 'windmove-right)
(bind-key "<S-s-left>"  'windmove-left)
(bind-key "<S-s-up>"    'windmove-up)
(bind-key "<S-s-down>"  'windmove-down)

For active regions

In quite a few situations I would like to use keybindings which are only valid when a selection/region is active.

We can bind keys into the selected-keymap if we want keys to be active on every region. When only needed for a certain major mode, define a keymap named selected-<major-mode>-map and bind keys into that.

Within orgmode I’d like to have the emphasis markers react when a region is active, so let’s create the necessary fragment for those bindings. These should in theory also work in org-journal mode, but they don’t.

During mail composition, when deleting text, this should be replace by ‘[…]’ to signal that we removed text from the original mail.

(use-package selected
  :diminish selected-minor-mode
  :after (org org-journal mu4e)                        ; all modes we supply binding for
  :demand t                             ; So we can use global mode enable in :config
  :commands (selected-global-mode selected-minor-mode) ; redundant?
  :init
  (setq selected-org-mode-map         (make-sparse-keymap)
        selected-mu4e-compose-mode-map (make-sparse-keymap))

  ;; Signal a deletion in compose
  ;; TODO  do something clever with the quote level?
  (defun mrb/compose-deletion ()
    (interactive)
    (delete-active-region)
    (insert "[...]"))

  :bind ( :map selected-keymap           ; bindings for all active regions
          ("C-q" . selected-off)
          ("C-u" . upcase-region)
          ("C-d" . downcase-region)
          :map selected-org-mode-map
          ("~" . (lambda () (interactive) (org-emphasize ?~)))
          ("_" . (lambda () (interactive) (org-emphasize ?_)))
          ("*" . (lambda () (interactive) (org-emphasize ?*)))
          ("C-b" . (lambda () (interactive) (org-emphasize ?*)))
          ("+" . (lambda () (interactive) (org-emphasize ?+)))
          ("=" . (lambda () (interactive) (org-emphasize ?=)))
          ("/" . (lambda () (interactive) (org-emphasize ?/)))
          :map selected-mu4e-compose-mode-map
          ("." . mrb/compose-deletion))

  :config
  (selected-global-mode))

Other key and mouse related settings

Emacs has, like for everything else, a peculiar idea on scrolling and moving from screen to screen.

These settings work better for me.

I have my keyboard repeat rate rather quick; this helps by moving the cursor fast. It also means that if I press a key like backspace things disappear quite quickly, so it’s important that what happens on the screen is ‘real-time’. The effect I want to prevent is that when releasing the backspace key, the cursor keeps on going and deletes way more than needed. I think, by default, this is properly configured, but I just want to make sure.

(setq redisplay-dont-pause t)

When scrolling, I don’t tend to think in half-screens like emacs does, I just want the text in the window to move up or down without having to guess where it’s going to be. Make sure we scroll 1 line and have a small, or none at all, scroll-margin. Having both at a value of 1 is intuitive.

(setq scroll-margin 1  scroll-step 1)

Make sure that the scroll wheel scrolls the window that the mouse pointer is over and that we are scrolling 1 line at a time. I don’t use any modifiers with the scroll wheel.

(xterm-mouse-mode)
(setq mouse-wheel-follow-mouse 't)
(setq mouse-wheel-scroll-amount '(1 ((shift) . 1)))
(setq focus-follows-mouse t) ;; i3wm changes focus automatically

Terminals, Character encodings and emulation

My policy is to use Emacs for as many things and use as little other programs as necessary. All this within reason obviously.

This sections describes how terminals and shells are used from within Emacs. In an ideal situation I won’t be needing any other terminal emulator program, other than the ones used directly from Emacs.

vterm

While eshell is the more emacsy solution, it has some limitations which makes me reach for xterm for daily work. However, there’s still a good option to benefit from all the emacs configuration in that case and that is the vterm package. This is an interface to the libvterm library creating a real terminal emulation using emacs for display.

The basic install is easy, but does need an emacs compiled with module support.

(use-package vterm
  :straight (:type built-in)
  :if (fboundp'module-load)
  :custom-face
  (vterm-color-black   ((t (:foreground "#2e3440"))))
  (vterm-color-red     ((t (:foreground "#bf616a"))))
  (vterm-color-green   ((t (:foreground "#a3be8c"))))
  (vterm-color-yellow  ((t (:foreground "#ebcb8b"))))
  (vterm-color-blue    ((t (:foreground "#81a1c1"))))
  (vterm-color-cyan    ((t (:foreground "#88c0c0"))))
  (vterm-color-magenta ((t (:foreground "#b48ead"))))
  :custom
  (vterm-timer-delay 0))

To benefit from some of the most useful features, there is a little bit of shell configuration involved. The package includes a shell script for that which I am sourcing in my .zshrc configuration file.

The vterm brings me some advantages, even close to being able to replace xterm for daily use, for example:

• magit is now effectively a terminal package everywhere available
• a terminal is an emacs buffer, so it’s easy to have it available anywhere in emacs
• most emacs commands will just work in a predictable way.

The most important downside is that if emacs does weird things, like hang or crash, this will affect all my terminals too, which is unworkable in some situations, so xterm is not going anywhere soon (but eshell is out I think)

Process handling

Sometimes processes get stuck and i want a way to delete those processes easily.

(defun mrb/delete-process-interactive ()
  "Based on an auto-completed list of process, choose one process to kill"
  (interactive)
  (let ((pname (completing-read "Process Name: "
                                    (mapcar 'process-name (process-list)))))
    (delete-process (get-process pname))))

Completion

There are 2 types of completion:

1. Input completion in the minibuffer
2. Inline completion in another buffer

I want completion to work as follows:

1. completion functions are always bound to a keybinding involving the TAB-key, with as little modifiers as possible;
2. completion should always produce something, even if emacs has no special semantic knowledge of the current mode, it should produce something which makes sense;
3. completion should be inline whenever possible.
4. for each mode, a specialization is ok, if that improves the situation; I expect to have many specializations to improve the auto-complete quality;
5. if a completion window must be opened, do this at the same place always and do not mess up other windows.
6. Completion should behave somewhat like real-time systems. An answer must be produced within a certain amount of time. If a completion answer takes longer than the amount of type to type it in full, the system has collapsed, so the time needs to be in the order of one third of the typing time.

The next sections deal with the above requirements

Ad 1. Bind completion always involves TAB-key

The package smart-tab seems to fit this bill, but the thing that I care about can be achieved fine without it (I only found this out after considerable time using smart-tab).

So, tab tries to indent, which is the main expectation, and if it can’t it tries to complete stuff.

(setq tab-always-indent 'complete)

There are some situations where tab does completion (which is good), but the type of completion is not what I want (which is bad). Currently this includes only quail-completion of which I don’t even know what it is for. This code prevents having a quail completion window when pressing tab inline.

; Bit of a kludge, but when quail loads, get rid of the completion
(with-eval-after-load 'quail (defun quail-completion ()))

In a standard emacs installation, TAB indents, depending on mode obviously. If indenting would not make sense, a TAB can be inserted or completion could start. The function completion-at-point is used in some situations. Ideally the corfu-complete function could take over in many cases. Here’s a simplistic approach to get me started:

1. if in minibuffer, do completion there like we are used to;
2. if cursor is at the end of a symbol, try to complete it with corfu;
3. else, indent according to mode.

This is probably incomplete or wrong even in some cases, but it’s a start.

This way, TAB always does completion or indent, unless corfu-mode is not active.

Ad 2. Completion should always produce something

Not sure if there is anything to do here.

Ad 3. Inline completion when possible

With inline completion I mean without opening a whole new **Completions** window if not needed.

Content that I want:

• languages: lisp, python, ruby, bash, C/C++ roughly in that order (function and argument completion)
• for all languages, function/method signature shorthands
• speed, slowness is killing here
• prevent minibuffer distractions, put info where my eyes are and that is the cursor in most cases.
• maybe: spelling suggestions
• nick completion in irc channels

Candidates:

auto-complete

http://cx4a.org/software/auto-complete/

company-mode

http://company-mode.github.io

corfu-mode

https://github.com/minad/corfu

All of these work fine, but corfu gets installed because it fits right in with the rest of the completion packages and does more than enough for what I need. Given the quick popup is nice to complete inline, but when still confused, press M-m to move the completion list to the minibuffer (or whatever vertico has been configured to) to show the list and have more info on the items in the list, while competion still continues.

(use-package corfu
  :custom
  (corfu-auto t)
  (corfu-quit-at-boundary 'separator)
  (corfu-cycle t)


  :bind ( :map corfu-map
          ("M-m" . mrb/corfu-move-to-minibuffer))
  :init
  ;; Enable it globally
  (global-corfu-mode)

  ;; But not for some
  (setq global-corfu-modes
        '(message-mode
          (not text-mode)
          t)))

Ad 4. Mode specialization

There’s always exceptions to the rule; with Emacs doubly so. Here’s the relevant exceptions for my configuration.

Ad 5. Open completion window predictably

The configuration of the completion is handled by vertico now, so nothing needs to be done here.

For most of the completion that is needed which is not inline (for which I am using the corfu package above, helm seems to be the most powerful solution, but feels isolated. For every integration something extra seems to be needed. Vertico seems the completing framework that fits the integration requirement best; it clearly states so in its goal to only depending on the Emacs API and not create a new one. Integrating it with others looks a lot more orthogonal than Helm.

(use-package vertico
  :demand t
  :straight (vertico :files (:defaults "extensions/*")
                     :includes (vertico-reverse vertico-multiform))

  :custom-face
  (vertico-current ((t (:background "#5e81ac"))))
  :config
  (vertico-mode))

As M-x or execute-extended-command is probably the most used command in Emacs, I want to give it a bit more prominent place on the scree than just the tiney minibuffer at the bottom. The vertico-posframe package turns the minibuffer into a floating popup sort of screen by using posframe. This puts the minibuffer more in my face and gives some better options to present the command information.

(use-package vertico-posframe
  ;; Customize border face to same color as cursor
  :custom-face
  (vertico-posframe-border ((t (:background "DarkOrange" :inherit default))))
  :config
  (setq vertico-posframe-parameters
        `((left-fringe . 18)
          (right-fringe . 18)
          (border-width . 4)
          (border-color . ,mrb/cursor-color)
          (child-frame-border-width . 4)))

  (setq vertico-multiform-commands
        '((consult-line
           posframe
           (vertico-posframe-fallback-mode . vertico-buffer-mode))
          (t posframe)))

  (vertico-multiform-mode 1))

The first thing to complement vertico is the orderless package. My main use is to have the matching for completing items match ‘first second’ as well as ‘second first’ regardless of their order. Note that this works for every completion, also within corfu which is especially useful.

(use-package orderless
  :custom-face
  (orderless-match-face-0 ((t (:foreground "dark orange"))))
  :custom
  (completion-styles '(orderless basic)))

The default list of candidates can be nicely augmented with marginalia which shows extra info about the matching items in the right aligned margin.

(use-package marginalia
  :custom
  (marginalia-align 'right)

  :init
  (marginalia-mode))

The consult package exposes a number of consult- commands which work with the completing framework and typically put an original command on steroids. Some of the commands support live previewing while browsing the selections.

(use-package consult
  ;; Replace some bindings with a consult command, so we get things on steroids
  :bind (("C-x b" . consult-buffer)))

Embark is the last of the set, and adds a new level of interaction. The basis principle normally in Emacs is that you specify a command and then tell what to act on. Like opening a file is the `find-file` command and then specifying what file to find.

Embark is the other way around. Something is active, depending on context and `embark-act` give a list of things you can do with it.

(use-package embark
  :straight (embark :files (:defaults "embark-org.el"))

  :bind
  (("C-." . embark-act)         ;; pick some comfortable binding
   ("C-;" . embark-dwim)        ;; good alternative: M-.
   ("C-h B" . embark-bindings)) ;; alternative for `describe-bindings'

  :init
  ;; Optionally replace the key help with a completing-read interface
  (setq prefix-help-command #'embark-prefix-help-command)

  :config
  ;; Hide the mode line of the Embark live/completions buffers
  (add-to-list 'display-buffer-alist
               '("\\`\\*Embark Collect \\(Live\\|Completions\\)\\*"
                 nil
                 (window-parameters (mode-line-format . none)))))

;; Consult users will also want the embark-consult package.
(use-package embark-consult
  :ensure t ; only need to install it, embark loads it after consult if found
  :hook
  (embark-collect-mode . consult-preview-at-point-mode))

Ad 6. Guaranteed response-time

I haven’t found a solution for this yet, but also have not found it to be a problem that needs solving in practice so far.

Editing control

Editing control are generic features which make editting easier, faster, more productive etcetera

Techniques in use by me:

multiple cursors

doing the same edit actions over multiple locations at once

templates

using a shortcut or abbreviation to insert templates/forms to fill in common constructs

Multiple cursors is a package which turns 1 cursor into many, depending on the active region, and enters insertions at all cursor locations.

(use-package multiple-cursors
  :bind (("C-c e" . 'mc/edit-lines)))

Tempel is a templating system, with templates being lisp data, which hooks into modes and autocompletion system. The folder ~/.emacs.d/templates holds *.eld files which specify templates which get autoloaded based on the major mode of the current buffer.

I set it up for relevant modes as a CAPF handler (for programming and orgmode for now)

(use-package tempel
  :custom
  ;; This would start completion directly, which I do NOT want
  (tempel-trigger-prefix nil)
  (tempel-path `(,(expand-file-name "templates/*.eld" user-emacs-directory)))

  :bind (("M-<insert>" . tempel-complete))

  :after (org)

  :init

  (setq org-tab-before-tab-emulation-hook nil)
  (defun mrb/tempel-setup-capf()
    ;; Remove org structured templates, which are just tempo templates as well
    ;; TODO make this smarter? (there might be other things in there which I *do* need)
    (setq org-tab-before-tab-emulation-hook nil)
    (setq-local completion-at-point-functions
                (cons #'tempel-complete ; use tempel-expand if only exact matches wanted
                      completion-at-point-functions)))

  ;; Do template name completion in programming, textmodes and orgmode
  ;; TODO why not global?
  :hook ((prog-mode text-mode org-mode) . mrb/tempel-setup-capf))

Navigation

Navigating pieces of text effectively is probably the best optimization to make in the emacs configuration.

In my browser I use vimium to jump to links with the keyboard. In emacs avy does something similar. On pressing the hotkey, all matches of the first char typed are highlighted (frame wide) by a single character. By pressing that character the cursor is place there directly. This makes within-frame navigation a 3-key operation, which is considerably faster than anything else.

;; Bind super-j globally to jump quickly to anything in view
(use-package avy
  :bind (("s-j" . 'avy-goto-char)))

In a more general sense, evil-mode, the VI emulation layer on top of emacs, is the ultimate navigational package. I have tried this for about 6 months, but it’s not for me (yet).

Remote editing

As my default shell is zsh tramp does not work out of the box for me. It gets confused by my prompt, so on top of my .zshrc file I have the following line:

# Immediately bail out if we are coming in with a dumb terminal
# which, in my case, mostly means TRAMP is asking for something.
[[ $TERM == "dumb" ]] && unsetopt zle && PS1='$ ' && return

Once that is in place files can be opened with /ssh:/host:/path/to/file syntax. Surprisingly, the syntax I got used to in the past (i.e. /host:/path/to/file) does not work for me anymore.

(use-package tramp
  :straight (:type built-in)            ; tramp is closely tied to emacs, use builtin
  :custom
  (default-tramp-method "sshx")
  (tramp-syntax 'default)
  (tramp-terminal-type "dumb")

  :config
  (eval-after-load 'tramp '(setenv "SHELL" "/bin/sh"))
  (add-to-list 'tramp-connection-properties
                   (list ".*" "locale" "LC_ALL=C")))

Browser integration

My default browser, as set in the OS, should be started automatically on opening http: like stuff and html files. The only link to the outside world is specifying that xdg-open should figure out what program to use.

(setq browse-url-browser-function 'browse-url-generic)
(setq browse-url-generic-program "xdg-open")

There’s a special place for the nyxt browser here. It is particularly interesting because its design was inspired by emacs. The two things that are interesting to me are that it is programmed in common-lisp and thus has all the niceties of that, including an option to connect a REPL to the browser and change the program while it is running. So, let’s start with a convenience function to make a connection to a running Nyxt instance.

(defun mrb/nyxt-connected-p ()
  "Is nyxt connected"
  (sly-connected-p))

(defun mrb/nyxt ()
  "Connect the the nyxt browser with sly"
  (interactive)

  (unless (sly-connected-p)
    (sly-connect "localhost" "4006")))

What other options I need for this is as of yet unclear. Things which come to mind are: controlling window refreshments when something was published (blog postings for example) or monitoring web page changes, capturing information from the current web view buffer to emacs.

Not exactly browser integration, but ‘openstreetmap browsing’ is close enough. I would like to use this for planning trips, so something like a ‘set of view settings’ for this package would be nice. This would allow me to gather maps views, augment them with a gpx route and bookmarks. Not sure if gpx waypoints/POI sets would work.

The package as is, is already useful for searching locations and storing links to locations.

(use-package osm
  :straight (:host github :repo "minad/osm")
  :commands (osm osm-mode)
  :bind ( :map osm-mode-map
          ("q" . (lambda() (interactive) (quit-window t))))
  :init
  ;; Load Org link support
  (with-eval-after-load 'org
    (require 'osm-ol))

  :custom
  (osm-outdoor-url
   (concat "https://tile.thunderforest.com/outdoors/{%z}/{%x}/{%y}.png?apikey="
           (password-store-get "API/tile.thunderforest.com")))
  (map5-base (concat "https://s.map5.nl/map/"
                     (password-store-get "API/map5.nl")
                     "/tiles/"))
  (osm-map5-opentopo-url
   (concat map5-base "opentopo/EPSG900913/{%z}/{%x}/{%y}.jpeg"))

  (osm-map5-opensimpletopo-url
   (concat map5-base "opensimpletopo/EPSG900913/{%z}/{%x}/{%y}.jpeg"))

  (osm-map5-openlufo-url
   (concat map5-base "openlufo/EPSG900913/{%z}/{%x}/{%y}.jpeg"))

  (osm-server-list
   `((default
      :name "Mapnik"
      :description "Standard Mapnik map provided by OpenStreetMap"
      :url "https://%s.tile.openstreetmap.org/%z/%x/%y.png"
      :group "Standard")
     (outdoor :name "Outdoor"
              :description "Outdoor focussed maps"
              :url ,osm-outdoor-url
              :group "Personal")
     (opentopo :name "OpenTopo NL"
               :description "Map5.nl OpenTopo"
               :url ,osm-map5-opentopo-url
               :group "Personal")
     (opensimpletopo  :name "OpenSimpleTopo NL"
                      :description "Map5.nl OpenSimpleTopo"
                      :url ,osm-map5-opensimpletopo-url
                      :group "Personal")
     (openlufo  :name "OpenLufo NL"
                :description "Map5.nl OpenLufo"
                :url ,osm-map5-openlufo-url
                :group "Personal")))

  ;; set proper home location, TODO  read this from private store
  (osm-home '(51.6441759 4.4377029 17))
  (osm-copyright nil))

Messaging and chatting

Mail

This section describes a search for the mail setup I want in Emacs. There are a few mail related packages and I’m unsure at what I want to use; reading the feature list doesn’t cut it. So, I’m having a setup where multiple mail handling packages may exist and hopefully one will float to the top as being the one

Composing mail

Composing mail is often an out of band activity, like creating a tweet or a capture, so I would like to have roughly the same behavior. This is by default provided by compose-mail-other-frame, which in turn calls the right mua to do the job. Oddly enough, it is still required to have mu4e-compose-in-new-frame set to actually have another frame, so I’m writing it with mu4e-compose directly.

  :bind (("C-x m" . 'mrb/compose-mail)
         ( :map mu4e-compose-mode-map
           ("C-;" . mu4e-compose-context-switch)))

  :custom
  (mu4e-compose-format-flowed t)
  (mu4e-compose-switch nil)
  (mu4e-compose-dont-reply-to-self t)
  (mu4e-compose-complete-only-after nil) ; no limit on contact completion for now

  :config
  (defun mrb/compose-mail (&optional mailto-url)
    "Run mail-compose, use mailto URI if it is given."
    (interactive)

    ;; If we have a mailto argument, parse and use it
    (if (and (stringp mailto-url)
             (string-match "\\`mailto:" mailto-url))
        (browse-url-mail mailto-url)
      ;; No mailto, argument, just run the mua
      (mu4e-compose-new)))

  (defun mrb/mailfile()
    "Compose mail message from current buffer file, typically a pdf is viewed for my use-case"
    (interactive)
    (let* ((file (buffer-file-name))
           (mimetype (mm-default-file-type file)))
      (mu4e-compose-new)
      (mml-attach-file file mimetype (concat mimetype " attachment") "attachment")))

;; Context switching in compose
<<mu4e-context-switch-in compose>>

;; Extend attaching files with extra info insertion
  <<extend-attach>>

To be able to use the mrb/compose-mail function as a mailto handler we need to be able to call it from outside of emacs. Let’s define a small shell script that does exactly this. The SRC attributes tangle it into my bin directory where is will be in the path. In the OS, this script will need to be set as the default mail handler.

# Emacs mailto URI handler

# Make sure mailto: is always prepended to $1
mailto="mailto:${1#mailto:}"
mailto=$(printf '%s\n' "$mailto" | sed -e 's/[\"]/\\&/g')

# Call the elisp function handling our mailto URI
elisp_expr="(mrb/compose-mail \"$mailto\")"

# Do not create new frame, because mu4e does that already
# which is a simpler solution in this case
edit-noframe --eval "$elisp_expr"

On top of that, I want a manual capture script which is basically the same as the mailto handler.

edit -e '(mrb/compose-mail)'

Attachments

When attaching files, it is sometimes useful to extract some info from the attached file and include it in the message. My main usecase is extracting PDF annotations (mostly for people who cannot process the annotations in their PDF viewer directly.

The way I have implemented this is by advicing the function mml-insert-empty-tag which is used by the mml-attach-file function. At that point in the code the file and point in the active buffer are known, and we just have to process the extra actions. The advice function extracts the annotations from the pdf file (unconditionally at the moment) and inserts a rendering of them just before inserting the attachment itself.

:config
;; TODO  return info, do not use insert directly, so we have more control
(defun mrb/pdf-extract-info (file)
  "Extract and render the pdf annotations in FILE"
  (mapc
   (lambda (annot) ;; traverse all annotations
     (progn
       (let ((page (cdr (assoc 'page annot)))
             (highlighted-text
              (if (pdf-annot-get annot 'markup-edges)
                  (let ((highlighted-text
                         (pdf-info-gettext (pdf-annot-get annot 'page)
                                           (pdf-tools-org-edges-to-region
                                            (pdf-annot-get annot 'markup-edges))
                                           t
                                           file)))
                    (replace-regexp-in-string "\n" " " highlighted-text))
                nil))
             (note (pdf-annot-get annot 'contents))
             (type (pdf-annot-get annot 'type))) ; underline, strike-through etc.
         ;; Stuff gathered, start rendering
         (when (or highlighted-text (> (length note) 0))
           (insert (format "\n- page %s" page))

           (when highlighted-text
             (insert (format ": “%s”\n" highlighted-text)))

           (if (> (length note) 0)
               (insert (format "\n  %s\n" note))
             (insert "\n" ))))))
   (cl-remove-if                          ; don't process links?
    (lambda (annot) (member (pdf-annot-get-type annot) (list 'link)))
    (pdf-info-getannots nil  (expand-file-name file)))))


(defun mrb/extended-attach-file (name &rest plist)
  "Advice `:before `mml-insert-empty-tag to grab additional info from the attachment."
  (let* ((file (plist-get plist 'filename))
         (mimetype (if file (mm-default-file-type file) nil)))
    (when (string= mimetype "application/pdf")
      ;; Extract and render annotations
      (mrb/pdf-extract-info file))))

(advice-add
 'mml-insert-empty-tag                  ; a lot easier because we already know the file
 :before
 'mrb/extended-attach-file
 '((name . "extended-attach")))

Footnotes

While composing mail referring to external resources through links is done very often. While creating links in HTML mail, which I do not use, gives the option to name a link, there’s no such thing in plain text. This usually works out fine, but when there are many links I fall back to creating ‘footnotes’ which actuall hold the links while in the text there is just an identifier to point the reader to the proper footnote.

Of course these footnotes can be used for other things as well.

Here’s what I would like:

• easy shortcut to start adding a footnote
• easy shortcut to return to the main text
• insert the proper numbers automatically, optionally renumber when I put an extra in later on.
• insert superscript like ¹ or ² into the text
• insert captured content at bottom of current text (but not below quotes)

Footnote mode seems to do exactly that, configured properly. Its prefix key is a bit awkward, so I have changed it and having the unicode parentheses for start and end tag is not ideal, but the package requires there to be something; I’d prefer there to be nothing but that messes up renumbering and other operations.

• Key shortcuts are a bit awkward, as ! is awkward A: changed the prefix to "C-c n" which makes more sense
• auto enable in message mode, mu4e compose

  (use-package footnote
    :straight (:type built-in)
  
    ;; Use a better prefix than the default 'C-c !'
    :bind-keymap ("C-c n" . footnote-mode-map)
  
    :hook (message-mode . footnote-mode)  ; I use it in mu4e
    :config
    (setq footnote-section-tag "" ; No introduction needed?
          footnote-start-tag "⁽"  ; doc says not to use empty string
          footnote-end-tag "⁾"
          footnote-style 'unicode         ; this makes footnotes use superscript numbering
          footnote-body-tag-spacing 1     ; between the \sup1 and the footnote text
          ))
      

Sending mail

Sending mail through smtp, using the smtpmail package

(use-package smtpmail
  :custom
  (smtpmail-default-smtp-server "localhost")
  (smtpmail-service 25)
  (smtpmail-local-domain user-domain)
  (smtpmail-sendto-domain user-domain)
  ;; User agent style is message mode by default, specific mail packages may override this
  (mail-user-agent 'message-user-agent)
  (send-mail-function 'smtpmail-send-it) ;This is for mail-mode
  (message-send-mail-function 'message-smtpmail-send-it) ; This is for message-mode

  (password-cache t)            ; default is true, so no need to set this actually
  (password-cache-expiry 28800)) ; default is 16 seconds, which is ridiculously low

At times we want to send signed and/or encrypted mail

(use-package mml-sec
  :straight (:type built-in)
  :config
  ;; Use my key to sign messages and make it safe if other keys exist
  (add-to-list 'mml-secure-openpgp-signers user-gpg-key)
  (setq mml-secure-key-preferences
        '((OpenPGP
           (sign)
           (encrypt
            (user-mail-address user-gpg-key)))
          (CMS
           (sign)
           (encrypt))))

        ;; Add my bcc address to list of safe addresses in bcc for secure message
        (add-to-list 'mml-secure-safe-bcc-list mrb/bcc-address)

        ;; Always encrypt to self, so I can read my own messages
        (setq mml-secure-openpgp-encrypt-to-self `(,user-gpg-key)))

Generic mail message settings

It’s not entirely clear which package is exactly responsible for what, but there are a few settings which are related to the message package

(use-package message
  :demand t ; mu4e won't load otherwise
  :straight (:type built-in)

  :init
  (defun mrb/message-mode-hook ()
    "This configures message mode and thus other modes which inherit message mode."
    ;; We have to make sure that the messages we produce are
    ;; format=flowed compatible. This happens on encoding when sending
    ;; it. However, during compose we need to make sure that what we
    ;; offer to encode is suitable for it.

    ;; Let's do the compose part first No reflowing is possible without
    ;; having hard newlines, so lets always enable those
    (use-hard-newlines t 'always)

    ;; Use visual line mode and visually break at word boundaries which
    ;; is the same as the email encoding on send (see below)
    (visual-line-mode 1)
    (auto-fill-mode 0)                    ; Makes no sense in visual mode
    (setq adaptive-fill-mode nil)         ; This messes up fill-flowed-encode
    ;; FIXME: adaptive-fill-mode is global, is just want it off here, and not touch global

    (setq visual-fill-column-width nil)   ; Make sure `fill-column gets used
    (setq fill-column 80)                 ; Not the same as encode column below!
    (visual-fill-column-mode 1)           ; Show it

    ;; Next, do the encoding and sending part
    ;; Set the fill column on encoding (send)
    ;; 66 would be the proper value, but to help rendering
    ;; on the other end 998 is the max which helps a bit
    (setq fill-flowed-encode-column 66)

    ;; This enables the f=f encoding for sending
    (setq mml-enable-flowed t))

  :hook (message-mode . mrb/message-mode-hook)

  :config
  (setq
   message-signature-directory mrb/maildir
   message-signature-file mrb/default-signature-file

   ;; Register my alternative email-adresses, other than the default
   ;; This steers From header when replying and as such has some overlap with mu4e contexts
   message-alternative-emails (regexp-opt mrb/private-addresses)

   ;; When citing, remove the senders signature.
   ;; TODO  often I want to remove a lot more, like the disclaimer
   message-cite-function 'message-cite-original-without-signature

   ;; Citing/Quoting when replying
   message-yank-prefix "> "             ; Add '> ' when quoting lines
   message-yank-cited-prefix ">"        ; Add '>' for cited lines
   message-yank-empty-prefix ""         ; Add nothing for empty lines
   )

  ;; Define the headers that will be sent

  ;; Add openpg preferences
  (setq message-openpgp-header
        '("235E5C8CF5E8DFFB"
          "https://keys.openpgp.org/vks/v1/by-fingerprint/77DDA1B68D04792A8F85D855235E5C8CF5E8DFFB"
          "signencrypt"))
  (add-hook 'message-header-setup-hook 'message-add-openpgp-header)

  ;; This will have an effect in mu4e starting from 1.11.23
  (setq
        ;; NOTE: mu4e sets message-hidden-headers to a local value based on by mu4e-compose-hidden-headers
        message-hidden-headers '(not "To:" "From:" "Cc:" "Subject:")

        ;; How to attribute
        message-citation-line-format "[%N]:"
        message-citation-line-function 'message-insert-formatted-citation-line

        message-kill-buffer-on-exit t))

Mu4e specific settings

After a journey with many different MUAs I seem to be settling on mu4e.

;; TODO  check for system packages meson, g++, pkg-config,
;;        cmake, libglib2.0-dev, libgmime3.0-dev, libxapian-dev, texinfo
;; (names are debian package names)
(use-package mu4e
  ;; For guix
  :straight ( :type built-in :includes (mu4e-icalendar))

  ;; Straight needs some tweaking
  ;; :straight ( :type git :host github :repo "djcb/mu"
  ;;             :pre-build (("./autogen.sh") ("ninja" "-C" "build"))
  ;;             :files (:defaults "build/mu4e/*.el")
  ;;             :includes (mu4e-icalendar))

  :custom-face
  (mu4e-header-highlight-face ((t (:inherit hl-line :underline nil
                                            :weight normal :foreground unspecified))))
  (mu4e-unread-face           ((t (:inherit nil :underline nil :weight bold))))
  (mu4e-context-face          ((t (:inherit mu4e-link-face))))
  (mu4e-flagged-face          ((t (:foreground "#ebcb8b" ))))
  (mu4e-title-face            ((t (:foreground "#5e81ac"))))

  :custom
  (mu4e-mu-binary (expand-file-name "/home/mrb/.guix-profile/bin/mu" ))
  ;(mu4e-mu-binary (expand-file-name "/home/mrb/dat/src/emacs/packages/mu/build/mu/mu" ))

  :after (message)
  :commands (mu4e mrb/compose-mail mrb/mailfile)
  :hook ((mu4e-view-mode . mrb/mu4e-view-mode-hook)
         (mu4e-headers-mode . (lambda () (eldoc-mode -1))))

  :config
  (defun mrb/mu4e-view-mode-hook ()
    (interactive)
    (turn-on-visual-line-mode)
    ;; Reflow messages
    (setq mm-fill-flowed t))

  :bind ( :map mu4e-search-minor-mode-map
          ("/" . mu4e-search)
          ("s" . nil)
          ("e" . mu4e-search-edit))

  ;; Configuration sections
  <<mu4e-composeconfig>>
  <<mu4e-main-config>>
  <<mu4e-headers-config>>
  <<mu4e-view-config>>
  <<mu4e-packages>>

  ;; TODO  move these up
  (defun mrb/setsigfile (ctxname)
    (expand-file-name
     (concat ".signature-" (downcase ctxname))
     mrb/maildir))

  (setq
   ;; Override the default mail user agent with the mu4e specific one
   mail-user-agent 'mu4e-user-agent
   mu4e-sent-folder        "/Sent"
   mu4e-drafts-folder      "/Drafts"
   mu4e-trash-folder       "/Trash"
   mrb/mu4e-junk-folder    "/Junk"         ; Added for my specific config
   mrb/mu4e-archive-folder "/Archives"
   mu4e-attachment-dir     "~/Downloads"   ; Absolute folder here, not relative to Maildir!

   ;; Refiling to /Archives/YYYY
   mu4e-refile-folder (lambda (msg)
                        (let ((archive_base (concat mrb/mu4e-archive-folder "/"))
                              (msgyear (format-time-string "%Y" (mu4e-message-field msg :date))))
                          (cond
                           ;; Order: most specific to general, end with the 't' case
                           ;; Archive messages per year if it has a date field (basically, always)
                           (msgyear   (concat archive_base msgyear))
                           ;; The default refile location
                           (t  archive_base))))


   mu4e-use-fancy-chars t
   mu4e-get-mail-command "mbsync quick" ; Just get inbox and important stuff, not everything
   mu4e-decryption-policy 'ask
   mu4e-update-interval 120             ; we just index, no retrieve, so 2mins is fine

   ;; Set completing function to standard, so completing frameworks can use it
   mu4e-read-option-use-builtin nil
   mu4e-completing-read-function 'completing-read
   mu4e-save-multiple-attachments-without-asking t

   ;; Use same hidden setting as message mode
   mu4e-compose-hidden-headers message-hidden-headers

   ;; Context definitions
   ;; Goal 1. use global config, unless private address is used
   ;; TODO  Can we use the one that matched?
   mu4e-context-policy 'pick-first      ; When nothing matches only!
   mu4e-compose-context-policy 'nil     ; for compose, use current if nothing matches

   mu4e-search-results-limit -1         ; until we run into performance problems
   mu4e-search-include-related nil      ; do not included related by default


   mu4e-contexts
   `(,(let* ((ctxname "Default")
             (sigfile (mrb/setsigfile ctxname)))
        (make-mu4e-context
         :name ctxname
         :match-func (lambda (msg)
                       (when msg
                         (mu4e-message-contact-field-matches
                          msg `(:to :cc :from) mrb/default-email-address)))
         ;; Use global config, but restore what was changed in other contexts
         :vars `((user-mail-address      . ,mrb/default-email-address)
                 (message-signature-file    . ,sigfile)
                 (mu4e-compose-signature . (with-current-buffer
                                               (find-file-noselect message-signature-file)
                                             (buffer-string))))))
     ,(let* ((ctxname "Private")
             (sigfile (mrb/setsigfile ctxname)))
        (make-mu4e-context
         :name "Private"
         :match-func (lambda (msg)
                       (when msg
                         (mu4e-message-contact-field-matches
                          msg `(:to :cc :from) mrb/private-addresses)))
         :vars `((user-mail-address     . ,mrb/default-private-address)
                 (message-signature-file   . ,sigfile)
                 (mu4e-compose-signature . (with-current-buffer
                                               (find-file-noselect message-signature-file)
                                             (buffer-string))))))
     )
   )

  ;; Rendering of html mail is done with shr, but I prefer not to show it at all
  (with-eval-after-load "mm-decode"
    (add-to-list 'mm-discouraged-alternatives "text/html")
    (add-to-list 'mm-discouraged-alternatives "text/richtext"))

  (defun mrb/mu4e-shr2text (msg)
    "Overridden mu4e-shr2text"
    (mu4e~html2text-wrapper
     (lambda ()
       (let ((shr-inhibit-images nil)   ;
             (shr-width nil)            ; Use whole window width for rendering
             (shr-use-colors nil)       ; Don't use colors from html, they ugly
             (shr-bullet "• "))         ; Original was "* "
         (shr-render-region (point-min) (point-max)))) msg))

  (setq mu4e-html2text-command 'mrb/mu4e-shr2text))

Main screen

When mu4e start, it always opens in its main screen, which contains the main operations, visible bookmarks.

:bind ( :map mu4e-main-mode-map
        ("q" . mrb/mu4e-quit) ; just close the menu
        ("Q" . mu4e-quit)     ; really Quit
        (";" . nil)           ; this was too easy to press by mistake
        ("G" . (lambda () (interactive) (mu4e-update-mail-and-index 1)))
        ("C-;" . mu4e-context-switch))
:init
;; Claim the whole screen in mu4e main window
(defun mrb/mu4e-quit ()
  (interactive)
  (bury-buffer))

:config
(setq mu4e-main-buffer-hide-personal-addresses t ; no need to see my own addresses
      mu4e-main-hide-fully-read nil
      mu4e-confirm-quit nil

      mu4e-bookmarks (list
                      '( :name "Unread messages"
                         :query "flag:unread and not flag:trashed"
                         :key ?u)
                      '( :name "INBOX"
                         :query "(flag:unread or flag:new or maildir:/INBOX) and not (flag:draft or flag:flagged or maildir:/Trash or flag:trashed)"
                         :favorite t
                         :key ?i)
                      '( :name "TODO list"
                         :query "flag:flagged and not flag:trashed"
                         :key ?+)
                      '( :name "Today's messages"
                         :query "date:today..now"
                         :key ?d)
                      '( :name "Trash"
                         :query "maildir:/Trash or flag:trashed"
                         :key ?t)
                      '( :name "Junk"
                         :query "maildir:/Junk"
                         :key ?j)))

Header view

The first view after searching is a list of mail headers. Set config vars for this view and define the visualisation of the marks. I need an extra action to mark mail as junk which, on processing, moves the marked messages to the junk folder

:bind ( :map mu4e-headers-mode-map
        ("s" . mu4e-headers-mark-for-junk)
        ("r" . mu4e-compose-reply)
        ("R" . mu4e-compose-wide-reply)
        ("A" . mu4e-headers-mark-for-refile)
        ("SPC" . mu4e-headers-mark-for-something)
        ("X" . (lambda () (interactive) (mu4e-mark-execute-all t))))

:custom
(mu4e-headers-include-related nil)

(mu4e-headers-date-format "%F" "ISO format yyyy-mm-dd")
(mu4e-headers-results-limit -1)  ; For now, needed to have related search results
(mu4e-headers-visible-lines 20)  ; Default of 10 is a bit small
(mu4e-headers-auto-update nil)   ; Prevent confusion

;; Make header configuration explicit
(mu4e-headers-fields '((:flags . 6)
                       (:human-date . 12)
                       (:mailing-list . 10)
                       (:from . 22)
                       (:subject . nil)))
(mu4e-use-fancy-chars t)


;; Define marks, possibly redundant, but I'd like to see them here explicitly
;; Note that these are not custom vars, why not?? (they wont work as :custom entries)
:config
(setq mu4e-headers-attach-mark    '("a" . "📎 "))
(setq mu4e-headers-calendar-mark  '("c" . "📅"))
(setq mu4e-headers-draft-mark     '("D" . "🚧 "))
(setq mu4e-headers-encrypted-mark '("x" . "🔑 "))
(setq mu4e-headers-flagged-mark   '("F" . "🚩 "))
(setq mu4e-headers-list-mark      '("s" . "🔈"))
(setq mu4e-headers-new-mark       '("N" . "✨ "))
(setq mu4e-headers-passed-mark    '("P" . "↪ "))
(setq mu4e-headers-personal-mark  '("p" . "👨"))
(setq mu4e-headers-replied-mark   '("R" . "↩ "))
(setq mu4e-headers-seen-mark      '("S" . " "))
(setq mu4e-headers-signed-mark    '("s" . "🖊 "))
(setq mu4e-headers-trashed-mark   '("T" . "🗑️"))
(setq mu4e-headers-unread-mark    '("u" . "📩 "))

;; Add one specifically for marking as junk
(add-to-list 'mu4e-marks
             '(junk
               :char ("J" . "💀") :prompt "Mark as junk"
               :show-target (lambda (dyn-target) "Junk")
               :ask-target (lambda () mrb/mu4e-junk-folder)
               :action (lambda (docid msg target)
                         (mu4e--server-move
                          docid
                          (mu4e--mark-check-target target) "+S-N-u"))))

;; Let mu4e define functions for it
(mu4e~headers-defun-mark-for junk)
(mu4e--view-defun-mark-for junk)

Message view

After selecting a message in the header view, a message view will open.

:bind ( :map mu4e-view-mode-map
        ("s" . mu4e-view-mark-for-junk)
        ("r" . mu4e-compose-reply)
        ("A" . mu4e-view-mark-for-refile)
        ("X" . (lambda () (interactive) (mu4e~view-in-headers-context (mu4e-mark-execute-all t)))))

:custom
(mu4e-view-use-old nil)
(mu4e-view-show-images t)           ; This creates connections, possible privacy issue?
(mu4e-view-image-max-width 800)
(mu4e-view-image-max-height 600)
(mu4e-view-show-addresses t)        ; Show actual addresses instead of names

(mu4e-view-actions '(("c - capture message" . mu4e-action-capture-message)
                          ("v - view in browser" . mu4e-action-view-in-browser)
                          ("s - save attachments" . mu4e-view-save-attachments)
                          ("t - show this thread" . mu4e-action-show-thread)))

Addon packages

There are quite a few mu4e packages out there. Let’s load them in one group for now, so it will be easy to disable plugins if we need to.

:config
;; Show an alert when new mails come in, this relies on mu4e being active though
(use-package mu4e-alert
  :hook (after-init . mu4e-alert-enable-notifications)
  :config
  ;; Just show subjects, not counts
  (setq mu4e-alert-email-notification-types '(subjects))
  (mu4e-alert-set-default-style 'libnotify))

;; Support icalendar Yes/No/Maybe reactions to calendar invites
;; TODO not needed anymore now?
(use-package mu4e-icalendar
  :config
  (mu4e-icalendar-setup))

(use-package mu4e-query-fragments)  ; Support reusable query pieces '%junk' etc.
(use-package mu4e-jump-to-list)     ; Shortcut key 'l' to directly focus on list threads

;; Patch highlighting inside messages
(use-package message-view-patch
  :after (magit)
  :hook (gnus-part-display . message-view-patch-highlight))

;(use-package org-mu4e :after org)

Sieve

Sieve is used on the server to filter our mail and this is accessed by the sieve-manage package. For some reason I can’t get the ‘^M’ characters out of the process, so here’s a hack for that.

(use-package sieve
  :init
  (defun dos2unix ()
    "Replace DOS eolns CR LF with Unix eolns CR"
    (interactive)
    (goto-char (point-min))
    (while (search-forward "\r" nil t) (replace-match "")))
  :commands sieve-manage
  :hook (sieve-mode . dos2unix)
  :custom
  (sieve-manage-authenticators '(plain digest-md5 cram-md5 scram-md5 ntlm login)))

which basically goes over the whole sieve script and removes the ‘^M’ characters from the buffer

Elfeed

Elfeed handles my RSS feeds, which I specify using an orgmode file feeds.org

(use-package notifications)

(use-package elfeed
  :after notifications
  :commands elfeed
  :bind (("C-c f" . 'elfeed)
         :map elfeed-show-mode-map
         ("w" . 'mrb/elfeed-show-toggle-watchlater)
         ("v" . 'mrb/elfeed-play-with-mpv)
         :map elfeed-search-mode-map
         ("v" . 'mrb/elfeed-play-with-mpv)
         ("w" . 'mrb/elfeed-search-toggle-watchlater))
  :init
  (setf url-queue-timeout 30
        elfeed-db-directory "~/.elfeed/")

  :custom-face
  (elfeed-search-tag-face ((t (:foreground "#a3be8c"))))
  (elfeed-search-feed-face ((t (:foreground "#ebcb8b"))))
  (elfeed-search-date-face ((t (:foreground "#88c0d0"))))
  :config
  (defun mrb/elfeed-search-toggle-tag(tag)
    (let ((entries (elfeed-search-selected)))
      (cl-loop for entry in entries do
               (if (elfeed-tagged-p tag entry)
                   (elfeed-untag entry tag)
                 (elfeed-tag entry tag)))
      (mapc #'elfeed-search-update-entry entries)
      (unless (use-region-p) (forward-line))))

  (defun mrb/elfeed-search-toggle-watchlater()
    (interactive)
    (mrb/elfeed-search-toggle-tag 'watchlater))

  (defun mrb/elfeed-show-toggle-tag(tag)
    (interactive)
    (if (elfeed-tagged-p tag elfeed-show-entry)
        (elfeed-show-untag tag)
      (elfeed-show-tag tag)))

  (defun mrb/elfeed-show-toggle-watchlater()
    (interactive)
    (mrb/elfeed-show-toggle-tag 'watchlater))

  ;; umpv maintains a playlist, so adding more videos wil automatically queue
  (defun mrb/elfeed-play-with-mpv ()
    "Play elfeed link in mpv"
    (interactive)
    (notifications-notify
     :title "Elfeed action"
     :body "Playing video with MPV"
     :app-name "Elfeed")

    (start-process "elfeed-mpv" nil
                   "~/bin/umpv"
                   (elfeed-entry-link (elfeed-search-selected t))))


  ;; New entry hook allows meta information manipulation
  ;; without directly having to change elfeed-feeds
  (add-hook 'elfeed-new-entry-hook
            (elfeed-make-tagger :feed-url "youtube\\.com"
                                :add '(video youtube)))
  (add-hook 'elfeed-new-entry-hook
            (elfeed-make-tagger :feed-url "vimeo\\.com"
                                :add '(video vimeo))))

Elfeed allows to interactively subscribe to a feed (defaulting to what is in the clipboard. Managing elfeed-feeds as a variable is kinda clumsy, although very flexible. There is a middle ground which fits me even better. At the cost of the bare-bone flexibility of having the feeds directly in lisp, I’m using an orgmode file to record the feeds i want.

(use-package elfeed-org
  :after elfeed
  :init
  (setq rmh-elfeed-org-files (list (concat elfeed-db-directory "feeds.org")))
  (elfeed-org))

Quite a few of my feeds are youtube channels, the elfeed-tube package has some extra features for reading those feeds

(use-package elfeed-tube
  :after elfeed
  :config
  (elfeed-tube-setup)

  :bind (:map elfeed-show-mode-map
              ("F" . elfeed-tube-fetch)
              ([remap save-buffer] . elfeed-tube-save)
              :map elfeed-search-mode-map
              ("F" . elfeed-tube-fetch)
              ([remap save-buffer] . elfeed-tube-save)))

IRC

I used to run a weechat relay on a vps for IRC. The main reason for this was that the relay allowed multiple clients to be active at the same time an have at least an attempt at saving the state betwee multiple devices.

The major downside is that a special weechat client is needed to make this work, which is not the same as a standard IRC client. This made the choice for clients very limited, but luckily there was an emacs client.

After using it some time and maintaining some local patches; the main program is largely unmaintained, it was time to look for an alternative.

While having the weechat relay is very nice, I also want to have a “standard” IRC configuration where everything is happening in the client, possibly augmented by having an IRC bouncer to maintain persistent connections. ZNC might almost have the featureset to do what weechat does, or very closely to it.

ERC seems to be the most used IRC client for emacs and included with it, so it would make sense to use it. I like the simplicity of circe though, so I opted to create a config for that.

(use-package circe
  :commands (circe mrb/chat)
  :custom
  (circe-default-nick "mrvdb")
  (circe-network-options
        `(("libera"
           :host "chat.hsdev.com" :port 6667
           :pass ,(concat "mrb@emacs/libera:"
                          (password-store-get "chat.hsdev.com"))
           :channels ("#emacs"
                      "#talos-workstation"
                      "#vikings"))
          ("oftc"
           :host "chat.hsdev.com" :port 6667
           :pass ,(concat "mrb@emacs/oftc:"
                          (password-store-get "chat.hsdev.com"))
           :channels ("#osm"))))

  ;; Simplify and improve formatting, remove some details
  (circe-format-say "{nick:10s}: {body}" "right align nicks at 10th pos")
  (circe-format-self-say "{nick:10s}: {body}")
  (circe-format-server-part "*** Part: {nick} left {channel}")
  (circe-format-server-quit-channel "*** Quit: {nick} left {channel}")
  (circe-format-server-quit "*** Quit: {nick}")
  (circe-format-server-join "*** Join: {nick}")
  (circe-format-server-rejoin "*** Re-joined: {nick}")
  (lui-time-stamp-position 'right-margin)
  (lui-fill-type nil)
  (circe-reduce-lurker-spam t "dont show notices for non talkers")
  (circe-default-part-message "" "no parting reason")
  (circe-default-quit-message "" "no quitting reason")

  :config
  (enable-lui-track)                   ; show indicator to where has been read
  (enable-circe-display-images)        ; turn links to images into images
  (enable-circe-color-nicks)           ; color nicks

  ;; Faces after enables, so the faces are there
  ;; TODO  do not use color names here, but functional names, which we can change centrally
  (set-face-attribute 'circe-prompt-face   nil :foreground mrb/cursor-color :height 1.3)
  (set-face-attribute 'circe-originator-face nil :height 0.8)
  (set-face-attribute 'lui-time-stamp-face nil :height 0.8)
  (set-face-attribute 'lui-track-bar       nil :background "LawnGreen") ; fix this one!

  ;; Gather buffer local and line UI settings
  (defun mrb/lui-setup ()
    (setq fringes-outside-margins t     ; first margins, then fringes
          right-margin-width 7          ; [hh:mm]
          word-wrap t                   ;
          wrap-prefix "            ")   ; line this up with size of nick in `circe-format-say
    (setf (cdr (assoc 'continuation fringe-indicator-alist)) nil))
  (add-hook 'lui-mode-hook 'mrb/lui-setup)

  ;; Set the prompt properly on entering the chat mode
  (defun mrb/set-circe-prompt ()
    (lui-set-prompt (propertize (concat "[" circe-chat-target "]➜ ")
                                'face 'circe-prompt-face)))
  (add-hook 'circe-chat-mode-hook 'mrb/set-circe-prompt)

  ;; Make a convenient connect function
  ;; TODO make this re-entrant
  (defun mrb/chat()
    (interactive)
    (circe "libera")
    (circe "oftc")))

Mastodon

Not messaging or chatting perse, but a micro blog social network. The ‘capture a toot’ is hot-keyed through xbindkeys like other capture commands.

;; Mastodon in emacs
(use-package mastodon
  :straight (mastodon :host codeberg :repo "martianh/mastodon.el"
                      :branch "develop")
  :custom
  (mastodon-instance-url "https://mastodon.nl")
  (mastodon-active-user "mrb")
  ;; finds mastodon.nl in password-store and expects user field to be present
  (mastodon-auth-source-file 'password-store)

  :config
  ;; Convenience function
  (defun mrb/capture-toot()
    (interactive)
    (mastodon-toot)))

edit-noframe --eval '(mrb/capture-toot)'

Development settings

Some settings which aid in development tasks.

Generic

Let’s start with some generic development related settings and packages

(use-package ggtags)                    ; not used very much

(use-package cmake-mode)                ; adjusts auto-mode-alist on load

(use-package yaml-mode)                 ; used by ansible, magit(?)

For most of the languages I use (Bash, python, C, Go, HTML Haskell, Lua), the Language Server Protocol seems to be supported by eglot, which is now a built-in package, so I’m setting this up generally here and enable it for each language I’m using if it is supported.

(use-package eglot
  :straight (:type built-in))

Make our source look a bit more attractive by enabling prettify-symbol-mode in all programming modes

(use-package prog-mode
  :straight (:type built-in)
  ;; TODO move hooks to their defining packages?
  :hook ((prog-mode                     ; all modes that derive from prog-mode
          lisp-interaction-mode) . mrb/prettify-symbols)
  :init

  ;; TODO Move this to visual?
  ;; TODO this is a bit messy, do this in mode packages and per mode
  (defun mrb/prettify-symbols ()
    (interactive)
    ;; set buffer local variable to map the symbols
    (setq prettify-symbols-alist
        '(("lambda" . ?λ)
          ("lambda*" . (?λ (Br . Bl) ?*)) ; just in scheme really
          ("-lambda" . (?- (Br . Bl) ?λ))
          ;("map"    . ?↦)     ; this one is a bit of a pain actually
          ("->"     . ?⟶)
          ("<-"     . ?⟵)
          ("=>"     . ?⟹)
          ;("#t"     . ?⟙)     ; bad idea
          ;("#f"     . ?⟘)     ; bad idea
          ("|>"     . ?▷)
          ("<|"     . ?◁)
          ("->>"    . ?↠)
          ("<="     . ?≤)
          (">="     . ?≥)))
    (prettify-symbols-mode)))

In general, the syntax highlighting of emacs is sufficient, but is based on defining regular expression to match language contents to color them. There is a better way to do this; based on generating a semantic tree representation of the language in question. There is a tool called tree-sitter which does this.

The main advantages of the method that tree-sitter uses are:

• the same tree representation is used for all languages, so adding new languages is relatively easy
• using the representation for syntax highlighting is just one application: code folding or semantic extending a region, all of which are available in emacs in other ways, is a lot better using the tree-sitter method.
• it’s super fast. fast enough to keep up with typing.

As of emacs 29 the treesit package is built-in, so let’s use that in any case for highlighting where it is possible. I think over time the mapping of the modes below will just disappear and everything will be treesitter based.

;; Use the built-in treesit and load all language grammars
(use-package treesit
  :straight (:type built-in)
  :config
  (setq treesit-extra-load-path
        `(,(expand-file-name "var/tree-sitter/langs" user-emacs-directory)))
  ;; Replace relevant modes with the treesitter variant
  (dolist (mode
           '((bash-mode       . bash-ts-mode)
             (c-mode          . c-ts-mode)
             (c++-mode        . c++-ts-mode)
             (css-mode        . css-ts-mode)
             (dockerfile-mode . dockerfile-ts-mode)
             (go-mode         . go-ts-mode)
             (javascript-mode . js-ts-mode)
             (js-json-mode    . json-ts-mode)
             (python-mode     . python-ts-mode)
             (typescript-mode . typescript-ts-mode)
             (yaml-mode       . yaml-ts-mode)))
    (add-to-list 'major-mode-remap-alist mode)))

To have some control over the environment for specific projects I use direnv, so let’s install emacs support for it as well.

(use-package envrc
  :config
  (envrc-global-mode))

Reference & documentation

While emacs itself has a very nicely documentation system, I still don’t understand why this is not interpolated into every other mode and or language.

Example: C-h-f in lisp-mode describes the function to me. If I am in a python file, I want it to do exactly the same, but for the python function. There’s a page on emacs wiki dealing with this a bit: https://www.emacswiki.org/emacs/Context_sensitive_help

There’s a package ghelp (generic help I guess) which promises to do what I want, but I can’t get it installed properly with use-package. That package can use the helpful package as back-end which is also helpful on it own, so I’m installing that anyways and replace the internal help commands from emacs with them.

(use-package helpful
  :bind (("C-h f" . #'helpful-callable)
         ("C-h h" . #'helpful-at-point) ; I don't care about the emacs 'hello' file
         ("C-h v" . #'helpful-variable)
         ("C-h k" . #'helpful-key)))

Anyways, here’s what I have configured related to reference information and documentation

Context sensitive

Regardless of (programming) language there’s certain information that’s needed on the spot. These are the function signatures, the specific syntax when typing Math symbols, orgmode oddities etc.

I want that type of information directly at hand. One part of the solution is ElDoc. This shows documentation in the echo area for a language. ElDoc is built into Emacs so that’s a good start. Let’s enable it for the modes that have support for it.

(dolist
    (the_mode
     (list 'emacs-lisp-mode-hook
           'lisp-interaction-mode-hook
           'ielm-mode-hook
           'python-mode))
  (add-hook the_mode 'eldoc-mode))
(diminish 'eldoc-mode)

The location of the eldoc information in the message area is a bit far away from the editing point. I have looked at eldoc-overlay and others to resolve that, but haven’t found a satisfying solution yet.

Reference information

If the inline documentation presented by ElDoc is not sufficient, I want a way to spawn reference documentation based on the context.

The closest thing I found is zeal or dash which allow docsets to be searched. There is a GUI and helm-dash is an interface to the docsets for emacs. Typically I install docsets through the GUI or manually (the folder names are sometimes not the same when I use helm-dash to install docsets). This solution gives me at least access to reference information for most of the programming languages I use. The direct access to the sources is missing, or I don’t know how to do this yet.

;; Unify reference documentation with dash
(use-package helm-dash
  :after helm
  :commands (helm-dash helm-dash-at-point)
  :config
  ;; Make sure docset directory exists
  (make-directory helm-dash-docsets-path t)
  :custom
  (dash-docs-enable-debugging nil)
  (helm-dash-browser-func 'eww "Within dash, keep browser links within emacs")
  (helm-dash-common-docsets (helm-dash-installed-docsets)))

Dash show documentation inside emacs, which is my preferred method. There are times however I need the desktop GUI Zeal. For one, it’s a lot easier to see which docsets are installed and manage them.

;; Global keybinding ot open documentation
  ;; This should probably be somewhere else
  (use-package zeal-at-point
     :bind
     ("s-d" . zeal-at-point))

RFC document reader

I often need to consult RFC documents, so not having to leave emacs for that and be able to use all tools on RFC documents is very helpful.

;; Special mode to read rfc documents locally
(use-package rfc-mode
:custom
(rfc-mode-directory (expand-file-name "~/dat/doc/rfc/"))
(rfc-mode-index-path (concat rfc-mode-directory"rfc-index.txt")))

Man pages

There’s a builtin package to read man pages, let’s make it explicit.

(use-package man)

Not really man pages as such, but meant as their advanced cousin for GNU programs is the info format. As I tend to forget all the key shortcuts, install casual-info as well to help with that. C-o seems to be the defacto standord to summon the transient menu in a mode.

(use-package info
  :straight (:type built-in)
  :config
  (use-package casual-info
    :bind (:map Info-mode-map ("C-o" . 'casual-info-tmenu))))

Epub documents

Not often, but increasingly so, reference books are in epub format. So, for that we need a reader package. nov seems to be the best option

(use-package nov
  :mode ("\\.epub" . nov-mode))

Coding styles

Different projects use different coding styles. The ones I need I’ll gather here for now.

My personal style will be called mrb and basing it on the linux style.

;; Basing it on k&r, no real reason, does it really matter?
(c-add-style "mrb"
             '("k&r"
               (indent-tabs-mode . nil)
               (c-basic-offset . 2)
               (c-cleanup-list . (scope-operator
                                  space-before-funcall))))

;; Make my style the default
(setq c-default-style
      '((java-mode . "java")
        (awk-mode  . "awk")
        (other     . "mrb")))

;; EditorConfig support, not used much by the looks of it
(use-package editorconfig
  :disabled t
  :diminish
  :config
  (editorconfig-mode 1))

Language support

Python

Just enabling eglot for now.

(use-package python
  :straight (:type built-in)
  :hook (python-mode . eglot-ensure))

Haskell

I am just starting out with haskell, but the two things that are probably needed in any case are a mode to use when editing Haskell source (*.hs files) and the ghc-mod package to help with completion and showing syntax errors.

(use-package haskell-mode
  :hook (;(haskell-mode . interactive-haskell) ;; Produces elc load error
         (haskell-mode . turn-on-haskell-doc)
         (haskell-mode . haskell-indentation-mode))

  :mode "\‌\.hs\\'"
  :custom
  (haskell-font-lock-symbols t)
  (haskell-interactive-popup-errors nil)
  (haskell-process-type 'stack-ghci)
  :config
  ;; Replace ⇒ with ⇉
  (delete '("=>" . "⇒") haskell-font-lock-symbols-alist)
  (add-to-list 'haskell-font-lock-symbols-alist '("=>" . "⇉")))

Enable the language server for haskell

(use-package lsp-haskell
  :after (haskell-mode lsp-mode)
  :hook ((haskell-mode  haskell-literate-mode) . lsp))

Because the haskell environment can be different for each project, we need a way to adjust to that for certain things. One such thing is flycheck which will give false-negatives when checking haskell files if the specific build environment is not taken into account. The package flycheck-haskell automatically adjusts flycheck.

(use-package flycheck-haskell
  :after flycheck
  :hook (flycheck-mode . flycheck-haskell-setup))

For creating web apps I use yesod, which is supported by a number of packages

;; Yesod's html like files
(use-package hamlet-mode)

Go

(use-package go-mode
  ;; TODO  godef is fairly heavy on my little machines, make optional?
  :ensure-system-package godef

  :init
  ;; Set up before-save hooks to format buffer and add/delete imports.
  ;; Make sure you don't have other gofmt/goimports hooks enabled.
  ;; TODO  would this not add hooks for all files that are saved??
  (defun lsp-go-install-save-hooks ()
    (add-hook 'before-save-hook #'lsp-format-buffer t t)
    (add-hook 'before-save-hook #'lsp-organize-imports t t))
  (add-hook 'go-mode-hook #'lsp-go-install-save-hooks))

Rust

(use-package rust-mode)

Lisp-like

There are a number of languages which are lisp-like, not in the least Emacs lisp itself. It makes sense to group the configuration for these languages together. In the first part there will be configuration for all lisp-like languages (like the puni configuration for example). For each specific implementation there may be augmentations to the config, or even a completely separate.

For all of those languages, I want to use puni which makes editing, longer term, a lot more productive. For autopairing, which puni does not do itself unlike paredit, I use the built-in electric-pair-mode Both of these support multiple modes, so are really generic packages, but lisp-like modes are the modes typically benefitting the most from them.

(use-package puni
  :init
  (puni-global-mode)                    ; perhaps too much, we'll see

  :bind ( :map puni-mode-map
          ("C-<right>" . puni-slurp-forward) ; foo (bar|) baz  ->  foo (bar| baz)
          ("C-<left>"  . puni-barf-forward)  ; foo (bar| baz)  ->  foo (bar|) baz
          )

  :hook ((org-mode . puni-disable-puni-mode)
         (prog-mode . electric-pair-local-mode)))

Evaluating an expression with the cursor after it is often used, I’d like being able to do the same when the cursor is in front of such an expression

;; The reciprocate of C-x C-e
;; As in:  _cursor_(...eval what is in here...)
(defun mrb/eval-next-sexp ()
  (interactive)
  (save-excursion
    (forward-sexp)
    (eval-last-sexp nil)))

Not sure how to bind this though. I’d like it close to C-x C-e obviously

When evaluating expressions, the result is way down in the status area. I would like to have it inline in the buffer in an overlay. The package cider does this, but I have no need for the complete clojure environment. There is a specific package for emacs-lisp though, called eros

(use-package eros
  :config
  (eros-mode 1) ;; ok for global active, just puts advices around the eval
  ;; TODO set the face
  )

Scheme

The generic scheme support is the builtin scheme package, let’s tell it that I use guile as my default scheme program.

(use-package scheme
  :custom
  (scheme-program-name "/home/mrb/.guix-profile/bin/guile"))

Geiser is, among other things, a repl for schemes. I try to use one, mostly for educational purposes. I chose guile because it also claims to support emacs-lisp and might be the future vm on which emacs-lisp will run.

;; Specific implementations depend on geiser
(use-package geiser-guile
    :straight (:host gitlab :repo "emacs-geiser/guile")
    :demand t
    :hook ((geiser-repl-mode . mrb/prettify-symbols))

    :custom
    (geiser-guile-binary "/home/mrb/.guix-profile/bin/guile")
    (geiser-active-implementations '(guile))
    (geiser-default-implementation 'guile)

    ;; Do NOT evaluate on return when inside an expression
    (geiser-repl-send-on-return-p nil)

    ;; start repl and eval results inline in buffer
    ;; TODO I do want this eval result to be transient, not inserted!
    (geiser-mode-start-repl-p t)
    (geiser-mode-eval-last-sexp-to-buffer t)
    (geiser-mode-eval-to-buffer-prefix " ;;=> ")

    )

Similar to using eros for emacs-lisp, I use geiser-eros for languages that geiser supports.

(use-package geiser-eros
  :after (eros geiser)
  :straight '(:type git :host sourcehut :repo "sokolov/geiser-eros")
  :config
  ;; Make sure geiser does not insert eval into buffer
  (setq geiser-mode-eval-last-sexp-to-buffer nil)
  (geiser-eros-mode 1))

While we’re in the scheme section, let’s configure guix and friends, which use scheme heavily

(use-package guix
  :config
  ;; Assuming the Guix checkout is in ~/src/guix.
  (with-eval-after-load 'geiser-guile
    (add-to-list 'geiser-guile-load-path "~/dat/src/guix"))

  ;; Use tempel templates
  (with-eval-after-load 'tempel
    ;; Ensure tempel-path is a list -- it may also be a string.
    (add-to-list 'tempel-path "~/dat/src/guix/etc/snippets/tempel/*"))

  ;; guix uses debbugs, but this should probably be closer to mu4e and gnus config
  (use-package debbugs))

Common lisp

The defacto standard for a development environment in Emacs for common-lisp is either the slime or sly package. The latter seems a bit more modern and easier to configure, so trying that one

(use-package sly
  :hook ((sly-mrepl-mode . sly-mrepl-font-lock-setup))
  :commands (mrb/nyxt)
  :init

  ;; Copy the keywords from lisp mode
  (defvar sly-mrepl-font-lock-keywords lisp-font-lock-keywords-2)

  ;; Set them up
  (setq sly-mrepl-font-lock-keywords
        (cons '(sly-mrepl-font-lock-find-prompt
                . 'sly-mrepl-prompt-face)
              sly-mrepl-font-lock-keywords))
  (defun sly-mrepl-font-lock-setup ()
    (setq font-lock-defaults
          '(sly-mrepl-font-lock-keywords
            ;; From lisp-mode.el
            nil nil (("+-*/.<>=!?$%_&~^:@" . "w")) nil
            (font-lock-syntactic-face-function
             . lisp-font-lock-syntactic-face-function))))

  :custom
  (inferior-lisp-program "sbcl")
  (sly-lisp-implementations
   '((sbcl ("sbcl" "--core" "/home/mrb/.sbcl/sbcl.core-for-sly"))))
  (sly-net-coding-system 'utf-8-unix)

  :config
  ;; Nyxt exposes a repl we can use
  <<nyxt-sly-config>>

  ;; Correct the REPL prompt
  (defun sly-mrepl-font-lock-find-prompt (limit)
    ;; Rough: (re-search-forward "^\\w*>" limit t)
    (let (beg end)
      (when (setq beg (text-property-any
                       (point) limit 'sly-mrepl-prompt-face t))
        (setq end (or (text-property-any
                       beg limit 'sly-mrepl-prompt-face nil)
                      limit))
        (goto-char beg)
        (set-match-data (list beg end))
        t))))

Gcode

I’m not in the habit of editing gcode, but every now and then I need to look at it and I want it reasonably decent. My use-case is just sending gcode to my 3D-printer which runs Marlin firmware.

Using define-generic-mode is more than enough to define something useful for me.

(use-package generic-x
  :straight (:type built-in))

(define-generic-mode marlin-gcode-mode
  '((";"))   ;; ; starts a comment
  (apply 'append
         (mapcar #'(lambda (s) (list (upcase s) (downcase s) (capitalize s)))
                 '("keywords?" "exist?")))
  '(("\\([GM]\\)[0-9]+" (1 font-lock-function-name-face))    ; code letters
    ("\\([ABCDEFHIJKLNOPQRSTUVWXYZ]\\)[-]?[0-9]+" (1 font-lock-string-face)) ; parameter letters
    ("\\([\-+]?[0-9]*\\.[0-9]+\\)" (1 font-lock-constant-face)) ; 9.9 type numbers
    ("\\([\-+]?[0-9]+\\)" (1 font-lock-constant-face)))         ; 999 type numbers
  '("\\.gcode\\'")
  nil
  "Mode for marlin g-code files.")

Openscad

Openscad is a script based 3d parametric drawing program. Obviously I’m editing those scripts in emacs.

(use-package scad-mode
  :custom
  (scad-keywords '("return" "true" "false" "include")))

SQL

SQL has a number of different dialects I use, depending on the database software in use.

(use-package sql
  :straight (:type built-in)

  :custom
  (sql-server "dbserver.hsdev.com")
  (sql-postgres-options '("-P" "pager=off" "-p 5434")))

GIT integration

A common factor in all my development is the use of git. For emacs this automatically means Magit. I’ve used eshell in the past but that config didn’t work out.

Magit

For most, if not all development work (and some other work too) I use git as the revision control system. In emacs that translates to using magit, so let’s begin with bringing that in.

Magit forge is a sub-module for magit which helps in managing repositories in forges (notably github and gitlab). This looks very promising, albeit slow.

Features I’m interested in: issue commenting, pull requests

(use-package magit
  :straight (:host github :reo "magit/magit")

  :demand t
  :after (org fullframe)
  :commands magit-status
  :bind
  ("C-c m" . magit-status)

  :init
  (fullframe magit-status magit-mode-quit-window)

  :custom-face
  (diff-refine-added                 ((t (:background "#003000"))))
  (diff-refine-removed               ((t (:background "#300000"))))
  (magit-diff-added-highlight        ((t (:foreground "#a3be8c"))))
  (magit-diff-removed-highlight      ((t (:foreground "#bf616a"))))
  (magit-diff-added                  ((t (:foreground "#a3be8c"))))
  (magit-diff-removed                ((t (:foreground "#bf616a"))))
  (magit-diff-hunk-heading-highlight ((t (:background "#5e81ac"))))
  (magit-diff-hunk-heading           ((t (:background "#5e81ac"))))

  :custom
  (magit-last-seen-setup-instructions "1.4.0")
  (magit-diff-refine-hunk 'all)

  :config
  ;; Enable links to magit fromm org
  (use-package orgit)

  (use-package forge
    :disabled
    :custom
    (forge-add-pullreq-refspec 'ask)
    (forge-pull-notifications t)
    (forge-topic-list-limit '(60 . 0))

    :config
    ;; Only show assigned stuff
    (magit-add-section-hook
     'magit-status-sections-hook
     'forge-insert-assigned-issues   nil t)
    (magit-add-section-hook
     'magit-status-sections-hook
     'forge-insert-assigned-pullreqs   nil t)))

Most of the magit work is done through its status screen and I would like to see issues and todo’s and fixme’s and other work that needs to be done in that screen. The forge package does that from a remote forge, like github, but many tasks are hiddenn in the sources already by myself. The magit-todos package scans for those things and show them in the magit screen

(use-package magit-todos
  :after (magit)
  :config
  (magit-todos-mode 1))

Committing automatically

I have lost a number of changes in the past because I reverted a file, made a mistake or whatever. Some of these mistakes can be reverted easily if saves are automatically committed

Rather than using an after save hook, there is a minor git-auto-commit mode package which does just what I need.

There is not much to configure for this minor mode. There are a couple of ways to enable it:

1. file-local variable (put it in the file to be autocommitted)
2. directory-local variable (make a .dir-locals.el file); this enables it for all files in the directory
3. as a hook

I’m using the first method on relevant files. The disadvantage of this method is that you have to think about it for each file, so perhaps a .dir-locals.el is a better solution.

I am considering using a generic hook again to enable the method and either using git commit --amend and commit squashing if working on more structured commits. For files that I really do not want autocommit to run I can use a file local variable to disable the hook (or the minor mode)

(use-package git-auto-commit-mode
  :demand t
  :hook ((org-journal-mode . git-auto-commit-mode))

  :init
  (defun mrb/git-auto-commit-mode ()
    (interactive)
    ;; Make sure, possibly redundant, to NOT enable git autocommit in my config file
    (unless (string= (buffer-file-name) config-file)
      (git-auto-commit-mode))
    ))

Miscellaneous

A collection of loosely development related things.

Online pastebin services

I use github gists sometimes, but rather have more open options as well. These are mostly use to communicate longer pieces of text in chat programs, notably IRC

(use-package webpaste
  :config
  (add-to-list 'webpaste-providers-alist
               '("dpaste.org-custom"
                 :uri "https://dpaste.org/api/"
                 :post-data (("expires" . 86400))
                 :post-field "content"
                 :post-lang-field-name "lexer"
                 :lang-overrides ((emacs-lisp-mode . "clojure")
                                  (scheme-mode . "clojure"))
                 :success-lambda webpaste--providers-success-returned-string))
  :custom
  (webpaste-provider-priority '("dpaste.org-custom"))
  (webpaste-paste-confirmation t)
  (webpaste-open-in-browser t))

Grepping files

Searching for terms in files is a hard problem, but the vast majority of my use-cases can just be solved by searching for terms in the current project root, almost always defined by having a .git directory in a parent directory. So, let’s start there and see what else I’ll need.

I’ve chosen ag as the engine for grepping the files because it’s very fast. I would like to have a better UI within Emacs though (consult maybe?)

(use-package ag)

Ansible

Not really a development setting, but closely related to it, as eventually the programs written need to be deployed. Most of the time, while deploying, ansible plays a role in this and it’s now worth having a dedicated configuration for editing the YAML files for it.

For now, just the ansible package, and some access to its documentation will suffice

(use-package ansible
  :after yaml-mode
  :init
  (use-package ansible-doc
    :hook (yaml-mode . ansible-doc-mode))
  :hook (yaml-mode . (lambda () (ansible 1))))

Finale

When we are all done with this, provide it.

(provide 'mrb)
;;; mrb ends here