Dynamically, recursively, and intelligently reorganize and display any type data in custom outlines
Customize how data displayed and shortcuts to interact with it
Create three dimensional outlines by displaying the same headings in multiple locations
Automatically update outlines when the underlying data is changes
Mix different kinds of data (e.g., org-mode, email, files, etc.) in the same outline
Create modules to expand capabilities to display and interact with any kind of data
[2024-02-26 Mon] Emacs 29 has brought on some problems…
[2024-02-25 Sun]
Reorg allows you to highly customize the sorting, grouping, displaying, and interacting with any type of data.
For now, Reorg is most useful if you use org-mode. It serves as a potential complete–and far more powerful–relacement for org-agenda (no disrepect to org-agenda).
And as the example modules included here show, it can also be used for file heirarchies (see, e.g., reorg-files.el, .leo files (see, e.g., reorg-leo.el, json data (see, e.g., reorg-json.el), source code (see, e.g., reorg-elisp.el), and limitless other types (so long as someone writes a module).
I am still developing this package. With the exception of reorg-org.el, the modules I’ve supplied are minimal working examples.
The reorg-org.el file is messy. The parsers (i.e., those that call reorg-create-data-type) change often, and contain many overlaps. I am convinced there is no good way to handle every type of timestamp the user might want to access. I am convinced there is no universal way to parse all data from an orgmode heading that serves all purposes.
If anyone hacks on one of the modules, beware that if you add or change a data type, you’ll need to reevaluate the entire buffer, not just that single data type definition. (This is a workaround for a bug in the class and data creation macros.)
Anyone who uses Reorg will have to know some elisp. If, when writing a template, you get an invalid function error with a reference to a .variable always try (quote .variable) or '.variable. There are layers of macros and backquoted lambdas in this code and it sometimes gets confusing. The benefit is that the level of abstraction and customization allows you to focus on how exactly how you want data (including orgmode, files, emails, or anything for which a module has been written) to be sorted and displayed.
Module writing is harder when it comes to interacting with the underlying data. See reorg-org.el as an example (and keep in mind that programatically parsing and editing orgmode files is, at best, tricky), which uses the reorg-org--with-source-and-sync macro. There are not similar macros written for other modules yet.
The key is that once you have a piece of parsed data, reorg will delete the old entries and insert new ones without refreshing the entire outline. See the (poorly named) function reorg--insert-new-heading.
Though I would not advise it, because it is so easy to write a parser it would be simple to use Reorg to create your own custom markup language so that you could have your own plain text note taking system that can be parsed into usable outlines.
Reorg was originally designed as a way to view and interact with org-mode files. Think of it as a combination of org-super-agenda, org-sidebar, org-ql, and taxy.el (although it does not rely on any of these packages).
First, (require 'reorg-launch). Then open reorg-test.el and see what you can get to work. You can look at reorg-jeff.el to see some of the ways I am using it for my agenda.
For example, take the following orgmode file which contains a few notes, TODO items, and dates.
* [2023-01-11 Wed 13:16] @Brian told @Alex that @Olivia ate three guitars for breakfast. :aaa:
* TODO [#A] Research the effect of nylon strings on gut health. :aaa:
* [2023-01-11 Wed 14:00] @Olivia told @Brian that she dreams of being a potato. :bbb:
* [2023-01-11 Wed 15:00] @Alex learns that @Olivia wants to be a potato and he begins pacing erratically. :bbb:
* [2023-01-08 Sun 13:19] @Olivia decides that she wants to mess with @Brian and @Alex because she is bored :aaa:
* TODO [#B] Think of a better example file :bbb:
* DONE [#A] Write a terrible example file
Suppose you want to display this information in the following way:
- Group all entries with a timestamp under a single heading and sort them by date
- Group all entries that mention an @name under a heading for each @name (regardless of whether those entries were already inserted into the previous heading)
- Within each of those headings, group the entries by tag with the tags sorted alphabetically
- But don’t display the timestamps
- Group all TODO entries together, sorting by tag, but display the headlines in all caps
- Show all the entries in the file sorted by org-id, but only display the org-id
This produces:
One of the key features is that entries can appear in multiple places in the outline.
Clones are a key feature of Reorg. Clones are entries that appear in multiple places in an outline. See, e.g., Leo Editor’s use of clones. See also a prototype package I wrote some time ago called org-clones. In my mind, an outline with clones is a three-dimensional outline which means that it can be viewed from different perspectives. More on that later.
In Reorg, a heading can appear in different locations and can have different appearances. Take the original example. Each of the headings in a red box are identical:
If you select any of these headings and render the underlying data, you will be taken to the exact same location in the org-mode file. If you edit these headings from the org-mode buffer, they will all be deleted, re-parsed from the new source, and re-inserted into the outline at the appropriate location(s).
Note that some of the headlines in the above example contained multiple @name references. If Reorg encounters data that is a list, it can be told to clone those entries once for each piece of data in the list. That means that a heading that includes @Olivia and @Brian will appear under both of their headings. (This is explained further in the section on writing templates.) See below discussing the .@ operator (the fact that I used @names as an example and the operator being a .@ prefix is purely a coincidence).
Reorg isn’t just for org-mode, so let’s add some more to the original example. In addition to the previous we want to:
- Display all of the variable declarations in reorg.el, but don’t dislpay the reorg- prefix; sort them in reverse alphabetical order. Group them by the type of variable.
- Show all of the .el files in the reorg directory, but group them by the number of characters in the filename and sort the groups in descending order. If a filename contains the letter “a”, put a happy face next to it.
Okay. You asked for it.
Note that these examples keep the types of data separate. It is just as easily possible to mix data so that files, org-mode entries, etc., all exist within the same headings.
These outlines are defined by Reorg’s template system, which is explained below. For now, let’s talk about the outlines themselves.
The keybindings, usable by any Reorg outline no matter the data type, are:
| n or down | next heading |
| p or up | previous heading |
| u or left | goto parent |
| U | next parent |
| b | previous sibling |
| f | next sibling |
| tab | fold/unfold subtree |
| shift+tab | fold/unfold all |
| R | Reload |
| g | Refresh heading at point |
| c | jump to next clone |
| C | jump to previous clone |
| RET | Display source buffer |
Reorg itself does not provide a way to interact with the data being displayed in the outline because it has no idea what that data is. All interactions are handled by the various modules. For example, if the header at point is based on an org-mode file, the reorg-org module uses the following keybindings:
| SPC | reorg-org–open-agenda-day |
| h | edit headline |
| t | edit todo state |
| a | edit tags |
| d | edit deadline |
| s | edit scheduled timestamp |
| r | set properties |
| i | set priority |
Note that the keybindings in reorg-org.el are constantly changing. These are only examples.
If you run any of these commands and change the underlying org-mode file, Reorg will edit the source buffer, delete the heading (and its clones) in the reorg tree, re-parse the heading, and re-insert the data into the tree. It does this without reloading the entire tree, so the change is immediate.
Note that refreshing the a single element in the outline is tricky. If you encounter a bug, please report it. You can always press R to reload the entire outline if needed.
A reorg buffer can be viewed in a side window so that it is like a sidebar. Otherwise, it can be viewed in a normal window as you would any buffer.
If you view reorg as a sidebar (the default shortcut to toggle the sidebar is C-; r), it will automatically render the source for the header at point. For example, here it is interacting with the preceding outline:
Reorg will automatically detect if it’s in a side window or not, and adjust its display behavior accordingly. If in a side window, it will try to render the source of the heading at point. If in a main window, it will only try to render the source if you press RET.
(Please forgive my haphazard GIF.)
Templates are used to create custom outlines and have a simple syntax. Be forewarned that the user must be familiar with basic lisp (i.e., enough to navigate a config file) to competently write a template.
Templates use variables that are stored when Reorg parses the underlying data. These variables can be accessed using dotted symbols identical those used by the built-in let-alist macro. Templates can use any data that has been parsed by Reorg.
Extra data types can be added using the reorg-create-data-type macro which is explained below.
As the reorg-org module stands now (i.e., hacky, in testing, etc.), if the first heading in the original example is parsed, it returns an alist:
Note: Now, when reorg generates an outline, it scans the template and only parses the data that is needed for that specific outline. This allows uses to define additional data types without worrying about slowing down outlines that don’t use that data type.
Also note: these are for illustration and the actual types defined in reorg-org.el are constantly changing.
((ts-any . "[2023-01-11 Wed 13:16]")
(ts-ts . #s(ts 13 16 [....] 1673460960.0))
(timestamp-type)
(ts-day-name . "Wednesday")
(ts-day . 11)
(ts-month-num . 1)
(ts-month . "January")
(ts-year . "2023")
(at-names "Brian" "Alex" "Olivia")
(root-ts-inactive)
(root . "[2023-01-11 Wed 13:16] @Brian told @Alex that @Olivia ate three guitars for breakfast.")
(org-level . 1)
(order . 1)
(buffer . #<buffer test.org>)
(buffer-name . "test.org")
(filename . "~/.emacs.d/lisp/reorg/TEST/test.org")
(category . "test")
(category-inherited)
(id . "7038a596-f2c3-414d-a68a-fadbc9ef61ad")
(timestamp-range)
(timestamp-ia-range)
(timestamp-ia . "[2023-01-11 Wed 13:16]")
(link-file-path)
(link-file-name)
(link)
(links)
(timestamp)
(todo)
(tags . ":aaa:")
(headline . "[2023-01-11 Wed 13:16] @Brian told @Alex that @Olivia ate three guitars for breakfast")
(scheduled)
(deadline)
(body)
(priority . "B")
(timestamp-all)
(ts)
(ts-pretty)
(tag-list "aaa")
(delegatee)
(class . org))At all points within an outline template, the user can access any of these variables with dotted notation. In other words, .priority is the same as (alist-get 'priority DATA). (See below for an explanation about DATA).
If you use company, you can M-x reorg-enable-completion and you should get auto complete for the dotted prefix
Here is a basic template that will display each heading that has a “TODO” todo state for each file in your org-agenda-files.
(reorg-open-sidebar `( :sources ((org . ,(org-agenda-files)))
:group "Example template"
:children (( :group (when (equal .todo "TODO" ) "TODO")
:format-results (.stars " " .todo " " .headline)))))Here is another example that will create a date tree from your agenda files. Note: this requires ts.el.
(reorg-open-sidebar `( :sources ((org . ,(org-agenda-files)))
:children (( :group
.ts-year
:sort-groups
string<
:children
(( :group
.ts-month
:sort-groups
(lambda (a b)
(let ((seq '("January"
"February"
"March"
"April"
"May"
"June"
"July"
"August"
"September"
"October"
"November"
"December")))
(< (seq-position seq a 'string=)
(seq-position seq b 'string=))))
:sort-results
((.ts-day . <))
:format-results
(.stars " " .headline " " .tag-string)))))))Here’s an explanation:
:sources is an alist where the key is the name of a class and the value is the actual source. The value can also be a list of sources, e.g., in the above code (org-agenda-files) can return multiple org-mode files. You can use multiple sources. For example:
'( :sources ((org . "~/path/to/org/file.org")
(org . "~/path/to/a/different/org.org")
(email . "email search terms")
(files . "/path/to/directory")))(For a source to be available, a module has to be created using reorg-create-class-type and reorg-create-data-type as described below.)
Group is an elisp form that determines what data to include in the outline, and what data should be available to any subtrees within the outline. The rule is easy: if :group returns nil, then the data is excluded from the outline. If :group returns non-nil, then the data is grouped by return value.
In the previous example, there is only one non-nil return value for group: “TODO”
:group (when (equal .todo "TODO" ) "TODO")But what if you wanted to create groups for each todo state?
:group .todo ;; Remember: .todo will either be a string (the todo state) or it will be nil (if there is no todo keyword)What if you wanted to create a group called “GROUP A” if there is a timestamp, and “GROUP B” if there is a “DONE” todo state, but you want the timestamp to take precedence?
:group (cond (.timestamp "GROUP A")
((when (and .todo
(equal .todo "DONE")))
"GROUP B"))The preceding example has a problem: what if a heading has a timestamp, and a DONE todo state? Maybe you want it to appear under both headings. In that case, you have to create sibling groups using the :children keyword:
:children (( :group (when .timestamp "GROUP A"))
( :group (when (and .todo
(equal .todo "DONE"))
"GROUP B")))The :group keyword allows a second kind of dotted symbol: the .@symbol. If a reorg group contains a symbol prefixed with .@ it is a signal to reorg that: (1) you anticipate the value of that data will be a list; and (2) you want to create clones of the data which are identical except for that one piece of data.
I am not explaining this well, but it is shown in the main example screenshots. This means that if you have, for example, an orgmode entry with multiple dates in it, and you create a date tree, it will appear at each entry (note the use of .@ts-all-flat)
(defun jrf/reorg-diary ()
(interactive)
(reorg-org-capture-enable)
(reorg-open-sidebar
`( :sources ((org . ,(org-agenda-files))
:format-results (
(propertize
(reorg--truncate-and-pad
(car (s-split "," .root))
12 13)
'face '((t ( :foreground "black"
:background "light gray"
:box t))))
" "
(propertize
(reorg--truncate-and-pad .headline 50 55 "...")
'face
'((t ( :foreground "black"
:background "light gray"
:box t))))
.clocked-time)
:group (when .ts-all-flat
(substring .@ts-all-flat 0 4))
:sort-groups string>
:children (( :group (reorg-org--format-time-string .ts-all-flat "%B")
:bullets "-"
:folded-bullets ">"
:sort-groups (lambda (a b)
(reorg--sort-by-list a b '("January"
"February"
"March"
"April"
"May"
"June"
"July"
"August"
"September"
"October"
"November"
"December")))
:children (( :group (reorg-org--format-time-string .ts-all-flat "%e %A")
:sort-groups string<)))))))I generally call this a “drill.” If you prefix a symbol in a template (which should be an ordered list) with .!, then reorg will assume the list of names of branches in a tree, and create an outline placing each element at the appropriate spot. This means that reorg can easily display, for example, a file system heirarchy.
For example, here is a minimal file class definition with a couple shortcuts. Note that this definition requires the user to supply a shell command to generate a list of files:
(reorg-create-class-type
:name files
:getter (cl-loop for each in (s-split "\n" (shell-command-to-string
;; (concat
;; "find "
SOURCE
;; " -type f"
)
t)
collect (PARSER each))
:keymap (("e" . (lambda ()
(interactive)
(let ((file (reorg--get-prop 'fullname)))
(reorg--select-main-window)
(find-file file))))
("o" . (lambda () (interactive)
(xdg-open (reorg--get-prop 'path))))))
(reorg-create-data-type
:name filename
:class files
:parse (f-filename data))
(reorg-create-data-type
:name parent-dirs
:class files
:parse (butlast (s-split "/" data t)))
(provide 'reorg-files)Note that .parent-dirs will contain an ordered list showing each file’s parent directories. Now, we will create a template that groups the files based on their parent directories. Note that we’ve decided for the user the input will be a directory and the command will be find DIR -type f.
(require 'reorg-files)
(defun reorg-files (&optional dir)
(interactive "D")
(reorg-open-sidebar
`( :sources ((files . ,(concat "find "
dir
" -type f")))
:format-results (.filename)
:group .!parent-dirs
:sort-results ((.filename . string<))
:sort-groups string<)))Obviously you could do a great deal of customizing how files are displayed without much trouble.
You can mix groups generated by the .! operator with other traditional groups. For example, you can group something first and then drill the results. Here, we create two groups for files with an even number of characters and those with an odd number, then display the full file heirarchy:
(defun reorg-group-then-drill (&optional dir)
(interactive "D")
(reorg-open-sidebar
`( :sources ((files . ,(concat "find "
dir
" -type f")))
:format-results (.filename)
:sort-results ((.filename . string<))
:sort-groups string<
:group (if (= (mod (length .filename) 2) 0)
"EVEN"
"ODD")
:children (( :group .!parent-dirs)))))Or, you can put everything into a heirarchy and then group the items further:
(defun reorg-drill-then-group (&optional dir)
(interactive "D")
(reorg-open-sidebar
`( :sources ((files . ,(concat "find "
dir
" -type f")))
:format-results (.filename)
:group .!parent-dirs
:sort-results ((.filename . string<))
:sort-groups string<
:children (( :group (if (= (mod (length .filename) 2) 0)
"EVEN"
"ODD"))))))You can mix different groups of things, for example, you could mix your orgmode entries and your files by sorting them according to the same rules and have the results exist in the same outline.
Let’s return to our template and make it group all of the todo entries in your agenda files and sort them alphabetically:
(reorg-open-sidebar `( :sources ((org . ,(org-agenda-files)))
:group "Example template"
:children (( :group .todo
:format-results (.stars " " .todo " " .headline)))))I’ve decided that I do not like having the root heading there. Luckily you do not need a root heading and you can skip the first :group declaration:
(reorg-open-sidebar `( :sources ((org . ,(org-agenda-files)))
:children (( :group .todo
:format-results (.stars " " .todo " " .headline)))))Now, all of the TODO keywords will be root headings instead of part of a subtree.
Let’s sort the todo keywords. :sort-groups accepts a function that takes two argument and returns t if the first should come before the second. The arguments to the function are the heading strings returned by the :group parameter.
(reorg-open-sidebar `( :sources ((org . ,(org-agenda-files)))
:children (( :group .todo
:sort-groups string<
:format-results (.stars " " .todo " " .headline)))))Unlike some template components, :sort-groups is not inherited. It will only apply to the group in which it is declared.
It may be ideal to pass the function accepted by :sort-groups some metadata about the group instead of only the heading string, but because the outline is still being generated at the time :sort-groups is called, it’s not clear whether it would be useful.
Format results tells reorg how to display the data. It is a list that contains either strings or dotted symbols. After the values of the dotted symbols are substituted into the list, the string is concatted together with concat. (Note: it is okay if the dotted symbols evaluate to nil.)
:format-results can transform the results in any way, e.g., adding text properties, overlays, performing calculations, transforming values.
:format-results ((replace-regexp-in-string (rx "reorg-"
(zero-or-one "-"))
""
.form-name)
(propertize " " 'display
`(space . (:align-to 70)))
(f-filename .file)))))Again, all that matters is that each form within it returns a string or nil.
:format-results arguments are inherited. For example, in this template, botih groups would be rendered with the same result formatter.
:format-results (.stars " " .todo " " .priority " " .headline)
:children (( :group (when .timestamp "GROUP A"))
( :group (when (and .todo
(equal .todo "DONE"))
"GROUP B")))But in this tempalte, only the second would use the formatter (the first would have to use one declared higher in the template, or the fallback format reorg-headline-format.
:children (( :group (when .timestamp "GROUP A"))
( :group (when (and .todo
(equal .todo "DONE"))
"GROUP B")
:format-results (.stars " " .todo " " .priority " " .headline)))Finally the .stars symbol you have seen refers to org-mode style stars showing the depth in the outline. It can be included or omitted. (Group headings always have leading stars, because Reorg relies on functions from outline-mode to handle folding. This reliance is out of laziness; in reality there is no need for Reorg’s folding or display to be confined to an outline and that code should be written as it would help free Reorg from org-mode styled outline trees.)
Currently, groups can only sorted by a single function. But results can be multi-sorted.
:sort-results accepts an alist in the form ‘((FORM . PREDICATE)) where FORM is code that determines what arguments are passed to the predicate function. For example:
:sort-results (( .todo . string<)
((downcase .headline) . string>))Result sorters are inherited through the subtree. If additional result sorters appear within a subtree, they are added to the previously declared sorters such that the previous declarations have a higher precedence.
This are strings that are used to replace the leading stars for each heading. For example, youc could use “->” and “–” to show if a heading is folded or not.
hidden options
Try to use reorg-org-capture-enable. It should update the single entry and then then find all of the places the edited entry should appear. No shit. It took me a long time to figure this out. It works for me. This means after you capture a note it will automatically propogate to the outline without reloading the entire outline.
It’s probably best to look at the examples provided if you’re interested in this.
A class is created using the reorg-create-class macro. Here are two examples of class definitions. First, the class definition for org files:
(reorg-create-class-type
:name org
:getter (org-ql-select SOURCE nil :action #'PARSER))Second, the class definition for files from the file system:
(reorg-create-class-type
:name files
:getter (cl-loop for each in (s-split "\n" (shell-command-to-string
SOURCE)
t)
collect (PARSER each))A class definition has two required components: the name of the new class and a “getter.” It also has two optional components: a render function (keyword :render-func) and a keymap (keyword :keymap).
The :name argument is the name of the class that will be used then declaring a source in a template and when creating parsers for the data.
The job of the “getter” is to fetch the data call tell Reorg where when and how the parser should be used on that data. There are two pre-defined variables that you must use when writing the getter: SOURCE and PARSER.
The SOURCE variable refers to the input from the template that is supplied by the user. For example, in the above org-mode class, SOURCE is simply the path of an org-mode file.
This means that to use org-mode data in an outline, the user would specify the source like this:
'(:sources ((org . "~/.emacs.d/lisp/reorg/TEST/test.org")))In the files example, SOURCE is a bash command that outputs a list of file paths (e.g., it could be find ~/ -type f). In a template using the files class, the user would specify the source like this:
'(:sources ((files . "find ~/.emacs.d -type f")))But suppose that you did not want the user to have to type in a shell command to retrieve a list of files; instead, you only want the user to supply a directory. In other words, you want the template to look like this:
'(:sources ((files . "~/.emacs.d")))Then the class getter would be defined this way:
(reorg-create-class-type
:name files
;; . . .
:getter (cl-loop for each in (s-split "\n" (shell-command-to-string
(concat "find "
SOURCE
" -type f"))
t)
collect (PARSER each))As you can see, SOURCE simply refers to the anticipated input from the template.
See below.
This parses and displays json data. It renders the underlying json file by narrowing it to the relevant region.
(reorg-create-class-type
:name json
:getter (with-current-buffer (find-file-noselect SOURCE)
;; This is the quickest and dirtiest way
;; I found to parse a json file which I have
;; no reason to ever do. It will properly
;; parse the test file at least.
(let ((json-array-type 'list)
(json-key-type 'symbol)
(json-object-type 'alist)
(json-null nil)
(json-false nil)
(file (buffer-file-name)))
(save-excursion
(goto-char (point-min))
(cl-loop for each in (json-read)
collect (append (list (cons 'file file)) each))))))Then, to use that class:
(reorg-open-sidebar '( :sources ((json . "~/.emacs.d/lisp/reorg/TEST/y77d-th95.json"))
:group (if .year (substring .year 0 4) "Unknown year")
:sort-groups string>
:format-results (.mass "\t" .name " " .geolocation.type)
:sort-results (((if .mass (string-to-number .mass) "") . <))
:children (( :group (if .mass
(if (> (string-to-number .mass) 1000)
"Mass > 1000"
"Mass <= 1000")
"Mass Unknown")
:sort-groups string<))))Within the class creation macro, PARSER refers to a function that parsers the data. For the purposes of writing a class definition, you do not worry about how you are parsing the data; only worry about when you are parsing it. (We’ll deal with writing the parsers later using a separate macro called reorg-create-data-type, see infra.)
There two primary ways to fetch get data and call the PARSER.
First, for the org-mode class above, all parsing is done by physically moving through the buffer and calling the parsing function at each heading.
(reorg-create-class-type
:name org
;; . . .
:getter (with-current-buffer (find-file SOURCE)
(org-map-entries #'PARSER)))You could, alternatively, use org-ql which does the same thing:
(reorg-create-class-type
:name org
;; . . .
:getter (org-ql-select SOURCE nil :action #'PARSER))In these examples, PARSER is called with no arguments because it is does not need any data; it gathers the data by examining a buffer.
But sometimes the data is not contained in a buffer. The second way to parse data is to call PARSER, with the data passed to it as an argument. For example, in the files example above which runs a shell command and receives a list of file paths, PARSER is called on each member of that list individually. In that case, PARSER is called with the data to be parsed as an argument:
(reorg-create-class-type
:name files
;; . . .
:getter (cl-loop for each in (s-split "\n" (shell-command-to-string
(concat "find "
SOURCE
" -type f"))
t)
collect (PARSER each))There are two other parts to writing a class definition: (1) writing a function that, when Reorg is used as a sidebar, displays the data at point in the main buffer; and (2) defining keyboard shortcuts to interact with the data. Because these are both optional, they are addressed below. The next section explains how to define what the PARSER actually does.
In the above examples, we know that PARSER is being called either at an org-mode heading or with the path of a file name. The question is what information we want to store from those sources and, optionally, how we want to display that data. This is accomplished with the reorg-create-data-type marco.
Suppose that we’ve created the same minimal org-mode class set out above:
(reorg-create-class-type
:name org
:getter (org-ql-select SOURCE nil :action #'PARSER))Suppose we want to store the TODO state of an org entry for use in our outline.
(reorg-create-data-type
:class org
:name todo
:parse (org-entry-get (point) "TODO"))This tells Reorg that we are adding data named “todo” to the org class. The macro will automatically create a function definition, add that function to an internal parser list, and call that function each time the PARSER function is invoked by the org class we defined above.
Reorg will store all parsed data in an alist, so the above data type would generate:
'((todo . "TODO")) ;;or "DONE" or whateverYou are not limited to pre-defined parts of the org-mode heading like TODO statets and priorities. For example, here is one that searching the heading for any words prefixed with an @ symbol and that stores a list of those words:
(reorg-create-data-type
:class org
:name at-names
:parse (let ((headline (org-get-heading t t t t)))
(cl-loop with start = 0
while (setq start (and (string-match "@\\([[:word:]]+\\)" headline start)
(match-end 1)))
collect (match-string-no-properties 1 headline))))Note that this will return a list with, potentially, multiple entries. That is not a problem because Reorg can clone these so each @name can appear in multiple places in the outline, as in the first example in this readme. See also deadling with multivalued data types, supra.
There is no limit to the data you can extract and save for later display.
The approach is slightly different when you are dealing with data that is passed to the PARSER as an argument. For example, in the files example, supra, the PARSER was called with each path as an argument.
(reorg-create-class-type
:name files
:getter (cl-loop for each in (s-split "\n" (shell-command-to-string
SOURCE)
t)
collect (PARSER each))To access the information passed to the parser when using the reorg-create-data-type macro, we use the variable data.
For example, suppose you were creating an outline and you wanted to sort a list of files by extension. Then we need to parse the file extension:
(reorg-create-data-type
:class files
:name extension
:parse (f-ext data))The variable data will always refer to the original data passed to PARSER.
For either approach, you can also reference all the data that has has been generated by the other previous parsers within the same class. For example, suppose you parse and store a ts object that represents the deadline of a heading:
(reorg-create-data-type
:class org
:name ts-deadline
:parse (when-let ((deadline (org-entry-get (point) "DEADLINE")))
(ts-parse-org deadline)))And suppose you also want to parse and store the name of the day associated with the deadline, so that your outline can include headings that include the name of the day of the week. Instead of parsing another ts object, you can simply refer to the previous data using the dot notation that is used in the template system:
(reorg-create-data-type
:name ts-day-name
:class org
:parse (when .ts-deadline
(ts-day .ts-deadline))
:append t)Alternatively, you can use the variable DATA which is the alist of all previously parsed data. Since Reorg stores all parsed data as an alist, so DATA is just an alist:
(reorg-create-data-type
:name ts-day-name
:class org
:parse (when-let ((ts (alist-get 'ts-deadline DATA)))
(ts-day ts))
:append t)Note the use of :append in both examples. If you want to refer to previously parsed data, you must ensure that parser is run after the parser generating the data you want to use. If the first parser that is run attempts to reference DATA, it will be nil because there will be nothing there to reference.
When a user writes a template and tells Reorg how to format and display the results, the user uses dot notation. But sometimes that is not satisfactory. For example, (org-entry-get (point) "PRIORITY") will return “A”, “B”, or “C”. But no one wants to see A, B, or C in their outline because it would be ugly. To change how data is display, use the :display keyword. Like the parser, the display keyword can use dot-notation to refer to any previously parsed data (including the data generated by the current parser):
(reorg-create-data-type
:class org
:name priority
:parse (org-entry-get (point) "PRIORITY")
:display (pcase .priority
("A" "⚡")
("B" "➙")
("C" "﹍")
(_ " ")))Now, if the user creates a template that displays the priority of an org heading, the data will be transformed and shown as ⚡, ➙, or ﹍ instead of A, B, or C. But the underlying data remains unchanged.
Note: the :display parameter is not the only way to customize how data is displayed because templates can also alter how data is displayed. The :display parameter is most useful if the data being parsed is not a string. For example, if you create a parser that stores an integer value, you should dislay it as a string.
For example, here is a parser for the files class that stores the depth of the file:
(reorg-create-data-type
:name depth
:class files
:parse (f-depth data))If the user attempts to display the depth data in their outline by using .depth in the template, they’ll have to ensure that it’s transformed into a string. For example suppose the user tries to use a tempalte that formats the results like this:
:format-results ("Depth: " .depth " " .filename)Reorg will err. It cannot concat .depth because it is an integer. Instead, the user’s template will have to convert it to a string:
:format-results (.stars " Depth: " (number-to-string .depth) " " .filename)This is not best practice. The user should be able to assume that any stored data can be safely displayed.
But also assume (for whatever reason) you want to keep the data stored as an integer (or any other object). You do not want it stored as a string. You can avoid this mess with a definition that uses the :display keyword:
(reorg-create-data-type
:name depth
:class files
:parse (f-depth data)
:display (number-to-string .depth))When a Reorg buffer is displayed in a side window, it can automatically render the data at point as the user moves through the outline. For Reorg to know how to display the data, it needs to be told how to do so. If no render function is provided in a class definition, then Reorg will not attempt to render the data. If a render function is provided, and the Reorg buffer is in a side window, then it will render the underlying data each time the user selects a heading in the outline.
When you define a keyboard shortcut in a class, it will apply any entry in the outline that belongs to that class. See reorg-org.el, reorg-files.el, and other modules to see how this works.
For example hoisting headings / inverting the outline / multiple views
Allow the user to use nested dots referencing the class name, e.g., .org.filename and .files.filename to allow the user (as it’s often desired behavior) to avoid collusions when grouping
[fn:1] For all I know this could be a better back end; it was published as I was completing the core grouping and sorting functions and I was already on my own path.