Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requery #1558

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Requery #1558

wants to merge 6 commits into from

Conversation

mjg
Copy link
Contributor

@mjg mjg commented Dec 30, 2020

This mini-series addresses two issues related to the way threads are requeried: the initial query is by a search term, and further queries are by thread id.

The first patch addresses what I consider a bug: If, say, you respond to an e-mail then the search view for the inbox will show the date of your reply but sort the thread by the date of the latest message matching the inbox query. This can look strange.

The second patch uses the solution from the first patch (remembering the query) to achieve a look closer to notmuch's search result list: display numbers of matched messages within a thread in a search view.

The latter one may need some way to configure. [In fact I think I submitted or discusssed this before but couldn't find it, sorry.]

Copy link
Owner

@pazz pazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't see the problem that you're solving, sorry.
In any case, adding querystrings to Thread objects looks excessive at first glance.

@mjg
Copy link
Contributor Author

mjg commented Jan 1, 2021

I really don't see the problem that you're solving, sorry.

That means I failed to describe it ;)

In a search buffer, all displayed threads originally come from matching a query. In that sense only some of the messages in a thread are actual matches since not all messages necessarily match the query - a thread is "matched" as soon as one the messages in a thread matches the query.

notmuch search date:today gives me a line thread:000000000001cf1a 11 mins. ago [6/12] Patrick Totzke; [pazz/alot] ... for example. The thread with 12 messages "mathched" because at least 1 message (in fact 6) match the query.

Ultimately, that is what I want to have in the search buffer (patch 2). But even if you don't care about that display:

If you ask for the newest datetime in a thread then it makes a difference whether you query for the thread ID alone or for that AND the original query. Before patch 1, alot sorts the seach buffer by one of those dates and displays the other. This can look quite confusing. After patch 1 always the same date is used, namely the one from the newest matching message.

@pazz
Copy link
Owner

pazz commented Jan 1, 2021 via email

@mjg
Copy link
Contributor Author

mjg commented Jan 1, 2021

I think I see now. For instance if I have messages A<B<C in two threads {A,C} and {B} and search mode sorts them by newest first then I have {A,C} newer than {B} but its date stamp derives from the oldest msg C. Can this not be solved more easily by making sure that search mode displays the "right" date for a thread? Notice that Thread has get_newest_date and get_oldest_date. So you could make sure that if search mode sorts threads to have the newer ones above then it should display the newest date for each thread instead of the oldest one..

No. It's not a matter of "new" vs. "old". It's a matter of "all messages in thread" vs. "matched messages in thread". They are different sets of messages, and so you might get different dates (whether newest or oldest).

Now, one could forget about the search completely, of course (i.e. fix the sorting and leave the display). But if I have a search buffer say for tag:inbox then I care about the newest arrivals in my inbox much more than about my newest replies to (possible ancient) messages in my inbox. So I would say the existing sorting is correct (and the display is not).

After all, it comes down to how you view the search buffer/widget and notmuch search. Since all notmuch queries really are about messages I consider the output of notmuch search as well as the display of the search buffer to be a "thread condensed view of messages", but still a list of matched messages more than it is a list of threads. Similarly, if you run notmuch count or notmuch tag on a query then both will apply to matched messages only, not all messages from threads with matched messages. That's why there is the relatively new thread:{<notmuch query>} search term.

@mjg
Copy link
Contributor Author

mjg commented Jan 4, 2021

I'll check what it would take to fix the sorting issue on the other end (leave the display as is but sort accordingly); achieving the count (matched/total) could be independent of that.

@mjg
Copy link
Contributor Author

mjg commented Jan 4, 2021

OK, so the man page for notmuch search says this:

Search for messages matching the given search terms, and display as results the threads containing the matched messages.
The output consists of one line per thread, giving a thread ID, the date of the newest (or oldest, depending on the sort option) matched message in the thread,
the number of matched messages and total messages in the thread, the names of all participants in the thread, and the subject of the newest (or oldest) message.

And this is exactly what db.threads() does and - thus - the sort order which the search buffer in alot shows. But as soon as threads are queried by a search for thread id all dates are computed based on that query (i.e. all messages in the thread).

So, I guess the first question is whether the search buffer in alot should correspond to the output of notmuch search closely. I would say yes, and this PR achieves that (in terms of the displayed date and the counts; sort order unchanged).

Now, it might be useful to have a search view corresponding to all messages in threads with matched messages, especially in terms of sort order: if you tag only the first message in a thread (as is usual) then notmuch and alot (with or without this PR) will sort by that first message's date even if there are much newer messages in the thread. This might not be what you want and makes it difficult to look for something visually.

The solution is thread:{querystring}, of course (again, independent of this PR). Though you can use this already (if you know where to put quotes) it might be worthwhile to have a command "break out to all messages" or "list all messages in matched threads" which simply takes the current query of a search buffer and opens a new search buffer for thread:"{querystring}". This would round things up.

@mjg
Copy link
Contributor Author

mjg commented Feb 2, 2021

So, I have been using this feature a lot ( ;) ), and before merging (however likely or unlikely that is), I would like to amend this series by one more feature. It touches the same question as the question whether the whole series makes sense:

Is alot search (the search buffer view) supposed to mirror notmuch search?

My answer is obviously yes, and the series is a consequence. Note that this point of view affects also the answer to the question: Which messages should a tag action in a search buffer affect? notmuch tag <+-tags> <query> affects the matched messsages (not threads). Alot's tag action works differently from notmuch before this series, but in the same way after. And this is what I would like to address with one more commit (--full-thread or such for the tag actions), because often you want to act on the "full thread in a search buffer" without opening that thread first and then using the existing --all. I would even expect the standard key bindings to use --full-thread in order to preserve the current behavior. Added bonus: this shows up in the key help as a reminder.

Currently, the thread list for the search widget is created by
re-querying the db by thread ids. This loses the "match" flag on
individual messages in a thread, so that, e.g., get_newest_date()
lookest at the newest message in the thread instead of the newest match,
so that dates may look out of order. (The list is sorted by matched
dates.)

Thus, remember the querystring and re-query by thread id and the
original query. This restores dates.
Now that thread objects remember the original query, enhance the
mailcount to show the number of matched and total messages in a thread
in the search widget analogous to `notmuch search`.

The original width in the default theme was 5: two parentheses plus up
to three digits. We need to increase this 2+3+3+1=9 now (parentheses,
three digits for each of the counts, slash).
Implement a new `fullthreads` command which turns the current query
quickly into a query for all messages in all matched threads.
Since the search buffer is a view to `notmuch search`, the tag commands
act on matched messages in the same ways as `notmuch tag` does.

Implement an option `--fullthread` that makes the tag commands act on
all messages in a thread instead. It's often the more useful variant in
the UI, and this was the behaviour when alot treated the search buffer
as a buffer of threads (as opposed to a buffer of matched messages).
Depending on your point of view, `--all` means something different or
the same for the tag commands in search and thread buffers.

Provide an alias `--fullthread` for the `--all` option of tag commands
in a thread buffer so that one can use the same key binding for acting
on one full thread in either buffer.
Typically, a user wants to act an a full thread in a search buffer. This
used to be the behaviour before the switch to the cffi notmuch bindings
and the requery series.

Use the new option to restore the old behaviour for the default key
bindings.
@pazz
Copy link
Owner

pazz commented Mar 29, 2021

I find this new "fullthreads" search mode command a bit strange: it just seems to replace the current query string, or rather opens up a new search buffer for a slightly modified one.
Perhaps it'd be more consistent to beef up the current "prompt" command so that it can also insert the current query string in buffer mode? That way you could simulate the fullthreads command
by prompt search thread:%s ?

@mjg
Copy link
Contributor Author

mjg commented Mar 29, 2021

I think you can consider most of my PR's on hold, except for "fix-toggletags", which I guess is merged now.

Where is a good place for a more fundamental discussion? By that I mean design principles such as:

  • A search buffer for QUERY displays the results of notmuch search QUERY (which are messages sorted by thread, not threads).
  • The tag command on a THREAD in a search buffer for QUERY operates on the same messages as notmuch tag QUERY AND thread:THREAD.
  • ...

Even before the switch to nm2, I found alot's behaviour surprising in some cases; the switch added some more. I tried to "fix" some of this, but I guess I lack a clear picture of what behaviour to aim for.

I'm not completely happy with the behaviour of --full-thread (and requery) yet, but before working on it further it would be better to clarify the overall desigb goal.

As for the fullthread command: It turns a search buffer for QUERY (i.e. a buffer for all messages matching QUERY, i.e. those matched by notmuch search QUERY) into a search buffer matching all threads matching QUERY (i.e. a buffer for all messages in threads with messages messaging query, i.e. those matched by notmuch search thread:{QUERY}). In general this gives you different message counts and different messages to operate on (if the commands operate like notmuch does).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants