Buffer IDs do not persist

[franburstall]

Describe the bug

Once more I do not know if this is a bug or a feature request.

I would like to persist a list of favoured buffers and had thought that I could simply save a list of buffer id's to disk.

However, I discover that the mapping from id's to buffers does not survive a restart. That is, the buffers in a restored session have different ids after a restart..

Should this be the case? I thought ids used to persist.

If not, is there a way to accomplish my goal?

Information

• OS name+version: debian 11
• Desktop environment / Window manager name+version: exwm
• How you installed Nyxt (Guix pack, package manager, build from source): source
• Information from show-system-information:

Nyxt version: 3-pre-release-1-478-g870c8d28
Renderer version: GI-GTK
Operating system kernel: Linux 5.10.0-18-amd64
Lisp implementation: SBCL 2.1.1 (Dynamic space size: 3221225472)
Features: (:QUICKLISP :NYXT-GI-GTK :NYXT-GTK :WEBKIT2 :WEBKIT2-2.36
 :WEBKIT2-CORS-ALLOWLIST :WEBKIT2-PASTE-PLAINTEXT :WEBKIT2-TRACKING
 :WEBKIT2-MUTE :WEBKIT2-EMOJI :WEBKIT2-MEDIA :WEBKIT2-SANDBOXING :GTK-3-22
 :GTK-3-20 :GTK-3-18 :GTK-3-16 :GTK-3-14 :GTK-3-12 :GTK-3-10 :GTK-3-8 :GTK-3-6
 :GTK-3-4 :GTK :GDK-3-22 :GDK-3-20 :GDK-3-18 :GDK-3-16 :GDK-3-14 :GDK-3-12
 :GDK-3-10 :GDK-3-8 :GDK-3-6 :GDK-3-4 :CAIRO-1-10 :CAIRO-1-12 :GDK-PIXBUF
 :GLIB-2-30 :GLIB-2-32 :GLIB-2-34 :GLIB-2-36 :GLIB-2-38 :GLIB-2-40 :GLIB-2-42
 :GLIB-2-44 :GLIB-2-46 :GLIB-2-48 :GLIB-2-50 :GLIB-2-52 :GLIB-2-54 :GLIB-2-56
 :GLIB-2-58 :GLIB :NYXT-3.1 :NYXT-3 :CUSTOM-HASH-TABLE-NATIVE :SLYNK
 :PLUMP-UTF-32 :PARENSCRIPT :FSET-EXT-STRINGS :GLOBAL-VARS :DECLARE-TYPES
 :NAMED-READTABLES :SWANK :LPARALLEL :21BIT-CHARS :CLOSER-MOP :CL-PPCRE-UNICODE
 :CL-UNICODE :CL-PPCRE :CHUNGA :FLEXI-STREAMS :BORDEAUX-THREADS
 :LPARALLEL.WITH-CLTL2 :LPARALLEL.WITH-CAS :LPARALLEL.WITH-STEALING-SCHEDULER
 :SPLIT-SEQUENCE CFFI-FEATURES:FLAT-NAMESPACE CFFI-FEATURES:X86-64
 CFFI-FEATURES:UNIX :CFFI CFFI-SYS::FLAT-NAMESPACE ALEXANDRIA::SEQUENCE-EMPTYP
 :FAST-IO-SV :FAST-IO :CL-JSON-CLOS :CL-JSON :SBCL-USES-SB-ROTATE-BYTE
 CHIPZ-SYSTEM:GRAY-STREAMS :THREAD-SUPPORT :ASDF3.3 :ASDF3.2 :ASDF3.1 :ASDF3
 :ASDF2 :ASDF :OS-UNIX :NON-BASE-CHARS-EXIST-P :ASDF-UNICODE :X86-64 :GENCGC
 :64-BIT :ANSI-CL :COMMON-LISP :ELF :IEEE-FLOATING-POINT :LINUX :LITTLE-ENDIAN
 :PACKAGE-LOCAL-NICKNAMES :SB-LDB :SB-PACKAGE-LOCKS :SB-THREAD :SB-UNICODE
 :SBCL :UNIX)

ASDF version: 3.3.1
ASDF registries: (NYXT-SOURCE-REGISTRY ENVIRONMENT-SOURCE-REGISTRY USER-SOURCE-REGISTRY
                      USER-SOURCE-REGISTRY-DIRECTORY
                      DEFAULT-USER-SOURCE-REGISTRY SYSTEM-SOURCE-REGISTRY
                      SYSTEM-SOURCE-REGISTRY-DIRECTORY
                      DEFAULT-SYSTEM-SOURCE-REGISTRY)
Critical dependencies: (/home/fran/common-lisp/nyxt/_build/cl-cffi-gtk/gtk/cl-cffi-gtk.asd
 /home/fran/common-lisp/nyxt/_build/cl-gobject-introspection/cl-gobject-introspection.asd
 /home/fran/common-lisp/nyxt/_build/cl-webkit/webkit2/cl-webkit2.asd)

[aartaka]

Does store-history-by-name work for your use-case?

[franburstall]

No. I want to persist a subset of the buffers: really I want to persist an ordered list of up to 9 buffers. See #1695 for the context: I want to save the "tagged" state of some buffers.

I thought I could just save a vector of buffer-ids but those ids point to different buffers after a restart.

[Ambrevar]

Indeed the buffer IDs have only one guarantee: to be unique. They are random on each start, including when restoring a session. This is by design.

Do you want to save a session or a subset of the global history tree? I guess that latter would be more general.

We could tweak store-history-by-name to prune unselected buffers. That's quite easy.

Does the buffer ID really matter though? Why do you want to keep the buffers ordered?
If it's tagging that you want, then we need to introduce buffer names, IDs are not the right concept here.

[franburstall]

Does the buffer ID really matter though? Why do you want to keep the buffers ordered?
If it's tagging that you want, then we need to introduce buffer names, IDs are not the right concept here.

I just want to tag a subset of buffers with some numbers in the range 1 through 9 so I can quickly switch to them via keys C-3 etc and persist these choices. Everything else is an implementation detail: my current implementation is simply a vector of buffers and C-3 switches to the buffer in the third entry of the vector. This does not serialise and I was hoping that buffer ids would be a simple alternative. I now understand that they aren't and wonder if there is any existing alternative for my use-case.

[jmercouris]

You could write a function that checks for a buffer, and if it doesn't exist, opens a new buffer with said URL.

Pseudocode:

look for buffer with URL
if found, switch to buffer
if not found open new buffer with URL

then you could map that to C-1 -> C-9. Unless it matters that it be restored buffers with some state from previous executions.

[franburstall]

Unless it matters that it be restored buffers with some state from previous executions.

Well, state like buffer history would be nice!

I am surprised that the (somewhat opaque to me) history mechanism in nyxt does not supply a simple way to persist a reference to a buffer!

[aartaka]

Well, Nyxt history library is designed in such a way that it both does not need to and does not know of buffers. But I'm pretty sure we can still sort and store it in a buffer-aware fashion. Just need to experiment with the code (no promises, I'm buzy with lots of stuff atm xD)

EDIT: lits -> lots

[Ambrevar]

@franburstall I believe the best way to achieve what you want is to introduce buffer names. This way you can uniquely identify buffers, and from there do what you want.

Introducing buffer names is a simple as adding a new name slot to the buffer class. Pull request welcome :)

[aartaka]

@franburstall I believe the best way to achieve what you want is to introduce buffer names. This way you can uniquely identify buffers, and from there do what you want.

Introducing buffer names is a simple as adding a new name slot to the buffer class. Pull request welcome :)

@Ambrevar, tagged buffers are a different, and as appealing of a use-case! See #1695 (also by @franburstall), for example (it's 3.0-labelled, but the way!)

[Ambrevar]

I know, I know, here I'm just suggesting to introduce a name slot, not a tag slot.

For instance, buffer with ID 345 would have name "web-buffer2" by default, and the name could be settable by the user at run-time. The name is guaranteed to be unique, like in Emacs.

When restoring the session, the name would be restored, but the ID would be different as is currently the case.

Tags are different because they are a list of strings, and each tag can be shared by multiple buffers.

[franburstall]

When restoring the session, the name would be restored, but the ID would be different as is currently the case.

I am having some difficulty getting my mind around persisting any new slot of the buffer class, be it a name or a list of tags. I am assuming that I should use the history mechanism for this. IIUC, the entity in a history-tree data structure corresponding to a buffer is an owner but the place for storing data is an entry inside a node belonging to the owner (perhaps the current-node of the owner?). But that will not work for this use-case because the same node can have many owners!

It is entirely likely I have misunderstood many things here but the following implementation-independent question remains: how to persist (that is save and later restore) data that belong to a buffer rather than one of the web-pages recorded by whatever history mechanism?

[Ambrevar]

Oh right, I forgot that we only store the IDs and we haven't fixed #2064 yet.
So with #2064 in mind, we would store more:

• ID
• The new name slot.
• The modes.

[aartaka]

Wait, why store modes? #2490 should fix #2064.

[Ambrevar]

@aartaka In the end it seems that your last comment was wrong: #2490 does not fix #2064. Correct?

So we still need to store modes.

I thought a bit more about this: still unclear we need buffer names and how we would use it. However tags seem in higher demand (see also #1695), so maybe we can focus on tags for now.

To fix it all in one go, here is what I suggest:

• Add a tags slot to buffer.
• Add a tags and a modes slot to history-entry.
• Fill the slots with the corresponding buffer values in history-add.
• Restore the tags and modes restore-history-buffers.

At this point, #2064 and this issue should be fixed.
#1695 would still not be completely fixed as we need to derive and API for buffer tags. Maybe some simple commands like buffer-set-tags would do for now.

[aartaka]

Sounds like a plan!

[Ambrevar]

And... this obviously does not work, because tags and modes are buffer-specific, not URL-specific. So we must store it either:

• in a new structure that will serve as "owner ID". That means breaking backward-compatibility with previous history files.
• add a data slot to htree:owner, like we do for htree;entry.

Thoughts?

[Ambrevar]

The problem is that while the buffer ID is constant for a given owner, the modes and the tags may change, and we don't have an htree API point to update them. Trickier than I thought...

[Ambrevar]

See #2627 for an untested design draft.