2.10.3

[vassilit]

Discussions about what should go to the next (yet unnamed) bugfix release.

[RayOei]

I ran into this comment, thought to share it here: #624 (comment)
cebtenzzre explaining why he was a bit overwhelmed.

[RayOei]

Take this one #593 too?

[RayOei]

@vassilit Just to align: were you planning to look into #687 or shall I have a stab at it?
As far as I can see the work done is not complete in the PR, some nose references still exists and the first test I checked seems to expect outdated function calls, by the looks of it.
Either way: before we are wasting each others time ... 😁

[vassilit]

@vassilit Just to align: were you planning to look into #687

Yes

or shall I have a stab at it?

Go ahead :)

As far as I can see the work done is not complete in the PR, some nose references still exists

Which ones ?

05/03 17:47 vt@luga ~/dev/contrib/rmlint% git grep nose
05/03 17:47 vt@luga ~/dev/contrib/rmlint% git status
On branch remove_nose

[RayOei]

Hmmmmm.... seems to be a rebase thing when I merged his branch on top of my local rmlint master.
Not too bad, though. I have the formatters cleaned up again.

[RayOei]

And more recent tests (mostly addressing regressions) which are not part of #603 are also still using nose.
Also updating the usage of fixtures.

[vassilit]

And more recent tests (mostly addressing regressions) which are not part of #603 are also still using nose. Also updating the usage of fixtures.

I rebased #603 to master, including the recent tests

[vassilit]

We should be able to make a new release soon, to hopefully get the package (re)integrated into distributions, not breaking users that depends on it.

It would be good also to review the downstream patches, to see if there is something reasonable to include here:

https://salsa.debian.org/debian/rmlint/-/tree/debian/master/debian/patches?ref_type=heads
https://gitlab.archlinux.org/archlinux/packaging/packages/rmlint
https://cgit.freebsd.org/ports/tree/sysutils/rmlint
https://src.fedoraproject.org/rpms/rmlint
https://gitweb.gentoo.org/repo/gentoo.git/tree/app-misc/rmlint?id=392900cef25d31d5c622e542d636ba37e7a0b71a
https://bugs.debian.org/cgi-bin/pkgreport.cgi?src=rmlint
https://github.com/macports/macports-ports/tree/master/sysutils/rmlint
https://github.com/barracuda156/powerpc-ports/tree/master/sysutils/rmlint
https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/testing/rmlint
https://github.com/Homebrew/homebrew-core/blob/master/Formula/r/rmlint.rb
https://github.com/NixOS/nixpkgs/tree/master/pkgs/tools/misc/rmlint
https://github.com/DragonFlyBSD/DPorts/tree/master/sysutils/rmlint
https://github.com/void-linux/void-packages/tree/master/srcpkgs/rmlint
https://github.com/getsolus/packages/tree/main/packages/r/rmlint
https://copr-dist-git.fedorainfracloud.org/cgit/rodoma92/rmlint/rmlint.git
https://git.slackbuilds.org/slackbuilds/plain/misc/rmlint/
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/disk.scm#n1288
https://git.launchpad.net/ubuntu/+source/rmlint/tree/
https://summer.exherbolinux.org/packages/sys-apps/rmlint/index.html

https://github.com/tldr-pages/tldr/blob/main/pages/common/rmlint.md?plain=1

Also, the pain point before release might be to fix the GUI, if there is something to fix. I've never used it to be honest, but it looks nice on the screenshots.

[vassilit]

TODO for the release: fix *64 calls, use -D_FILE_OFFSET_BITS=64 instead. Eventually undefine _LARGEFILE64_SOURCE

[vassilit]

@sahib Hello, could you recap in two words what was wrong with libc fts ? What was the rational for importing musl/NetBSD ?

[RayOei]

the pain point before release might be to fix the GUI

Yeah... I have tried it but it doesn't work for me at this point. I thought I noticed something but that was a red herring. I have seen that cebtenzzre also tinkered with it. It may have introduced a problem with the generated bootstrap file. Not sure yet.
At this point it seems there is also an Debian installation issue which is also mentioned in the readme. It could be that #673 is related to that? Will try a few things 😉

[vassilit]

I think #673 is not actual anymore and was fixed in e811a34

[vassilit]

@sahib Hello, could you recap in two words what was wrong with libc fts ? What was the rational for importing musl/NetBSD ?

The idea behind my question would be to simplify the codebase. E.G. using openssl and libc when possible and concentrate on rmlint own features.

I don't think maintaining own hash implementations in rmlint makes sense, and we miss on many architectures optimisations.

[RayOei]

Doesn't look like that? It doesn't work on 2.10.2 for me on Ubuntu 24. That is why I thought that change might have been the culprit. I got the same SyntaxError: source code cannot contain null bytes error at first with 2.9. With 2.10.2 (or the latest develop) I get No module named shredder error. When I run the gui\setup.py another module (gi = gobject-introspection) is also missing so it looks like the install is no longer(?) installing everything?

As a side note: it also doesn't work on macOS and by the looks of it the brew formula is incomplete. I think a comment/warning in the readme would be helpful as users are still commenting on #253
And the different solutions, which are no solutions for end-users anyway, don't solve it as far as I can see. Also here, even with some tinkering, the python gi module seems a problem.

[vassilit]

I got the same SyntaxError: source code cannot contain null bytes error at first with 2.9.

This bug has been fixed, I'm sure of it. It was due to a C-string written to disk including its NULL-termination byte.

With 2.10.2 (or the latest develop) I get No module named shredder error.

That's another bug, but distribution-specific it seems, as the GUI runs fine on my system (it is installed in /usr/lib/python3.13/site-packages/shredder/)

[RayOei]

What are you using as distribution?

[RayOei]

(and yes, I agree with the null terminator fix)

[vassilit]

What are you using as distribution?

As rmlint is a swiss-army knife, I sometimes use it a bit everywhere, FreeBSD, Debian, Centos, macOS, Raspberry Pi OS, etc.
I've only tested the GUI on Arch.

[RayOei]

I got GUI to work on my Ubuntu 24 when applying the full build stack for GI. I did run into a default configuration issue which results in rmlint not actually scanning but the Gui just implies it is done. Have to get the exact situation clear, though.

I don't get it to work on macOS (15) and I tried several recommendations and it keeps its missing module gi. At this point I suspect that something in the python install clashes with my pyEnv.
I think a disclaimer for macOS may be warranted, for the time being? As I understand it the brew installation is lacking (see also #253 (comment)) in regards to the GUI option.

Either way: it is clear that the GUI part needs attention across the board.

I am planning to test with a fresh OS (Ubuntu & Pi OS) to see how that behaves, at this stage it is a bit hard to tell which problem I see is related to my messy environment or a real issue.

[RayOei]

I noticed this warning in the builds:

lib/pathtricia.c:105:5: warning: '__builtin_strncpy' specified bound 4096 equals destination size [-Wstringop-truncation]
  105 |     strncpy(iter->path_buf, path, PATH_MAX);
      |     ^

I am not sure this is ignored on purpose or just not enforced by the current settings?
Isn't it also better to avoid strncpy altogether?

[vassilit]

This is accounted for. There used to be memset(iter->path_buf, 0, PATH_MAX);, so the NUL-padding of strncpy might be useful. I haven't checked.

In the next lines, we check for truncation:

    if (iter->path_buf[PATH_MAX - 1]) {
        iter->path_buf[PATH_MAX - 1] = 0;
        rm_log_error("path too long, truncated to: %s", iter->path_buf);
    }

This is not ideal, but better than before and maybe enough for 2.10.3. What do you think ?

[vassilit]

In the future, we could see if strncpy() could be replaced by our own simpler function as:

/* simpler strcpy implementation without the downsides of strncpy and strlcpy */
static bool rm_strcpy(char *dst, const char *src, size_t len) {
	for (size_t i = 0; i < len; i++)
		if ('\0' == (dst[i] = src[i]))
			return true;

	return false;
}

[RayOei]

Thanks for the clarification.
It isn't used a lot, strncpy, but replacing by a safe function seems useful and/or wise. 😁

[RayOei]

We could add a _PRAGMA for the time being? But maybe that is too much effort for little gains.

[vassilit]

Not a lot, but we must check every case.

11/03 21:23 vt@luga ~/dev/contrib/rmlint% git grep strncpy
lib/cmdline.c:    strncpy(size_spec_copy, size_spec, sizeof(size_spec_copy)-1);
lib/cmdline.c:            strncpy(format_name, pair, sizeof(format_name)-1);
lib/cmdline.c:            strncpy(format_name, extension, sizeof(format_name)-1);
lib/cmdline.c:        strncpy(format_name, pair, MIN((long)sizeof(format_name), separator - pair));
lib/cmdline.c:    strncpy(cfg->rank_criteria, criteria, sizeof(cfg->rank_criteria)-1);
lib/formats/progressbar.c:    strncpy(self->last_eta, eta_info, sizeof(self->last_eta) - 1);
lib/pathtricia.c:    strncpy(iter->path_buf, path, PATH_MAX);
lib/utilities.c:                strncpy(diskname, entry->fsname, sizeof(diskname)-1);
lib/utilities.c:                strncpy(diskname, entry->fsname, until_slash);
lib/utilities.c:                strncpy(diskname, "unknown", sizeof(diskname));
lib/utilities.c:                strncpy(diskname, entry->fsname, sizeof(diskname)-1);

[vassilit]

Ignoring the false-positive warning would give:

#pragma GCC diagnostic push
#ifndef __clang__
#pragma GCC diagnostic ignored "-Wstringop-truncation"
#endif
    strncpy(iter->path_buf, path, PATH_MAX);
#pragma GCC diagnostic pop

Not very clear.

[RayOei]

You're already half-way changing the function itself 🤣
Anyway, it's fine for now, it is not that important at this time (for 2.10.3).
Let's put it up for a future code improvement.

[RayOei]

I have been tinkering with the badges on the readme.rst:

• there was a broken badge, still referencing Travis. I have replaced that for the workflow one
• Also expanded the release badge (now with name)
• removed the download badge as that always returns 0, so it seems
• removed the chat reference - although we may add it again when discussion is available?
• added commit to indicate there is activity

I also noticed the Please see the AUTHORS distributed along rmlint. comment: where does that refer to?

Any comments on the result?

[vassilit]

Either way: it is clear that the GUI part needs attention across the board.

Could you test #710 and tell me if it works for you ?

[RayOei]

I think the GUI uninstall needs verifying. For reference: #710 (comment)

[RayOei]

Installing GUI should be seamless for a user. For reference: #710 (comment)
Either this is handled by the Python setup or at least a readme with pointers?