Upgrade 7-Zip version?

[teoberi]

Describe the bug

The 7-Zip version included in Clamav 0.105.0 is 9.20 from 2010-11-18 (https://github.com/Cisco-Talos/clamav/blob/main/libclamav/7z/7zVersion.h)
The latest stable version is 21.07 according to the history.txt
Is there any particular reason why the latest 7-Zip version is not used with all updates including security updates?

[val-ms]

We've been procrastinating upgrading the 7z LZMA-SDK for many years largely because of significant customizations made to our vendored copy back in the 2011-2012 time-frame.

We intend to switch from the C version to the C++ version soon, to get some additional features (see: #542). We will of course pick up the latest version during that switch.

[teoberi]

What @HenkPoley says in #542 may also apply to the use of codecs not yet included in Igor Pavlov's original 7-Zip version. Viruses that pass the Clamav scan can be packaged this way.
For the 7-Zip version 21.07 by Igor Pavlov

Codecs:
4ED 303011B BCJ2
EDF 3030103 BCJ
EDF 3030205 PPC
EDF 3030401 IA64
EDF 3030501 ARM
EDF 3030701 ARMT
EDF 3030805 SPARC
EDF 20302 Swap2
EDF 20304 Swap4
ED 40202 BZip2
ED 0 Copy
ED 40109 Deflate64
ED 40108 Deflate
EDF 3 Delta
ED 21 LZMA2
ED 30101 LZMA
ED 30401 PPMD
D 40301 Rar1
D 40302 Rar2
D 40303 Rar3
D 40305 Rar5
EDF 6F10701 7zAES
EDF 6F00181 AES256CBC

For the 7-Zip version 21.07 by cielavenir

Codecs:
4ED 303011B BCJ2
EDF 3030103 BCJ
EDF 3030205 PPC
EDF 3030401 IA64
EDF 3030501 ARM
EDF 3030701 ARMT
EDF 3030805 SPARC
EDF 20302 Swap2
EDF 20304 Swap4
ED 40202 BZip2
ED 0 Copy
ED 40109 Deflate64
ED 40108 Deflate
EDF 3 Delta
ED 21 LZMA2
ED 30101 LZMA
ED 30401 PPMD
ED 4F71101 ZSTD
ED 4F71104 LZ4
ED 4F71102 BROTLI
ED 4F71106 LIZARD
ED 4F71105 LZ5
ED 4F71001 LZHAM
D 40301 Rar1
D 40302 Rar2
D 40303 Rar3
D 40305 Rar5
EDF 6F10701 7zAES
EDF 6F00181 AES256CBC
ED 4010A PKImplode
ED 21 FLZMA2

[teoberi]

7-Zip 22.01
Upgrade from UnRAR 6.0.7 -> 6.1.7 (fix a path traversal vulnerability).
Compile failure with llvm 14.
With all these 3 problems solved we will be able to use Clamav again (which is now suspended).

[val-ms]

@teoberi we am working on the UnRAR upgrade (#634) and will include it in patch versions for 0.103, 0.104, and 0.105 in the coming week(s).

The other issues (llvm14, and 7zip) are not critical and should not prevent anyone from using ClamAV:

• You can still build with prior versions of LLVM. If they are not available to you, you can build with the bytecode interpreter (the default option) instead of llvm, and bytecode signatures will still work, although a little slower.
• The current 7zip support is better than no 7zip scanning. Work is planned to improve 7zip support and should be included in the next feature release.

[teoberi]

Fair enought, I will wait for that.
Building with the bytecode interpreter generates quite a few warnings, I avoided this in the last builds.
Regarding to 7-zip, I consider that the update to the latest version (now that the 7-zip development is going quite well) is necessary and important.
On my servers I use Clamav integrated in Amavis to scan email messages together with a commercial solution from Sophos which will have EOL next year. So I still have time to wait for Clamav to fix this problems if it will remains the only solution for my operating system (Slackware).

[val-ms]

I did some additional research today. I found the blog post describing the unrar CVE. https://blog.sonarsource.com/zimbra-pre-auth-rce-via-unrar-0day/

After some intense debugging and reading over the article, I think we may actually be affected by this CVE, but ONLY when using the --leave-temps option. I was able to modify a test.rar RAR archive and change the name of an embedded file to have the name ..\..\../t. In my testing that caused it to try to extract to this path:

unrar_extract_file: Extracted file to: /tmp/20220721_101601-test.rar.051dfeb4f4/test.rar.be3b9aca25/..\..\..\t.ae45e02c57

According to the blog post, the bug in libunrar is specifically for symbolic links, though. For these, it will convert those backslashes after they're passed in to the extraction function over to / which turns them into actual path separators which means writing outside the temp directory. My very brief review of the libunrar code confirms this. For ClamAV's use of libunrar, we append a hash suffix, like you see above (t.ae45e02c57), so even if you can get the path traversal issue to affect ClamAV in --leave-temps-mode, would be highly unlikely to be useful.

When not using --leave-temps, the filename for the extracted file is totally random, so it extracts here instead:

unrar_extract_file: Extracted file to: /tmp/20220721_104349-scantem.3f66db77da/clamav-1c8d878fa9dbe5535b71297e49fa100b.tmp

That is excellent. Almost nobody uses --leave-temps except when using clam to analyze files, or when debugging a ClamAV bug.

TL;DR: ClamAV appears to be only very very slightly affected. It is still best for us to to upgrade libunrar in ClamAV, and I will continue to work on getting that done for a patch release next week.

[val-ms]

For reference: From discussion in discord on 2023/12/16, @CTRLRLTY is working on this.

[teoberi]

I'm glad to know that!

I would be even more happy if the support for LLVM 14+ would also be resolved!

[val-ms]

Re: LLVM 14+ support, it's on our radar but actively focusing on LLVM 14+ support for the bytecode compiler, first. And after that we have to focus on some archive support improvements before we can look at it. So it's a ways down the backlog.

Others are welcome to help if they want -- though you can also use the bytecode interpreter for bytecode functionality whenever LLVM is not available.

[Sanesecurity]

Just to add useful 7zip feature updates...

HISTORY of the 7-Zip

24.01 2024-01-31

• 7-Zip now can unpack ZSTD archives (.zst filename extension).
• 7-Zip now can unpack ZIP, SquashFS and RPM archives that use ZSTD compression method.
• 7-Zip now supports fast hash algorithm XXH64 that is used in ZSTD.
• 7-Zip now can unpack RAR archives (that use larger than 4 GB dictionary) created by new WinRAR 7.00.
• 7-Zip now can unpack DMG archives that use XZ (ULMO/LZMA) compression method.
• 7-zip now can unpack NTFS images with cluster size larger than 64 KB.
• 7-zip now can unpack MBR and GDP images with 4 KB sectors.
• Speed optimizations for archive unpacking: rar, cab, wim, zip, gz.
• Speed optimizations for hash caclulation: CRC-32, CRC-64, Blake2sp.