Release 3.2.3-1.14

- Sync fork-load-daemon script with http-nbd-transfer (v1.5.0)
- Fix coalesce process for LINSTOR SRs
- Many code improvements for issues detected by mypy

Signed-off-by: Ronan Abhamon <[email protected]>

SOURCES/0001-Update-xs-sm.service-s-description-for-XCP-ng.patch

@@ -1,7 +1,7 @@
 From de38d2672887c47a4e0600426d87137185243d38 Mon Sep 17 00:00:00 2001
 From: Samuel Verschelde <[email protected]>
 Date: Thu, 13 Aug 2020 15:22:17 +0200
-Subject: [PATCH 01/31] Update xs-sm.service's description for XCP-ng
+Subject: [PATCH 01/60] Update xs-sm.service's description for XCP-ng
 
 This was a patch added to the sm RPM git repo before we had this
 forked git repo for sm in the xcp-ng github organisation.

SOURCES/0002-feat-drivers-add-CephFS-and-GlusterFS-drivers.patch

@@ -1,7 +1,7 @@
 From 1809f021bf3af5fa18419e248469cd970bded9f3 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Mon, 20 Jul 2020 16:26:42 +0200
-Subject: [PATCH 02/31] feat(drivers): add CephFS and GlusterFS drivers
+Subject: [PATCH 02/60] feat(drivers): add CephFS and GlusterFS drivers
 
 ---
  Makefile               |   2 +

SOURCES/0003-feat-drivers-add-XFS-driver.patch

@@ -1,7 +1,7 @@
 From 4517b34462a3285630dc134de99eb84f6aa4fb58 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Mon, 20 Jul 2020 16:26:42 +0200
-Subject: [PATCH 03/31] feat(drivers): add XFS driver
+Subject: [PATCH 03/60] feat(drivers): add XFS driver
 
 Originally-by: Ronan Abhamon <[email protected]>
 

SOURCES/0004-feat-drivers-add-ZFS-driver-to-avoid-losing-VDI-meta.patch

@@ -1,7 +1,7 @@
 From cd3cd24251e847eb20ac5b32a374109718c8cca0 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Wed, 12 Aug 2020 11:14:33 +0200
-Subject: [PATCH 04/31] feat(drivers): add ZFS driver to avoid losing VDI
+Subject: [PATCH 04/60] feat(drivers): add ZFS driver to avoid losing VDI
  metadata (xcp-ng/xcp#401)
 
 ---

SOURCES/0005-feat-drivers-add-LinstorSR-driver.patch

@@ -1,7 +1,7 @@
 From ee88ac2658f10fe367a1a299f8b17b10209388ff Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Mon, 16 Mar 2020 15:39:44 +0100
-Subject: [PATCH 05/31] feat(drivers): add LinstorSR driver
+Subject: [PATCH 05/60] feat(drivers): add LinstorSR driver
 
 Some important points:
 

SOURCES/0006-feat-tests-add-unit-tests-concerning-ZFS-close-xcp-n.patch

@@ -1,7 +1,7 @@
 From 52c6e98da6238b6a624f7e59b4e0d896ca0728bc Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Tue, 27 Oct 2020 15:04:36 +0100
-Subject: [PATCH 06/31] feat(tests): add unit tests concerning ZFS (close
+Subject: [PATCH 06/60] feat(tests): add unit tests concerning ZFS (close
  xcp-ng/xcp#425)
 
 - Check if "create" doesn't succeed without zfs packages

SOURCES/0007-Added-SM-Driver-for-MooseFS.patch

@@ -1,7 +1,7 @@
 From 1dc43ae24403979db03fb7bbf90aeffe81d6c28b Mon Sep 17 00:00:00 2001
 From: Aleksander Wieliczko <[email protected]>
 Date: Fri, 29 Jan 2021 15:21:23 +0100
-Subject: [PATCH 07/31] Added SM Driver for MooseFS
+Subject: [PATCH 07/60] Added SM Driver for MooseFS
 
 Co-authored-by: Piotr Robert Konopelko <[email protected]>
 Signed-off-by: Aleksander Wieliczko <[email protected]>

SOURCES/0008-Avoid-usage-of-umount-in-ISOSR-when-legacy_mode-is-u.patch

@@ -1,7 +1,7 @@
 From 0f993508af755b52d46b216b503e81914eb570ef Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Thu, 2 Dec 2021 09:28:37 +0100
-Subject: [PATCH 08/31] Avoid usage of `umount` in `ISOSR` when `legacy_mode`
+Subject: [PATCH 08/60] Avoid usage of `umount` in `ISOSR` when `legacy_mode`
  is used
 
 `umount` should not be called when `legacy_mode` is enabled, otherwise a mounted dir

SOURCES/0009-MooseFS-SR-uses-now-UUID-subdirs-for-each-SR.patch

@@ -1,7 +1,7 @@
 From 1f67bfaffbd431c166a5e61322ea7d179ee4d687 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Wed, 18 May 2022 17:28:09 +0200
-Subject: [PATCH 09/31] MooseFS SR uses now UUID subdirs for each SR
+Subject: [PATCH 09/60] MooseFS SR uses now UUID subdirs for each SR
 
 A sm-config boolean param `subdir` is available to configure where to store the VHDs:
 - In a subdir with the SR UUID, the new behavior

SOURCES/0010-Fix-is_open-call-for-many-drivers-25.patch

@@ -1,7 +1,7 @@
 From d32bf0ccfe58c67ba87e9bab5b76f9eead92735d Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Thu, 23 Jun 2022 10:36:36 +0200
-Subject: [PATCH 10/31] Fix is_open call for many drivers (#25)
+Subject: [PATCH 10/60] Fix is_open call for many drivers (#25)
 
 Ensure all shared drivers are imported in `_is_open` definition to register
 them in the driver list. Otherwise this function always fails with a SRUnknownType exception.

SOURCES/0011-Remove-SR_CACHING-capability-for-many-SR-types-24.patch

@@ -1,7 +1,7 @@
 From 3c420bca8372e1441f977562b61439ff2235003f Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Thu, 23 Jun 2022 10:37:07 +0200
-Subject: [PATCH 11/31] Remove SR_CACHING capability for many SR types (#24)
+Subject: [PATCH 11/60] Remove SR_CACHING capability for many SR types (#24)
 
 SR_CACHING offers the capacity to use IntelliCache, but this
 feature is only available using NFS SR.

SOURCES/0012-Fix-code-coverage-regarding-MooseFSSR-and-ZFSSR-29.patch

@@ -1,7 +1,7 @@
 From 471b7b2742da807aca012c6609c8917cbe786507 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Mon, 19 Sep 2022 10:31:00 +0200
-Subject: [PATCH 12/31] Fix code coverage regarding MooseFSSR and ZFSSR (#29)
+Subject: [PATCH 12/60] Fix code coverage regarding MooseFSSR and ZFSSR (#29)
 
 Signed-off-by: Ronan Abhamon <[email protected]>
 ---

SOURCES/0013-py3-simple-changes-from-futurize-on-XCP-ng-drivers.patch

@@ -1,7 +1,7 @@
 From 395daa86f0b4dcbc4eeabb42fed4f818d46afc36 Mon Sep 17 00:00:00 2001
 From: Yann Dirson <[email protected]>
 Date: Wed, 8 Mar 2023 10:13:18 +0100
-Subject: [PATCH 13/31] py3: simple changes from futurize on XCP-ng drivers
+Subject: [PATCH 13/60] py3: simple changes from futurize on XCP-ng drivers
 
 * `except` syntax fixes
 * drop `has_key()` usage

SOURCES/0014-py3-futurize-fix-of-xmlrpc-calls-for-CephFS-GlusterF.patch

@@ -1,7 +1,7 @@
 From af592f003d7fb8bc2572455271bf143e97a10c81 Mon Sep 17 00:00:00 2001
 From: Yann Dirson <[email protected]>
 Date: Wed, 8 Mar 2023 10:28:10 +0100
-Subject: [PATCH 14/31] py3: futurize fix of xmlrpc calls for CephFS,
+Subject: [PATCH 14/60] py3: futurize fix of xmlrpc calls for CephFS,
  GlusterFS, MooseFS, Linstore
 
 Signed-off-by: Yann Dirson <[email protected]>

SOURCES/0015-py3-use-of-integer-division-operator.patch

@@ -1,7 +1,7 @@
 From e1335b1ca83771142e6a6f831b5b4aa888e2ff18 Mon Sep 17 00:00:00 2001
 From: Yann Dirson <[email protected]>
 Date: Wed, 8 Mar 2023 10:32:37 +0100
-Subject: [PATCH 15/31] py3: use of integer division operator
+Subject: [PATCH 15/60] py3: use of integer division operator
 
 Guided by futurize's "old_div" use
 

SOURCES/0016-test_on_slave-allow-to-work-with-SR-using-absolute-P.patch

@@ -1,7 +1,7 @@
 From 2e7fa79a4d7d4ebbee00959326b8f8151a816889 Mon Sep 17 00:00:00 2001
 From: Yann Dirson <[email protected]>
 Date: Wed, 8 Mar 2023 13:53:21 +0100
-Subject: [PATCH 16/31] test_on_slave: allow to work with SR using absolute
+Subject: [PATCH 16/60] test_on_slave: allow to work with SR using absolute
  PROBE_MOUNTPOINT
 
 PROBE_MOUNTPOINT in a some drivers is a relative path, which is resolved

SOURCES/0017-py3-switch-interpreter-to-python3.patch

@@ -1,7 +1,7 @@
 From 2df6ea2d37f2a117f70f844105db2771d9c88517 Mon Sep 17 00:00:00 2001
 From: Yann Dirson <[email protected]>
 Date: Mon, 27 Mar 2023 15:30:46 +0200
-Subject: [PATCH 17/31] py3: switch interpreter to python3
+Subject: [PATCH 17/60] py3: switch interpreter to python3
 
 ---
  drivers/CephFSSR.py             | 2 +-

SOURCES/0018-Support-recent-version-of-coverage-tool.patch

@@ -1,7 +1,7 @@
 From 09aa72c4d2655ba1027dc831b1580bde87643447 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Thu, 4 May 2023 10:24:22 +0200
-Subject: [PATCH 18/31] Support recent version of coverage tool (coverage
+Subject: [PATCH 18/60] Support recent version of coverage tool (coverage
  7.2.5)
 
 Without these changes many warns/errors are emitted:

SOURCES/0019-feat-LinstorSR-import-all-8.2-changes.patch

@@ -1,7 +1,7 @@
 From 98d8c31104c3eae9ea62ea106d09fb801a784f4e Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Fri, 20 Nov 2020 16:42:52 +0100
-Subject: [PATCH 19/31] feat(LinstorSR): import all 8.2 changes
+Subject: [PATCH 19/60] feat(LinstorSR): import all 8.2 changes
 
 Signed-off-by: Ronan Abhamon <[email protected]>
 ---

SOURCES/0020-feat-LinstorSR-is-now-compatible-with-python-3.patch

@@ -1,7 +1,7 @@
 From 1354e78a5409805f08f118910db32b89bfc6c780 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Fri, 30 Jun 2023 12:41:43 +0200
-Subject: [PATCH 20/31] feat(LinstorSR): is now compatible with python 3
+Subject: [PATCH 20/60] feat(LinstorSR): is now compatible with python 3
 
 Signed-off-by: Ronan Abhamon <[email protected]>
 ---

SOURCES/0021-Remove-SR_PROBE-from-ZFS-capabilities-36.patch

@@ -1,7 +1,7 @@
 From d778817b78c72a04d419073ee8e98099e2a960a0 Mon Sep 17 00:00:00 2001
 From: BenjiReis <[email protected]>
 Date: Fri, 4 Aug 2023 12:10:37 +0200
-Subject: [PATCH 21/31] Remove `SR_PROBE` from ZFS capabilities (#36)
+Subject: [PATCH 21/60] Remove `SR_PROBE` from ZFS capabilities (#36)
 
 The probe method is not implemented so we
 shouldn't advertise it.

SOURCES/0022-Repair-coverage-to-be-compatible-with-8.3-test-env.patch

@@ -1,7 +1,7 @@
 From 18fe0f0b6c645b314d49b377602cc681167422e6 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Fri, 22 Sep 2023 11:11:27 +0200
-Subject: [PATCH 22/31] Repair coverage to be compatible with 8.3 test env
+Subject: [PATCH 22/60] Repair coverage to be compatible with 8.3 test env
 
 Impacted drivers: LINSTOR, MooseFS and ZFS.
 - Ignore all linstor.* members during coverage,

SOURCES/0023-Support-IPv6-in-Ceph-Driver.patch

@@ -1,7 +1,7 @@
 From e46f99246bd1c9d1e06e6dbc49b1d95dbaaf8eac Mon Sep 17 00:00:00 2001
 From: BenjiReis <[email protected]>
 Date: Mon, 25 Sep 2023 16:13:13 +0200
-Subject: [PATCH 23/31] Support IPv6 in Ceph Driver
+Subject: [PATCH 23/60] Support IPv6 in Ceph Driver
 
 Signed-off-by: BenjiReis <[email protected]>
 ---

SOURCES/0024-lvutil-use-wipefs-not-dd-to-clear-existing-signature.patch

@@ -1,7 +1,7 @@
 From 709b30f2d5d8c85c87da2f375a032a0d004b4298 Mon Sep 17 00:00:00 2001
 From: Yann Dirson <[email protected]>
 Date: Wed, 5 Jul 2023 16:57:26 +0200
-Subject: [PATCH 24/31] lvutil: use wipefs not dd to clear existing signatures
+Subject: [PATCH 24/60] lvutil: use wipefs not dd to clear existing signatures
  (xapi-project#624)
 
 Signed-off-by: Yann Dirson <[email protected]>

SOURCES/0025-feat-LargeBlock-introduce-largeblocksr-51.patch

@@ -1,7 +1,7 @@
 From 47d5d8691b40f9c4bd8a77c00d851443e68ac887 Mon Sep 17 00:00:00 2001
 From: Damien Thenot <[email protected]>
 Date: Fri, 12 Apr 2024 15:08:59 +0200
-Subject: [PATCH 25/31] feat(LargeBlock): introduce largeblocksr (#51)
+Subject: [PATCH 25/60] feat(LargeBlock): introduce largeblocksr (#51)
 
 A SR inheriting from a EXTSR allowing to use a 4KiB blocksize device as
 SR.

SOURCES/0026-feat-LVHDSR-add-a-way-to-modify-config-of-LVMs-60.patch

@@ -1,7 +1,7 @@
 From db0a6cb684e3a24ef1c47094154d19c03a6f0a5c Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Tue, 30 Jul 2024 15:31:23 +0200
-Subject: [PATCH 26/31] feat(LVHDSR): add a way to modify config of LVMs (#60)
+Subject: [PATCH 26/60] feat(LVHDSR): add a way to modify config of LVMs (#60)
 
 With this change the driver supports a "lvm-conf" param on "other-config".
 For now The configuration is only used by "remove" calls from LVMCache.

SOURCES/0027-reflect-upstream-changes-in-our-tests.patch

@@ -1,7 +1,7 @@
 From df0d4cd2dcafb2d866ee121d8d7903711ffa4456 Mon Sep 17 00:00:00 2001
 From: Benjamin Reis <[email protected]>
 Date: Tue, 13 Aug 2024 11:11:39 +0200
-Subject: [PATCH 27/31] reflect upstream changes in our tests
+Subject: [PATCH 27/60] reflect upstream changes in our tests
 
 Signed-off-by: Benjamin Reis <[email protected]>
 ---

SOURCES/0028-CA-398425-correctly-check-for-multiple-targets-in-iS.patch

@@ -1,7 +1,7 @@
 From 006a3727a18eed16a581251553b829667efceefd Mon Sep 17 00:00:00 2001
 From: Mark Syms <[email protected]>
 Date: Fri, 30 Aug 2024 10:13:27 +0100
-Subject: [PATCH 28/31] CA-398425: correctly check for multiple targets in
+Subject: [PATCH 28/60] CA-398425: correctly check for multiple targets in
  iSCSI
 
 Signed-off-by: Mark Syms <[email protected]>

SOURCES/0029-Synchronization-with-8.2-LINSTOR-before-a-stable-rel.patch

@@ -1,7 +1,7 @@
 From 6d2b2d1b0dbe44f52033949d0a2f758c89972a97 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Thu, 26 Sep 2024 18:53:04 +0200
-Subject: [PATCH 29/31] Synchronization with 8.2 LINSTOR before a stable
+Subject: [PATCH 29/60] Synchronization with 8.2 LINSTOR before a stable
  release (#68)
 
 Last commit: 9207abe1f2e1ff1795cdba1a0aeb76574412a583

SOURCES/0030-fix-getAllocatedSize-is-incorrect-75.patch

@@ -1,7 +1,7 @@
 From 32d0de18367a53e1e0f6c4d53615825cd1fa7833 Mon Sep 17 00:00:00 2001
 From: Damien Thenot <[email protected]>
 Date: Thu, 28 Nov 2024 15:42:57 +0100
-Subject: [PATCH 30/31] fix: getAllocatedSize is incorrect (#75)
+Subject: [PATCH 30/60] fix: getAllocatedSize is incorrect (#75)
 
 The `VDI.canLiveCoalesce` method can manipulates sizes of different units because of this change:
 ```

SOURCES/0031-fix-LinstorSR-sync-fork-load-daemon-with-http-nbd-tr.patch

@@ -1,7 +1,7 @@
 From 122b4a10e878567ac59082d4129a6477eb5b3db5 Mon Sep 17 00:00:00 2001
 From: Ronan Abhamon <[email protected]>
 Date: Thu, 28 Nov 2024 15:12:21 +0100
-Subject: [PATCH 31/31] fix(LinstorSR): sync fork-load-daemon with
+Subject: [PATCH 31/60] fix(LinstorSR): sync fork-load-daemon with
  http-nbd-transfer (v1.5.0) (#73)
 
 Signed-off-by: Ronan Abhamon <[email protected]>

SOURCES/0032-fix-linstorvhdutil-coalesce-helper-returns-the-secto.patch

@@ -0,0 +1,42 @@
+From 65021acb5918cfe002dc53232a1b5569efce0d13 Mon Sep 17 00:00:00 2001
+From: Ronan Abhamon <[email protected]>
+Date: Wed, 11 Dec 2024 11:14:27 +0100
+Subject: [PATCH 32/60] fix(linstorvhdutil): coalesce helper returns the sector
+ count now
+
+Without this change we have an error in cleanup.py that interrupts
+the coalesce algorithm.
+
+Signed-off-by: Ronan Abhamon <[email protected]>
+---
+ drivers/linstor-manager   | 3 +--
+ drivers/linstorvhdutil.py | 2 +-
+ 2 files changed, 2 insertions(+), 3 deletions(-)
+
+diff --git a/drivers/linstor-manager b/drivers/linstor-manager
+index 47c434a3..8ee6f149 100755
+--- a/drivers/linstor-manager
++++ b/drivers/linstor-manager
+@@ -539,8 +539,7 @@ def set_parent(session, args):
+ def coalesce(session, args):
+     try:
+         device_path = args['devicePath']
+-        vhdutil.coalesce(device_path)
+-        return ''
++        return str(vhdutil.coalesce(device_path))
+     except Exception as e:
+         util.SMlog('linstor-manager:coalesce error: {}'.format(e))
+         raise
+diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py
+index 046c9695..b3df004a 100644
+--- a/drivers/linstorvhdutil.py
++++ b/drivers/linstorvhdutil.py
+@@ -376,7 +376,7 @@ class LinstorVhdUtil:
+
+     @linstormodifier()
+     def force_coalesce(self, path):
+-        return self._call_method(vhdutil.coalesce, 'coalesce', path, use_parent=True)
++        return int(self._call_method(vhdutil.coalesce, 'coalesce', path, use_parent=True))
+
+     @linstormodifier()
+     def force_repair(self, path):

SOURCES/0033-Prevent-wrong-mypy-error-regarding-_linstor-member-n.patch

@@ -0,0 +1,44 @@
+From 94214801928d2a07fcc247cb8fa0bca07eb8b718 Mon Sep 17 00:00:00 2001
+From: Ronan Abhamon <[email protected]>
+Date: Wed, 16 Oct 2024 15:02:30 +0200
+Subject: [PATCH 33/60] Prevent wrong mypy error regarding `_linstor` member
+ not set
+
+Signed-off-by: Ronan Abhamon <[email protected]>
+---
+ drivers/LinstorSR.py | 9 +++++----
+ 1 file changed, 5 insertions(+), 4 deletions(-)
+
+diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py
+index 829c48f8..7640f93d 100755
+--- a/drivers/LinstorSR.py
++++ b/drivers/LinstorSR.py
+@@ -613,16 +613,16 @@ class LinstorSR(SR.SR):
+                 logger=util.SMlog
+             )
+             self._vhdutil = LinstorVhdUtil(self.session, self._linstor)
+-        except Exception as e:
+-            util.SMlog('Failed to create LINSTOR SR: {}'.format(e))
+-            raise xs_errors.XenError('LinstorSRCreate', opterr=str(e))
+
+-        try:
+             util.SMlog(
+                 "Finishing SR creation, enable drbd-reactor on all hosts..."
+             )
+             self._update_drbd_reactor_on_all_hosts(enabled=True)
+         except Exception as e:
++            if not self._linstor:
++                util.SMlog('Failed to create LINSTOR SR: {}'.format(e))
++                raise xs_errors.XenError('LinstorSRCreate', opterr=str(e))
++
+             try:
+                 self._linstor.destroy()
+             except Exception as e2:
+@@ -637,6 +637,7 @@ class LinstorSR(SR.SR):
+         util.SMlog('LinstorSR.delete for {}'.format(self.uuid))
+         cleanup.gc_force(self.session, self.uuid)
+
++        assert self._linstor
+         if self.vdis or self._linstor._volumes:
+             raise xs_errors.XenError('SRNotEmpty')
+