Skip to content

Commit

Permalink
Add 0030-fix-cleanup.py-bad-live-coalesce-check-regarding-Fil.patch (#71
Browse files Browse the repository at this point in the history
)

Signed-off-by: Damien Thenot <[email protected]>
  • Loading branch information
Nambrok authored Nov 27, 2024
1 parent 16ed1cb commit b3dbc37
Show file tree
Hide file tree
Showing 31 changed files with 121 additions and 39 deletions.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From de38d2672887c47a4e0600426d87137185243d38 Mon Sep 17 00:00:00 2001
From: Samuel Verschelde <[email protected]>
Date: Thu, 13 Aug 2020 15:22:17 +0200
Subject: [PATCH 01/29] Update xs-sm.service's description for XCP-ng
Subject: [PATCH 01/30] Update xs-sm.service's description for XCP-ng

This was a patch added to the sm RPM git repo before we had this
forked git repo for sm in the xcp-ng github organisation.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 1809f021bf3af5fa18419e248469cd970bded9f3 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Mon, 20 Jul 2020 16:26:42 +0200
Subject: [PATCH 02/29] feat(drivers): add CephFS and GlusterFS drivers
Subject: [PATCH 02/30] feat(drivers): add CephFS and GlusterFS drivers

---
Makefile | 2 +
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0003-feat-drivers-add-XFS-driver.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 4517b34462a3285630dc134de99eb84f6aa4fb58 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Mon, 20 Jul 2020 16:26:42 +0200
Subject: [PATCH 03/29] feat(drivers): add XFS driver
Subject: [PATCH 03/30] feat(drivers): add XFS driver

Originally-by: Ronan Abhamon <[email protected]>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From cd3cd24251e847eb20ac5b32a374109718c8cca0 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Wed, 12 Aug 2020 11:14:33 +0200
Subject: [PATCH 04/29] feat(drivers): add ZFS driver to avoid losing VDI
Subject: [PATCH 04/30] feat(drivers): add ZFS driver to avoid losing VDI
metadata (xcp-ng/xcp#401)

---
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0005-feat-drivers-add-LinstorSR-driver.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From ee88ac2658f10fe367a1a299f8b17b10209388ff Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Mon, 16 Mar 2020 15:39:44 +0100
Subject: [PATCH 05/29] feat(drivers): add LinstorSR driver
Subject: [PATCH 05/30] feat(drivers): add LinstorSR driver

Some important points:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 52c6e98da6238b6a624f7e59b4e0d896ca0728bc Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Tue, 27 Oct 2020 15:04:36 +0100
Subject: [PATCH 06/29] feat(tests): add unit tests concerning ZFS (close
Subject: [PATCH 06/30] feat(tests): add unit tests concerning ZFS (close
xcp-ng/xcp#425)

- Check if "create" doesn't succeed without zfs packages
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0007-Added-SM-Driver-for-MooseFS.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 1dc43ae24403979db03fb7bbf90aeffe81d6c28b Mon Sep 17 00:00:00 2001
From: Aleksander Wieliczko <[email protected]>
Date: Fri, 29 Jan 2021 15:21:23 +0100
Subject: [PATCH 07/29] Added SM Driver for MooseFS
Subject: [PATCH 07/30] Added SM Driver for MooseFS

Co-authored-by: Piotr Robert Konopelko <[email protected]>
Signed-off-by: Aleksander Wieliczko <[email protected]>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 0f993508af755b52d46b216b503e81914eb570ef Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Thu, 2 Dec 2021 09:28:37 +0100
Subject: [PATCH 08/29] Avoid usage of `umount` in `ISOSR` when `legacy_mode`
Subject: [PATCH 08/30] Avoid usage of `umount` in `ISOSR` when `legacy_mode`
is used

`umount` should not be called when `legacy_mode` is enabled, otherwise a mounted dir
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 1f67bfaffbd431c166a5e61322ea7d179ee4d687 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Wed, 18 May 2022 17:28:09 +0200
Subject: [PATCH 09/29] MooseFS SR uses now UUID subdirs for each SR
Subject: [PATCH 09/30] MooseFS SR uses now UUID subdirs for each SR

A sm-config boolean param `subdir` is available to configure where to store the VHDs:
- In a subdir with the SR UUID, the new behavior
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0010-Fix-is_open-call-for-many-drivers-25.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From d32bf0ccfe58c67ba87e9bab5b76f9eead92735d Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Thu, 23 Jun 2022 10:36:36 +0200
Subject: [PATCH 10/29] Fix is_open call for many drivers (#25)
Subject: [PATCH 10/30] Fix is_open call for many drivers (#25)

Ensure all shared drivers are imported in `_is_open` definition to register
them in the driver list. Otherwise this function always fails with a SRUnknownType exception.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 3c420bca8372e1441f977562b61439ff2235003f Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Thu, 23 Jun 2022 10:37:07 +0200
Subject: [PATCH 11/29] Remove SR_CACHING capability for many SR types (#24)
Subject: [PATCH 11/30] Remove SR_CACHING capability for many SR types (#24)

SR_CACHING offers the capacity to use IntelliCache, but this
feature is only available using NFS SR.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 471b7b2742da807aca012c6609c8917cbe786507 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Mon, 19 Sep 2022 10:31:00 +0200
Subject: [PATCH 12/29] Fix code coverage regarding MooseFSSR and ZFSSR (#29)
Subject: [PATCH 12/30] Fix code coverage regarding MooseFSSR and ZFSSR (#29)

Signed-off-by: Ronan Abhamon <[email protected]>
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 395daa86f0b4dcbc4eeabb42fed4f818d46afc36 Mon Sep 17 00:00:00 2001
From: Yann Dirson <[email protected]>
Date: Wed, 8 Mar 2023 10:13:18 +0100
Subject: [PATCH 13/29] py3: simple changes from futurize on XCP-ng drivers
Subject: [PATCH 13/30] py3: simple changes from futurize on XCP-ng drivers

* `except` syntax fixes
* drop `has_key()` usage
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From af592f003d7fb8bc2572455271bf143e97a10c81 Mon Sep 17 00:00:00 2001
From: Yann Dirson <[email protected]>
Date: Wed, 8 Mar 2023 10:28:10 +0100
Subject: [PATCH 14/29] py3: futurize fix of xmlrpc calls for CephFS,
Subject: [PATCH 14/30] py3: futurize fix of xmlrpc calls for CephFS,
GlusterFS, MooseFS, Linstore

Signed-off-by: Yann Dirson <[email protected]>
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0015-py3-use-of-integer-division-operator.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From e1335b1ca83771142e6a6f831b5b4aa888e2ff18 Mon Sep 17 00:00:00 2001
From: Yann Dirson <[email protected]>
Date: Wed, 8 Mar 2023 10:32:37 +0100
Subject: [PATCH 15/29] py3: use of integer division operator
Subject: [PATCH 15/30] py3: use of integer division operator

Guided by futurize's "old_div" use

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 2e7fa79a4d7d4ebbee00959326b8f8151a816889 Mon Sep 17 00:00:00 2001
From: Yann Dirson <[email protected]>
Date: Wed, 8 Mar 2023 13:53:21 +0100
Subject: [PATCH 16/29] test_on_slave: allow to work with SR using absolute
Subject: [PATCH 16/30] test_on_slave: allow to work with SR using absolute
PROBE_MOUNTPOINT

PROBE_MOUNTPOINT in a some drivers is a relative path, which is resolved
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0017-py3-switch-interpreter-to-python3.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 2df6ea2d37f2a117f70f844105db2771d9c88517 Mon Sep 17 00:00:00 2001
From: Yann Dirson <[email protected]>
Date: Mon, 27 Mar 2023 15:30:46 +0200
Subject: [PATCH 17/29] py3: switch interpreter to python3
Subject: [PATCH 17/30] py3: switch interpreter to python3

---
drivers/CephFSSR.py | 2 +-
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0018-Support-recent-version-of-coverage-tool.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 09aa72c4d2655ba1027dc831b1580bde87643447 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Thu, 4 May 2023 10:24:22 +0200
Subject: [PATCH 18/29] Support recent version of coverage tool (coverage
Subject: [PATCH 18/30] Support recent version of coverage tool (coverage
7.2.5)

Without these changes many warns/errors are emitted:
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0019-feat-LinstorSR-import-all-8.2-changes.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 98d8c31104c3eae9ea62ea106d09fb801a784f4e Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Fri, 20 Nov 2020 16:42:52 +0100
Subject: [PATCH 19/29] feat(LinstorSR): import all 8.2 changes
Subject: [PATCH 19/30] feat(LinstorSR): import all 8.2 changes

Signed-off-by: Ronan Abhamon <[email protected]>
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 1354e78a5409805f08f118910db32b89bfc6c780 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Fri, 30 Jun 2023 12:41:43 +0200
Subject: [PATCH 20/29] feat(LinstorSR): is now compatible with python 3
Subject: [PATCH 20/30] feat(LinstorSR): is now compatible with python 3

Signed-off-by: Ronan Abhamon <[email protected]>
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From d778817b78c72a04d419073ee8e98099e2a960a0 Mon Sep 17 00:00:00 2001
From: BenjiReis <[email protected]>
Date: Fri, 4 Aug 2023 12:10:37 +0200
Subject: [PATCH 21/29] Remove `SR_PROBE` from ZFS capabilities (#36)
Subject: [PATCH 21/30] Remove `SR_PROBE` from ZFS capabilities (#36)

The probe method is not implemented so we
shouldn't advertise it.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 18fe0f0b6c645b314d49b377602cc681167422e6 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Fri, 22 Sep 2023 11:11:27 +0200
Subject: [PATCH 22/29] Repair coverage to be compatible with 8.3 test env
Subject: [PATCH 22/30] Repair coverage to be compatible with 8.3 test env

Impacted drivers: LINSTOR, MooseFS and ZFS.
- Ignore all linstor.* members during coverage,
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0023-Support-IPv6-in-Ceph-Driver.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From e46f99246bd1c9d1e06e6dbc49b1d95dbaaf8eac Mon Sep 17 00:00:00 2001
From: BenjiReis <[email protected]>
Date: Mon, 25 Sep 2023 16:13:13 +0200
Subject: [PATCH 23/29] Support IPv6 in Ceph Driver
Subject: [PATCH 23/30] Support IPv6 in Ceph Driver

Signed-off-by: BenjiReis <[email protected]>
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 709b30f2d5d8c85c87da2f375a032a0d004b4298 Mon Sep 17 00:00:00 2001
From: Yann Dirson <[email protected]>
Date: Wed, 5 Jul 2023 16:57:26 +0200
Subject: [PATCH 24/29] lvutil: use wipefs not dd to clear existing signatures
Subject: [PATCH 24/30] lvutil: use wipefs not dd to clear existing signatures
(xapi-project#624)

Signed-off-by: Yann Dirson <[email protected]>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 47d5d8691b40f9c4bd8a77c00d851443e68ac887 Mon Sep 17 00:00:00 2001
From: Damien Thenot <[email protected]>
Date: Fri, 12 Apr 2024 15:08:59 +0200
Subject: [PATCH 25/29] feat(LargeBlock): introduce largeblocksr (#51)
Subject: [PATCH 25/30] feat(LargeBlock): introduce largeblocksr (#51)

A SR inheriting from a EXTSR allowing to use a 4KiB blocksize device as
SR.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From db0a6cb684e3a24ef1c47094154d19c03a6f0a5c Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Tue, 30 Jul 2024 15:31:23 +0200
Subject: [PATCH 26/29] feat(LVHDSR): add a way to modify config of LVMs (#60)
Subject: [PATCH 26/30] feat(LVHDSR): add a way to modify config of LVMs (#60)

With this change the driver supports a "lvm-conf" param on "other-config".
For now The configuration is only used by "remove" calls from LVMCache.
Expand Down
2 changes: 1 addition & 1 deletion SOURCES/0027-reflect-upstream-changes-in-our-tests.patch
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From df0d4cd2dcafb2d866ee121d8d7903711ffa4456 Mon Sep 17 00:00:00 2001
From: Benjamin Reis <[email protected]>
Date: Tue, 13 Aug 2024 11:11:39 +0200
Subject: [PATCH 27/29] reflect upstream changes in our tests
Subject: [PATCH 27/30] reflect upstream changes in our tests

Signed-off-by: Benjamin Reis <[email protected]>
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 006a3727a18eed16a581251553b829667efceefd Mon Sep 17 00:00:00 2001
From: Mark Syms <[email protected]>
Date: Fri, 30 Aug 2024 10:13:27 +0100
Subject: [PATCH 28/29] CA-398425: correctly check for multiple targets in
Subject: [PATCH 28/30] CA-398425: correctly check for multiple targets in
iSCSI

Signed-off-by: Mark Syms <[email protected]>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
From 93a8128e234613c4973d83bbb37c84d2a9cf5d87 Mon Sep 17 00:00:00 2001
From 6d2b2d1b0dbe44f52033949d0a2f758c89972a97 Mon Sep 17 00:00:00 2001
From: Ronan Abhamon <[email protected]>
Date: Mon, 5 Feb 2024 23:16:16 +0100
Subject: [PATCH 29/29] Synchronization with 8.2 LINSTOR before a stable
release
Date: Thu, 26 Sep 2024 18:53:04 +0200
Subject: [PATCH 29/30] Synchronization with 8.2 LINSTOR before a stable
release (#68)

Last commit: 9207abe1f2e1ff1795cdba1a0aeb76574412a583
"fix(linstor): check if resource is tiebreaker (#62)"

Signed-off-by: Ronan Abhamon <[email protected]>
---
dev_requirements.txt | 1 +
drivers/LinstorSR.py | 100 +++---
drivers/LinstorSR.py | 102 +++---
drivers/linstor-manager | 183 ++++++++---
drivers/linstorvhdutil.py | 60 +++-
drivers/linstorvolumemanager.py | 349 +++++++++++++++------
drivers/tapdisk-pause | 6 +-
{tests/mocks => mocks}/linstor/__init__.py | 0
7 files changed, 497 insertions(+), 202 deletions(-)
7 files changed, 499 insertions(+), 202 deletions(-)
rename {tests/mocks => mocks}/linstor/__init__.py (100%)

diff --git a/dev_requirements.txt b/dev_requirements.txt
Expand All @@ -29,7 +29,7 @@ index 104316ba..f25f7686 100644
bitarray
+python-linstor
diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py
index fe6d01d4..8b8be6c9 100755
index fe6d01d4..829c48f8 100755
--- a/drivers/LinstorSR.py
+++ b/drivers/LinstorSR.py
@@ -362,9 +362,6 @@ class LinstorSR(SR.SR):
Expand Down Expand Up @@ -247,7 +247,23 @@ index fe6d01d4..8b8be6c9 100755
)
raise xs_errors.XenError('VDIClone', opterr=str(e))

@@ -2739,7 +2748,7 @@ class LinstorVDI(VDI.VDI):
@@ -2550,6 +2559,7 @@ class LinstorVDI(VDI.VDI):
[FORK_LOG_DAEMON] + arguments,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
+ universal_newlines=True,
# Ensure we use another group id to kill this process without
# touch the current one.
preexec_fn=os.setsid
@@ -2635,6 +2645,7 @@ class LinstorVDI(VDI.VDI):
[FORK_LOG_DAEMON] + arguments,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
+ universal_newlines=True,
# Ensure we use another group id to kill this process without
# touch the current one.
preexec_fn=os.setsid
@@ -2739,7 +2750,7 @@ class LinstorVDI(VDI.VDI):

# 0. Fetch drbd path.
must_get_device_path = True
Expand All @@ -256,7 +272,7 @@ index fe6d01d4..8b8be6c9 100755
# We are on a slave, we must try to find a diskful locally.
try:
volume_info = self._linstor.get_volume_info(self.uuid)
@@ -2754,7 +2763,7 @@ class LinstorVDI(VDI.VDI):
@@ -2754,7 +2765,7 @@ class LinstorVDI(VDI.VDI):
must_get_device_path = hostname in volume_info.diskful

drbd_path = None
Expand All @@ -265,7 +281,7 @@ index fe6d01d4..8b8be6c9 100755
# If we are master, we must ensure we have a diskless
# or diskful available to init HA.
# It also avoid this error in xensource.log
@@ -2812,37 +2821,6 @@ class LinstorVDI(VDI.VDI):
@@ -2812,37 +2823,6 @@ class LinstorVDI(VDI.VDI):
self._kill_persistent_nbd_server(volume_name)
self._kill_persistent_http_server(volume_name)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
From e207c0a18ee7e35c92c2868bb5e047e94208d0f0 Mon Sep 17 00:00:00 2001
From: Damien Thenot <[email protected]>
Date: Tue, 26 Nov 2024 16:45:24 +0100
Subject: [PATCH 30/30] fix(cleanup.py): bad live coalesce check regarding
FileSR

The `VDI.canLiveCoalesce` method can manipulates sizes of different units because of this change:
```
CP-40871: use VHD allocation size in checking canLiveCoalesce
2f863b9fce6f2978499892d8c019bb3ab7ad72c5
```
As a result, the `canLiveCoalesce` method can return True and cause coalesce attempts
resulting in "Timed out" exceptions.

Only drivers deriving from `FileSR` are impacted.
The size of `self._sizeAllocated` is calculated correctly when `vhdutil.getAllocatedSize`
is called but there is a problematic case where `getVHDInfo` is used instead.
And this function does not convert `info.sizeAllocated` from block size to bytes.
This bug is caused by the call to `FileVDI.load` in cleanup.py.

Signed-off-by: Damien Thenot <[email protected]>
Co-authored-by: Ronan Abhamon <[email protected]>
---
drivers/vhdutil.py | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/vhdutil.py b/drivers/vhdutil.py
index 723f3af2..6a71804c 100755
--- a/drivers/vhdutil.py
+++ b/drivers/vhdutil.py
@@ -94,6 +94,11 @@ def ioretry(cmd, text=True):
errlist=[errno.EIO, errno.EAGAIN])


+def convertAllocatedSizeToBytes(size):
+ # Assume we have standard 2MB allocation blocks
+ return size * 2 * 1024 * 1024
+
+
def getVHDInfo(path, extractUuidFunction, includeParent=True, resolveParent=True):
"""Get the VHD info. The parent info may optionally be omitted: vhd-util
tries to verify the parent by opening it, which results in error if the VHD
@@ -118,7 +123,7 @@ def getVHDInfo(path, extractUuidFunction, includeParent=True, resolveParent=True
vhdInfo.parentUuid = extractUuidFunction(fields[nextIndex])
nextIndex += 1
vhdInfo.hidden = int(fields[nextIndex].replace("hidden: ", ""))
- vhdInfo.sizeAllocated = int(fields[nextIndex+1])
+ vhdInfo.sizeAllocated = convertAllocatedSizeToBytes(int(fields[nextIndex+1]))
vhdInfo.path = path
return vhdInfo

@@ -277,8 +282,7 @@ def setSizePhys(path, size, debug=True):
def getAllocatedSize(path):
cmd = [VHD_UTIL, "query", OPT_LOG_ERR, '-a', '-n', path]
ret = ioretry(cmd)
- # Assume we have standard 2MB allocation blocks
- return int(ret) * 2 * 1024 * 1024
+ return convertAllocatedSizeToBytes(int(ret))

def killData(path):
"zero out the disk (kill all data inside the VHD file)"
Loading

0 comments on commit b3dbc37

Please sign in to comment.