How to retain original image metadata properties?

[marbetschar]

How can I retain the original image metadata properties when I distribute images using lxd-image-server ...? I do have some custom logic, which does some magic based upon the image variant. But this property seems to get lost when the image is downloaded:

How to reproduce

I'm building different variants of the same distribution using distrobuilder and upload them to lxd-image-server using scp. After upload I'm executing lxd-image-server update to rebuild the json files. Then, I pull the images to a local lxd using the following commands:

$ lxc remote add desktops https://images.desktop-linuxcontainers.org --public --protocol simplestreams
$ lxc image copy desktops:ubuntu/focal/amd64/xfce local:

And display the image metadata with:

$ lxc image show 30ccdd7a5809
auto_update: false
properties:
  architecture: amd64
  description: ubuntu focal amd64 (20200603_11:06)
  os: ubuntu
  release: focal
  serial: "20200603_11:06"
public: false

As you can see, there is no variant property. However, if you download the lxd.tar.xz file, unpack it and look at the metadata.yaml its there (the description is different as well):

$ wget -O lxd.tar.xz https://images.desktop-linuxcontainers.org/images/ubuntu/focal/amd64/xfce/20200603_11:06/lxd.tar.xz
$ tar xf lxd.tar.xz
$ head -25 metadata.yaml
architecture: amd64
creation_date: 1591185795
expiry_date: 1593777795
properties:
  architecture: amd64
  description: Ubuntu focal amd64 (xfce) (20200603_1119)
  name: ubuntu-focal-amd64-xfce-20200603_1119
  os: ubuntu
  release: focal
  serial: "20200603_1119"
  variant: xfce
templates:
...

[marbetschar]

@carlosduelo I'm happy to provide a PR, but don't know where to start. Would be awesome if you can point me towards where to start, because this issue is a blocker for me.

FWIW: Executing the following commands show the image metadata is missing when requested through lxd-image-server - although it is part of the image itself stored on lxd-image-server as shown before:

$ lxc remote add desktops https://images.desktop-linuxcontainers.org --public --protocol simplestreams
$ lxc image show desktops:ubuntu/focal/amd64/xfce
auto_update: false
properties:
  architecture: amd64
  description: ubuntu focal amd64 (20200603_11:06)
  os: ubuntu
  release: focal
  serial: "20200603_11:06"
public: true

[mkg20001]

It seems like lxd-image-server generates it's own metadata based on the directory structure, thus completly ignoring the image's metadata

I think the best way would be to:

• add a flag to directly use the image's metadata.yaml (--honor-metadata-yaml / --use-metadata-from-image)
  • this would also mean the directory structure would just be arbitrary 1 folder=1 image
• when updating the metadata extract the metadata.yaml from each image, store it in a cache (or use cached version) and then write the list of images with their actual extracted metadata