Merge pull request #708 from mikedep333/release_3.7.7

Release 3.7.7

README.md

@@ -5,7 +5,7 @@ Pulp 3 Ansible Installer
 
 A collection of roles to install or upgrade Pulp 3. Can also reconfigure or add plugins to an existing install.
 
-See [https://pulp-installer.readthedocs.io](https://pulp-installer.readthedocs.io) for full documentation.
+See [https://docs.pulpproject.org/pulp_installer](https://docs.pulpproject.org/pulp_installer/) for full documentation.
 
 
 How to File an Issue

docs/CHANGES.md

@@ -13,6 +13,32 @@ Changelog
 
 <!-- TOWNCRIER -->
 
+3.7.7 (2021-07-28)
+==================
+
+
+Features
+--------
+
+- Add git repo and revision to pulcore and plugin installer.
+  [#6547](https://pulp.plan.io/issues/6547)
+
+
+Bugfixes
+--------
+
+- Fix the "markuppy" `pkg_resources.DistributionNotFound` error on the task
+  `pulp_common : Collect static content`.
+  This occurs when installing from RPM packages on EL8 (ever since EPEL8 released
+  python-tablib-3.0.0-1.el8 on approximately 2021-07-23).
+  [#9166](https://pulp.plan.io/issues/9166)
+- If you upgrade from older pulpcore to pulpcore 3.7 from RPMs, `pulp_common: Collect static content` may fail due to dynaconf being too old (3.0.0rc1 is older than 3.0.0 final). If this happens, you can now workaround it by setting `pulp_pkg_upgrade_all: true` (or upgrading the RPM "python3-dynaconf").
+  [#9181](https://pulp.plan.io/issues/9181)
+
+
+----
+
+
 3.7.6 (2021-04-29)
 ==================
 

docs/index.md

@@ -5,9 +5,9 @@ The Pulp 3 Ansible installer is a collection of Ansible roles that you can use t
 
 Each Ansible role installs and configures a component of Pulp.
 
-This version of the installer, 3.7.6, installs Pulp 3.7.6 specifically.
+This version of the installer, 3.7.7, installs Pulp 3.7.7 specifically.
 
-If run against an older version of Pulp 3, it will upgrade it to 3.7.6.
+If run against an older version of Pulp 3, it will upgrade it to 3.7.7.
 
 System Requirements
 -------------------
@@ -93,18 +93,18 @@ Recommended Workflows for Pulpcore & Plugin Versioning
 Initial installation:
 
 1. Make sure you are running the latest version of the installer, which installs the latest version
-   of pulpcore (3.7.6).
+   of pulpcore (3.7.7).
 1. Confirm that all the latest stable releases of your desired plugins are compatible with pulpcore
-   3.7.6, such as by reading the release announcement email thread for pulpcore 3.7.6, reading the
+   3.7.7, such as by reading the release announcement email thread for pulpcore 3.7.7, reading the
 plugins README, or as a last resort, reading their `setup.py`.
 1. Run `pulp_installer`.
 
 Upgrading your installation:
 
 1. Observe what is the latest version of `pulp_installer`, and what version of pulpcore it installed
-   (3.7.6).
+   (3.7.7).
 1. Confirm that all the latest stable releases of **currently installed** plugins are compatible
-   with pulpcore 3.7.6, such as by reading the release announcement email thread for pulpcore 3.7.6,
+   with pulpcore 3.7.7, such as by reading the release announcement email thread for pulpcore 3.7.7,
 reading the plugins README, or as a last resort, reading their setup.py.
 1. If they are not all compatible yet, **wait** for the plugins to be updated for
    compatibility.
@@ -116,9 +116,9 @@ reading the plugins README, or as a last resort, reading their setup.py.
 
 Initial installation:
 
-1. Observe the latest branch of `pulp_installer`, and what version of pulpcore it installs (3.7.6).
+1. Observe the latest branch of `pulp_installer`, and what version of pulpcore it installs (3.7.7).
 1. Confirm that all the latest stable releases of your desired plugins are compatible with pulpcore
-   3.7.6, such as by reading the release announcement email thread for pulpcore 3.7.6, reading the
+   3.7.7, such as by reading the release announcement email thread for pulpcore 3.7.7, reading the
 plugins README, or as a last resort, reading their setup.py.
 1. If they are not all compatible yet, try the last version of the installer that installs pulpcore
    3.6.z . Then confirm that there exist stable releases of your desired plugins that are compatible
@@ -130,9 +130,9 @@ with pulpcore 3.6.z. If there are none, try pulpcore 3.5.z, and repeate.
 Upgrading your install:
 
 1. Observe what the latest version of `pulp_installer` is, and what version of pulpcore it installed
-   (3.7.6). (Even if there is no update, you can still upgrade your plugins.)
+   (3.7.7). (Even if there is no update, you can still upgrade your plugins.)
 1. Confirm that all the latest stable releases of **currently installed** plugins are compatible
-   with pulpcore 3.7.6, such as by reading the release announcement email thread for pulpcore 3.7.6,
+   with pulpcore 3.7.7, such as by reading the release announcement email thread for pulpcore 3.7.7,
 reading the plugins README, or as a last resort, reading their setup.py.
 1. If they are not all compatible yet, try the last version of the installer that installs pulpcore
    3.6.z . Then confirm that there exist stable releases of your desired plugins that are compatible

docs/letsencrypt.md

@@ -36,7 +36,7 @@ HTTP 01 verification.
 The 3rd-party role listed below,
 [lexa-uw.letsencrypt](https://galaxy.ansible.com/lexa-uw/letsencrypt), is an example. The variables listed below are meant for it, and can serve as a guide for other roles.
 
-You can learn more about the variables on the [roles section](https://pulp-installer.readthedocs.io/en/latest/roles/pulp/#role-variables)
+You can learn more about the variables on the [roles section](https://docs.pulpproject.org/pulp_installer/roles/pulp_common/#role-variables)
 
 1 -  Install the `pulp_installer` collection:
 ```

docs/quickstart.md

@@ -31,7 +31,8 @@ they can be set in the playbook. See the Ansible docs for more flexible idiomati
 Example Playbook for Installing Plugins
 ---------------------------------------
 As an example, we are going to write a playbook for installing `pulp_container` and `pulp_rpm`.
-You can learn more about the variables on the [roles section](https://pulp-installer.readthedocs.io/en/latest/roles/pulp/#role-variables)
+
+You can learn more about the variables on the [roles section](https://docs.pulpproject.org/pulp_installer/roles/pulp_common/#role-variables)
 
 1 -  Install the `pulp_installer` collection:
 ```

galaxy.yml

@@ -10,7 +10,7 @@ namespace: pulp
 name: pulp_installer
 
 # The version of the collection. Must be compatible with semantic versioning
-version: 3.7.6
+version: 3.7.7
 
 # The path to the Markdown (.md) readme file. This path is relative to the root of the collection
 readme: README.md
@@ -58,7 +58,7 @@ dependencies: {}
 repository: https://github.com/pulp/pulp_installer
 
 # The URL to any online docs
-documentation: https://pulp-installer.readthedocs.io/
+documentation: https://docs.pulpproject.org/pulp_installer/
 
 # The URL to the homepage of the collection/project
 homepage: https://pulpproject.org/

mkdocs.yml

@@ -2,7 +2,7 @@
 site_name: Pulp Installer
 site_description: Pulp 3 Ansible Installer
 site_author: Pulp Team
-site_url: https://pulp-installer.readthedocs.io/
+site_url: https://docs.pulpproject.org/pulp_installer/
 repo_name: pulp/pulp_installer
 repo_url: https://github.com/pulp/pulp_installer
 theme: readthedocs

molecule/packages-dynamic/group_vars/all

@@ -8,4 +8,4 @@ pulp_install_plugins:
 pulp_settings:
   secret_key: secret
   content_origin: "https://{{ ansible_fqdn }}"
-pulp_pkg_repo: "https://yum.theforeman.org/pulpcore/3.6/el{{ ansible_distribution_major_version }}/$basearch/"
+pulp_pkg_repo: "https://yum.theforeman.org/pulpcore/3.7/el{{ ansible_distribution_major_version }}/$basearch/"

molecule/packages-static/group_vars/all

@@ -10,4 +10,4 @@ pulp_install_plugins:
 pulp_settings:
   secret_key: secret
   content_origin: "https://{{ ansible_fqdn }}"
-pulp_pkg_repo: "https://yum.theforeman.org/pulpcore/3.6/el{{ ansible_distribution_major_version }}/$basearch/"
+pulp_pkg_repo: "https://yum.theforeman.org/pulpcore/3.7/el{{ ansible_distribution_major_version }}/$basearch/"

molecule/packages-upgrade/group_vars/all

@@ -8,5 +8,5 @@ pulp_install_plugins:
 pulp_settings:
   secret_key: secret
   content_origin: "https://{{ ansible_fqdn }}"
-pulp_pkg_repo: "https://yum.theforeman.org/pulpcore/3.6/el{{ ansible_distribution_major_version }}/$basearch/"
+pulp_pkg_repo: "https://yum.theforeman.org/pulpcore/3.7/el{{ ansible_distribution_major_version }}/$basearch/"
 pulp_pkg_upgrade_all: true

molecule/release-dynamic/group_vars/all

@@ -13,7 +13,8 @@ pulp_install_plugins:
   # pulp-maven: {}
   # pulp-npm: {}
   # pulp-python: {}
-  pulp-rpm: {}
+  pulp-rpm:
+    version: 3.11.1
 pulp_settings:
   secret_key: secret
   content_origin: "https://{{ ansible_fqdn }}"

molecule/release-static/group_vars/all

@@ -15,7 +15,8 @@ pulp_install_plugins:
   # pulp-maven: {}
   # pulp-npm: {}
   # pulp-python: {}
-  pulp-rpm: {}
+  pulp-rpm:
+    version: 3.11.1
 pulp_settings:
   secret_key: secret
   content_origin: "https://{{ ansible_fqdn }}"

molecule/release-static/prepare.yml

@@ -7,6 +7,20 @@
         dest: /etc/systemd/system/redis-server.service
       when: ansible_distribution == "Debian"
 
+    # These 2 tasks exist in playbooks/example-use/playbook.yml , but we need
+    # them here so that yum/dnf commands can succeed.
+    - name: Check if pulpcore repo exists
+      stat:
+        path: /etc/yum.repos.d/pulpcore.repo
+      register: pulpcorerepo
+
+    - name: Remove pulpcore fedorapeople
+      lineinfile:
+        path: /etc/yum.repos.d/pulpcore.repo
+        regexp: 'fedorapeople'
+        line: "baseurl = {{ pulp_pkg_repo }}"
+      when: pulpcorerepo.stat.exists
+
     - name: Ensure dnf is not installed, which would conflict with ansible-pulp-rpm
       yum:
         name:
@@ -19,3 +33,27 @@
       when:
         - ansible_os_family == "RedHat"
         - ansible_distribution_major_version|int == 7
+
+    # Needed for the 1st time we run a package or apt task
+    - name: Update apt package index
+      apt:
+        update_cache: yes
+      when: ansible_facts.distribution == 'Debian' and ansible_facts.distribution_major_version == '10'
+      # We aren't really changing the system, so let's minimize the reporting of CHANGED tasks.
+      changed_when: False
+
+    # Probably needed due to a mismatch of systemd versions/capabilities between the Ubuntu GHA
+    # host and the container
+    - name: Update systemd to fix the Ansible systemd module, and inspec (verify phase)
+      package:
+        name: systemd
+        state: latest  # noqa 403
+      # Fixes ansible systemd module on Debian & Fedora, fixes inspec on EL8
+      when: (ansible_facts.distribution == 'Debian' and ansible_facts.distribution_major_version == '10') or
+            (ansible_facts.distribution == 'Fedora' and ansible_facts.distribution_major_version == '31') or
+            (ansible_facts.os_family == 'RedHat' and ansible_facts.distribution_major_version == '8')
+
+    # Possibly not necessary
+    - name: Make the systemd update take effect
+      systemd:
+        daemon_reexec: true

molecule/release-upgrade/group_vars/all

@@ -6,7 +6,7 @@ pulp_install_plugins:
   pulp-file:
     version: 1.6.1
   pulp-rpm:
-    upgrade: true
+    version: 3.11.1
 pulp_settings:
   secret_key: secret
   content_origin: "https://{{ ansible_fqdn }}"

molecule/source-dynamic/group_vars/all

@@ -1,6 +1,8 @@
 ---
 pulp_default_admin_password: password
 pulp_source_dir: '/var/lib/pulp/devel/pulpcore'
+pulp_git_url: "https://github.com/pulp/pulpcore"
+pulp_git_revision: "3.7"
 pulp_install_plugins:
   # galaxy-ng:
   #   source_dir: "/var/lib/pulp/devel/galaxy_ng"
@@ -15,6 +17,8 @@ pulp_install_plugins:
   # pulp-deb:
   #   source_dir: "/var/lib/pulp/devel/pulp_deb"
   pulp-file:
+    git_url: "https://github.com/pulp/pulp_file"
+    git_revision: "1.6"
     source_dir: "/var/lib/pulp/devel/pulp_file"
   # pulp-gem:
   #   source_dir: "/var/lib/pulp/devel/pulp_gem"
@@ -25,6 +29,8 @@ pulp_install_plugins:
   # pulp-python:
   #   source_dir: "/var/lib/pulp/devel/pulp_python"
   pulp-rpm:
+    git_url: "https://github.com/pulp/pulp_rpm"
+    git_revision: "3.11"
     source_dir: "/var/lib/pulp/devel/pulp_rpm"
 developer_user_home: /var/lib/pulp
 developer_user: pulp

molecule/source-static/group_vars/all

@@ -1,6 +1,8 @@
 ---
 pulp_default_admin_password: password
 pulp_source_dir: '/var/lib/pulp/devel/pulpcore'
+pulp_git_url: "https://github.com/pulp/pulpcore"
+pulp_git_revision: "3.7"
 pulp_api_bind: "unix:/var/run/pulpcore-api/pulpcore-api.sock"
 pulp_content_bind: "unix:/var/run/pulpcore-content/pulpcore-content.sock"
 pulp_install_plugins:
@@ -17,6 +19,8 @@ pulp_install_plugins:
   # pulp-deb:
   #   source_dir: "/var/lib/pulp/devel/pulp_deb"
   pulp-file:
+    git_url: "https://github.com/pulp/pulp_file"
+    git_revision: "1.6"
     source_dir: "/var/lib/pulp/devel/pulp_file"
   # pulp-gem:
   #   source_dir: "/var/lib/pulp/devel/pulp_gem"
@@ -27,6 +31,8 @@ pulp_install_plugins:
   # pulp-python:
   #   source_dir: "/var/lib/pulp/devel/pulp_python"
   pulp-rpm:
+    git_url: "https://github.com/pulp/pulp_rpm"
+    git_revision: "3.11"
     source_dir: "/var/lib/pulp/devel/pulp_rpm"
 developer_user_home: /var/lib/pulp
 developer_user: pulp

molecule/source-static/host_vars/debian-10

@@ -13,7 +13,9 @@ pulp_install_plugins:
   # pulp-deb:
   #   source_dir: "/var/lib/pulp/devel/pulp_deb"
   pulp-file:
+    git_url: "https://github.com/pulp/pulp_file"
     source_dir: "/var/lib/pulp/devel/pulp_file"
+    git_revision: "1.6"
   # pulp-gem:
   #   source_dir: "/var/lib/pulp/devel/pulp_gem"
   # pulp-maven:

molecule/source-static/prepare.yml

@@ -14,21 +14,6 @@
         home: '{{ developer_user_home }}'
       when: developer_user is defined
 
-    - name: Clone pulp repository
-      git:
-        repo: 'https://github.com/pulp/pulpcore.git'
-        dest: '{{ pulp_source_dir }}'
-        version: master
-        update: yes
-
-    - name: Clone pulp plugin repositories
-      git:
-        repo: '{{ item.value.git_repo | default( "https://github.com/pulp/" + item.key | replace("-", "_") ) }}'
-        dest: '{{ item.value.source_dir }}'
-        version: master
-        update: yes
-      with_dict: '{{ pulp_install_plugins }}'
-
     - name: Make devel directory world-writable
       file:
         path: /var/lib/pulp/

molecule/source-upgrade/group_vars/all

@@ -2,15 +2,21 @@
 __pulp_database_config_run_once: false
 pulp_default_admin_password: password
 pulp_source_dir: '/var/lib/pulp/devel/pulpcore'
+pulp_git_url: "https://github.com/pulp/pulpcore"
+pulp_git_revision: "3.7"
 pulp_upgrade: true
 # Needed to determine whether or not pulpcore was actually upgraded, to
 # trigger the handler.
 pulp_pip_editable: false
 pulp_install_plugins:
   pulp-file:
+    git_url: "https://github.com/pulp/pulp_file"
+    git_revision: "1.6"
     upgrade: true
     source_dir: "/var/lib/pulp/devel/pulp_file"
   pulp-rpm:
+    git_url: "https://github.com/pulp/pulp_rpm"
+    git_revision: "3.11"
     upgrade: true
     source_dir: "/var/lib/pulp/devel/pulp_rpm"
 developer_user_home: /var/lib/pulp

molecule/source-upgrade/host_vars/debian-10

@@ -2,6 +2,8 @@ ansible_python_interpreter: /usr/bin/python3
 pulp_install_plugins:
   pulp-file:
     upgrade: true
+    git_url: "https://github.com/pulp/pulp_file"
+    git_revision: "1.6"
     source_dir: "/var/lib/pulp/devel/pulp_file"
   # pulp-rpm:
   #   upgrade: true

roles/pulp_common/README.md

@@ -45,6 +45,10 @@ Role Variables
         source_dir: "/var/lib/pulp/pulp_three" # path to the plugin source code
       pulp-four:
         prereq_role: "pulp.pulp_four_role" # role to run immediately before the venv is created
+      pulp-five:
+        source_dir: "/path/to/source/code/"
+        git_url: "https://github..."  # Optional. URL to the git repo from where plugin will be pulled.
+        git_revision: "v3.1.1"   # Optional. The specific git branch/tag/commit to be cheked out.
     ```
 * `pulp_cache_dir`: Location of Pulp cache. Defaults to "/var/lib/pulp/tmp".
 * `pulp_config_dir`: Directory which will contain Pulp configuration files.
@@ -55,6 +59,15 @@ Role Variables
 * `pulp_source_dir`: Optional. Absolute path to pulpcore source code. If
   present, pulpcore will be installed from source in editable mode. Also accepts
   a pip VCS URL, to (for example) install the master branch.
+* `pulp_git_url`: Optional. URL to the git repository from where pulpcore will be checked out if
+  doesn't exists already on `source_dir`.
+  > **WARNING** when `pulp_git_url` is defined this role will clone the repo if doesn't already
+  > exist in the `pulp_source_dir` location and also checkout/update to specified `pulp_git_revision` if
+  > provided. In the case of existing local repo with unstaged changes the update will NOT be forced.
+  > For development purposes the recommendation is to NOT provide `pulp_git_url` and manage local
+  > branches manually.
+* `pulp_git_revision`: Optional. The specific git branch/tag/commit to be cheked out
+  if git_url is provided.
 * `pulp_user`: User that owns and runs Pulp. Defaults to "pulp".
 * `pulp_user_id`: Integer value of uid for the `pulp_user`. Defaults to nothing and uid is assigned
   by the system.