Ansible fails to find needed python modules when using Python 3.12 vs Python 3.11

[disaac]

Describe the bug

Ansible fails to deploy algo vpn when using Python 3.12 issue not reproducible on Python 3.11. Environment versions shown below.

ansible-playbook [core 2.13.13]
configured module search path = ['/home/sdgeek/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/sdgeek/algo-repo/.env/lib/python3.12/site-packages/ansible
ansible collection location = /home/sdgeek/.ansible/collections:/usr/share/ansible/collections
executable location = /home/sdgeek/algo-repo/.env/bin/ansible-playbook
python version = 3.12.0 (main, Oct 2 2023, 15:16:58) [GCC 11.4.0]
jinja version = 3.0.3
libyaml = True

To Reproduce

Steps to reproduce the behavior:

1. Inside repo run the following with python 3.12 used
2. create virtual environment and install requirements

python -m virtualenv --python="$(command -v python)" .env \
    && source .env/bin/activate \
    && python -m pip install -U pip virtualenv \
    && python -m pip install -r requirements.txt`

3. Activate virtualenv and deploy using ansible-playbook

ansible-playbook main.yml -e "provider=digitalocean
                                  server_name=sdgeek-algo
                                  ondemand_cellular=false
                                  ondemand_wifi=false
                                  dns_adblocking=false
                                  ssh_tunneling=true
                                  store_pki=false
                                  region=sfo3
                                  do_token=${DIGITALOCEAN_ACCESS_TOKEN}" || exit_status=$?

Expected behavior

Deployment would succeed as it does when using Python 3.11 but it fails.

Additional context

Issue is not reproducible when using latest version of python 3.11

Configuration that succeeds with Python 3.11

ansible-playbook [core 2.13.13]
config file = /home/sdgeek/algo-repo/ansible.cfg
configured module search path = ['/home/sdgeek/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/sdgeek/algo-repo/.env/lib/python3.11/site-packages/ansible
ansible collection location = /home/sdgeek/.ansible/collections:/usr/share/ansible/collections executable location = /home/sdgeek/algo-repo/.env/bin/ansible-playbook
python version = 3.11.6 (main, Oct 3 2023, 04:42:57) [GCC 11.4.0]
jinja version = 3.0.3
libyaml = True

Full log

PLAY [localhost] ***************************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [Playbook dir stat] *******************************************************
ok: [localhost]

TASK [Ensure Ansible is not being run in a world writable directory] ***********
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
[DEPRECATION WARNING]: Use 'ansible.utils.ipaddr' module instead. This feature 
will be removed from ansible.netcommon in a release after 2024-01-01. 
Deprecation warnings can be disabled by setting deprecation_warnings=False in 
ansible.cfg.
Warning: : The value '' is not a valid IP address or network, passing this
value to ipaddr filter might result in breaking change in future.

TASK [Ensure the requirements installed] ***************************************
ok: [localhost]

TASK [Set required ansible version as a fact] **********************************
ok: [localhost] => (item=ansible==6.1.0)

TASK [Just get the list from default pip] **************************************
ok: [localhost]

TASK [Verify Python meets Algo VPN requirements] *******************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [Verify Ansible meets Algo VPN requirements] ******************************
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}
Warning: : Found variable using reserved name: no_log

PLAY [Ask user for the input] **************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [Set facts based on the input] ********************************************
ok: [localhost]

TASK [Set facts based on the input] ********************************************
ok: [localhost]

PLAY [Provision the server] ****************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [Display the invocation environment] **************************************
changed: [localhost]

TASK [Install the requirements] ************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'pkg_resources'
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (setuptools) on fv-az245-440's Python /home/sdgeek/algo-repo/.env/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}

TASK [include_tasks] ***********************************************************
included: /home/sdgeek/algo-repo/playbooks/rescue.yml for localhost

TASK [debug] *******************************************************************
ok: [localhost] => {
    "fail_hint": [
        "Sorry, but something went wrong!",
        "Please check the troubleshooting guide.",
        "https://trailofbits.github.io/algo/troubleshooting.html"
    ]
}

TASK [Fail the installation] ***************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed as requested from task"}

PLAY RECAP *********************************************************************
localhost                  : ok=15   changed=1    unreachable=0    failed=1    skipped=8    rescued=1    ignored=0   

Ansible completed with an exit status of 2