[Enabler] [conftest.py] Hot fix for testing the collection in Ansible 2.17.1

[rexemin]

SUMMARY

Fixes #1670.

ISSUE TYPE

• Enabler Pull Request

COMPONENT NAME

• tests/conftest.py

ADDITIONAL INFORMATION

Instead of passing the environment variables through the interpreter string, now we decorate the Ansible engine function that computes the environment string.

Some changes were needed to fix a couple of tests. Most changes were related to escaping shell commands and were failures even when running on dev with Ansible 2.16. Changes directly related to this PR are the ones made to tests for zos_copy and zos_fetch, as well as the users.py helper.

Changes to the config

This changes how the config should look like in Jenkins. When testing on Ansible >=2.17.1, we need to remove the PYTHONSTDINENCODING variable and change the value of _CEE_RUNOPTS to "FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)" (notice how we will not use an extra pair of single quotes now).

The new config for Jenkins should be like this:

_BPXK_AUTOCVT: "ON"
_CEE_RUNOPTS: "FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
_TAG_REDIR_ERR: txt
_TAG_REDIR_IN: txt
_TAG_REDIR_OUT: txt
LANG: C
ZOAU_HOME: "ZOAUDIR"
ZOAU: "ZOAUDIR/bin"
ZOAU_ROOT: "ZOAUDIR"
LIBPATH: "ZOAUDIR/lib:PYTHONDIR/lib:/lib:/usr/lib:."
ZOAUTIL_DIR: "ZOAUDIR"
PYTHONPATH: "ZOAUDIR/lib"
PYTHON_HOME: "PYTHONDIR"
PYTHON: "PYTHONDIR/bin"
PATH: "ZOAUDIR/bin:PYTHONDIR/bin:/bin:/usr/sbin:/var/bin:/usr/lpp/java/java180/J8.0_64/bin"

When running Ansible 2.16 or 2.15, the only change needed is to remove the pair of single quotes from _CEE_RUNOPTS, and thus the config should look like this:

_BPXK_AUTOCVT: "ON"
_CEE_RUNOPTS: "FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
_TAG_REDIR_ERR: txt
_TAG_REDIR_IN: txt
_TAG_REDIR_OUT: txt
LANG: C
ZOAU_HOME: "ZOAUDIR"
ZOAU: "ZOAUDIR/bin"
ZOAU_ROOT: "ZOAUDIR"
LIBPATH: "ZOAUDIR/lib:PYTHONDIR/lib:/lib:/usr/lib:."
ZOAUTIL_DIR: "ZOAUDIR"
PYTHONPATH: "ZOAUDIR/lib"
PYTHON_HOME: "PYTHONDIR"
PYTHON: "PYTHONDIR/bin"
PATH: "ZOAUDIR/bin:PYTHONDIR/bin:/bin:/usr/sbin:/var/bin:/usr/lpp/java/java180/J8.0_64/bin"
PYTHONSTDINENCODING: "cp1047"

Running locally

To run locally, the same config should work, but in case you're getting garbage as output when using Ansible 2.16 or 2.15, try removing PYTHONSTDINENCODING and see if that fixes the issue.

[rexemin]

---

[rexemin]

---

[rexemin]

---

[rexemin]

[fernandofloresg]

@rexemin I see this PR is ready for review but no reviewers were assigned. I'm reviewing this, just one question, I see the screenshots are tested with 2.17.1, is this not breaking testing functionality for 2.15 and 2.16 ?

[rexemin]

Thanks for reviewing, it should still be on draft mode, that's a mistake on my part. I've been testing the changes for 2.16, but I still need to make sure it works fine with 2.15 too

[rexemin]

The whole test suite passed on 2.16.7. Still testing on 2.15 to make sure nothing breaks.

[AndreMarcel99]

Your change also cover a lot of warning we had on pytest, great

[rexemin]

The test suite was successful on Ansible 2.15.5.

[fernandofloresg]

lgtm

[ddimatos]

Hi @rexemin - I see this was approved, my questions are really just for peace of mind.

1. I see the shell commands are now using multi-line doc strings, is this because of the way the env vars are being injected into the engine? e.g. """dls "{0}.*" """.format(HLQ)) , I ask so that we ensure going forward the team is aware of this and if we end up later down the road addressing this differently we recall this change.

2. From what I can tell, it does not seem like your solution differentiates between either fixture, --zinventory-raw or --zinventory , so likely there is no impact to anything using --zinventory-raw like ce.py. Were you able to test ce.py , if not I could give it a test run.

3. Similar to question 2, the changes don't look like they will impact the use of the --zinventory-raw used by ce.py, not sure if its been tested.

        zoau_path = environment_vars.get("ZOAU_HOME")
        pythonpath = environment_vars.get("PYTHONPATH")

This was a good option to inject our vars, good solution.

[rexemin]

@ddimatos regarding point 1: not really, we had a PR that changed most commands to use multi-line strings a couple of months ago, but the ones I've changed in this PR were overlooked at the time. I think we have changed all commands in the test suite at this point.

For points 2 and 3: I have not tested with ce.py.