Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactoring interface common routines #1079

Draft
wants to merge 1 commit into
base: devel
Choose a base branch
from
Draft

Refactoring interface common routines #1079

wants to merge 1 commit into from

Conversation

matburt
Copy link
Member

@matburt matburt commented May 19, 2022

This should also fix up direct role execution when run from the command line using an execution environment.

Still need to sort out some tests and make sure the cli entrypoint is doing the right thing in all circumstances.

@matburt
Refactoring interface common routines
4fb5cfe 
This should also fix up direct role execution when run from the command line using an execution environment
@github-actions github-actions bot added the needs_triage New item that needs to be triaged label May 20, 2022
matburt
matburt commented May 20, 2022
View reviewed changes
ansible_runner/interface.py
@@ -52,10 +52,10 @@ def init_runner(**kwargs):
'''
# If running via the transmit-worker-process method, we must only extract things as read-only
# inside of one of these commands. That could be either transmit or worker.
if kwargs.get('streamer') not in ('worker', 'process'):
if kwargs.get('process_isolation') or kwargs.get('streamer') not in ('worker', 'process'):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lines are what I was really looking for in here. Trying to use direct role execution from the CLI using an execution environment didn't work because of the path assignment behavior.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoa, wait. Just on the surface of it, it seems like making this change would break the transmit phase of streaming runner.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... or I misread the logic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reconsidering, though, I'm not sure how it helps to do the dump_artifacts writing to disk in the worker and processor cases when we have process isolation? The worker should already have everything by unzipping everything that came over the wire from the transmit phase, and I don't think that the processor phase needs any of it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice, artifacts shouldn't be getting dumped because of suppress_env_files

@Akasurde Akasurde removed the needs_triage New item that needs to be triaged label May 31, 2022
@matburt matburt mentioned this pull request Jun 1, 2022
Merged
AlanCoding
AlanCoding reviewed Jun 1, 2022
View reviewed changes
ansible_runner/__main__.py
@@ -824,52 +721,82 @@ def main(sys_args=None):
if vargs.get('command') in ('transmit', 'worker', 'process'):
streamer = vargs.get('command')

# TODO: Remove / Refactor unused role context manager
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like it was removed?

@AlanCoding
Copy link
Member

I tested in conjunction with AWX / AAP controller, and normal jobs error with this from the worker:

{'private_data_dir': '/tmp/awx_91_2oshmw_9', 'ident': UUID('91c14a59-dd60-4e57-b589-cc430806c44b'), 'binary': None, 'playbook': None, 'module': None, 'module_args': None, 'host_pattern': None, 'verbosity': None, 'quiet': False, 'rotate_artifacts': 0, 'ignore_logging': False, 'json_mode': False, 'omit_event_data': False, 'only_failed_event_data': False, 'inventory': None, 'forks': None, 'project_dir': None, 'artifact_dir': None, 'role': None, 'roles_path': None, 'process_isolation': None, 'process_isolation_executable': None, 'process_isolation_path': None, 'process_isolation_hide_paths': None, 'process_isolation_show_paths': None, 'process_isolation_ro_paths': None, 'container_image': None, 'container_volume_mounts': None, 'container_options': None, 'directory_isolation_base_path': None, 'cmdline': None, 'limit': None, 'streamer': 'worker', 'suppress_env_files': False}
{"status": "starting", "runner_ident": "91", "command": ["podman", "run", "--rm", "--tty", "--interactive", "--workdir", "/runner/project", "-v", "/tmp/awx_91_2oshmw_9/:/runner/:Z", "-v", "/tmp/ansible_runner_plugins_ztvw3gwc/callback/:/home/runner/.ansible/plugins/callback/:Z", "--authfile=/tmp/ansible_runner_registry_91_sk4ys67n/auth.json", "-v", "/etc/pki/ca-trust/:/etc/pki/ca-trust/:O", "-v", "/opt/tmpawx/:/opt/tmpawx/:z", "-v", "/usr/share/pki/:/usr/share/pki/:O", "-v", "/var/lib/awx/projects/_80__project_texthat/:/var/lib/awx/projects/_80__project_texthat/:z", "-v", "/var/lib/awx/projects/.__awx_cache/_80__project_texthat/:/var/lib/awx/projects/.__awx_cache/_80__project_texthat/:z", "--env-file", "/tmp/awx_91_2oshmw_9/artifacts/91/env.list", "--quiet", "--name", "ansible_runner_91", "--user=root", "--network", "slirp4netns:enable_ipv6=true", "brew.registry.redhat.io/rh-osbs/ansible-automation-platform-22-ee-supported-rhel8:latest", "ansible-playbook", "-t", "update_git,install_roles,install_collections", "-i", "/runner/inventory/hosts", "-e", "@/runner/env/extravars", "main.json"], "env": {"ANSIBLE_UNSAFE_WRITES": "1", "AWX_ISOLATED_DATA_DIR": "/runner/artifacts/91", "ANSIBLE_FORCE_COLOR": "True", "ANSIBLE_HOST_KEY_CHECKING": "False", "ANSIBLE_INVENTORY_UNPARSED_FAILED": "True", "ANSIBLE_PARAMIKO_RECORD_HOST_KEYS": "False", "AWX_PRIVATE_DATA_DIR": "/tmp/awx_91_2oshmw_9", "ANSIBLE_RETRY_FILES_ENABLED": "False", "ANSIBLE_ASK_PASS": "False", "ANSIBLE_BECOME_ASK_PASS": "False", "DISPLAY": "", "TMP": "/tmp", "PROJECT_UPDATE_ID": "91", "ANSIBLE_ROLES_PATH": "roles", "ANSIBLE_STDOUT_CALLBACK": "awx_display", "RUNNER_OMIT_EVENTS": "False", "RUNNER_ONLY_FAILED_EVENTS": "False"}, "cwd": "/runner/project"}
{"status": "running", "runner_ident": "91"}
{"event": "verbose", "uuid": "1a72a67a-a609-4f4e-8178-ee6b772b7489", "counter": 1, "stdout": "\u001b[0;31mERROR! role definitions must contain a role name\u001b[0m", "start_line": 0, "end_line": 1, "runner_ident": "91", "created": "2022-06-01T20:38:57.916933"}
{"status": "failed", "runner_ident": "91"}
{"zipfile": 2110}
UEsDBBQAAAAIANqkwVQwb5Ix7AIAAJYGAAAHAAAAY29tbWFuZJVU70/bMBD9V1A/k2YwxKAS0tIkHVnTpEpSWDdNlpuaxjSJM9tpKYj/fef8QCkUbXyqz/feu+e7S596McsynC97g6NfvYIt4dw7PurxsvrRNJ7Vv1Lu6gPNJeE4lnRD6ost4+sl5SrQgZYTrhec3ZNYVvlNlZBZoePtA7o8QadMJNkWXeqDFj74+RqZC7pICarzqEjLFc0FepSb7efVNtZjnKYLHK9BImEZaXX6DU9vCB1cXUHDpUzuaEquDpXhZEWF5DtlUqzPduL8S64rRv9esLxjkchYL9YU5DXJSyHBxtsrv0NghVQFoQEA7QaPHVApuC4SzEklNHgdd/U2mOspXaiOtr0WOrr4hFATIUkeZIKVsw9gH/9Voo+QGmKM44R8qN5/EOviGsk3mprQu1uDuaR3WIlenuiA7qcwtJr7p6SkOeY4qyRejfjypE6XgvArzliLJlJtsQpESnlxBhe5GJAcKy4tNudXMNVKcMHJtt9uChyWYL5Pmc4TjYmFaHdKbRrLsKQs14oUyzvGM+30VCNEE2VRMC7JUuMJSS8GkCX1C1ouEHYLxio/WpUpiyWg0IrKY9hrCWuNOEuJeIlilqbQTignKhbtfo4035BcMr6Dr0XIGlA95muLgD7qMAiOYXpVPsM0r9f+N0TxVv0/HPq8gQiZp57hhc7QtdHMC42RjW4DJ7JDxakabtz+QE7ou0ZkW8gyIgNZTtBV7A61IjRyIz8wbWT6rl/Bo2YIbfraDyM0tufIvLbNseN9U6ARTsUeyvFubC/ygzm4mxpBCB5GhuPa1iFJABgTZ+yjwDb9wHopEe5Lw4OmgXMDD9p/z4GN7aoHdgQ2RlA8RLZnDBsTbywb4RichOHB5BCcTd7BWE44dY25ulNhNJm2tlQ4Dfzvthmh2dRSzp2q+H7HA195mxrRtcpVS9ZNh5HlzyJkGq47NMyxwqjnLqlQS6uQwczz7AD5EydCtmr8vsE27bnzZgpvUM/PfwFQSwMEFAAAAAgA2qTBVCLrAkISAQAA6AEAAAgAAABlbnYubGlzdGWQzU7DMBAG73mZCG49+LCxN9TEsa2105LTKoIiKvVPSavC2+OgNrRw9VjfjBZs0IVBbmyAEnlJOmIQDxksX1gHZyCiYgURWGkSeX/a7VZ93vXH9Xv3ehzyWfp6mSgdSWTpjCMR+9NqAnMXIlfYspyjrLR9EmW3GX65tgu00VGbKjxQSMYStEF1P5MQ1LpyTCgdqWk2XOdSsie9SMk3xcftIe/Onzx74Mf98LE982xaJIxJWiZVYLRQjMr7NAhV8obw57lIBfU/qnTwBlqRxdr/iDNP7hll5MarsUorcXMvcqPXQ5yLfr9ZDRMIUbkmsgRjCpCVGPPf1sNh031l1FiLxK7WkXE821V+Bda0l+vd8W9QSwMEFAAAAAgA26TBVLWxEZlpAQAAswQAABAAAABjb2xsZWN0aW9ucy5qc29uhZTdcoIwEIXv+xQO151FQNtpX8aJYZWtIaG7AbSO714UdShBewHk58tJds+GYxTXwrEUijFWVmhtMNbOGNSenJXb2GowFn3OjpEq1Y+zoNq+2yBLN9e1owxSmEen11l0XQvaWc/n9TxmFwFr0WtXlpfpkew8RFvHuzGYBJqVE9qHWDbCBHXN5A8hOd659mSCuFN4g+QP1pLNXZigBD5Gegdlt092ZRKvAF2gtLwzmkQ7UKIep61HKFSZQPaBU2PI7p8KGVfnMG1kCsk1Txtm6J4QuMnQuoRvVrl6wnzVlirkazEIdP1nJ9vV6wuM0h2PMdRNIbuArkLbSEteFzBoT0XTKzPmhfJAnafbwk+4Pof3IXgWlYI2/l9JLppQ7Q2WQ0aUR2PIBxFl9zoX5IY0WtcCeSmnLkQfulSmtjvAiRK/pdEz2rwkzQ5yxKq7Oo/Zpmy7vwv0nxWjTEa86NmDEzi/Hjh4Or38AlBLAwQUAAAACADcpMFUFAebPxgAAAAWAAAAEwAAAGFuc2libGVfdmVyc2lvbi50eHRLzCvOTMpJVYhOzi9KVTDSMzTWM4jlAgBQSwMEFAAAAAgA3aTBVHeji54IAAAABgAAAAYAAABzdGF0dXNLS8zMSU0BAFBLAwQUAAAACADdpMFUt+/cgwMAAAABAAAAAgAAAHJjMwQAUEsDBBQAAAAAANqkwVQAAAAAAAAAAAAAAAALAAAAam9iX2V2ZW50cy9QSwECFAMUAAAACADapMFUMG+SMewCAACWBgAABwAAAAAAAAAAAAAAgIEAAAAAY29tbWFuZFBLAQIUAxQAAAAIANqkwVQi6wJCEgEAAOgBAAAIAAAAAAAAAAAAAACkgREDAABlbnYubGlzdFBLAQIUAxQAAAAIANukwVS1sRGZaQEAALMEAAAQAAAAAAAAAAAAAACkgUkEAABjb2xsZWN0aW9ucy5qc29uUEsBAhQDFAAAAAgA3KTBVBQHmz8YAAAAFgAAABMAAAAAAAAAAAAAAKSB4AUAAGFuc2libGVfdmVyc2lvbi50eHRQSwECFAMUAAAACADdpMFUd6OLnggAAAAGAAAABgAAAAAAAAAAAAAAgIEpBgAAc3RhdHVzUEsBAhQDFAAAAAgA3aTBVLfv3IMDAAAAAQAAAAIAAAAAAAAAAAAAAICBVQYAAHJjUEsBAhQDFAAAAAAA2qTBVAAAAAAAAAAAAAAAAAsAAAAAAAAAAAAQAMBBeAYAAGpvYl9ldmVudHMvUEsFBgAAAAAHAAcAhwEAAKEGAAAAAA=={"eof": true}

In this case, it should be running a normal playbook, but it seems to somehow interpret that as direct role execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AlanCoding AlanCoding AlanCoding left review comments

@jbradberry jbradberry jbradberry left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@matburt @AlanCoding @jbradberry @Akasurde