[BUG] Jinja2 3.0 support breaking changes #60188
Assignees
Labels
Bug
broken, incorrect, or confusing behavior
Regression
The issue is a bug that breaks functionality known to work in previous releases.
severity-high
2nd top severity, seen by most users, causes major problems
Silicon v3004.0
Release code name
Milestone
Comments
Oloremo
added
Feature
|
As mentioned in Slack: I'm wondering if this should actually be considered to be a bug rather than a feature request. We've started seeing failures across various formulas and I did a test run to determine that 32 of around 100 formulas are now failing: https://saltstack-formulas.zulipchat.com/#narrow/stream/239693-CI/topic/myii-ci.2F2021-W20d/search/fail. Here's an example of a PR that fixes the issue: saltstack-formulas/postgres-formula#311. I'm not so sure this will work everywhere, though. Should we patch our 32 formulas or should we wait for a resolution in the main Salt repo? |
|
I agree this is a bug. The I made a little test where
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
include:
- .direct-import
- .indirect-import
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{%- from "foo/libsaltcli.jinja" import cli %}
test-direct-import-of-cli:
test.nop:
- name: {{ cli }}
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{%- from "foo/test.jinja" import test_cli %}
test-indirect-import-of-cli:
test.nop:
- name: {{ test_cli }}
# -*- coding: utf-8 -*-
# vim: ft=jinja
{#- Get the relevant values from the `opts` dict #}
{%- set opts_cli = opts.get('__cli', '') %}
{%- set opts_masteropts_cli = opts | traverse('__master_opts__:__cli', '') %}
{#- Determine the type of salt command being run #}
{%- if opts_cli == 'salt-minion' %}
{%- set cli = 'minion' %}
{%- elif opts_cli == 'salt-call' %}
{%- set cli = 'ssh' if opts_masteropts_cli in ('salt-ssh', 'salt-master') else 'local' %}
{%- else %}
{%- set cli = 'unknown' %}
{%- endif %}
{%- do salt['log.debug']('[libsaltcli] the salt command type has been identified to be: ' ~ cli) %}
# -*- coding: utf-8 -*-
# vim: ft=jinja
{%- from "foo/libsaltcli.jinja" import cli %}
{%- set test_cli = cli %}DEBUG log |
It works if I modify --- foo/test.jinja.orig 2021-05-18 12:07:27.592339605 +0200
+++ foo/test.jinja 2021-05-18 12:06:25.054300769 +0200
@@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# vim: ft=jinja
-{%- from "foo/libsaltcli.jinja" import cli %}
+{%- from "foo/libsaltcli.jinja" import cli with context %}
{%- set test_cli = cli %} |
|
This is a major change since we may want to have the |
Merged
19 tasks
saltstack-formulas-github
pushed a commit
to netmanagers/salt-image-builder
that referenced
this issue
May 19, 2021
Various discussions about the underlying issue: * saltstack/salt#60188 * saltstack-formulas/postgres-formula#311 * saltstack-formulas/docker-formula#290
|
For the Formulas' team, we're rebuilding our CI images to force the downgrade of |
|
If/when this gets addressed, could we get #35398 addressed at the same time? |
sagetherage
added
Bug
sagetherage
changed the title
|
Now affecting openSUSE Tumbleweed |
saltstack-formulas-github
pushed a commit
to netmanagers/salt-image-builder
that referenced
this issue
Jul 22, 2021
Note, `Jinja2` wasn't downgraded, as per the upstream issue (saltstack/salt#60188) -- appears that this will need to be done for the `master` pre-salted boxes.
And now also affects our newly built pre-salted
Update: I've uploaded version |
|
@myii for openSUSE Tumbleweed users, they can download the following rpm to downgrade: And then add lock on it, so it won't get accidentally upgraded again. |
Thanks, @aboe76 -- we've already got that in the Salt image builder: However, the problem is that the history is only around for ~25 days. |
sagetherage
added
Phosphorus v3005.0
This was referenced Aug 22, 2021
|
I tried to look at the code to see if I can see something but it's a little too complex for my skills, I'm lost between I can see the grains, pillar and opts variables passed and keyword arguments merged as a context variable and used in the rendering call which is then passed as template globals. I'll try to find some time to setup a Jinja3 salt environment and see how I could follow the evolution of Regards. |
|
A minimal example to test
I added some logging added to some logssalt-call --local --versions-report |
|
As I don't fully understand the code, I'm wondering few things:
I don't really see how to ask jinja2 developpers what's going one without being rude to force them to look at salt code. I don't see obvious hints in Jinja2 3.0 changelog, maybe that message about the use of globals but as I don't masterise the code I'm just wondering ;-) That's all I can find. |
This was referenced Aug 30, 2021
baby-gnu
added a commit
to eole/template-formula
that referenced
this issue
Aug 31, 2021
A change in Jinja2 3.0 cause troubles in rendering templates where salt special variables are available only on first import but not subsequent ones (see saltstack/salt#60188) We could workaround that bug by explicitly import jinja libraries and load YAML files `with context`. * TEMPLATE/map.jinja: import `libmapstack.jinja` and `post-map.jinja` with context. * TEMPLATE/libmapstack.jinja: import `libmatchers.jinja` with context. Load YAML files with context. * TEMPLATE/libmatchers.jinja: import `libsaltcli.jinja` with context.
baby-gnu
added a commit
to eole/template-formula
that referenced
this issue
Aug 31, 2021
A change in Jinja2 3.0 cause troubles in rendering templates where salt special variables are available only on first import but not subsequent ones (see saltstack/salt#60188) We could workaround that bug by explicitly import jinja libraries and load YAML files `with context`. * TEMPLATE/map.jinja: import `libmapstack.jinja` and `post-map.jinja` with context. * TEMPLATE/libmapstack.jinja: import `libmatchers.jinja` with context. Load YAML files with context. * TEMPLATE/libmatchers.jinja: import `libsaltcli.jinja` with context.
19 tasks
|
Looks like the fix in #60811 solves this issue in additional to another one. |
3 tasks
garethgreenaway
added
Silicon v3004.0
|
This regression has been fixed going forward but there are still issues across various platforms, which I have shown a breakdown for in #60811 (comment).
|
saltstack-formulas-github
pushed a commit
to netmanagers/salt-image-builder
that referenced
this issue
Apr 3, 2022
Check for problems associated with this bug: * saltstack/salt#60188 - Affects use of `grains`, `opts`, `pillar` & `salt` in Jinja. Fix was first available in `3004`: * saltstack/salt#60811 Use tests to apply the Jinja2 downgrade only where necessary (and to ensure that it is actually working).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jinja2 3.0 was released a few days ago and seems like Salt can't work with it.
We have issues like this:
The text was updated successfully, but these errors were encountered: