Project

General

Profile

Actions

Bug #18552

closed

Jinja2 UTF-8 rendering Problem

Added by Alexander Brunhirl over 3 years ago. Updated over 3 years ago.

Status:
Released
Priority:
N/A
Category:
System techniques
Target version:
Severity:
Critical - prevents main use of Rudder | no workaround | data loss | security
UX impact:
User visibility:
Operational - other Techniques | Rudder settings | Plugins
Effort required:
Priority:
74
Name check:
To do
Fix check:
Checked
Regression:

Description

Hi,

if a jinja2 template contains any non ASCI characters e.g. German umlaute the template rendering stopps.

Best regards,
Alex

node:~ # rudder agent run
Rudder agent 6.1.3.release
Node uuid: ff11c6ec-292d-4740-a513-0d8eb537788b
Start execution with config [20201111-144730-d9896543]

M| State         Technique                 Component                 Key                Message
   warning       fileTemplate              None                      /tmp/failed.jinja  Jinja templating failed with the following error:
E| repaired      fileTemplate              Load Template from a fil| /tmp/failed.jinja  The copy of the file /var/rudder/configuration-repository/shared-files/failed.jinja.tpl from the policy server to /var/rudder/tmp/templates/failed.jinja.tpl was repaired
E| repaired      fileTemplate              Load Template from a fil| /tmp/success.jinja The copy of the file /var/rudder/configuration-repository/shared-files/success.jinja.tpl from the policy server to /var/rudder/tmp/templates/success.jinja.tpl was repaired
E| error         fileTemplate              Expand template           /tmp/failed.jinja  The expansion of the template /var/rudder/tmp/templates/failed.jinja.tpl in /tmp/failed.jinja could not be repaired
E| repaired      fileTemplate              Expand template           /tmp/success.jinja The expansion of the template /var/rudder/tmp/templates/success.jinja.tpl in /tmp/success.jinja was repaired
E| error         fileTemplate              Put permissions           /tmp/failed.jinja  The application of the rights 600 on the file /tmp/failed.jinja, owner=root  group=root could not be repaired
E| repaired      fileTemplate              Put permissions           /tmp/success.jinja The application of the rights 600 on the file /tmp/success.jinja, owner=root  group=root was repaired

## Summary #####################################################################
Not all components were displayed because we are not in full compliance mode. Please run with -g to force full compliance mode.
   => 6 components in Enforce mode
      -> 4 repaired
      -> 2 error
Execution time: 21.12s
################################################################################
Reports sent.
node:~ # head /var/rudder/tmp/templates/*.jinja.tpl
==> /var/rudder/tmp/templates/failed.jinja.tpl <==
# Test comment: äeiöü
{{ vars.sys.fqhost }}

==> /var/rudder/tmp/templates/success.jinja.tpl <==
# Test comment: aeiou
{{ vars.sys.fqhost }}
node:~ # ls -l /tmp/*.jinja
-rw-------. 1 root root 42 Nov 11 14:54 /tmp/success.jinja
node:~ # head /tmp/*.jinja
# Test comment: aeiou
node.example.com
node:~ #

Related issues 1 (0 open1 closed)

Related to Rudder - Bug #18441: Jinja2 templating fails with python 3 with unicode dataReleasedFélix DALLIDETActions
Actions #1

Updated by François ARMAND over 3 years ago

  • Target version set to 6.1.7
  • Severity set to Critical - prevents main use of Rudder | no workaround | data loss | security
  • User visibility set to Operational - other Techniques | Rudder settings | Plugins
  • Priority changed from 0 to 76

That's... Interesting. UTF-8 chars seems OK but the expansion is not done aftwared. Wow.

Actions #2

Updated by Nicolas CHARLES over 3 years ago

I think this is a duplicate of https://issues.rudder.io/issues/18441

Could you test with the nightly Alex?

Actions #3

Updated by Nicolas CHARLES over 3 years ago

  • Related to Bug #18441: Jinja2 templating fails with python 3 with unicode data added
Actions #4

Updated by Alexander Brunhirl over 3 years ago

Nicolas CHARLES wrote in #note-2:

I think this is a duplicate of https://issues.rudder.io/issues/18441

Could you test with the nightly Alex?

Hi Nicolas,
I've tested this with an updated Rudder master and client ( version: 6.1.7.rc1.git202011120312 ).
But I got the same error message like before.

Best regards,
Alex


node:~ # rm /tmp/*.jinja /var/rudder/tmp/templates/*.jinja.tpl
node:~ # rudder agent run -qu
Rudder agent 6.1.7.rc1.git202011120312
Node uuid: ff11c6ec-292d-4740-a513-0d8eb537788b
ok: Rudder agent policies were updated.
Start execution with config [20201112-090959-f36c3121]

M| State         Technique                 Component                 Key                Message
   warning       fileTemplate              None                      /tmp/failed.jinja  Jinja templating failed with the following error:
E| repaired      fileTemplate              Load Template from a fil| /tmp/failed.jinja  The copy of the file /var/rudder/configuration-repository/shared-files/failed.jinja.tpl from the policy server to /var/rudder/tmp/templates/failed.jinja.tpl was repaired
E| repaired      fileTemplate              Load Template from a fil| /tmp/success.jinja The copy of the file /var/rudder/configuration-repository/shared-files/success.jinja.tpl from the policy server to /var/rudder/tmp/templates/success.jinja.tpl was repaired
E| error         fileTemplate              Expand template           /tmp/failed.jinja  The expansion of the template /var/rudder/tmp/templates/failed.jinja.tpl in /tmp/failed.jinja could not be repaired
E| repaired      fileTemplate              Expand template           /tmp/success.jinja The expansion of the template /var/rudder/tmp/templates/success.jinja.tpl in /tmp/success.jinja was repaired
E| error         fileTemplate              Put permissions           /tmp/failed.jinja  The application of the rights 600 on the file /tmp/failed.jinja, owner=root  group=root could not be repaired
E| repaired      fileTemplate              Put permissions           /tmp/success.jinja The application of the rights 600 on the file /tmp/success.jinja, owner=root  group=root was repaired

## Summary #####################################################################
Not all components were displayed because we are not in full compliance mode. Please run with -g to force full compliance mode.
   => 6 components in Enforce mode
      -> 4 repaired
      -> 2 error
Execution time: 22.82s
################################################################################
Reports sent.
node:~ #
Actions #5

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.1.7 to 6.1.8
Actions #6

Updated by Félix DALLIDET over 3 years ago

  • Status changed from New to In progress
  • Assignee set to Félix DALLIDET
Actions #7

Updated by Félix DALLIDET over 3 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Félix DALLIDET to Alexis Mousset
  • Pull Request set to https://github.com/Normation/ncf/pull/1277
  • Priority changed from 76 to 75
Actions #8

Updated by Félix DALLIDET over 3 years ago

  • Status changed from Pending technical review to Pending release

Applied in changeset ncf:commit:c15a4433d2d6274329bdcfa9022e63cb17d7909b.

Actions #9

Updated by François ARMAND over 3 years ago

  • Priority changed from 75 to 74
  • Fix check changed from To do to Checked
Actions #10

Updated by Vincent MEMBRÉ over 3 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 6.1.8 and 6.2.1 which were released today.

Actions

Also available in: Atom PDF