Project

General

Profile

Actions

Bug #8826

closed

Reporting generated for rudder techniques has broken reporting promises ( log with 2 params )

Added by Dmitry Svyatogorov almost 5 years ago. Updated over 4 years ago.

Status:
Released
Priority:
N/A
Category:
Technique editor - API
Target version:
Severity:
User visibility:
Effort required:
Priority:

Description

After upgrade from 3.2.4 to 3.2.5 (ubuntu 14.04, dpkg from rudder repo, "apt-get upgrade"), each user technique brokes promices from been calculated.
Claims for "log_rudder" call with invalid number of arguments:

Policy update process was stopped due to an error:

⇨ Policy update error for process '972' at 2016-08-09 04:09:27
⇨ Cannot write configuration node
⇨ The generated promises are invalid

Show technical details

Technical details
⇨ cf-promise check fails for promises generated at '/var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community'
⇨ Command to check generated promises is: '/var/rudder/cfengine-community/bin/cf-promises -f /var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community/promises.cf'
⇨ /var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community/TZ_UTC/1.0/rudder_reporting.cf:7:0: error: Conflicting arity in calling bundle log_rudder, expected 4 arguments, 2 given
⇨ /var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community/TZ_UTC/1.0/rudder_reporting.cf:11:0: error: Conflicting arity in calling bundle log_rudder, expected 4 arguments, 2 given

I tried to save the user technique again under 3.2.5, but with no luck.
So, it seems, that some ("git pull"?) step is forbidden in package scripts.

And now, what I have to do to recover user techniques? (Workaround?)


Files


Related issues

Related to Rudder - Bug #8845: ncf.conf logger issues when updating from 3.1 to 3.2Rejected2016-08-15Actions
Related to Rudder - Bug #9416: Transient error during upgrade from 3.1 to 4.0RejectedActions
Actions #1

Updated by Dmitry Svyatogorov almost 5 years ago

Moreover: the new technique creation is now broken at all:

An Error occured! Unknown internal error during technique writing
Details: Cause: expected string or buffer Traceback (most recent call last): File "/usr/share/ncf/api/ncf_api_flask_app/views.py", line 154, in create_technique ncf.write_technique(technique,path) File "/usr/share/ncf/tools/ncf.py", line 606, in write_technique content = generate_technique_content(technique_metadata, methods) File "/usr/share/ncf/tools/ncf.py", line 499, in generate_technique_content check = ncf_constraints.check_parameter(arg, arg_constraint) File "/usr/share/ncf/tools/ncf_constraints.py", line 121, in check_parameter if re.search(r'[$@][{(][$@{(a-zA-Z0-9[\]_.-]+[})]', parameter_value) is not None : File "/usr/lib/python2.7/re.py", line 142, in search return _compile(pattern, flags).search(string) TypeError: expected string or buffer 

Actions #2

Updated by Vincent MEMBRÉ almost 5 years ago

Hello Dmitry, thanks for reporting this issue! and sorry about this ...

What is the version of the ncf package installed ?

Is technique 'TZ_UTC' a new technique or an existing technique ?

I'm gonna try to reproduce it

Actions #3

Updated by Dmitry Svyatogorov almost 5 years ago

TZ_UTC is old technique (but that I tried to resave).
Now I try to create completelly new technique as a hot workaround.

# dpkg -l | grep ncf
ii  ncf                                  1.0.0.201607292058-trusty1       all          ncf - CFEngine framework
ii  ncf-api-virtualenv                   3.2.5-trusty1                    all          ncf - CFEngine framework
  • The error in the comment above appears on saving e.g. "Command execution result" with not all fields filled. (So, it's bacause of invalid pre-check, not a total destroy.)
Actions #4

Updated by Vincent MEMBRÉ almost 5 years ago

Found the first issue (on logger arity), as stated in the error messages, it generate wrong bundle call for rudder_reproting, making a patch.

On the second one, in your new technique, what methods have you added ? I created techniques without problems, but did'nt tried with all methods ...

Actions #5

Updated by Vincent MEMBRÉ almost 5 years ago

Thanks, so you kept blank fields in some 'command execution result' and you got an error on saving ?

The error is ok (you cannot have blank fields in this methods) but the error message is really not helpful, and the UI should prevent you from saving

Actions #6

Updated by Vincent MEMBRÉ almost 5 years ago

I'll improve the error message and also the UI checks!

Actions #7

Updated by Vincent MEMBRÉ almost 5 years ago

as an easy workaround, you can modify that file /var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community/TZ_UTC/1.0/rudder_reporting.cf on line 7 and 11

The line should look line this

'log_rudder(param1, param2)'

You should change them to:

'log_rudder(param1, param2, "", "")'

Then launch an update on Rudder webapp

Actions #8

Updated by Vincent MEMBRÉ almost 5 years ago

I'll give you a patch to fix how those files are generated

Actions #9

Updated by Vincent MEMBRÉ almost 5 years ago

  • Project changed from Rudder to ncf
  • Category changed from Web - Technique editor to Technique editor - API
  • Assignee set to Vincent MEMBRÉ
  • Target version changed from 3.2.6 to master
Actions #10

Updated by Vincent MEMBRÉ almost 5 years ago

  • Status changed from New to In progress
Actions #11

Updated by Vincent MEMBRÉ almost 5 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Vincent MEMBRÉ to Jonathan CLARKE
  • Pull Request set to https://github.com/Normation/ncf/pull/416
Actions #12

Updated by Vincent MEMBRÉ almost 5 years ago

Here is the patch:

To apply it on your server Rudder:

cp 0001-Fixes-8826-Technique-Editor-is-broken-in-Rudder-3.2..patch  /usr/share/ncf
cd /usr/share/ncf
patch -p1 < 0001-Fixes-8826-Technique-Editor-is-broken-in-Rudder-3.2..patch 
service apache2 restart

Saving your technique will not break anymore your policy generation.

I'll open another issue on the error message/detection and will provide new package soon

Actions #13

Updated by Dmitry Svyatogorov almost 5 years ago

#find /var/rudder/share/ -name rudder_reporting.cf | wc -l
100

# find /var/rudder/share/ -name rudder_reporting.cf | xargs -I XXX sed -ri 's/(log_rudder\([^)]+)/\1, "", ""/g' XXX

… Oh, the patch is ready, I'll try it now.
Actions #14

Updated by Dmitry Svyatogorov almost 5 years ago

Ok, the patch works, but all user techniques have to be saved again (to implement ncf_rudder.py).
So, there although must be "runtime" patch to update existing directives (techniques) during rudder upgrade.

Actions #15

Updated by Vincent MEMBRÉ almost 5 years ago

Do your other rudder_reporting.cf file contains log_rudder with 2 parameters, log_rudder with 4 parameters or logger_rudder with 2 parameters?

First one is wrong (and was generated by our current cnf ...
Second one is what we should generate ( And I thought it was ok, but a merge broke this ...)
Third one is OK (use a deprecated method, but that what we use on version 3.1 of Rudder)

If you have first One then we need a migration process else other are OK for now (Though if we migrated third case automatically it whould be better ...)

Actions #16

Updated by Vincent MEMBRÉ almost 5 years ago

If I read my history correctly:

Rudder 3.2.5, generates log_rudder with 2 parameters => Wrong, corrected here
Rudder 3.2.4, generated log_rudder with 4 paramters => What we want ( was fixed in #8475, but broken by merge 04c18f )
Before Rudder 3.2.4, logger_rudder with 2 parameters => Ok

Actions #17

Updated by Dmitry Svyatogorov almost 5 years ago

On those which are saved in Technique editor after patch:

    !(ubuntu)::
      "dummy_report" usebundle => _classes_noop("command_execution_result_timedatectl___grep_Timezone___grep__q_UTC");
      "dummy_report" usebundle => log_rudder("Command execution result timedatectl | grep Timezone | grep -q UTC if ubuntu", "command_execution_result_timedatectl___grep_Timezone___grep__q_UTC", "", "");

ok.

On old ncf:

    !(ubuntu)::
      "dummy_report" usebundle => _classes_noop("file_ensure_block_in_section__etc_multipath_conf");
      "dummy_report" usebundle => log_rudder("File ensure block in section /etc/multipath.conf if ubuntu", "file_ensure_block_in_section__etc_multipath_conf");

causing the same error.

Actions #18

Updated by Vincent MEMBRÉ almost 5 years ago

  • Status changed from Pending technical review to Pending release
  • % Done changed from 0 to 100
Actions #19

Updated by François ARMAND over 4 years ago

  • Related to Bug #8845: ncf.conf logger issues when updating from 3.1 to 3.2 added
Actions #20

Updated by Vincent MEMBRÉ over 4 years ago

  • Subject changed from Technique Editor is broken in Rudder 3.2.5 to Reporting generated for rudder techniques has broken reporting promises ( log with 2 params )
Actions #21

Updated by Vincent MEMBRÉ over 4 years ago

  • Status changed from Pending release to Released

This bug has been fixed in ncf bundled with Rudder 3.2.6 which was released today.

Actions #22

Updated by Nicolas CHARLES over 4 years ago

  • Related to Bug #9416: Transient error during upgrade from 3.1 to 4.0 added
Actions

Also available in: Atom PDF