Bug #8826
closedReporting generated for rudder techniques has broken reporting promises ( log with 2 params )
Description
After upgrade from 3.2.4 to 3.2.5 (ubuntu 14.04, dpkg from rudder repo, "apt-get upgrade"), each user technique brokes promices from been calculated.
Claims for "log_rudder" call with invalid number of arguments:
Policy update process was stopped due to an error: ⇨ Policy update error for process '972' at 2016-08-09 04:09:27 ⇨ Cannot write configuration node ⇨ The generated promises are invalid Show technical details Technical details ⇨ cf-promise check fails for promises generated at '/var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community' ⇨ Command to check generated promises is: '/var/rudder/cfengine-community/bin/cf-promises -f /var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community/promises.cf' ⇨ /var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community/TZ_UTC/1.0/rudder_reporting.cf:7:0: error: Conflicting arity in calling bundle log_rudder, expected 4 arguments, 2 given ⇨ /var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community/TZ_UTC/1.0/rudder_reporting.cf:11:0: error: Conflicting arity in calling bundle log_rudder, expected 4 arguments, 2 given
I tried to save the user technique again under 3.2.5, but with no luck.
So, it seems, that some ("git pull"?) step is forbidden in package scripts.
And now, what I have to do to recover user techniques? (Workaround?)
Files
Updated by Dmitry Svyatogorov over 8 years ago
Moreover: the new technique creation is now broken at all:
An Error occured! Unknown internal error during technique writing Details: Cause: expected string or buffer Traceback (most recent call last): File "/usr/share/ncf/api/ncf_api_flask_app/views.py", line 154, in create_technique ncf.write_technique(technique,path) File "/usr/share/ncf/tools/ncf.py", line 606, in write_technique content = generate_technique_content(technique_metadata, methods) File "/usr/share/ncf/tools/ncf.py", line 499, in generate_technique_content check = ncf_constraints.check_parameter(arg, arg_constraint) File "/usr/share/ncf/tools/ncf_constraints.py", line 121, in check_parameter if re.search(r'[$@][{(][$@{(a-zA-Z0-9[\]_.-]+[})]', parameter_value) is not None : File "/usr/lib/python2.7/re.py", line 142, in search return _compile(pattern, flags).search(string) TypeError: expected string or buffer
Updated by Vincent MEMBRÉ over 8 years ago
Hello Dmitry, thanks for reporting this issue! and sorry about this ...
What is the version of the ncf package installed ?
Is technique 'TZ_UTC' a new technique or an existing technique ?
I'm gonna try to reproduce it
Updated by Dmitry Svyatogorov over 8 years ago
TZ_UTC is old technique (but that I tried to resave).
Now I try to create completelly new technique as a hot workaround.
# dpkg -l | grep ncf ii ncf 1.0.0.201607292058-trusty1 all ncf - CFEngine framework ii ncf-api-virtualenv 3.2.5-trusty1 all ncf - CFEngine framework
- The error in the comment above appears on saving e.g. "Command execution result" with not all fields filled. (So, it's bacause of invalid pre-check, not a total destroy.)
Updated by Vincent MEMBRÉ over 8 years ago
Found the first issue (on logger arity), as stated in the error messages, it generate wrong bundle call for rudder_reproting, making a patch.
On the second one, in your new technique, what methods have you added ? I created techniques without problems, but did'nt tried with all methods ...
Updated by Vincent MEMBRÉ over 8 years ago
Thanks, so you kept blank fields in some 'command execution result' and you got an error on saving ?
The error is ok (you cannot have blank fields in this methods) but the error message is really not helpful, and the UI should prevent you from saving
Updated by Vincent MEMBRÉ over 8 years ago
I'll improve the error message and also the UI checks!
Updated by Vincent MEMBRÉ over 8 years ago
as an easy workaround, you can modify that file /var/rudder/share/42a34eff-213c-42fd-abf7-6fef5cca8d27/rules.new/cfengine-community/TZ_UTC/1.0/rudder_reporting.cf on line 7 and 11
The line should look line this
'log_rudder(param1, param2)'
You should change them to:
'log_rudder(param1, param2, "", "")'
Then launch an update on Rudder webapp
Updated by Vincent MEMBRÉ over 8 years ago
I'll give you a patch to fix how those files are generated
Updated by Vincent MEMBRÉ over 8 years ago
- Project changed from Rudder to 41
- Category changed from Web - Technique editor to Technique editor - API
- Assignee set to Vincent MEMBRÉ
- Target version changed from 3.2.6 to master
Updated by Vincent MEMBRÉ over 8 years ago
- Status changed from New to In progress
Updated by Vincent MEMBRÉ over 8 years ago
- Status changed from In progress to Pending technical review
- Assignee changed from Vincent MEMBRÉ to Jonathan CLARKE
- Pull Request set to https://github.com/Normation/ncf/pull/416
Updated by Vincent MEMBRÉ over 8 years ago
- File 0001-Fixes-8826-Technique-Editor-is-broken-in-Rudder-3.2..patch 0001-Fixes-8826-Technique-Editor-is-broken-in-Rudder-3.2..patch added
Here is the patch:
To apply it on your server Rudder:
cp 0001-Fixes-8826-Technique-Editor-is-broken-in-Rudder-3.2..patch /usr/share/ncf cd /usr/share/ncf patch -p1 < 0001-Fixes-8826-Technique-Editor-is-broken-in-Rudder-3.2..patch service apache2 restart
Saving your technique will not break anymore your policy generation.
I'll open another issue on the error message/detection and will provide new package soon
Updated by Dmitry Svyatogorov over 8 years ago
#find /var/rudder/share/ -name rudder_reporting.cf | wc -l 100 # find /var/rudder/share/ -name rudder_reporting.cf | xargs -I XXX sed -ri 's/(log_rudder\([^)]+)/\1, "", ""/g' XXX
… Oh, the patch is ready, I'll try it now.
Updated by Dmitry Svyatogorov over 8 years ago
Ok, the patch works, but all user techniques have to be saved again (to implement ncf_rudder.py).
So, there although must be "runtime" patch to update existing directives (techniques) during rudder upgrade.
Updated by Vincent MEMBRÉ over 8 years ago
Do your other rudder_reporting.cf file contains log_rudder with 2 parameters, log_rudder with 4 parameters or logger_rudder with 2 parameters?
First one is wrong (and was generated by our current cnf ...
Second one is what we should generate ( And I thought it was ok, but a merge broke this ...)
Third one is OK (use a deprecated method, but that what we use on version 3.1 of Rudder)
If you have first One then we need a migration process else other are OK for now (Though if we migrated third case automatically it whould be better ...)
Updated by Vincent MEMBRÉ over 8 years ago
If I read my history correctly:
Rudder 3.2.5, generates log_rudder with 2 parameters => Wrong, corrected here
Rudder 3.2.4, generated log_rudder with 4 paramters => What we want ( was fixed in #8475, but broken by merge commit:04c18f )
Before Rudder 3.2.4, logger_rudder with 2 parameters => Ok
Updated by Dmitry Svyatogorov over 8 years ago
On those which are saved in Technique editor after patch:
!(ubuntu):: "dummy_report" usebundle => _classes_noop("command_execution_result_timedatectl___grep_Timezone___grep__q_UTC"); "dummy_report" usebundle => log_rudder("Command execution result timedatectl | grep Timezone | grep -q UTC if ubuntu", "command_execution_result_timedatectl___grep_Timezone___grep__q_UTC", "", "");
ok.
On old ncf:
!(ubuntu):: "dummy_report" usebundle => _classes_noop("file_ensure_block_in_section__etc_multipath_conf"); "dummy_report" usebundle => log_rudder("File ensure block in section /etc/multipath.conf if ubuntu", "file_ensure_block_in_section__etc_multipath_conf");
causing the same error.
Updated by Vincent MEMBRÉ over 8 years ago
- Status changed from Pending technical review to Pending release
- % Done changed from 0 to 100
Applied in changeset commit:de7730486e047d47cd79b503d7c721cb79491d6a.
Updated by François ARMAND about 8 years ago
- Related to Bug #8845: ncf.conf logger issues when updating from 3.1 to 3.2 added
Updated by Vincent MEMBRÉ about 8 years ago
- Subject changed from Technique Editor is broken in Rudder 3.2.5 to Reporting generated for rudder techniques has broken reporting promises ( log with 2 params )
Updated by Vincent MEMBRÉ about 8 years ago
- Status changed from Pending release to Released
This bug has been fixed in ncf bundled with Rudder 3.2.6 which was released today.
Updated by Nicolas CHARLES about 8 years ago
- Related to Bug #9416: Transient error during upgrade from 3.1 to 4.0 added
Updated by Alexis Mousset over 2 years ago
- Target version changed from master to ncf-1.2
- Priority set to 0
Updated by Alexis Mousset over 2 years ago
- Project changed from 41 to Rudder
- Category changed from Technique editor - API to Web - Technique editor