Actions
Bug #16699
closed
Error messages while upgrading from 5.0.16.rc1.git202002060302-1.EL.7 to 6.0.3.rc1.git202002100103-1.EL.7
Pull Request:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
To do
Fix check:
To do
Regression:
Description
When upgrading the Rudder server from 5.0 nightly to 6.0 nightly, I get the following error message
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : audit-libs-2.8.5-4.el7.x86_64 1/29
Updating : 1398866025:rudder-agent-6.0.3.rc1.git202002100103-1.EL.7.x86_64 2/29
INFO: Scheduling an inventory during next run...
ls: cannot access /var/backups/rudder/agent.cert-*: No such file or directory
INFO: Agent certificate is missing, creating it... Done
INFO: No disable file detected and no agent executor process either. Restarting agent service...rudder-cf-serverd: enabled
rudder-cf-execd: enabled
ok: stop service rudder-agent succeeded
rudder-cf-serverd: enabled
rudder-cf-execd: enabled
ok: start service rudder-agent succeeded
Done
INFO: Inventory older than 3 days, resending ...Rudder agent 5.0.16.rc1.git2020020703136.0.3.rc1.git202002100103
Node uuid: root
error: Anomalous ending '
error: Anomalous ending '
error: Anomalous ending '
error: Anomalous ending '
error: Anomalous ending '
error: Anomalous ending '
error: Anomalous ending '
Start execution with config [20200207-103445-8c127128]
M| State Technique Component Key Message
E| compliant Inventory inventory The inventory has been successfully sent
info Rudder agent was run on a subset of policies - not all policies were checked
## Summary #####################################################################
1 components verified in 3 directives
=> 1 components in Enforce mode
-> 1 compliant
Execution time: 3.30s
################################################################################
Done
FINISH: Rudder agent check ran properly, please look at messages above to see if there has been any error.
Installing : audit-libs-python-2.8.5-4.el7.x86_64 3/29
Updating : policycoreutils-2.5-33.el7.x86_64 4/29
Installing : libcgroup-0.41-21.el7.x86_64 5/29
Installing : libsemanage-python-2.5-14.el7.x86_64 6/29
Updating : 1398866025:rudder-reports-6.0.3.rc1.git202002100103-1.EL.7.noarch 7/29
INFO: Setting PostgreSQL as a boot service... Done
INFO: Waiting for PostgreSQL to be up... Done
Installing : setools-libs-3.3.8-4.el7.x86_64 8/29
Installing : python-IPy-0.75-6.el7.noarch 9/29
Installing : checkpolicy-2.5-8.el7.x86_64 10/29
Installing : policycoreutils-python-2.5-33.el7.x86_64 11/29
Updating : 1398866025:rudder-server-relay-6.0.3.rc1.git202002100103-1.EL.7.x86_64 12/29
INFO: Setting Apache httpd as a boot service... Done
INFO: Creating users ... Done
INFO: Configuring systemd services... Done
Adding password for user rudder
Adding password for user rudder
INFO: Starting services... Done
INFO: rudder-server-relay setup complete.
getfacl: Removing leading '/' from absolute path names
Installing : 1398866025:rudder-webapp-6.0.3.rc1.git202002100103-1.EL.7.x86_64 13/29
warning: /opt/rudder/etc/openldap/slapd.conf created as /opt/rudder/etc/openldap/slapd.conf.rpmnew
warning: /opt/rudder/etc/rudder-web.properties created as /opt/rudder/etc/rudder-web.properties.rpmnew
warning: user rudder-slapd does not exist - using root
warning: group rudder-slapd does not exist - using root
INFO: Creating groups ...INFO: Creating users ... Done
INFO: Setting up systemd ...Created symlink from /etc/systemd/system/multi-user.target.wants/rudder-jetty.service to /usr/lib/systemd/system/rudder-jetty.service.
Created symlink from /etc/systemd/system/rudder-server.service.requires/rudder-jetty.service to /usr/lib/systemd/system/rudder-jetty.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/rudder-slapd.service to /usr/lib/systemd/system/rudder-slapd.service.
Created symlink from /etc/systemd/system/rudder-server.service.requires/rudder-slapd.service to /usr/lib/systemd/system/rudder-slapd.service.
INFO: Restarting rudder-slapd... Done
INFO: Launching script to check if a migration is needed ...
Done
enable /opt/rudder/share/plugins/scale-out-relay/scale-out-relay.jar
[]
Restarting jetty
$ service rudder-jetty restart
Stopping Jetty: OK
Setting umask to 0007
Starting Jetty: . . . . OK Mon Feb 10 15:27:18 UTC 2020
done
INFO: Make sure Rudder webapp is started... Done
INFO: Make sure Apache HTTPd is started... Done
INFO: Applying selinux policy... Done
Updating : 1398866025:rudder-server-root-6.0.3.rc1.git202002100103-1.EL.7.noarch 14/29
Created symlink from /etc/systemd/system/multi-user.target.wants/rudder-server.service to /usr/lib/systemd/system/rudder-server.service.
Updating : audit-2.8.5-4.el7.x86_64 15/29
Cleanup : 1398866025:rudder-server-root-5.0.16.rc1.git202002070313-1.EL.7.noarch 16/29
Cleanup : 1398866025:rudder-webapp-5.0.16.rc1.git202002070313-1.EL.7.noarch 17/29
warning: file /var/rudder/configuration-repository/ncf/ncf-hooks.d/post.write_technique.50_rudderify.sh: remove failed: No such file or directory
warning: file /var/rudder/configuration-repository/ncf/ncf-hooks.d/post.write_technique.10_commit.sh: remove failed: No such file or directory
warning: file /var/rudder/configuration-repository/ncf/ncf-hooks.d: remove failed: No such file or directory
Erasing : 1398866025:rudder-inventory-endpoint-5.0.16.rc1.git202002070313-1.EL.7.noarch 18/29
Erasing : 1398866025:ncf-api-virtualenv-5.0.16.rc1.git202002060302-1.EL.7.noarch 19/29
INFO: Removing the ncf-api-venv user... Done
INFO: Removing ncf-api-virtualenv selinux policy...libsemanage.semanage_direct_remove_key: Removing last ncf-api-virtualenv module (no other ncf-api-virtualenv module exists at another priority).
Done
Cleanup : 1398866025:rudder-server-relay-5.0.16.rc1.git202002070313-1.EL.7.x86_64 20/29
Erasing : 1398866025:rudder-techniques-5.0.16.rc1.git202002070313-1.EL.7.noarch 21/29
Cleanup : policycoreutils-2.5-29.el7_6.1.x86_64 22/29
Cleanup : audit-2.8.4-4.el7.x86_64 23/29
Erasing : 1398866025:ncf-5.0.16.rc1.git202002060302-1.EL.7.noarch 24/29
warning: file /usr/share/ncf/tools/ncf_constraints.pyc: remove failed: No such file or directory
INFO: A back up copy of the /opt/rudder/etc/uuid.hive has been created in /var/backups/rudder
INFO: A back up copy of the /var/rudder/cfengine-community/policy_server.dat has been created in /var/backups/rudder
INFO: A back up copy of the /var/rudder/cfengine-community/ppkeys has been created in /var/backups/rudder
Cleanup : 1398866025:rudder-agent-5.0.16.rc1.git202002070313-1.EL.7.x86_64 25/29
Erasing : 1398866025:rudder-inventory-ldap-5.0.16.rc1.git202002070313-1.EL.7.x86_64 26/29
Stopping Jetty: OK
Erasing : 1398866025:rudder-jetty-5.0.16.rc1.git202002070313-1.EL.7.noarch 27/29
Cleanup : 1398866025:rudder-reports-5.0.16.rc1.git202002070313-1.EL.7.noarch 28/29
Cleanup : audit-libs-2.8.4-4.el7.x86_64 29/29
Verifying : 1398866025:rudder-server-root-6.0.3.rc1.git202002100103-1.EL.7.noarch 1/29
Verifying : 1398866025:rudder-webapp-6.0.3.rc1.git202002100103-1.EL.7.x86_64 2/29
Verifying : audit-libs-python-2.8.5-4.el7.x86_64 3/29
Verifying : audit-libs-2.8.5-4.el7.x86_64 4/29
Verifying : checkpolicy-2.5-8.el7.x86_64 5/29
Verifying : python-IPy-0.75-6.el7.noarch 6/29
Verifying : setools-libs-3.3.8-4.el7.x86_64 7/29
Verifying : 1398866025:rudder-server-relay-6.0.3.rc1.git202002100103-1.EL.7.x86_64 8/29
Verifying : policycoreutils-python-2.5-33.el7.x86_64 9/29
Verifying : audit-2.8.5-4.el7.x86_64 10/29
Verifying : 1398866025:rudder-reports-6.0.3.rc1.git202002100103-1.EL.7.noarch 11/29
Verifying : libsemanage-python-2.5-14.el7.x86_64 12/29
Verifying : policycoreutils-2.5-33.el7.x86_64 13/29
Verifying : 1398866025:rudder-agent-6.0.3.rc1.git202002100103-1.EL.7.x86_64 14/29
Verifying : libcgroup-0.41-21.el7.x86_64 15/29
Verifying : 1398866025:rudder-jetty-5.0.16.rc1.git202002070313-1.EL.7.noarch 16/29
Verifying : 1398866025:ncf-api-virtualenv-5.0.16.rc1.git202002060302-1.EL.7.noarch 17/29
Verifying : 1398866025:rudder-server-relay-5.0.16.rc1.git202002070313-1.EL.7.x86_64 18/29
Verifying : policycoreutils-2.5-29.el7_6.1.x86_64 19/29
Verifying : audit-libs-2.8.4-4.el7.x86_64 20/29
Verifying : 1398866025:rudder-webapp-5.0.16.rc1.git202002070313-1.EL.7.noarch 21/29
Verifying : audit-2.8.4-4.el7.x86_64 22/29
Verifying : 1398866025:rudder-server-root-5.0.16.rc1.git202002070313-1.EL.7.noarch 23/29
Verifying : 1398866025:rudder-inventory-ldap-5.0.16.rc1.git202002070313-1.EL.7.x86_64 24/29
Verifying : 1398866025:ncf-5.0.16.rc1.git202002060302-1.EL.7.noarch 25/29
Verifying : 1398866025:rudder-inventory-endpoint-5.0.16.rc1.git202002070313-1.EL.7.noarch 26/29
Verifying : 1398866025:rudder-techniques-5.0.16.rc1.git202002070313-1.EL.7.noarch 27/29
Verifying : 1398866025:rudder-reports-5.0.16.rc1.git202002070313-1.EL.7.noarch 28/29
Verifying : 1398866025:rudder-agent-5.0.16.rc1.git202002070313-1.EL.7.x86_64 29/29
Installed:
rudder-webapp.x86_64 1398866025:6.0.3.rc1.git202002100103-1.EL.7
Dependency Installed:
audit-libs-python.x86_64 0:2.8.5-4.el7 checkpolicy.x86_64 0:2.5-8.el7 libcgroup.x86_64 0:0.41-21.el7 libsemanage-python.x86_64 0:2.5-14.el7 policycoreutils-python.x86_64 0:2.5-33.el7 python-IPy.noarch 0:0.75-6.el7
setools-libs.x86_64 0:3.3.8-4.el7
Updated:
rudder-agent.x86_64 1398866025:6.0.3.rc1.git202002100103-1.EL.7 rudder-reports.noarch 1398866025:6.0.3.rc1.git202002100103-1.EL.7 rudder-server-relay.x86_64 1398866025:6.0.3.rc1.git202002100103-1.EL.7
rudder-server-root.noarch 1398866025:6.0.3.rc1.git202002100103-1.EL.7
Dependency Updated:
audit.x86_64 0:2.8.5-4.el7 audit-libs.x86_64 0:2.8.5-4.el7 policycoreutils.x86_64 0:2.5-33.el7
Replaced:
ncf.noarch 1398866025:5.0.16.rc1.git202002060302-1.EL.7 ncf-api-virtualenv.noarch 1398866025:5.0.16.rc1.git202002060302-1.EL.7 rudder-inventory-endpoint.noarch 1398866025:5.0.16.rc1.git202002070313-1.EL.7
rudder-inventory-ldap.x86_64 1398866025:5.0.16.rc1.git202002070313-1.EL.7 rudder-jetty.noarch 1398866025:5.0.16.rc1.git202002070313-1.EL.7 rudder-techniques.noarch 1398866025:5.0.16.rc1.git202002070313-1.EL.7
Complete!
Updated by Nicolas CHARLES almost 5 years ago
Note that the scale-out-relay plugin was not disabled.
Webapp logs are thus full of errors
Updated by Nicolas CHARLES almost 5 years ago
Upgrading scale out relay plugin didn't fix the issue
Needed to restart slapd
at this point I have the errors
2020-02-10 15:45:57+0000] INFO inventory-processing - Watch new inventory file 'agent2-7b1378ec-cab0-4a38-a4fb-668402a169f0.ocs' with signature available: process.
[2020-02-10 15:46:00+0000] INFO policy.generation.timing - Policy generation failed after: 7 s
[2020-02-10 15:46:00+0000] ERROR policy.generation - Error when updating policy, reason was: Cannot write nodes configuration <- Unexpected: Error when executing hooks:
Exit code=1 for hook: '/opt/rudder/etc/hooks.d/policy-generation-node-ready/10-cf-promise-check'. [stdout: error: There are syntax errors in policy files
][stderr:/var/rudder/share/2e7b6735-cb4a-451d-b5b3-b729bf468a1b/rules.new/cfengine-community/promises.cf:735:15: error: syntax error
"${str}" not_matching => "true";
^
and
[2020-02-10 15:46:12+0000] WARN bootchecks - Could not authenticate in ncf API, maybe it was not initialized yet, retrying in 5 seconds, details Failure during authentication to ncf api: code 500: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <title>500 Internal Server Error</title> <h1>Internal Server Error</h1> <p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p> [2020-02-10 15:46:17+0000] WARN bootchecks - Could not authenticate in ncf API, maybe it was not initialized yet, retrying in 5 seconds, details Failure during authentication to ncf api: code 500: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <title>500 Internal Server Error</title> <h1>Internal Server Error</h1> <p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
Updated by Nicolas CHARLES almost 5 years ago
- Related to Bug #16707: After upgrade from 5.0 to 6.0 on centos7, rudder-slapd is not started added
Updated by Vincent MEMBRÉ almost 5 years ago
- Target version changed from 6.0.3 to 6.0.4
Updated by Vincent MEMBRÉ almost 5 years ago
- Target version changed from 6.0.4 to 6.0.5
Updated by Vincent MEMBRÉ over 4 years ago
- Target version changed from 6.0.5 to 6.0.6
Updated by Vincent MEMBRÉ over 4 years ago
- Target version changed from 6.0.6 to 6.0.7
Updated by Vincent MEMBRÉ over 4 years ago
- Target version changed from 6.0.7 to 6.0.8
Updated by François ARMAND over 4 years ago
- Status changed from New to Resolved
We corrected a lot of error for 6.1.1. So I'm closing that ticket and we will reopen ticket if needed.
Actions