Project

General

Profile

Bug #16699

Error messages while upgrading from 5.0.16.rc1.git202002060302-1.EL.7 to 6.0.3.rc1.git202002100103-1.EL.7

Added by Nicolas CHARLES 5 months ago. Updated 11 days ago.

Status:
New
Priority:
N/A
Assignee:
-
Category:
Packaging
Target version:
Pull Request:
Severity:
User visibility:
Effort required:
Priority:
0

Description

When upgrading the Rudder server from 5.0 nightly to 6.0 nightly, I get the following error message

Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                             1/29 
  Updating   : 1398866025:rudder-agent-6.0.3.rc1.git202002100103-1.EL.7.x86_64                                                                                                                                                           2/29 
INFO: Scheduling an inventory during next run...
ls: cannot access /var/backups/rudder/agent.cert-*: No such file or directory
INFO: Agent certificate is missing, creating it... Done
INFO: No disable file detected and no agent executor process either. Restarting agent service...rudder-cf-serverd: enabled
rudder-cf-execd: enabled
ok: stop service rudder-agent succeeded
rudder-cf-serverd: enabled
rudder-cf-execd: enabled
ok: start service rudder-agent succeeded
 Done
INFO: Inventory older than 3 days, resending ...Rudder agent 5.0.16.rc1.git2020020703136.0.3.rc1.git202002100103
Node uuid: root
   error: Anomalous ending '
   error: Anomalous ending '
   error: Anomalous ending '
   error: Anomalous ending '
   error: Anomalous ending '
   error: Anomalous ending '
   error: Anomalous ending '
Start execution with config [20200207-103445-8c127128]

M| State         Technique                 Component                 Key                Message
E| compliant     Inventory                 inventory                                    The inventory has been successfully sent
info     Rudder agent was run on a subset of policies - not all policies were checked

## Summary #####################################################################
1 components verified in 3 directives
   => 1 components in Enforce mode
      -> 1 compliant
Execution time: 3.30s
################################################################################
 Done
FINISH: Rudder agent check ran properly, please look at messages above to see if there has been any error.
  Installing : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                      3/29 
  Updating   : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                         4/29 
  Installing : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                              5/29 
  Installing : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                      6/29 
  Updating   : 1398866025:rudder-reports-6.0.3.rc1.git202002100103-1.EL.7.noarch                                                                                                                                                         7/29 
INFO: Setting PostgreSQL as a boot service... Done
INFO: Waiting for PostgreSQL to be up... Done
  Installing : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                           8/29 
  Installing : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                              9/29 
  Installing : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                             10/29 
  Installing : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                 11/29 
  Updating   : 1398866025:rudder-server-relay-6.0.3.rc1.git202002100103-1.EL.7.x86_64                                                                                                                                                   12/29 
INFO: Setting Apache httpd as a boot service... Done
INFO: Creating users ... Done
INFO: Configuring systemd services... Done
Adding password for user rudder
Adding password for user rudder
INFO: Starting services... Done
INFO: rudder-server-relay setup complete.
getfacl: Removing leading '/' from absolute path names
  Installing : 1398866025:rudder-webapp-6.0.3.rc1.git202002100103-1.EL.7.x86_64                                                                                                                                                         13/29 
warning: /opt/rudder/etc/openldap/slapd.conf created as /opt/rudder/etc/openldap/slapd.conf.rpmnew
warning: /opt/rudder/etc/rudder-web.properties created as /opt/rudder/etc/rudder-web.properties.rpmnew
warning: user rudder-slapd does not exist - using root
warning: group rudder-slapd does not exist - using root
INFO: Creating groups ...INFO: Creating users ... Done
INFO: Setting up systemd ...Created symlink from /etc/systemd/system/multi-user.target.wants/rudder-jetty.service to /usr/lib/systemd/system/rudder-jetty.service.
Created symlink from /etc/systemd/system/rudder-server.service.requires/rudder-jetty.service to /usr/lib/systemd/system/rudder-jetty.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/rudder-slapd.service to /usr/lib/systemd/system/rudder-slapd.service.
Created symlink from /etc/systemd/system/rudder-server.service.requires/rudder-slapd.service to /usr/lib/systemd/system/rudder-slapd.service.
INFO: Restarting rudder-slapd... Done
INFO: Launching script to check if a migration is needed ...
Done
enable /opt/rudder/share/plugins/scale-out-relay/scale-out-relay.jar
[]
Restarting jetty
 $ service rudder-jetty restart
Stopping Jetty: OK
Setting umask to 0007
Starting Jetty: . . . . OK Mon Feb 10 15:27:18 UTC 2020
done
INFO: Make sure Rudder webapp is started... Done
INFO: Make sure Apache HTTPd is started... Done
INFO: Applying selinux policy... Done
  Updating   : 1398866025:rudder-server-root-6.0.3.rc1.git202002100103-1.EL.7.noarch                                                                                                                                                    14/29 
Created symlink from /etc/systemd/system/multi-user.target.wants/rudder-server.service to /usr/lib/systemd/system/rudder-server.service.
  Updating   : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                 15/29 
  Cleanup    : 1398866025:rudder-server-root-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                   16/29 
  Cleanup    : 1398866025:rudder-webapp-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                        17/29 
warning: file /var/rudder/configuration-repository/ncf/ncf-hooks.d/post.write_technique.50_rudderify.sh: remove failed: No such file or directory
warning: file /var/rudder/configuration-repository/ncf/ncf-hooks.d/post.write_technique.10_commit.sh: remove failed: No such file or directory
warning: file /var/rudder/configuration-repository/ncf/ncf-hooks.d: remove failed: No such file or directory
  Erasing    : 1398866025:rudder-inventory-endpoint-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                            18/29 
  Erasing    : 1398866025:ncf-api-virtualenv-5.0.16.rc1.git202002060302-1.EL.7.noarch                                                                                                                                                   19/29 
INFO: Removing the ncf-api-venv user... Done
INFO: Removing ncf-api-virtualenv selinux policy...libsemanage.semanage_direct_remove_key: Removing last ncf-api-virtualenv module (no other ncf-api-virtualenv module exists at another priority).
 Done
  Cleanup    : 1398866025:rudder-server-relay-5.0.16.rc1.git202002070313-1.EL.7.x86_64                                                                                                                                                  20/29 
  Erasing    : 1398866025:rudder-techniques-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                    21/29 
  Cleanup    : policycoreutils-2.5-29.el7_6.1.x86_64                                                                                                                                                                                    22/29 
  Cleanup    : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                 23/29 
  Erasing    : 1398866025:ncf-5.0.16.rc1.git202002060302-1.EL.7.noarch                                                                                                                                                                  24/29 
warning: file /usr/share/ncf/tools/ncf_constraints.pyc: remove failed: No such file or directory
INFO: A back up copy of the /opt/rudder/etc/uuid.hive has been created in /var/backups/rudder
INFO: A back up copy of the /var/rudder/cfengine-community/policy_server.dat has been created in /var/backups/rudder
INFO: A back up copy of the /var/rudder/cfengine-community/ppkeys has been created in /var/backups/rudder
  Cleanup    : 1398866025:rudder-agent-5.0.16.rc1.git202002070313-1.EL.7.x86_64                                                                                                                                                         25/29 
  Erasing    : 1398866025:rudder-inventory-ldap-5.0.16.rc1.git202002070313-1.EL.7.x86_64                                                                                                                                                26/29 
Stopping Jetty: OK
  Erasing    : 1398866025:rudder-jetty-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                         27/29 
  Cleanup    : 1398866025:rudder-reports-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                       28/29 
  Cleanup    : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                            29/29 
  Verifying  : 1398866025:rudder-server-root-6.0.3.rc1.git202002100103-1.EL.7.noarch                                                                                                                                                     1/29 
  Verifying  : 1398866025:rudder-webapp-6.0.3.rc1.git202002100103-1.EL.7.x86_64                                                                                                                                                          2/29 
  Verifying  : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                      3/29 
  Verifying  : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                             4/29 
  Verifying  : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                              5/29 
  Verifying  : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                              6/29 
  Verifying  : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                           7/29 
  Verifying  : 1398866025:rudder-server-relay-6.0.3.rc1.git202002100103-1.EL.7.x86_64                                                                                                                                                    8/29 
  Verifying  : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                  9/29 
  Verifying  : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                 10/29 
  Verifying  : 1398866025:rudder-reports-6.0.3.rc1.git202002100103-1.EL.7.noarch                                                                                                                                                        11/29 
  Verifying  : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                     12/29 
  Verifying  : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                        13/29 
  Verifying  : 1398866025:rudder-agent-6.0.3.rc1.git202002100103-1.EL.7.x86_64                                                                                                                                                          14/29 
  Verifying  : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                             15/29 
  Verifying  : 1398866025:rudder-jetty-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                         16/29 
  Verifying  : 1398866025:ncf-api-virtualenv-5.0.16.rc1.git202002060302-1.EL.7.noarch                                                                                                                                                   17/29 
  Verifying  : 1398866025:rudder-server-relay-5.0.16.rc1.git202002070313-1.EL.7.x86_64                                                                                                                                                  18/29 
  Verifying  : policycoreutils-2.5-29.el7_6.1.x86_64                                                                                                                                                                                    19/29 
  Verifying  : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                            20/29 
  Verifying  : 1398866025:rudder-webapp-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                        21/29 
  Verifying  : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                 22/29 
  Verifying  : 1398866025:rudder-server-root-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                   23/29 
  Verifying  : 1398866025:rudder-inventory-ldap-5.0.16.rc1.git202002070313-1.EL.7.x86_64                                                                                                                                                24/29 
  Verifying  : 1398866025:ncf-5.0.16.rc1.git202002060302-1.EL.7.noarch                                                                                                                                                                  25/29 
  Verifying  : 1398866025:rudder-inventory-endpoint-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                            26/29 
  Verifying  : 1398866025:rudder-techniques-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                    27/29 
  Verifying  : 1398866025:rudder-reports-5.0.16.rc1.git202002070313-1.EL.7.noarch                                                                                                                                                       28/29 
  Verifying  : 1398866025:rudder-agent-5.0.16.rc1.git202002070313-1.EL.7.x86_64                                                                                                                                                         29/29 

Installed:
  rudder-webapp.x86_64 1398866025:6.0.3.rc1.git202002100103-1.EL.7                                                                                                                                                                            

Dependency Installed:
  audit-libs-python.x86_64 0:2.8.5-4.el7    checkpolicy.x86_64 0:2.5-8.el7    libcgroup.x86_64 0:0.41-21.el7    libsemanage-python.x86_64 0:2.5-14.el7    policycoreutils-python.x86_64 0:2.5-33.el7    python-IPy.noarch 0:0.75-6.el7   
  setools-libs.x86_64 0:3.3.8-4.el7        

Updated:
  rudder-agent.x86_64 1398866025:6.0.3.rc1.git202002100103-1.EL.7                 rudder-reports.noarch 1398866025:6.0.3.rc1.git202002100103-1.EL.7           rudder-server-relay.x86_64 1398866025:6.0.3.rc1.git202002100103-1.EL.7          
  rudder-server-root.noarch 1398866025:6.0.3.rc1.git202002100103-1.EL.7          

Dependency Updated:
  audit.x86_64 0:2.8.5-4.el7                                                audit-libs.x86_64 0:2.8.5-4.el7                                                policycoreutils.x86_64 0:2.5-33.el7                                               

Replaced:
  ncf.noarch 1398866025:5.0.16.rc1.git202002060302-1.EL.7                       ncf-api-virtualenv.noarch 1398866025:5.0.16.rc1.git202002060302-1.EL.7     rudder-inventory-endpoint.noarch 1398866025:5.0.16.rc1.git202002070313-1.EL.7    
  rudder-inventory-ldap.x86_64 1398866025:5.0.16.rc1.git202002070313-1.EL.7     rudder-jetty.noarch 1398866025:5.0.16.rc1.git202002070313-1.EL.7           rudder-techniques.noarch 1398866025:5.0.16.rc1.git202002070313-1.EL.7            

Complete!


Related issues

Related to Rudder - Bug #16707: After upgrade from 5.0 to 6.0 on centos7, rudder-slapd is not startedReleasedBenoît PECCATTEActions
#1

Updated by Nicolas CHARLES 5 months ago

Note that the scale-out-relay plugin was not disabled.
Webapp logs are thus full of errors

#2

Updated by Nicolas CHARLES 5 months ago

Upgrading scale out relay plugin didn't fix the issue
Needed to restart slapd
at this point I have the errors

2020-02-10 15:45:57+0000] INFO  inventory-processing - Watch new inventory file 'agent2-7b1378ec-cab0-4a38-a4fb-668402a169f0.ocs' with signature available: process.
[2020-02-10 15:46:00+0000] INFO  policy.generation.timing - Policy generation failed after:        7 s
[2020-02-10 15:46:00+0000] ERROR policy.generation - Error when updating policy, reason was: Cannot write nodes configuration <- Unexpected: Error when executing hooks:
 Exit code=1 for hook: '/opt/rudder/etc/hooks.d/policy-generation-node-ready/10-cf-promise-check'. [stdout:   error: There are syntax errors in policy files
][stderr:/var/rudder/share/2e7b6735-cb4a-451d-b5b3-b729bf468a1b/rules.new/cfengine-community/promises.cf:735:15: error: syntax error
      "${str}" not_matching => "true";
              ^

and

[2020-02-10 15:46:12+0000] WARN  bootchecks - Could not authenticate in ncf API, maybe it was not initialized yet, retrying in 5 seconds, details Failure during authentication to ncf api: code 500: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

[2020-02-10 15:46:17+0000] WARN  bootchecks - Could not authenticate in ncf API, maybe it was not initialized yet, retrying in 5 seconds, details Failure during authentication to ncf api: code 500: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

#3

Updated by Nicolas CHARLES 5 months ago

  • Related to Bug #16707: After upgrade from 5.0 to 6.0 on centos7, rudder-slapd is not started added
#4

Updated by Vincent MEMBRÉ 5 months ago

  • Target version changed from 6.0.3 to 6.0.4
#5

Updated by Vincent MEMBRÉ 4 months ago

  • Target version changed from 6.0.4 to 6.0.5
#6

Updated by Vincent MEMBRÉ 3 months ago

  • Target version changed from 6.0.5 to 6.0.6
#7

Updated by Vincent MEMBRÉ about 2 months ago

  • Target version changed from 6.0.6 to 6.0.7
#8

Updated by Vincent MEMBRÉ 11 days ago

  • Target version changed from 6.0.7 to 6.0.8

Also available in: Atom PDF