Project

General

Profile

Bug #7510

rudder-upgrade fails when *re*installing Rudder

Added by Nicolas CHARLES over 5 years ago. Updated about 5 years ago.

Status:
Released
Priority:
1
Category:
Packaging
Target version:
Severity:
User visibility:
Effort required:
Priority:

Description

While installing rudder-server-root, I got during the installation the message
INFO: Checking PostgreSQL service status............ FAILED during installation

Indeed, the package rudder-webapp was configured before package rudder-reports, so rudder database was not yet existent.

This message is so not relevant, as the database is not yet there. Script rudder-upgrade should skip this test if:
- rudder-reports is not yet configured
- we are installing rudder-server-root

(or have a way to configure furst rudder-reports and then rudder-webapp, but this sounds tricky to do that - as rudder-reports and rudder-webapp may not be on the same machine)


Related issues

Related to Rudder - Bug #7516: rudder-inventory-ldap and rudder-jetty don't stop their services on uninstall and rudder-slapd can fail to restart on upgradeReleasedBenoît PECCATTE2015-11-30Actions
#1

Updated by Jonathan CLARKE over 5 years ago

  • Subject changed from INFO: Checking PostgreSQL service status............ FAILED during installation to rudder-upgrade fails when *re*installing Rudder
  • Status changed from New to In progress
  • Target version changed from 3.2.0~beta1 to 3.1.5
  • How to reproduce updated (diff)

OK, we now know a lot more about this. See steps to reproduce.

The output during the second installation looks like this:

Running Transaction
  Installing : 1398866025:rudder-inventory-ldap-3.1.5.rc1.git201511291104-1.EL.6.x86_64                                                                            1/14 
INFO: Setting rudder-slapd as a boot service... Done
INFO: Reloading syslogd...  Done
INFO: Restarting rudder-slapd... Done
  Installing : postgresql-libs-8.4.20-4.el6_7.x86_64                                                                                                               2/14 
  Installing : 1398866025:rudder-jetty-3.1.5.rc1.git201511291104-1.EL.6.noarch                                                                                     3/14 
  Installing : 1398866025:ncf-0.201511291151-1.EL.6.noarch                                                                                                         4/14 
  Installing : postgresql-8.4.20-4.el6_7.x86_64                                                                                                                    5/14 
  Installing : postgresql-server-8.4.20-4.el6_7.x86_64                                                                                                             6/14 
  Installing : 1398866025:ncf-api-virtualenv-3.1.5.rc1.git201511291104-1.EL.6.noarch                                                                               7/14 
  Installing : 1398866025:rudder-techniques-3.1.5.rc1.git201511291104-1.EL.6.noarch                                                                                8/14 
  Installing : 1398866025:rudder-webapp-3.1.5.rc1.git201511291104-1.EL.6.noarch                                                                                    9/14 
INFO: Setting Apache HTTPd as a boot service... Done
INFO: Restarting syslog... Done
INFO: Stopping Apache HTTPd... Done
INFO: Creating group rudder... Done
INFO: Adding ncf-api-venv to the rudder group... Done
INFO: No usable SSL certificate detected for Rudder HTTP/S support, generating one automatically... Done
INFO: Starting Apache HTTPd... Done
INFO: Launching script to check if a migration is needed
INFO: Checking if rudder-web.properties database access credentials are all right... LDAP OK,  SQL Credentials updated
INFO: Checking if inventory-web.properties database access credentials are all right... non existant, skipping
INFO: Checking PostgreSQL service status............ FAILED
PostgreSQL service verification failed after 10 tries.
INFO: Checking LDAP service status............ FAILED
LDAP service verification failed after 10 tries.

ERROR: The migration has failed in some steps. Check previous error messages.
Please restart the failed service(s), and start the migration script again.
(on a single Rudder server, try service rudder restart)
Once it is working, run:
# /opt/rudder/bin/rudder-upgrade
INFO: End of migration script
Initialized empty shared Git repository in /var/rudder/configuration-repository/.git/
********************************************************************************
rudder-webapp has been upgraded, but for the upgrade to take effect, please
restart the jetty application server as follows:
# service rudder-jetty restart
********************************************************************************
  Installing : 1398866025:rudder-inventory-endpoint-3.1.5.rc1.git201511291104-1.EL.6.noarch                                                                       10/14 
INFO: Launching script to check if a migration is needed
INFO: End of migration script
Restarting syslogd ...
Shutting down system logger: [  OK  ]
Starting system logger: [  OK  ]
  Installing : rsyslog-pgsql-5.8.10-10.el6_6.x86_64                                                                                                               11/14 
  Installing : 1398866025:rudder-reports-3.1.5.rc1.git201511291104-1.EL.6.noarch                                                                                  12/14 
Initializing database: [  OK  ]
Starting postgresql service: [  OK  ]
INFO: Setting PostgreSQL as a boot service... Done
INFO: Waiting for PostgreSQL to be up... Done
INFO: Creating Rudder PostgreSQL user... Done
INFO: Creating Rudder PostgreSQL database... Done
  Installing : 1398866025:rudder-agent-3.1.5.rc1.git201511291104-1.EL.6.x86_64                                                                                    13/14 
Making sure that the permissions on the CFEngine key directory are correct...
CFEngine binaries copied to workdir
********************************************************************************
rudder-agent has been installed (not started). This host can be a Rudder node.
To get started, configure your Rudder server's hostname and launch the agent:
# echo 'rudder.server' > /var/rudder/cfengine-community/policy_server.dat
# service rudder-agent start
This node will then appear in the Rudder web interface under 'Accept new nodes'.
********************************************************************************
INFO: Creating keys for CFEngine agent...
INFO: Created a new key for CFEngine agent in /var/rudder/cfengine-community/ppkeys/
  Installing : 1398866025:rudder-server-root-3.1.5.rc1.git201511301450-1.EL.6.noarch                                                                              14/14 
565c58f9 hdb_db_open: database "cn=rudder-configuration": db_open(/var/rudder/ldap/openldap-data/id2entry.bdb) failed: No such file or directory (2).
565c58f9 backend_startup_one (type=hdb, suffix="cn=rudder-configuration"): bi_db_open failed! (2)
slap_startup failed
************************************************************
Rudder is now installed but not configured.
Please run /opt/rudder/bin/rudder-init
************************************************************

Key points to note here are:

  • rudder-reports is installed after rudder-webapp. This may be causing the SQL check in rudder-upgrade (that is launched as a postinst script to rudder-webapp) to fail.
  • rudder-slapd is not accessible because it was not correctly stopped after the uninstall, nor correctly restarted when reinstalled
#2

Updated by Jonathan CLARKE over 5 years ago

  • How to reproduce updated (diff)

Updated steps to reproduce.

This is at least the case in 3.1, probably older versions too.

#3

Updated by Jonathan CLARKE over 5 years ago

  • Blocked by Bug #7516: rudder-inventory-ldap and rudder-jetty don't stop their services on uninstall and rudder-slapd can fail to restart on upgrade added
#4

Updated by Jonathan CLARKE over 5 years ago

I'm going to take a double approach here: 1) fix as many of the root causes as possible, and 2) fix rudder-upgrade to not fail so easily.

Jonathan CLARKE wrote:

  • rudder-reports is installed after rudder-webapp. This may be causing the SQL check in rudder-upgrade (that is launched as a postinst script to rudder-webapp) to fail.

I don't see any way to make sure yum installs rudder-reports before rudder-webapp, aside from having the one depend on the other, but we can't do that to maintain compatibility with split-server setups. I tried changing the order of the "Requires" line in the .spec, but that didn't change anything.

  • rudder-slapd is not accessible because it was not correctly stopped after the uninstall, nor correctly restarted when reinstalled

This is indirectly related to #7515 (killall not working and pidof not used on RHEL 6, in the rudder-slapd init script). And directly fixed by #7516.

#5

Updated by Jonathan CLARKE over 5 years ago

Jonathan CLARKE wrote:

2) fix rudder-upgrade to not fail so easily.

This is what this ticket will fix.

#6

Updated by Jonathan CLARKE over 5 years ago

  • Target version changed from 3.1.5 to 2.11.17
#7

Updated by Jonathan CLARKE over 5 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Jonathan CLARKE to Benoît PECCATTE
  • Pull Request set to https://github.com/Normation/rudder-packages/pull/821
#8

Updated by Jonathan CLARKE over 5 years ago

  • Status changed from Pending technical review to Pending release
  • % Done changed from 0 to 100
#10

Updated by Vincent MEMBRÉ about 5 years ago

  • Blocked by deleted (Bug #7516: rudder-inventory-ldap and rudder-jetty don't stop their services on uninstall and rudder-slapd can fail to restart on upgrade)
#11

Updated by Vincent MEMBRÉ about 5 years ago

  • Related to Bug #7516: rudder-inventory-ldap and rudder-jetty don't stop their services on uninstall and rudder-slapd can fail to restart on upgrade added
#12

Updated by Vincent MEMBRÉ about 5 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 2.11.17, 3.0.12 and 3.1.5 which were released today.

Also available in: Atom PDF