Bug #21705
Updated by François ARMAND about 2 years ago
I was trying to migrate from rudder 6.2.17 to 7.2.0 (nightly from 5 september).
The migration process displayed some warning about non existing files, but @/var/log/rudder/install/rudder-server-@ contains an error about postgres:
<pre>
Mon Sep 5 12:24:03 CEST 2022 - Starting rudder-server post installation script
Mon Sep 5 12:24:03 CEST 2022 - Starting rudder-reports post installation step
Found postgres service name: postgresql
Mon Sep 5 12:24:04 CEST 2022 - Ending rudder-reports post installation step
Mon Sep 5 12:24:04 CEST 2022 - Starting rudder-webapp post installation step
Creating rudder users
Stopping apache before running upgrade scripts
Managing services
Initializing filesystem
Upgrading techniques
Correctly set option autoupdate-technique-library to value true
Configuring LDAP
Running upgrade script
INFO: Checking if rudder-web.properties database access credentials are all right... LDAP OK, SQL OK
INFO: Configuration property 'rudder.community.port' was updated to 'rudder.policy.distribution.port.cfengine' in /opt/rudder/etc/rudder-web.properties
Adding LDAP index on attribute: softwareId
Adding LDAP index on attribute: nodeId
Reindexing LDAP directory - this may take a few minutes... Done
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
Overriding existing techniques
INFO: Correcting permissions on /var/rudder/configuration-repository... Done
Techniques have been updated, and update branch set to current state of the Techniques
INFO: Checking PostgreSQL service status............ FAILED
PostgreSQL service verification failed after 10 tries.
INFO: Checking LDAP service status... OK
adding new entry "propertyName=rudder_setup_done,ou=Application Properties,cn=rudder-configuration"
Updating package index
Updating licences
No newer release compatible versions than 6.2-1.7 found for the plugin rudder-plugin-scale-out-relay.
No newer release compatible versions than 6.2-1.11 found for the plugin rudder-plugin-branding.
No newer release compatible versions than 6.2-1.11 found for the plugin rudder-plugin-auth-backends.
No newer release compatible versions than 6.2-1.12 found for the plugin rudder-plugin-datasources.
ERROR: The migration has failed in some steps. Check previous error messages.
Please restart the failed service(s), and start the migration script again.
(try service rudder-server restart)
Once it is working, run:
# /opt/rudder/bin/rudder-upgrade
INFO: Correcting permissions on /var/rudder/configuration-repository... Done
Running plugins upgrade scripts
Restoring file ACLs
/var/rudder/ncf/common/20_cfe_basics/services.cf:19:0: error: Duplicate definition of bundle service_knowledge with type common
/var/rudder/ncf/common/40_it_ops_knowledge/service/service_knowledge.cf:19:0: error: Duplicate definition of bundle service_knowledge with type common
WARNING: Policies invalid, reseting to initial policies and updating...ok: Rudder agent policies were updated.
ok: Rudder agent state has been reset.
Done
FINISH: Rudder agent check ran properly, please look at messages above to see if there has been any error.
Mon Sep 5 12:25:33 CEST 2022 - Ending rudder-webapp post installation step
Mon Sep 5 12:25:33 CEST 2022 - Ending rudder-server post installation script
</pre>
Postgres service is up and running, and I can connect to rudder database.
If I try to play again the rudder migration script, I get other pg-related errors:
<pre>
root@server:/opt/rudder# ./bin/rudder-upgrade
INFO: Checking if rudder-web.properties database access credentials are all right... LDAP OK, SQL OK
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
INFO: A Technique library reload is needed and has been scheduled.
Overriding existing techniques
INFO: Correcting permissions on /var/rudder/configuration-repository... Done
On branch master
nothing to commit, working tree clean
Techniques have been updated, and update branch set to current state of the Techniques
INFO: Checking PostgreSQL service status... Not installed
INFO: No local database found. During initial install, this is expected.
INFO: Checking LDAP service status... OK
Updating package index
Updating licences
No newer release compatible versions than 7.2.0~beta1-2.1 found for the plugin rudder-plugin-scale-out-relay.
No newer release compatible versions than 6.2-1.11 found for the plugin rudder-plugin-branding.
No newer release compatible versions than 6.2-1.11 found for the plugin rudder-plugin-auth-backends.
No newer release compatible versions than 6.2-1.12 found for the plugin rudder-plugin-datasources.
INFO: The migration has completed successfully.
root@server:/opt/rudder#
</pre>
The result is that no migration scrpit for database schema is played, and thus the DB is incorrect.
The most apparent symptom is that all nodes are missing reports, and you get errors in relayd (via journald) and webapp (via log files) about mising column @reportid@:
<pre>
Sep 05 14:13:28 server rudder-relayd[1809]: ERROR report{queue_id=2DD0A3D062A60B2595DA5FA8ECBA65F2}: rudder_relayd::processing::reporting: output error: column ruddersysevents.reportid does not exist
</pre>