Rudder

After a fresh install, i sometimes have no server inventory.
It is most likely a race condition, as the inventory is sent when the webapp is up an running, but with wrong ldap/postgres credential, so we have

[2016-11-09 15:05:12] INFO  com.normation.inventory.provisioning.endpoint.FusionReportEndpoint - New input inventory: 'ubuntu-root.ocs'
[2016-11-09 15:05:13] INFO  com.normation.inventory.provisioning.endpoint.FusionReportEndpoint - New input inventory: 'ubuntu-root.ocs'
[2016-11-09 15:05:14] INFO  com.normation.inventory.provisioning.endpoint.FusionReportEndpoint - Inventory 'ubuntu-root.ocs' parsed in 1 second and 570 milliseconds ms, now checking signature
[2016-11-09 15:05:14] INFO  com.normation.inventory.provisioning.endpoint.FusionReportEndpoint - Inventory 'ubuntu-root.ocs' parsed in 566 milliseconds ms, now checking signature
[2016-11-09 15:05:14] ERROR com.normation.ldap.sdk.RWPooledSimpleAuthConnectionProvider - Can't get a new LDAP connection
com.unboundid.ldap.sdk.LDAPException: invalid credentials
        at com.unboundid.ldap.sdk.LDAPConnection.bind(LDAPConnection.java:1937)
        at com.unboundid.ldap.sdk.LDAPConnectionPool.createConnection(LDAPConnectionPool.java:1138)

simultaneous agent run says:

Nov  9 15:05:06 ubuntu rudder[13726]: CFEngine(agent) rudder R: @@server-roles@@result_repaired@@server-roles@@server-roles-directive@@0@@Check LDAP in rudder-webapp.properties@@None@@2016-11-09 14:04:57+00:00##root@#The Rudder Webapp configuration files were updated with a new LDAP password
Nov  9 15:05:06 ubuntu rudder[13726]: CFEngine(agent) rudder R: @@server-roles@@result_repaired@@server-roles@@server-roles-directive@@0@@Check LDAP credentials@@None@@2016-11-09 14:04:57+00:00##root@#The OpenLDAP configuration file was updated with a new rootdn password
Nov  9 15:05:06 ubuntu rudder[13726]: CFEngine(agent) rudder R: @@server-roles@@log_info@@server-roles@@server-roles-directive@@0@@Check rudder-passwords.conf file@@None@@2016-11-09 14:04:57+00:00##root@#rudder-slapd has been restarted
Nov  9 15:05:06 ubuntu rudder[13726]: CFEngine(agent) rudder Method 'root_password_check_ldap' failed in some repairs
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder R: @@server-roles@@result_repaired@@server-roles@@server-roles-directive@@0@@Check SQL in rudder-webapp.properties@@None@@2016-11-09 14:04:57+00:00##root@#The Rudder Webapp configuration files were updated with a new SQL password
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder R: @@server-roles@@result_repaired@@server-roles@@server-roles-directive@@0@@Check SQL credentials@@None@@2016-11-09 14:04:57+00:00##root@#The Rudder PostgreSQL user account's password has been changed
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder R: @@server-roles@@result_repaired@@server-roles@@server-roles-directive@@0@@Check rudder-passwords.conf and pgpass files@@None@@2016-11-09 14:04:57+00:00##root@#The Rudder passwords file permissions were fixed
Nov  9 15:05:07 ubuntu systemd[1]: Stopped LSB: Jetty 7 webserver.
Nov  9 15:05:07 ubuntu systemd[1]: Starting LSB: Jetty 7 webserver...
Nov  9 15:05:07 ubuntu rudder-jetty[15810]: Setting umask to 0007
Nov  9 15:05:07 ubuntu rudder-jetty[15810]: Starting Jetty: already running.
Nov  9 15:05:07 ubuntu systemd[1]: Started LSB: Jetty 7 webserver.
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder Q: ".../systemctl rest": Failed to open /dev/tty: No such device or address
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder A variable seems to have been used for the name of the method. In this case, the promiser also needs to contain the unique name of the method
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder A method attempted to use a bundle '${configuration.enabled_loggers}' that was apparently not defined
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder Method '_log' failed in some repairs
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder Method 'command_execution' failed in some repairs
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder A variable seems to have been used for the name of the method. In this case, the promiser also needs to contain the unique name of the method
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder A method attempted to use a bundle '${configuration.enabled_loggers}' that was apparently not defined
Nov  9 15:05:07 ubuntu rudder[13726]: CFEngine(agent) rudder Method '_log' failed in some repairs
Nov  9 15:05:07 ubuntu systemd[1]: Stopping LSB: Jetty 7 webserver...
Nov  9 15:05:08 ubuntu rudder[15306]: CFEngine(agent) rudder Q: "...-inventory --lo": [info] FusionInventory instance: builtin
Q: "...-inventory --lo": [info] running task Inventory
Q: "...-inventory --lo": [info] Inventory saved in /var/rudder/tmp/inventory/ubuntu-root.ocs
Nov  9 15:05:08 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@Inventory@@log_debug@@inventory-all@@inventory-all@@00@@inventory@@None@@2016-11-09 14:05:04+00:00##root@#Running inventory
Nov  9 15:05:08 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@Inventory@@log_info@@inventory-all@@inventory-all@@00@@inventory@@None@@2016-11-09 14:05:04+00:00##root@#Curl installed
(snip)
ov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@server-roles@@result_success@@server-roles@@server-roles-directive@@0@@Check configuration-repository GIT lock@@None@@2016-11-09 14:05:04+00:00##root@#The /var/rudder/configuration-repository GIT lock file is not present or not older than 5 minutes
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder Finished command related to promiser '/usr/bin/curl --proxy '' --max-time 240 -s http://localhost:8080/rudder/api/status |/bin/grep -q OK' -- an error occurred, returned 1
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@server-roles@@result_error@@server-roles@@server-roles-directive@@0@@Check rudder status@@None@@2016-11-09 14:05:04+00:00##root@#This is the first time the http://localhost:8080/rudder/api/status web application failed to respond. Deferring the restart.
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder Method 'generic_alive_check' failed in some repairs
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@server-roles@@result_success@@server-roles@@server-roles-directive@@0@@Check endpoint status@@None@@2016-11-09 14:05:04+00:00##root@#The http://localhost:8080/endpoint/api/status web application is running
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@server-roles@@result_success@@server-roles@@server-roles-directive@@0@@Check slapd process@@None@@2016-11-09 14:05:04+00:00##root@#Check slapd process running was correct
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@server-roles@@result_success@@server-roles@@server-roles-directive@@0@@Check PostgreSQL configuration@@None@@2016-11-09 14:05:04+00:00##root@#There is no need of specific postgresql configuration on this system
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@server-roles@@result_success@@server-roles@@server-roles-directive@@0@@Check postgresql process@@None@@2016-11-09 14:05:04+00:00##root@#Check postgresql process running was correct
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@server-roles@@result_success@@server-roles@@server-roles-directive@@0@@Check postgresql boot script@@None@@2016-11-09 14:05:04+00:00##root@#Check postgresql boot starting parameters was correct
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@Common@@result_success@@hasPolicyServer-root@@common-root@@00@@Binaries update@@None@@2016-11-09 14:05:04+00:00##root@#The CFEngine binaries in /var/rudder/cfengine-community/bin are up to date
Nov  9 15:05:15 ubuntu rudder[15306]: CFEngine(agent) rudder R: @@Common@@log_info@@hasPolicyServer-root@@common-root@@00@@common@@EndRun@@2016-11-09 14:05:04+00:00##root@#End execution with config [0]
Nov  9 15:05:15 ubuntu systemd[1]: Reloading LSB: Apache2 web server.
Nov  9 15:05:15 ubuntu apache2[16242]:  * Reloading Apache httpd web server apache2
Nov  9 15:05:16 ubuntu apache2[16242]:  *
Nov  9 15:05:16 ubuntu systemd[1]: Reloaded LSB: Apache2 web server.
Nov  9 15:05:16 ubuntu rudder[15078]: CFEngine(agent) rudder R: @@server-roles@@result_repaired@@server-roles@@server-roles-directive@@0@@Check allowed networks configuration@@None@@2016-11-09 14:05:02+00:00##root@#The Rudder allowed networks configuration has been corrected
Nov  9 15:05:16 ubuntu rudder[15078]: CFEngine(agent) rudder R: @@server-roles@@log_info@@server-roles@@server-roles-directive@@0@@Check allowed networks configuration@@None@@2016-11-09 14:05:02+00:00##root@#Apache has been reloaded successfully
Nov  9 15:05:16 ubuntu rudder[15078]: CFEngine(agent) rudder R: @@Common@@log_info@@hasPolicyServer-root@@common-root@@00@@common@@EndRun@@2016-11-09 14:05:02+00:00##root@#End execution with config [0]
Nov  9 15:05:19 ubuntu CRON[14929]: (CRON) info (No MTA installed, discarding output)
Nov  9 15:05:39 ubuntu rudder-jetty[15889]: Stopping Jetty: OK
Nov  9 15:05:39 ubuntu systemd[1]: Stopped LSB: Jetty 7 webserver.
Nov  9 15:05:39 ubuntu systemd[1]: Starting LSB: Jetty 7 webserver...
Nov  9 15:05:39 ubuntu rudder-jetty[17534]: Setting umask to 0007
Nov  9 15:06:03 ubuntu rudder-jetty[17534]: Starting Jetty: . . . . . OK Wed Nov  9 15:06:03 CET 2016
Nov  9 15:06:03 ubuntu systemd[1]: Started LSB: Jetty 7 webserver.
Nov  9 15:06:03 ubuntu rudder[13726]: CFEngine(agent) rudder Method 'service_action' failed in some repairs

So, we have two concurrent cf-agent running, causing mayhem
Happens in 4.0, but most likely in 3.1