Bug #13256
Updated by François ARMAND over 6 years ago
A bug quite similar to #10646 was reported on the same heavy loaded installation, but with a different part of the system failing and that error message:
<pre>
[2018-08-21 18:35:57] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Policy generation completed in 1124 ms
[2018-08-21 18:35:57] ERROR com.normation.rudder.batch.AsyncDeploymentAgent$DeployerAgent - Error when updating policy, reason Cannot get the Configuration Cache <- Can't execute LDAP request
[2018-08-21 18:35:57] ERROR com.normation.rudder.batch.AsyncDeploymentAgent - Policy update error for process '13637' at 2018-08-21 18:35:57: Cannot get the Configuration Cache
[2018-08-21 18:36:05] INFO com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Start policy generation, checking updated rules
[2018-08-21 18:36:05] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Pre-policy-generation scripts hooks ran in 4 ms
[2018-08-21 18:36:05] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Pre-policy-generation modules hooks in 0 ms, start getting all generation related data.
[2018-08-21 18:36:06] ERROR com.normation.ldap.sdk.RWPooledSimpleAuthConnectionProvider - Can't execute LDAP request
com.unboundid.ldap.sdk.LDAPSearchException: The connection to server localhost:389 was closed while waiting for a response to search request SearchRequest(baseDN='cn=Nodes Configuration,ou=Rudder,cn=rudder-configuration', scope=BASE, deref=NEVER, sizeLimit=1, time
Limit=0, filter='(objectClass=*)', attrs={}): An I/O error occurred while trying to read the response from the server: IOException(message='The element indicated that it required 20972056 bytes to hold the value, but this is larger than the maximum of 20971520 b
ytes that the client has been configured to accept.', trace='readLength(ASN1StreamReader.java:390) / beginSequence(ASN1StreamReader.java:918) / readLDAPResponseFrom(LDAPMessage.java:1146) / run(LDAPConnectionReader.java:251)', revision=24201)
</pre>
The problem is linked to a default parameter in unboundid: https://docs.ldap.com/ldap-sdk/docs/javadoc/com/unboundid/ldap/sdk/LDAPConnectionOptions.html#PROPERTY_DEFAULT_MAX_MESSAGE_SIZE_BYTES which has a default value of 20971520 B (20MB).
It can be workaround by editing `/etc/default/rudder-jetty` on the line `JAVA_OPTIONS` to add `-Dcom.unboundid.ldap.sdk.LDAPConnectionOptions.defaultMaxMessageSizeBytes` with a sufficiently big value, for example:
<pre>
JAVA_OPTIONS="-Xss2M -Dcom.unboundid.ldap.sdk.LDAPConnectionOptions.defaultMaxMessageSizeBytes=209715200"
</pre>
And then restart rudder:
<pre>
service rudder-jetty restart
</pre>
You can then check that the parameter is correctly passed to Rudder with the command (parameter is on second line here):
<pre>
# ps aux | grep jetty
root 29889 23.7 19.2 3300344 394484 ? Sl 20:55 0:43 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Xss2M
-Dcom.unboundid.ldap.sdk.LDAPConnectionOptions.defaultMaxMessageSizeBytes=209715200 -server -Xms1024m -Xmx1024m -X
X:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -Dfile.encoding=UTF-8 -Drudder.configFile=/opt/rudder/etc/rudd
er-web.properties -Drudder.authFile=/opt/rudder/etc/rudder-users.xml -Dinventoryweb.configFile=/opt/rudder/etc/inv
entory-web.properties -Dlogback.configurationFile=/opt/rudder/etc/logback.xml -Drun.mode=production -Djetty.state=
/opt/rudder/jetty7/jetty.state -Djetty.logs=/var/log/rudder/webapp/ -Djetty.home=/opt/rudder/jetty7 -Djava.io.tmpd
ir=/var/rudder/tmp/jetty -jar /opt/rudder/jetty7/start.jar OPTIONS=Server etc/jetty-logging.xml etc/jetty-started.
xml --daemon
</pre>