Project

General

Profile

Bug #13256

Updated by François ARMAND over 5 years ago

A bug quite similar to #10646 was reported on the same heavy loaded installation, but with a different part of the system failing and that error message:  

 <pre> 
 [2018-08-21 18:35:57] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Policy generation completed in 1124 ms 
 [2018-08-21 18:35:57] ERROR com.normation.rudder.batch.AsyncDeploymentAgent$DeployerAgent - Error when updating policy, reason Cannot get the Configuration Cache <- Can't execute LDAP request 
 [2018-08-21 18:35:57] ERROR com.normation.rudder.batch.AsyncDeploymentAgent - Policy update error for process '13637' at 2018-08-21 18:35:57: Cannot get the Configuration Cache 
 [2018-08-21 18:36:05] INFO    com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Start policy generation, checking updated rules 
 [2018-08-21 18:36:05] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Pre-policy-generation scripts hooks ran in 4 ms 
 [2018-08-21 18:36:05] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Pre-policy-generation modules hooks in 0 ms, start getting all generation related data. 
 [2018-08-21 18:36:06] ERROR com.normation.ldap.sdk.RWPooledSimpleAuthConnectionProvider - Can't execute LDAP request 
 com.unboundid.ldap.sdk.LDAPSearchException: The connection to server localhost:389 was closed while waiting for a response to search request SearchRequest(baseDN='cn=Nodes Configuration,ou=Rudder,cn=rudder-configuration', scope=BASE, deref=NEVER, sizeLimit=1, time 
 Limit=0, filter='(objectClass=*)', attrs={}):    An I/O error occurred while trying to read the response from the server:    IOException(message='The element indicated that it required 20972056 bytes to hold the value, but this is larger than the maximum of 20971520 b 
 ytes that the client has been configured to accept.', trace='readLength(ASN1StreamReader.java:390) / beginSequence(ASN1StreamReader.java:918) / readLDAPResponseFrom(LDAPMessage.java:1146) / run(LDAPConnectionReader.java:251)', revision=24201) 
 </pre> 

 The problem is linked to a default parameter in unboundid: https://docs.ldap.com/ldap-sdk/docs/javadoc/com/unboundid/ldap/sdk/LDAPConnectionOptions.html#PROPERTY_DEFAULT_MAX_MESSAGE_SIZE_BYTES which has a default value of 20971520 B (20MB). 

 It can be workaround by editing `/etc/default/rudder-jetty` on the line `JAVA_OPTIONS` to add `-Dcom.unboundid.ldap.sdk.LDAPConnectionOptions.defaultMaxMessageSizeBytes` with a sufficiently big value, for example: 

 <pre> 
 JAVA_OPTIONS="-Xss2M -Dcom.unboundid.ldap.sdk.LDAPConnectionOptions.defaultMaxMessageSizeBytes=209715200" 
 </pre> 

 And then restart rudder:  

 <pre> 
 service rudder-jetty restart 
 </pre> 

 You can then check that the parameter is correctly passed to Rudder with the command (parameter is on second line here): 

 <pre> 
 # ps aux | grep jetty 
 root       29889 23.7 19.2 3300344 394484 ?        Sl     20:55     0:43 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Xss2M 
 -Dcom.unboundid.ldap.sdk.LDAPConnectionOptions.defaultMaxMessageSizeBytes=209715200 -server -Xms1024m -Xmx1024m -X 
 X:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -Dfile.encoding=UTF-8 -Drudder.configFile=/opt/rudder/etc/rudd 
 er-web.properties -Drudder.authFile=/opt/rudder/etc/rudder-users.xml -Dinventoryweb.configFile=/opt/rudder/etc/inv 
 entory-web.properties -Dlogback.configurationFile=/opt/rudder/etc/logback.xml -Drun.mode=production -Djetty.state= 
 /opt/rudder/jetty7/jetty.state -Djetty.logs=/var/log/rudder/webapp/ -Djetty.home=/opt/rudder/jetty7 -Djava.io.tmpd 
 ir=/var/rudder/tmp/jetty -jar /opt/rudder/jetty7/start.jar OPTIONS=Server etc/jetty-logging.xml etc/jetty-started. 
 xml --daemon 
 </pre>

Back