Bug #9736: Stack overflow on node generation - Rudder - Issue Tracker

Actions

Copy link

Bug #9736

closed

Stack overflow on node generation

Added by François ARMAND about 8 years ago. Updated over 6 years ago.

Status:

Rejected

Priority:

1 (highest)

Assignee:

Category:

Web - Config management

Target version:

4.1.11

Pull Request:

Severity:

Major - prevents use of part of Rudder | no simple workaround

UX impact:

User visibility:

Infrequent - complex configurations | third party integrations

Effort required:

Priority:

Name check:

Fix check:

Regression:

Description

On user found a case where generation is never ending, with the following stack trace:

[2016-11-24 10:16:39] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node's target configuration built in 734 ms, start to update rule values.
[2016-11-24 10:16:39] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - RuleVals updated in 65 ms, start to detect changes in node configuration.
[2016-11-24 10:16:39] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Checked node configuration updates leading to rules serial number updates and serial number updated in 11 ms
[2016-11-24 10:16:40] INFO  com.normation.rudder.services.policies.nodeconfig.NodeConfigurationServiceImpl - Configuration of following nodes were updated, their promises are going to be written: [7357f552-34ee-4b8d-9fa9-54854bbdece0, root]
[2016-11-24 10:16:40] ERROR net.liftweb.actor.ActorLogger - Actor threw an exception
java.lang.StackOverflowError: null
    at com.normation.rudder.services.policies.nodeconfig.NodeConfiguration.nodeInfo(NodeConfiguration.scala:67)
    at com.normation.rudder.services.policies.write.PathComputerImpl$$anonfun$com$normation$rudder$services$policies$write$PathComputerImpl$$recurseComputePath$2.apply(PathComputer.scala:140)
    at com.normation.rudder.services.policies.write.PathComputerImpl$$anonfun$com$normation$rudder$services$policies$write$PathComputerImpl$$recurseComputePath$2.apply(PathComputer.scala:139)
    at net.liftweb.common.Full.map(Box.scala:610)
    at com.normation.rudder.services.policies.write.PathComputerImpl.com$normation$rudder$services$policies$write$PathComputerImpl$$recurseComputePath(PathComputer.scala:139)
    at com.normation.rudder.services.policies.write.PathComputerImpl$$anonfun$com$normation$rudder$services$policies$write$PathComputerImpl$$recurseComputePath$3$$anonfun$apply$2.apply(PathComputer.scala:148)
    at com.normation.rudder.services.policies.write.PathComputerImpl$$anonfun$com$normation$rudder$services$policies$write$PathComputerImpl$$recurseComputePath$3$$anonfun$apply$2.apply(PathComputer.scala:141)
    at net.liftweb.common.Full.flatMap(Box.scala:612)
    at com.normation.rudder.services.policies.write.PathComputerImpl$$anonfun$com$normation$rudder$services$policies$write$PathComputerImpl$$recurseComputePath$3.apply(PathComputer.scala:141)
    at com.normation.rudder.services.policies.write.PathComputerImpl$$anonfun$com$normation$rudder$services$policies$write$PathComputerImpl$$recurseComputePath$3.apply(PathComputer.scala:139)
    at net.liftweb.common.Full.flatMap(Box.scala:612)
    [....]

We don't know the cause for now, but there is relay, and there was a lot of updates in the node/releay topologie. So Rudder must have badly updated something somewhere, and now there is an infinite loop when trying to generate policies.

The problem was seens in 4.0.0, but most likelly already exists in previous versions.

Related issues 2 (0 open — 2 closed)

Actions

Copy link

Updated by François ARMAND about 8 years ago

Is duplicate of Bug #9735: Stackoverflow on promise generation added

Actions

Copy link

Updated by François ARMAND about 8 years ago

So it happens that the node that is leading to the error (7357f552-34ee-4b8d-9fa9-54854bbdece0) is removed:

curl -k -H "X-API-Token: xxxxxxx"  https://yourserver/rudder/api/latest/nodes/7357f552-34ee-4b8d-9fa9-54854bbdece0
{"action": "nodeDetails","id": "7357f552-34ee-4b8d-9fa9-54854bbdece0","result": "success","data": {"nodes": [
{  "id": "7357f552-34ee-4b8d-9fa9-54854bbdece0" 
  , "hostname": "relay.bal.local" 
  , "status": "removed" 
  , "policyServerId": "root" 
  ...
}

Actions

Copy link

Updated by François ARMAND about 8 years ago

Fully removing by hand the node with the following LDAP command solved the generation problem:

ldapdelete -x -r -D "cn=Manager,cn=rudder-configuration" -W "nodeId=7357f552-34ee-4b8d-9fa9-54854bbdece0,ou=Nodes,ou=Removed Inventories,ou=Inventories,cn=rudder-configuration"

It is not clear at all why the node was taken into accound for policy generation, nor why there was a stack overflow.

One clue is that node : "relay.bal.local" does exsist but with a different id

Actions

Copy link

Updated by Vincent MEMBRÉ about 8 years ago

Target version changed from 4.0.1 to 4.0.2

Actions

Copy link

Updated by Vincent MEMBRÉ about 8 years ago

Target version changed from 4.0.2 to 4.0.3

Actions

Copy link

Updated by Nicolas CHARLES about 8 years ago

comment suppressed, it's not the correct issue that i'm checking

Actions

Copy link

Updated by Vincent MEMBRÉ almost 8 years ago

Target version changed from 4.0.3 to 4.0.4

Actions

Copy link

Updated by Benoît PECCATTE over 7 years ago

Severity set to Major - prevents use of part of Rudder | no simple workaround
User visibility set to Infrequent - complex configurations | third party integrations
Priority set to 24