Project

General

Profile

Actions
Copy link

Bug #11620

closed

Agent segfaults when merging non-container data variables

Added by Nicolas CHARLES over 6 years ago. Updated over 6 years ago.

Status:
Released
Priority:
N/A
Assignee:
Benoît PECCATTE
Category:
Agent
Target version:
4.1.9
Pull Request:
https://github.com/Normation/rudder-p...
Severity:
Major - prevents use of part of Rudder | no simple workaround
UX impact:
User visibility:
Infrequent - complex configurations | third party integrations
Effort required:
Priority:
40
Name check:
Fix check:
Regression:

Description

Coonsider the following /var/rudder/local/properties.d/zwq10.json file

{
    "env":"PROD10IT",
    "foo":"plop" 
}

Running the agent causes a segfault:

rudder    debug: Iteration engine finished   ---   WARPING OUT
rudder    debug: DeRefCopyPromise(): promiser:'namespace[${sorted_files}]'
rudder    debug: DeRefCopyPromise():     copying constraint: 'slist'
rudder    debug: Starting iteration engine with 2 wheels   ---   ENTERING WARP SPEED
rudder    debug: Evaluating function: getindices("property_${file_id[${sorted_files}]}")
rudder    debug: Evaluating vars promise: namespace[/var/rudder/cfengine-community/inputs/common/1.0/../../properties.d/aproperties.json]
rudder    debug: Evaluating function: getindices("property_${file_id[${sorted_files}]}")
rudder    debug: Evaluating vars promise: namespace[/var/rudder/cfengine-community/inputs/common/1.0/../../properties.d/properties.json]
rudder    debug: Evaluating function: getindices("property_${file_id[${sorted_files}]}")
rudder    debug: Evaluating vars promise: namespace[/var/rudder/cfengine-community/inputs/common/1.0/../../properties.d/zroperties.json]
rudder    debug: Evaluating function: getindices("property_${file_id[${sorted_files}]}")
rudder    debug: Evaluating vars promise: namespace[/var/rudder/local/properties.d/10.json]
rudder    debug: Evaluating function: getindices("property_${file_id[${sorted_files}]}")
rudder    debug: Evaluating vars promise: namespace[/var/rudder/local/properties.d/properties.json]
rudder    debug: Evaluating function: getindices("property_${file_id[${sorted_files}]}")
rudder    debug: Evaluating vars promise: namespace[/var/rudder/local/properties.d/zwq10.json]
rudder    debug: Iteration engine finished   ---   WARPING OUT
rudder    debug: DeRefCopyPromise(): promiser:'node.${namespace[${sorted_files}]}'
rudder    debug: DeRefCopyPromise():     copying constraint: 'data'
rudder    debug: Starting iteration engine with 2 wheels   ---   ENTERING WARP SPEED
rudder    debug: Evaluating function: parsejson("{}")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: parsejson("{}")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: parsejson("{}")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: parsejson("{}")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: parsejson("{}")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: parsejson("{}")
rudder    debug: Evaluating vars promise: node.env
rudder    debug: Evaluating function: parsejson("{}")
rudder    debug: Evaluating vars promise: node.foo
rudder    debug: Iteration engine finished   ---   WARPING OUT
rudder    debug: DeRefCopyPromise(): promiser:'node.${namespace[${sorted_files}]}'
rudder    debug: DeRefCopyPromise():     copying constraint: 'data'
rudder    debug: Starting iteration engine with 3 wheels   ---   ENTERING WARP SPEED
rudder    debug: Evaluating function: mergedata("node.${namespace[${sorted_files}]}","property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: mergedata("node.${namespace[${sorted_files}]}","property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: mergedata("node.${namespace[${sorted_files}]}","property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: mergedata("node.${namespace[${sorted_files}]}","property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: mergedata("node.${namespace[${sorted_files}]}","property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]")
rudder    debug: Evaluating vars promise: node.properties
rudder    debug: Evaluating function: mergedata("node.${namespace[${sorted_files}]}","property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]")
Segmentation fault

Version: Rudder agent 4.1.7-jessie0 (CFEngine Core 3.10.2)

Actions
Copy link
 #1

Updated by Nicolas CHARLES over 6 years ago

  • Target version set to 3.1.24

same with rudder agent 3.1

L:line 
L:line     # Read the files, 10MB maximum
L:line     # Read into a property_<canonified filename> because cfengine doesn't support to read data into property[<filename>]
L:line     "property_${file_id[${sorted_files}]}" data => readjson("${sorted_files}", "10000000");
    L:qstring "property_${file_id[${sorted_files}]}" 42
    P:bundle:agent:properties:vars:any promiser = property_${file_id[${sorted_files}]}
    L:id data 47
    P:bundle:agent:properties:vars:any:property_${file_id[${sorted_files}]} attribute = data
    L:assign 50
    P:=>
    L:id readjson 59
    P:bundle:agent:properties:any function id = readjson
    P:bundle:agent:properties begin givearglist for function readjson, level 1
    L:qstring "${sorted_files}" 77
    P:bundle:agent:properties:any function readjson, qstring arg = ${sorted_files}
    L:qstring "10000000" 89
    P:bundle:agent:properties:any function readjson, qstring arg = 10000000
    P:bundle:agent:properties end givearglist for function readjson, level 1
    P:bundle:agent:properties:any Finished with function, now at level 0
L:line 
L:line     # Get namespace names
L:line     "namespace[${sorted_files}]" slist => getindices("property_${file_id[${sorted_files}]}");
    L:qstring "namespace[${sorted_files}]" 32
    P:bundle:agent:properties:vars:any promiser = namespace[${sorted_files}]
    L:id slist 38
    P:bundle:agent:properties:vars:any:namespace[${sorted_files}] attribute = slist
    L:assign 41
    P:=>
    L:id getindices 52
    P:bundle:agent:properties:any function id = getindices
    P:bundle:agent:properties begin givearglist for function getindices, level 1
    L:qstring "property_${file_id[${sorted_files}]}" 91
    P:bundle:agent:properties:any function getindices, qstring arg = property_${file_id[${sorted_files}]}
    P:bundle:agent:properties end givearglist for function getindices, level 1
    P:bundle:agent:properties:any Finished with function, now at level 0
L:line 
L:line     # Initialize all namespace variables (mergedata refuses to merge undefined variables)
L:line     "node.${namespace[${sorted_files}]}" data => parsejson('{}');
    L:qstring "node.${namespace[${sorted_files}]}" 40
    P:bundle:agent:properties:vars:any promiser = node.${namespace[${sorted_files}]}
    L:id data 45
    P:bundle:agent:properties:vars:any:node.${namespace[${sorted_files}]} attribute = data
    L:assign 48
    P:=>
    L:id parsejson 58
    P:bundle:agent:properties:any function id = parsejson
    P:bundle:agent:properties begin givearglist for function parsejson, level 1
    L:qstring '{}' 63
    P:bundle:agent:properties:any function parsejson, qstring arg = {}
    P:bundle:agent:properties end givearglist for function parsejson, level 1
    P:bundle:agent:properties:any Finished with function, now at level 0
L:line 
L:line     # First level merge within each namespace, in file name order 
L:line     "node.${namespace[${sorted_files}]}" data => mergedata("node.${namespace[${sorted_files}]}",
    L:qstring "node.${namespace[${sorted_files}]}" 40
    P:bundle:agent:properties:vars:any promiser = node.${namespace[${sorted_files}]}
    L:id data 45
    P:bundle:agent:properties:vars:any:node.${namespace[${sorted_files}]} attribute = data
    L:assign 48
    P:=>
    L:id mergedata 58
    P:bundle:agent:properties:any function id = mergedata
    P:bundle:agent:properties begin givearglist for function mergedata, level 1
    L:qstring "node.${namespace[${sorted_files}]}" 95
    P:bundle:agent:properties:any function mergedata, qstring arg = node.${namespace[${sorted_files}]}
L:line                                                            "property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]");
    L:qstring "property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]" 128
    P:bundle:agent:properties:any function mergedata, qstring arg = property_${file_id[${sorted_files}]}[${namespace[${sorted_files}]}]
    P:bundle:agent:properties end givearglist for function mergedata, level 1
    P:bundle:agent:properties:any Finished with function, now at level 0
L:line 
L:line     # That's all, now we have a node.namespace data that contains the overriden values
L:line }
2017-10-18T13:05:57+0200    debug: /default/properties: Resolving variables in bundle 'agent' 'properties'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: Evaluating promise 'properties_files'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'findfiles'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: findfiles pattern '/var/rudder/cfengine-community/inputs/common/1.0/../../properties.d/*.json' found match '/var/rudder/cfengine-community/inputs/common/1.0/../../properties.d/properties.json'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: Evaluating promise '_sorted_files'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'sort'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'sort'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: Evaluating promise 'sorted_files'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'reverse'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: Evaluating promise 'file_id[${sorted_files}]'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'canonify'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: Evaluating promise 'property_${file_id[${sorted_files}]}'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'readjson'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'readjson'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: Evaluating promise 'namespace[${sorted_files}]'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'getindices'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: Evaluating promise 'node.${namespace[${sorted_files}]}'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'parsejson'
2017-10-18T13:05:57+0200  verbose: /default/properties/vars: Evaluating promise 'node.${namespace[${sorted_files}]}'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'mergedata'
2017-10-18T13:05:57+0200    debug: /default/properties/vars: Looking at arg for function-like object 'mergedata'

Actions
Copy link
 #2

Updated by Alexis Mousset over 6 years ago

Reproduced with:

bundle agent main {
  vars:

    "data" data => "{\"env\":\"test\"}";

    "node" data => mergedata("{}",
            "data[env]");
}
Actions
Copy link
 #4

Updated by Alexis Mousset over 6 years ago

  • Subject changed from If properties.json file is invalid, agent segfaults to Agent segfaults when merging non-container data variables
Actions
Copy link
 #5

Updated by Vincent MEMBRÉ over 6 years ago

  • Target version changed from 3.1.24 to 3.1.25
Actions
Copy link
 #6

Updated by Benoît PECCATTE over 6 years ago

  • Severity set to Major - prevents use of part of Rudder | no simple workaround
  • User visibility set to Infrequent - complex configurations | third party integrations
  • Priority changed from 0 to 41

Infrequent because it can happen only withing our internal cfengine code and with a not exactly formated json code.

Actions
Copy link
 #7

Updated by Alexis Mousset over 6 years ago

  • Status changed from New to In progress
Actions
Copy link
 #8

Updated by Alexis Mousset over 6 years ago

  • Status changed from In progress to New
  • Target version changed from 3.1.25 to 4.1.9

Patch is significantly different for 3.6, targeting Rudder 4.1 for now.

Actions
Copy link
 #9

Updated by Alexis Mousset over 6 years ago

  • Status changed from New to In progress
Actions
Copy link
 #10

Updated by Alexis Mousset over 6 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Alexis Mousset to Benoît PECCATTE
  • Pull Request set to https://github.com/Normation/rudder-packages/pull/1418
Actions
Copy link
 #11

Updated by Alexis Mousset over 6 years ago

  • Status changed from Pending technical review to Pending release
Actions
Copy link
 #12

Updated by Vincent MEMBRÉ over 6 years ago

  • Status changed from Pending release to Released
  • Priority changed from 41 to 40

This bug has been fixed in Rudder 4.1.9 and 4.2.3 which were released today.

Actions
Copy link

Also available in: Atom PDF