Rudder

Work in progess here: https://github.com/fanf/rudder/commit/bed63a9b5f46f78ac69935dcc5612f13060eb2bd

• logs in Trace for policy_generation and debig_timing
• ran the following command, on both 5.0 and 5.1 logs:
  

  
  sed -i "/is a Dsc node/d"  *
  sed -i "/discarding less priorize/d"  *
  sed -i "/Not updating changes by rule/d"  *
  sed -i "/their promises are going to be written/d"  *
  sed -i "/does not define any components/d"  *
  sed -i "/Not updating non-modified node/d"  *
  
  grep "debug_timing" * >> debug_timing.log
  
  grep "getNodeStatusReports" debug_timing.log >> getNodeStatusReports.log
  sed -i "/getNodeStatusReports/d"  debug_timing.log
  
  grep "computing policy status" debug_timing.log >> computePolicyStatus.log
  sed -i "/computing policy status/d"  debug_timing.log
  
  grep "Compute complianceForRun" debug_timing.log >> complianceForRun.log
  sed -i "/Compute complianceForRun/d"  debug_timing.log
  
  grep "compute compliance reports" debug_timing.log >> complianceReports.log
  sed -i "/compute compliance reports/d"  debug_timing.log
  
  grep "global cost" debug_timing.log >> globalCost.log
  sed -i "/global cost/d"  debug_timing.log
  
  grep "Cache for nodes info expire" debug_timing.log >> cacheExpire.log
  sed -i "/Cache for nodes info expire/d"  debug_timing.log
  
  grep "policy.generation" * >> policy_generation.log
  
  sed -i "/to merge multi-directives/d" policy_generation.log
  sed -i "/directive for unique techniques/d" policy_generation.log
  sed -i "/indep multi-directives/d" policy_generation.log
  sed -i "/Resolved policies for/d" policy_generation.log
  
  # Policy status comparision
  sed -i 's/.*: //' computePolicyStatus.log
  sed -i 's/ms//' computePolicyStatus.log
  
  sed -i 's/.*: //' getNodeStatusReports.log
  sed -i 's/ms//' getNodeStatusReports.log
  
  sed -i 's/.*: //' complianceReports.log
  sed -i 's/ms//' complianceReports.log
  
  sed -i 's/.*: //' cacheExpire.log
  sed -i 's/ms//' cacheExpire.log
  
  

(unfortunately global cost and Compute complianceForRun don't exist in 5.1)

Note: resultats for 5.0 are on a full night, 5.1 is on couple of hours, so it might skews the result a bit, but using only half a day for 5.0 show even worse stats for 5.0

After much more tests, the performance problem seems to lie in `toBox` and/or `toIO` (from box).
I'm not sure I undertand why, but avoiding that perf killer in hot spot leads to as good as 5.0 perf (or even better in some cases).

The hottest point is evalJS, which is done one time per directive param. Porting that to IOResult leads to excellent results.

For ex, in 5.0 I have:

===> Total out: 374 ms
===> Total in : 368 ms
===> Total exp: 34 ms

Run nodeconfigProg          : 129 ms

In 6.0 with evalJS in Box:

===> Total out: 3954 ms
===> Total in : 3904 ms
===> Total exp: 3234 ms
Run nodeconfigProg          : 4793 ms

And with evalJS in IORestult:

===> Total out: 270 ms
===> Total in : 261 ms
===> Total exp: 21 ms

config : 187 ms