Project

General

Profile

Actions
Copy link

Bug #16208

closed

On a loaded system, the compliance computation is fairly expensive

Added by Nicolas CHARLES over 4 years ago. Updated over 4 years ago.

Status:
Released
Priority:
N/A
Assignee:
François ARMAND
Category:
Performance and scalability
Target version:
5.0.15
Pull Request:
https://github.com/Normation/rudder/p...
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
To do
Fix check:
Checked
Regression:

Description

Receiving thousands of reports per second, with 4500 nodes and 321 directive leads to a costly compliance computation

timing_debug log in trace show

[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - unexpected directives computation: 0ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - expected directives computation: 0ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - prepare data: 0ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - get missing reports: 0ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - unexpected directives computation: 0ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - compute compliance : 1ms
[2019-11-18 22:22:24] DEBUG debug_timing - Compliance: mergeCompareByRule global cost : 296ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: getNodeStatusReports - computing compliance for node NodeId(28e55618-6886-45ed-8992-657a3022e448): 297ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: computing policy status for NodeId(28e55618-6886-45ed-8992-657a3022e448): 1ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: Compute complianceForRun map: 268ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - prepare data: 0ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - get missing reports: 27ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - unexpected directives computation: 0ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - expected directives computation: 0ms
[2019-11-18 22:22:24] TRACE debug_timing - Compliance: mergeCompareByRule - prepare data: 1ms

the Compliance: Compute complianceForRun map part is really expensive, as it iterate over each rules, each directive and each components, and manipulate a lot of collections, changing them and creating them

it's even more complex to debug, as each step in this map is 0ms ...

Actions
Copy link

Also available in: Atom PDF