Project

General

Profile

Actions

Bug #23456

closed

Agent run is slower in 8.0

Added by Nicolas CHARLES 8 months ago. Updated 8 months ago.

Status:
Resolved
Priority:
N/A
Category:
Web - Config management
Target version:
Severity:
Major - prevents use of part of Rudder | no simple workaround
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
To do
Fix check:
To do
Regression:
No

Description

It takes 22s to generate policies for 3 nodes, with 3 directives

2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Run pre-gen scripts hooks     :          1 ms
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Run pre-gen modules hooks     :          0 ms
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Fetch all information         :        133 ms
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Build current rule values     :          7 ms
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Build target configuration    :        174 ms
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Write node configurations     :      10021 ms
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Save expected reports         :          2 ms
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Run post generation hooks     :      12313 ms
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Number of nodes updated       :          3   
2023-09-20 13:50:56+0200 INFO  policy.generation.timing - Policy generation succeeded in:       23 s

policy generation validation hook seems to be offending
2023-09-20 13:50:42+0200 WARN  hooks.longexecution - Hook is taking more than 2 s to finish: '/opt/rudder/etc/hooks.d/policy-generation-node-ready/10-cf-promise-check' wit
h environment parameters: [[RUDDER_GENERATION_DATETIME:2023-09-20T13:50:33.184+02:00] [RUDDER_NODE_ID:a9e382e8-a7b0-4896-892a-f422fcf6128b] [RUDDER_NODE_HOSTNAME:agent1.ru
dder.local] [RUDDER_NODE_POLICY_SERVER_ID:77295fb0-3ea9-4604-a721-baff6a6c9920] [RUDDER_AGENT_TYPE:cfengine-community] [RUDDER_POLICIES_DIRECTORY_NEW:/var/rudder/share/772
95fb0-3ea9-4604-a721-baff6a6c9920/share/a9e382e8-a7b0-4896-892a-f422fcf6128b/rules.new/cfengine-community] [RUDDER_NODEID:a9e382e8-a7b0-4896-892a-f422fcf6128b] [RUDDER_NEX
T_POLICIES_DIRECTORY:/var/rudder/share/77295fb0-3ea9-4604-a721-baff6a6c9920/share/a9e382e8-a7b0-4896-892a-f422fcf6128b/rules.new/cfengine-community]]
Actions #1

Updated by François ARMAND 8 months ago

More qualification:

- the problem is only linked to agent speed, not webapp policy generation,
- it may be worse on on SLES
- it may be partially linked to policy update at each run (see #23464)

Actions #2

Updated by François ARMAND 8 months ago

  • Severity set to Major - prevents use of part of Rudder | no simple workaround

An dit is something that happened between 7.3 and 8.0, so effectively a regression.

Actions #3

Updated by Alexis Mousset 8 months ago

  • Subject changed from Policy generation in8.0 is *really* slow to Agent run is slower in 8.0
Actions #4

Updated by Alexis Mousset 8 months ago

  • Assignee set to Alexis Mousset
Actions #5

Updated by Alexis Mousset 8 months ago

  • Status changed from New to Resolved

I did some testing after the fixes that have been done. On clean nightly VMs I get, with empty policies:

  • 7.3 on ubuntu 22.04
    • local agent run: 2.2s
    • full policy generation: 4s
  • 8.0 on ubuntu 22.04
    • local agent run: 2.3s
    • full policy generation: 4s
  • 8.0 on sles 15 sp4
    • local agent run: 2.2s
    • full policy generation: 4s

I'm hence closing this issue, please reopen if the problem is still visible.

Actions

Also available in: Atom PDF