Project

General

Profile

Actions
Copy link

Bug #7265

closed

Writting promises should be parallelized

Added by François ARMAND about 9 years ago. Updated about 8 years ago.

Status:
Released
Priority:
N/A
Assignee:
Vincent MEMBRÉ
Category:
Performance and scalability
Target version:
2.11.15
Pull Request:
https://github.com/Normation/rudder/p...
Severity:
UX impact:
User visibility:
Effort required:
Priority:
Name check:
Fix check:
Regression:

Description

Writting promise has notoriously bad performance.

On first intention, that can be improved by parallelizing the actual technique parsing / writting by node (because, even if there is 1/ thread symchronization overhead and 2/ i/o limits, there is a big gain in actually trying to saturate i/o, and not blocking tens of little nodes because one nodes has more / longer promises).

Subtasks 1 (0 open — 1 closed)

Bug #7270: Fix compiling issue after merge of 7265 in branch 3.0ReleasedFrançois ARMAND2015-10-11Actions
Actions
Copy link
 #1

Updated by François ARMAND about 9 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Vincent MEMBRÉ
  • Pull Request set to https://github.com/Normation/rudder/pull/926
Actions
Copy link
 #2

Updated by François ARMAND about 9 years ago

  • Status changed from Pending technical review to Pending release
  • % Done changed from 0 to 100
Actions
Copy link
 #4

Updated by Vincent MEMBRÉ about 9 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 2.11.15, 3.0.10 and 3.1.3 which were released today.

Actions
Copy link
 #5

Updated by Nicolas CHARLES about 9 years ago

i'm just curious. Do we have metrics to mesure the performance, or do user see an improved generation time ?

Actions
Copy link
 #6

Updated by Dmitry Svyatogorov about 8 years ago

Still view serialized cf-promises in 3.2.5.
"Update took 4 min, 41 s" for distribution around group with ≈30 nodes with 18 rules, ≈70 directives.
Rudder-server node approximate config:
2 x CPU "Intel Xeon E312xx", 4GB RAM, 20GB HDD with about 100MB/s rw, 200 iops virtual host
jvm with rudder was not yet tuned (out-of-the-box "-Xms1024m -Xmx1024m" etc.).

As far as I view, cf-promises are launched one-by-one for each host.

root     30638 10:48:41 00:00:01 56.5 13444  34640  0.3 /var/rudder/cfengine-community/bin/cf-promises -f /var/rudder/share/5610a846-edbc-4ee4-8184-3b288b6a6d4f/rules.new/cfengine-community/promises.cf
root     31266 10:48:46 00:00:01 62.0 13744  34644  0.3 /var/rudder/cfengine-community/bin/cf-promises -f /var/rudder/share/a16024f1-137d-4951-8a57-8d06dd720637/rules.new/cfengine-community/promises.cf
etc.
etc.

More hosts in group → more lag.

It seems the good solution is to implement user-tunable parallelism at this point, m.b. one thread per core by default. Not very complicated.
The more complex, but mush more fast m.b. to provide homogeneous groups with cloned (in some manner) rules?

Actions
Copy link

Also available in: Atom PDF