Project

General

Profile

Bug #7265

Writting promises should be parallelized

Added by François ARMAND over 5 years ago. Updated over 4 years ago.

Status:
Released
Priority:
N/A
Category:
Performance and scalability
Target version:
Severity:
User visibility:
Effort required:
Priority:

Description

Writting promise has notoriously bad performance.

On first intention, that can be improved by parallelizing the actual technique parsing / writting by node (because, even if there is 1/ thread symchronization overhead and 2/ i/o limits, there is a big gain in actually trying to saturate i/o, and not blocking tens of little nodes because one nodes has more / longer promises).


Subtasks

Bug #7270: Fix compiling issue after merge of 7265 in branch 3.0ReleasedFrançois ARMAND2015-10-11Actions
#1

Updated by François ARMAND over 5 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Vincent MEMBRÉ
  • Pull Request set to https://github.com/Normation/rudder/pull/926
#2

Updated by François ARMAND over 5 years ago

  • Status changed from Pending technical review to Pending release
  • % Done changed from 0 to 100
#4

Updated by Vincent MEMBRÉ over 5 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 2.11.15, 3.0.10 and 3.1.3 which were released today.

#5

Updated by Nicolas CHARLES over 5 years ago

i'm just curious. Do we have metrics to mesure the performance, or do user see an improved generation time ?

#6

Updated by Dmitry Svyatogorov over 4 years ago

Still view serialized cf-promises in 3.2.5.
"Update took 4 min, 41 s" for distribution around group with ≈30 nodes with 18 rules, ≈70 directives.
Rudder-server node approximate config:
2 x CPU "Intel Xeon E312xx", 4GB RAM, 20GB HDD with about 100MB/s rw, 200 iops virtual host
jvm with rudder was not yet tuned (out-of-the-box "-Xms1024m -Xmx1024m" etc.).

As far as I view, cf-promises are launched one-by-one for each host.

root     30638 10:48:41 00:00:01 56.5 13444  34640  0.3 /var/rudder/cfengine-community/bin/cf-promises -f /var/rudder/share/5610a846-edbc-4ee4-8184-3b288b6a6d4f/rules.new/cfengine-community/promises.cf
root     31266 10:48:46 00:00:01 62.0 13744  34644  0.3 /var/rudder/cfengine-community/bin/cf-promises -f /var/rudder/share/a16024f1-137d-4951-8a57-8d06dd720637/rules.new/cfengine-community/promises.cf
etc.
etc.

More hosts in group → more lag.

It seems the good solution is to implement user-tunable parallelism at this point, m.b. one thread per core by default. Not very complicated.
The more complex, but mush more fast m.b. to provide homogeneous groups with cloned (in some manner) rules?

Also available in: Atom PDF