Project

General

Profile

Actions
Copy link

Bug #16382

closed

Improve performance of policy generation writer

Added by François ARMAND over 4 years ago. Updated about 4 years ago.

Status:
Released
Priority:
N/A
Assignee:
Nicolas CHARLES
Category:
-
Target version:
6.0.3
Pull Request:
https://github.com/Normation/rudder/p...
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Reviewed
Fix check:
Checked
Regression:

Description

We lose quite some performance in policy generation in 6.0.
The ticket (and companion pr) are here to try to see if we can get back some of it.

Subtasks 7 (0 open — 7 closed)

Bug #16503: Normalize loggers for timing performance on write promisesReleasedNicolas CHARLESActions
Bug #16504: Remove Box from STValidationReleasedNicolas CHARLESActions
Bug #16505: Change ZIO runtime ergonomicsReleasedNicolas CHARLESActions
Bug #16506: Improve Hook exec performanceReleasedNicolas CHARLESActions
Bug #16525: Improve hook performance caused test errorsReleasedNicolas CHARLESActions
Bug #16540: We still have one lonely Future which use yet another threadpoolReleasedVincent MEMBRÉActions
Bug #16628: Remove a deadlock in GitTechniqueReaderRejectedFrançois ARMANDActions

Related issues 2 (0 open — 2 closed)

Related to Rudder - Bug #16557: CachedFindRuleNodeStatusReports is a huge source of contentionReleasedNicolas CHARLESActions
Related to Rudder - Bug #16629: Use only one ZIO threadpool to enhance performancesRejectedNicolas CHARLESActions
Actions
Copy link
 #1

Updated by François ARMAND over 4 years ago

  • Status changed from New to In progress
  • Assignee set to François ARMAND
Actions
Copy link
 #2

Updated by François ARMAND over 4 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Nicolas CHARLES
  • Pull Request set to https://github.com/Normation/rudder/pull/2666
Actions
Copy link
 #3

Updated by François ARMAND over 4 years ago

  • Status changed from Pending technical review to Pending release
Actions
Copy link
 #4

Updated by François ARMAND over 4 years ago

  • Status changed from Pending release to In progress

I uncorrectly push the branch on a branch in "origin", and redmine thought it was corrected. I'm not sure how to remove all the links.

Actions
Copy link
 #5

Updated by François ARMAND over 4 years ago

It should be corrected now.

Actions
Copy link
 #6

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 6.0.1 to 6.0.2
Actions
Copy link
 #7

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 6.0.2 to 6.0.3
Actions
Copy link
 #8

Updated by François ARMAND over 4 years ago

Two things:

- movePromisesToFinalPosition(folders: Seq[NodePoliciesPaths]) is not parallel. It seems to be because the order seems to be important, certainly because of relay. It feels like it can be parallelised (at worse, we can parallelize each policy server direct children, recursively).

- the whole fillTemplate is monothreaded while we use a paralleleSequence. Now, the parallelism is made on fibers, not thread, but still it's strange that they don't use more thread. John advice to use MUCH bigger parallelism factor than core/2.

Actions
Copy link
 #9

Updated by François ARMAND over 4 years ago

  • Related to Bug #16557: CachedFindRuleNodeStatusReports is a huge source of contention added
Actions
Copy link
 #10

Updated by François ARMAND about 4 years ago

  • Status changed from In progress to Pending release
Actions
Copy link
 #11

Updated by François ARMAND about 4 years ago

  • Fix check changed from To do to Checked
Actions
Copy link
 #12

Updated by Alexis Mousset about 4 years ago

  • Name check changed from To do to Reviewed
Actions
Copy link
 #13

Updated by Vincent MEMBRÉ about 4 years ago

This bug has been fixed in Rudder 6.0.3 which was released today.

Actions
Copy link
 #14

Updated by Vincent MEMBRÉ about 4 years ago

  • Related to Bug #16629: Use only one ZIO threadpool to enhance performances added
Actions
Copy link
 #15

Updated by Vincent MEMBRÉ about 4 years ago

  • Status changed from Pending release to Released
Actions
Copy link

Also available in: Atom PDF