Bug #15858
closed
Computing dynamic groups is very memory intensive, and can lead to FGC or OOM
Added by Nicolas CHARLES over 4 years ago.
Updated over 4 years ago.
Category:
Performance and scalability
Description
When dynamic groups are updated, it uses a lot of memory, and can completly prevent a policy generation to finish
We need to find a way to make it less memory consuming
Note: my tests uses a lot of regex and and/or in software, it might be a cause of issue
Files
AgentTypes.scala:256 com.normation.inventory.domain.AgentInfoSerialisation$.parseJson(String, Option) 1585 2021260 847048904 is a strong contender in memory usage
- Status changed from New to In progress
- Assignee set to Nicolas CHARLES
- Status changed from In progress to Pending technical review
- Assignee changed from Nicolas CHARLES to François ARMAND
- Pull Request set to https://github.com/Normation/rudder/pull/2504
- Category deleted (
Performance and scalability)
- Target version deleted (
5.0.15)
- Name check deleted (
To do)
We could do much better than that:
process returns a Seq[Nodenfo], by converting the result of the QueryAndCheckNodeId, and converting the NodeEntry, InventoryEntry and Machine to a NodeInfo; but we are throwing it all away to keep only the id in computeDynGroup, so we could make a new method which would return only the NodeIds, and save a lot of hassle
- Target version set to 5.0.15
- Category set to Performance and scalability
ok, i don't have the finished computed part without, because the close tabs is so f***ing close to selecting the tab itself, but screenshot with and without the change
Note how with the change, only ldap uses memory, will without, the parsing takes half of allocated memory
So it tooks 1.5GB to compute dynamic group with the change, and more than 3.5 GB (it wasn't finished) without the change
- Status changed from Pending technical review to Pending release
- Name check changed from To do to Reviewed
- Fix check changed from To do to Checked
- Status changed from Pending release to Released
This bug has been fixed in Rudder 5.0.15 which was released today.
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by 
