Project

General

Profile

Actions

Architecture #14939

open

Inefficient storage of software in LDAP directory

Added by Nicolas CHARLES almost 5 years ago. Updated 5 days ago.

Status:
New
Priority:
N/A
Assignee:
-
Category:
Performance and scalability
Target version:
Effort required:
Name check:
Fix check:
Regression:

Description

Software details are store in a specific branch, with deduplication, so that's great, but we store the full dn of these software in a node.
So every node get 2000 entries like

softwareId=000ddc3b-6c0a-4394-bb52-ad9cf012658d,ou=Software,ou=Inventories,cn=rudder-configuration

there is about twice as much data as necessary there, we could simply have the uuid and not the full DN to the software

Impact is double:
  • ldap directory is bigger than necessary, so its slower than necessary
  • we need much more memory for everything:

when we get a node, we need to transport the thousands softwares dn, convert them to string, and manipulate it
memory impact is huge
we keep moving from softwareid that's an id to software dn that's a dn

Getting, from all the nodes, all the softwares entry, with 1000 nodes result in 1.5GB of memory, and a huge strain on GC

Graph shows big spike where all softwares are fetched, the frop is a GC


Files

graph.png (52.8 KB) graph.png memory usage Nicolas CHARLES, 2019-05-22 22:34
memory.png (34.3 KB) memory.png classes Nicolas CHARLES, 2019-05-22 22:34

Related issues 1 (1 open0 closed)

Related to Rudder - Architecture #14923: Dynamic groups with regex on software are long to build delaying generationNewFrançois ARMANDActions
Actions

Also available in: Atom PDF