Project

General

Custom queries

Profile

Actions

Architecture #14939

open

Inefficient storage of software in LDAP directory

Added by Nicolas CHARLES about 6 years ago. Updated 24 days ago.

Status:
New
Priority:
N/A
Assignee:
-
Category:
Performance and scalability
Target version:
Effort required:
Name check:
Fix check:
Regression:

Description

Software details are store in a specific branch, with deduplication, so that's great, but we store the full dn of these software in a node.
So every node get 2000 entries like

softwareId=000ddc3b-6c0a-4394-bb52-ad9cf012658d,ou=Software,ou=Inventories,cn=rudder-configuration

there is about twice as much data as necessary there, we could simply have the uuid and not the full DN to the software

Impact is double:
  • ldap directory is bigger than necessary, so its slower than necessary
  • we need much more memory for everything:

when we get a node, we need to transport the thousands softwares dn, convert them to string, and manipulate it
memory impact is huge
we keep moving from softwareid that's an id to software dn that's a dn

Getting, from all the nodes, all the softwares entry, with 1000 nodes result in 1.5GB of memory, and a huge strain on GC

Graph shows big spike where all softwares are fetched, the frop is a GC


Files

graph.png (52.8 KB) graph.png memory usage Nicolas CHARLES, 2019-05-22 22:34
memory.png (34.3 KB) memory.png classes Nicolas CHARLES, 2019-05-22 22:34

Related issues 1 (1 open0 closed)

Related to Rudder - Architecture #14923: Dynamic groups with regex on software are long to build delaying generationNewFrançois ARMANDActions
#1

Updated by Nicolas CHARLES about 6 years ago

#2

Updated by Nicolas CHARLES about 6 years ago

#3

Updated by Nicolas CHARLES about 6 years ago

  • Description updated (diff)
#4

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 6.0.0~beta1 to 6.0.0
#5

Updated by Alexis Mousset over 5 years ago

  • Target version changed from 6.0.0 to 6.0.1
#6

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 6.0.1 to 6.0.2
#7

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 6.0.2 to 6.0.3
#8

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 6.0.3 to 6.0.4
#9

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 6.0.4 to 6.0.5
#10

Updated by Vincent MEMBRÉ about 5 years ago

  • Target version changed from 6.0.5 to 6.0.6
#11

Updated by Vincent MEMBRÉ about 5 years ago

  • Target version changed from 6.0.6 to 6.0.7
#12

Updated by Vincent MEMBRÉ about 5 years ago

  • Target version changed from 6.0.7 to 6.0.8
#13

Updated by François ARMAND almost 5 years ago

  • Severity set to Major - prevents use of part of Rudder | no simple workaround
  • User visibility set to Operational - other Techniques | Rudder settings | Plugins
  • Priority changed from 0 to 43
#14

Updated by François ARMAND almost 5 years ago

  • Related to Architecture #14923: Dynamic groups with regex on software are long to build delaying generation added
#15

Updated by Vincent MEMBRÉ almost 5 years ago

  • Target version changed from 6.0.8 to 6.0.9
#16

Updated by Vincent MEMBRÉ almost 5 years ago

  • Target version changed from 6.0.9 to 6.0.10
#17

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 6.0.10 to 798
#18

Updated by Benoît PECCATTE about 4 years ago

  • Target version changed from 798 to 6.1.14
#19

Updated by Vincent MEMBRÉ almost 4 years ago

  • Target version changed from 6.1.14 to 6.1.15
#20

Updated by Vincent MEMBRÉ almost 4 years ago

  • Target version changed from 6.1.15 to 6.1.16
#21

Updated by Vincent MEMBRÉ almost 4 years ago

  • Target version changed from 6.1.16 to 6.1.17
#22

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.1.17 to 6.1.18
#23

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.1.18 to 6.1.19
#24

Updated by François ARMAND over 3 years ago

  • Tracker changed from Bug to Architecture
  • Severity deleted (Major - prevents use of part of Rudder | no simple workaround)
  • User visibility deleted (Operational - other Techniques | Rudder settings | Plugins)
  • Priority deleted (43)
#25

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.19 to 6.1.20
#26

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.20 to 6.1.21
#27

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.21 to old 6.1 issues to relocate
#28

Updated by Alexis Mousset over 2 years ago

  • Target version changed from old 6.1 issues to relocate to 8.0.0~alpha1
#29

Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 8.0.0~alpha1 to 8.0.0~beta1
#30

Updated by Alexis Mousset almost 2 years ago

  • Target version changed from 8.0.0~beta1 to 8.1.0~alpha1
#31

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 8.1.0~alpha1 to 8.1.0~beta1
#32

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 8.1.0~beta1 to 8.1.0~beta2
#33

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 8.1.0~beta2 to 8.1.0~rc1
#34

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 8.1.0~rc1 to 8.1.0
#35

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 8.1.0 to 8.1.1
#36

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 8.1.1 to 8.1.2
#37

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 8.1.2 to 8.1.3
#38

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 8.1.3 to 8.1.4
#39

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 8.1.4 to 8.1.5
#40

Updated by Vincent MEMBRÉ 12 months ago

  • Target version changed from 8.1.5 to 8.1.6
#41

Updated by Vincent MEMBRÉ 11 months ago

  • Target version changed from 8.1.6 to 8.1.7
#42

Updated by Vincent MEMBRÉ 9 months ago

  • Target version changed from 8.1.7 to 8.1.8
#43

Updated by Vincent MEMBRÉ 8 months ago

  • Target version changed from 8.1.8 to 8.1.9
#44

Updated by Vincent MEMBRÉ 7 months ago

  • Target version changed from 8.1.9 to 8.1.10
#45

Updated by Vincent MEMBRÉ 7 months ago

  • Target version changed from 8.1.10 to 8.1.11
#46

Updated by Vincent MEMBRÉ 6 months ago

  • Target version changed from 8.1.11 to 8.1.12
#47

Updated by Vincent MEMBRÉ 4 months ago

  • Target version changed from 8.1.12 to 8.1.13
#48

Updated by Vincent MEMBRÉ about 2 months ago

  • Target version changed from 8.1.13 to 8.2.6
#49

Updated by Vincent MEMBRÉ about 2 months ago

  • Target version changed from 8.2.6 to 8.2.7
#50

Updated by Vincent MEMBRÉ 24 days ago

  • Target version changed from 8.2.7 to 8.2.8
Actions

Also available in: Atom PDF