Project

General

Profile

Actions
Copy link

Bug #10405

closed

Bug #10373: Upgrading from 4.0 to 4.1 failed on Centos7.3, and purged LDAP directory

Bad way to calculate the size of mdb database for LDAP data

Bug #10405: Bad way to calculate the size of mdb database for LDAP data

Added by François ARMAND almost 9 years ago. Updated almost 9 years ago.

Status:
Released
Priority:
N/A
Assignee:
Alexis Mousset
Category:
System integration
Target version:
4.1.0
Pull Request:
https://github.com/Normation/rudder-p...
Severity:
Critical - prevents main use of Rudder | no workaround | data loss | security
UX impact:
User visibility:
Operational - other Techniques | Technique editor | Rudder settings
Effort required:
Small
Priority:
68
Name check:
Fix check:
Regression:

Description

With the parent task merged, we are now correctly setting the mdb database size during migration.

But we are not at all using the correct way to calculate it, because we are still using the old way which was for a cache, so depending on RAM, and not for a storage size.

The problem is that it is not trivial to update mdb storage size if limit is reached, and the consequences can be very grave (losing data).

So, we need to:

- find the normal database size for a set of production install,
- find a sufficiently high value so that, if possible, any installation can run a couple of thousand node without even knowing that that limit exists (tracked on that ticket)
- document all of that (tracked in #10404)

Updated by François ARMAND almost 9 years ago Actions
Copy link
 #1

  • Subject changed from By way to calcul to size of mdb database for LDAP data to Bad way to calcul to size of mdb database for LDAP data
  • Category set to System integration

Updated by Nicolas CHARLES almost 9 years ago Actions
Copy link
 #3

The parameter maxsize defines the maximum size the database is allowed to grow (see https://sites.google.com/site/bmaupinwiki/home/applications/ldap/optimizing-openldap )
There is no reason to not put a huge number, as it will only use what is necessary by the inventories and configurations

For instance, i put maxsize 24644718592 , but got a 436 MB file with 2000 auto-generated inventories
7000 real nodes translate to 4.3 GB

It would be really sad to have a catastrophic failure at some point because value was too conservative

Updated by François ARMAND almost 9 years ago Actions
Copy link
 #4

So, there doesn't seem to have a problem to set, say, 10GB (or even 100GB) for that? If so, why there is even a size?

Updated by Nicolas CHARLES almost 9 years ago Actions
Copy link
 #5

Don't know - someone more versed in openldap may have an answer

Updated by François ARMAND almost 9 years ago Actions
Copy link
 #6

Perhaps we should also add to the stats info we can collect that info, to have a better idea of the actual size.

Updated by François ARMAND almost 9 years ago Actions
Copy link
 #7

  • Subject changed from Bad way to calcul to size of mdb database for LDAP data to Bad way to calculate the size of mdb database for LDAP data

Updated by Jonathan CLARKE almost 9 years ago Actions
Copy link
 #8

  • User visibility set to Operational - other Techniques | Technique editor | Rudder settings
  • Effort required set to Small

Updated by François ARMAND almost 9 years ago Actions
Copy link
 #9

  • Assignee set to Benoît PECCATTE

So, given http://www.openldap.org/lists/openldap-technical/201208/msg00177.html ("set it to a few hundred GB") and the constraint that it must be adressable, we can set it to:

- 2 GB on 32 bits proc (note: it must be documented that 32bit proc won't support more than a couple thousands nodes)
- 100Go on 64 bits. The current size for ~7000 nodes is ~2,5 GB, so that is more than one order of magnitude above.

Updated by Benoît PECCATTE almost 9 years ago Actions
Copy link
 #10

  • Status changed from New to In progress

Updated by Benoît PECCATTE almost 9 years ago Actions
Copy link
 #11

  • Status changed from In progress to Pending technical review
  • Assignee changed from Benoît PECCATTE to Alexis Mousset
  • Pull Request set to https://github.com/Normation/rudder-packages/pull/1290

Updated by Benoît PECCATTE almost 9 years ago Actions
Copy link
 #12

  • Status changed from Pending technical review to Pending release

Updated by Benoît PECCATTE almost 9 years ago Actions
Copy link
 #13

  • Priority set to 68

Updated by Benoît PECCATTE almost 9 years ago Actions
Copy link
 #14

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 4.1.0 which was released today.

Actions
Copy link

Also available in: PDF Atom