Bug #10405
closed
Bug #10373: Upgrading from 4.0 to 4.1 failed on Centos7.3, and purged LDAP directory
Bad way to calculate the size of mdb database for LDAP data
Description
With the parent task merged, we are now correctly setting the mdb database size during migration.
But we are not at all using the correct way to calculate it, because we are still using the old way which was for a cache, so depending on RAM, and not for a storage size.
The problem is that it is not trivial to update mdb storage size if limit is reached, and the consequences can be very grave (losing data).
So, we need to:
- find the normal database size for a set of production install,
- find a sufficiently high value so that, if possible, any installation can run a couple of thousand node without even knowing that that limit exists (tracked on that ticket)
- document all of that (tracked in #10404)
Updated by François ARMAND almost 8 years ago
- Subject changed from By way to calcul to size of mdb database for LDAP data to Bad way to calcul to size of mdb database for LDAP data
- Category set to System integration
Updated by François ARMAND almost 8 years ago
For information/reference: https://sites.google.com/site/bmaupinwiki/home/applications/ldap/optimizing-openldap
Updated by Nicolas CHARLES almost 8 years ago
The parameter maxsize defines the maximum size the database is allowed to grow (see https://sites.google.com/site/bmaupinwiki/home/applications/ldap/optimizing-openldap )
There is no reason to not put a huge number, as it will only use what is necessary by the inventories and configurations
For instance, i put maxsize 24644718592 , but got a 436 MB file with 2000 auto-generated inventories
7000 real nodes translate to 4.3 GB
It would be really sad to have a catastrophic failure at some point because value was too conservative
Updated by François ARMAND almost 8 years ago
So, there doesn't seem to have a problem to set, say, 10GB (or even 100GB) for that? If so, why there is even a size?
Updated by Nicolas CHARLES almost 8 years ago
Don't know - someone more versed in openldap may have an answer
Updated by François ARMAND almost 8 years ago
Perhaps we should also add to the stats info we can collect that info, to have a better idea of the actual size.
Updated by François ARMAND almost 8 years ago
- Subject changed from Bad way to calcul to size of mdb database for LDAP data to Bad way to calculate the size of mdb database for LDAP data
Updated by Jonathan CLARKE almost 8 years ago
- User visibility set to Operational - other Techniques | Technique editor | Rudder settings
- Effort required set to Small
Updated by François ARMAND almost 8 years ago
- Assignee set to Benoît PECCATTE
So, given http://www.openldap.org/lists/openldap-technical/201208/msg00177.html ("set it to a few hundred GB") and the constraint that it must be adressable, we can set it to:
- 2 GB on 32 bits proc (note: it must be documented that 32bit proc won't support more than a couple thousands nodes)
- 100Go on 64 bits. The current size for ~7000 nodes is ~2,5 GB, so that is more than one order of magnitude above.
Updated by Benoît PECCATTE almost 8 years ago
- Status changed from New to In progress
Updated by Benoît PECCATTE almost 8 years ago
- Status changed from In progress to Pending technical review
- Assignee changed from Benoît PECCATTE to Alexis Mousset
- Pull Request set to https://github.com/Normation/rudder-packages/pull/1290
Updated by Benoît PECCATTE almost 8 years ago
- Status changed from Pending technical review to Pending release
Applied in changeset rudder-packages|c0d72c0189d5ae6b49c81107c24924eedcb0fedd.
Updated by Benoît PECCATTE over 7 years ago
- Status changed from Pending release to Released
This bug has been fixed in Rudder 4.1.0 which was released today.
- 4.1.0: Announce Changelog
- Download: https://www.rudder-project.org/site/get-rudder/downloads/