Bug #10424: LDAP error at upgrade - Cannot allocate memory - Rudder - Issue Tracker

Actions

Copy link

Bug #10424

closed

LDAP error at upgrade - Cannot allocate memory

Added by Alexis Mousset over 7 years ago. Updated over 7 years ago.

Status:

Released

Priority:

N/A

Assignee:

Benoît PECCATTE

Category:

Packaging

Target version:

4.1.0

Pull Request:

https://github.com/Normation/rudder-p...

Severity:

UX impact:

User visibility:

Effort required:

Priority:

Name check:

Fix check:

Regression:

Description

When upgrading from 3.1.18 to 4.1-nightly:

Installing: rudder-inventory-ldap-1398866025:4.1.0.rc2.git201703150133-1.SLES.11 [done]
Additional rpm output:
warning: /opt/rudder/etc/openldap/slapd.conf created as /opt/rudder/etc/openldap/slapd.conf.rpmnew
INFO: Setting rudder-slapd as a boot service... Done
INFO: Reloading syslogd...  Done
INFO: Stopping rudder-slapd... Done
rudder-slapd[5011]: [INFO] Using /etc/default/rudder-slapd for configuration
58c91e2f mdb_db_open: database "cn=rudder-configuration" cannot be opened: Cannot allocate memory (12). Restore from backup!
58c91e2f backend_startup_one (type=mdb, suffix="cn=rudder-configuration"): bi_db_open failed! (12)
slap_startup failed
ERROR: Failed to restore data from old format into the new format
You can reimport manually the data from backup file /var/rudder/ldap/backup/openldap-data-pre-upgrade-20170315115747.ldif
INFO: Restarting rudder-slapd... Done

INFO: Checking LDAP service status............ FAILED
LDAP service verification failed after 10 tries.

ERROR: The migration has failed in some steps. Check previous error messages.
Please restart the failed service(s), and start the migration script again.
(on a single Rudder server, try service rudder restart)
Once it is working, run:
# /opt/rudder/bin/rudder-upgrade
INFO: End of migration script
Setting umask to 0007
Starting Jetty: . . . . OK Wed Mar 15 11:59:02 CET 2017

Installing: rudder-server-root-1398866025:4.1.0.rc2.git201703150133-1.SLES.11 [done]
Additional rpm output:
58c91e77 mdb_db_open: database "cn=rudder-configuration" cannot be opened: Cannot allocate memory (12). Restore from backup!
58c91e77 backend_startup_one (type=mdb, suffix="cn=rudder-configuration"): bi_db_open failed! (12)
slap_startup failed

With

server:~ # rpm -qa | grep rudder-inventory-ldap
rudder-inventory-ldap-4.1.0.rc2.git201703150133-1.SLES.11

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Nicolas CHARLES over 7 years ago

oh :(
this seems OS specific: on a centos7

29057 root      20   0  9,027t 207676 202740 S   0,0  0,4   0:00.53 slapd

Actions

Copy link

Updated by Alexis Mousset over 7 years ago

server:~ # getconf LONG_BIT
64
server:~ # uname -a
Linux server 3.0.76-0.11-default #1 SMP Fri Jun 14 08:21:43 UTC 2013 (ccab990) x86_64 x86_64 x86_64 GNU/Linux
server:~ # ll -ahl /var/rudder/ldap/openldap-data
total 44K
drwxr-xr-x 2 root root 4.0K Mar 15 12:25 .
drwxr-xr-x 5 root root 4.0K Mar 15 11:57 ..
-rw-r--r-- 1 root root  580 Mar 15 02:03 DB_CONFIG
-rw------- 1 root root 100G Mar 15 11:57 data.mdb
-rw------- 1 root root 8.0K Mar 15 12:25 lock.mdb
-rw-r----- 1 root root  10M Mar 15 12:24 log.0000000001
server:~ # /opt/rudder/libexec/slapd -d 5 -h ldap://localhost:389 -n rudder-slapd -f /opt/rudder/etc/openldap/slapd.conf
58c92714 backend_startup_one: starting "cn=rudder-configuration" 
58c92714 mdb_db_open: "cn=rudder-configuration" 
58c92714 mdb_db_open: database "cn=rudder-configuration": dbenv_open(/var/rudder/ldap/openldap-data).
58c92714 mdb_db_open: database "cn=rudder-configuration" cannot be opened: Cannot allocate memory (12). Restore from backup!
58c92714 backend_startup_one (type=mdb, suffix="cn=rudder-configuration"): bi_db_open failed! (12)

Actions

Copy link

Updated by Nicolas CHARLES over 7 years ago

On a centos7:

# getconf LONG_BIT
64
# uname -a
Linux commented.site 3.10.0-514.10.2.el7.x86_64 #1 SMP Fri Mar 3 00:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
# ll -ahl /var/rudder/ldap/openldap-data
total 436M
drwxr-xr-x. 2 root root   55 15 mars  11:28 .
drwxr-xr-x. 5 root root   95  8 mars  10:07 ..
-rw-------. 1 root root 436M 15 mars  02:00 data.mdb
-rw-r--r--. 1 root root  580  8 mars  08:21 DB_CONFIG
-rw-------. 1 root root 8,0K 15 mars  11:43 lock.mdb

maxsize 9924644718592

Actions

Copy link

Updated by Alexis Mousset over 7 years ago

server:~ # ulimit -a
core file size          (blocks, -c) 1
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 23481
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) 2564192
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 23481
virtual memory          (kbytes, -v) 3633120
file locks                      (-x) unlimited
server:~ # ulimit -v unlimited

Seems to fix the issue.

Actions

Copy link

Updated by Alexis Mousset over 7 years ago

Status changed from New to In progress
Assignee set to Alexis Mousset

Actions

Copy link

Updated by Alexis Mousset over 7 years ago

Status changed from In progress to Pending technical review
Assignee changed from Alexis Mousset to Benoît PECCATTE
Pull Request set to https://github.com/Normation/rudder-packages/pull/1293

PR https://github.com/Normation/rudder-packages/pull/1293

Actions

Copy link