Rudder

Hello, I'm not able to reproduce it in unit tests. Would you mind:

- share a screenshot ? It may be not possible to add it here, can you copy/paste it to chat.rudder.io?
- give me an example of the memory value that works with =

The inconsistency error message is a bit strange, I wonder if there is not a problem with saved data.
If you don't mind, could you also share the result of the following ldap query executed from rudder root server?

% ldapsearch -o ldif-wrap=no -H "ldap://localhost:389" -x -D "cn=Manager,cn=rudder-configuration" -w XXXX -b "ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration" -s sub '(objectClass=virtualMachineLogicalElement)' vmName vmMemory

Where XXXX is the LDAP password that you can find in /opt/rudder/etc/rudder-passwords.conf for RUDDER_OPENLDAP_BIND_PASSWORD (or use -W) for interactively providing it)

You should get result like:

# extended LDIF
#
# LDAPv3
# base <ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration> with scope subtree
# filter: (objectClass=virtualMachineLogicalElement)
# requesting: vmName vmMemory
#

# aaaaaaaa-6f1d-4bc0-976d-7ebcfada4a0e, 000000002-55a2-4b97-8529-5154cbb63a18, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=aaaaaaaa-6f1d-4bc0-976d-7ebcfada4a0e,nodeId=000000002-55a2-4b97-8529-5154cbb63a18,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 100000
vmName: vm1

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Search query from from log is

[2022-01-20 15:44:03] DEBUG com.normation.rudder.services.queries.AcceptedNodesLDAPQueryProcessor - LDAP result: 0 entries obtained in 191ms for query { returnType:'NodeReturnType' with 'And' criteria [virtualMachineLogicalElement.vmMemory gt 0] }
[2022-01-20 15:44:03] DEBUG com.normation.rudder.services.queries.AcceptedNodesLDAPQueryProcessor - [post-filter:rudderNode] Found 0 nodes when filtering for info service existence and properties (4 ms)

ldap query returns

# e2fbebb1-6b9b-403c-ae0e-629f91920b56, e9dfe54a-2925-45f0-a174-455b1696d986, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=e2fbebb1-6b9b-403c-ae0e-629f91920b56,nodeId=e9dfe54a-2925-45f0-a174-455b1696d986,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 8388
vmName: TEST4

# 0b3e57bf-9017-4030-ae14-606caf1ae930, e9dfe54a-2925-45f0-a174-455b1696d986, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=0b3e57bf-9017-4030-ae14-606caf1ae930,nodeId=e9dfe54a-2925-45f0-a174-455b1696d986,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 16777
vmName: TEST22

# e2fbebb1-6b9b-403c-ae0e-629f91920b56, 0a287ab1-f190-4d29-aca0-3c7a99540e0a, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=e2fbebb1-6b9b-403c-ae0e-629f91920b56,nodeId=0a287ab1-f190-4d29-aca0-3c7a99540e0a,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 8388
vmName: TEST3

# 0b3e57bf-9017-4030-ae14-606caf1ae930, 0a287ab1-f190-4d29-aca0-3c7a99540e0a, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=0b3e57bf-9017-4030-ae14-606caf1ae930,nodeId=0a287ab1-f190-4d29-aca0-3c7a99540e0a,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 16777
vmName: TEST22

# extended LDIF
#
# LDAPv3
# base <ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration> with scope subtree
# filter: (objectClass=virtualMachineLogicalElement)
# requesting: vmName vmMemory
#

# 486c63ef-225d-4f3e-bfa4-31e226cf10a3, bdbab835-ef6f-4d81-8560-170eb67bf3b4, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=486c63ef-225d-4f3e-bfa4-31e226cf10a3,nodeId=bdbab835-ef6f-4d81-8560-170eb67bf3b4,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 4080
vmName: windows-hvm

[...]

# 9c4a0ca8-3ff7-4d96-aff7-b4953ed58098, 0ad68f11-540b-46cd-a617-6848fe98f238, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=9c4a0ca8-3ff7-4d96-aff7-b4953ed58098,nodeId=0ad68f11-540b-46cd-a617-6848fe98f238,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 3039
vmName: windows-hvm

# ec7d505e-ea88-43a5-9948-24817313fd01, 3d60095e-8816-4ac8-9cc4-674aad1804da, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=ec7d505e-ea88-43a5-9948-24817313fd01,nodeId=3d60095e-8816-4ac8-9cc4-674aad1804da,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 4063
vmName: windows-hvm

# 4928aa9c-356a-40d1-835f-40cf6886b188, ecf4c3c5-3d58-4bf0-b984-a038b09eeb17, Nodes, Accepted Inventories, Inventories, rudder-configuration
dn: virtualMachineUuid=4928aa9c-356a-40d1-835f-40cf6886b188,nodeId=ecf4c3c5-3d58-4bf0-b984-a038b09eeb17,ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration
vmMemory: 2048
vmName: ubuntu-pvm

# search result
search: 2
result: 0 Success

# numResponses: 725
# numEntries: 724

See pictures in
https://gitter.im/normation/rudder?at=61e985889a33545406316e63

as you can see it works with a fixed value... like on the pictures for example "=" "3039"

detailed query generated from rudder

[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.InternalLDAPQueryProcessor - [331905074922] Start search for { returnType:'NodeReturnType' with 'And' criteria [virtualMachineLogicalElement.vmMemory gteq 0] }
[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.InternalLDAPQueryProcessor - [331905074922] |-- LDAPObjectType(ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration,Sub,LDAPObjectTypeFilter((objectClass=virtualMachineLogicalElement)),Some((vmMemory>=0)),ParentDNJoin,Set())
[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.InternalLDAPQueryProcessor - [331905074922] |--- SearchRequest(baseDN='ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration', scope=SUB, deref=NEVER, sizeLimit=0, timeLimit=0, filter='(&(objectClass=virtualMachineLogicalElement)(vmMemory>=0))', attrs={1.1})
[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.InternalLDAPQueryProcessor - [331905074922] |---- after ldap search request 0 result(s)
[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.InternalLDAPQueryProcessor - [331905074922] |---- after post-processing: 0 result(s)
[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.InternalLDAPQueryProcessor - [331905074922] |--- results are:
[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.InternalLDAPQueryProcessor - [331905074922] |-- 0 sub-results (merged)
[2022-01-20 16:04:09] TRACE com.normation.rudder.services.queries.InternalLDAPQueryProcessor - [331905074922] |-- ObjectType: ou=Nodes,ou=Accepted Inventories,ou=Inventories,cn=rudder-configuration; ids: 
[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.AcceptedNodesLDAPQueryProcessor - LDAP result: 0 entries obtained in 8ms for query { returnType:'NodeReturnType' with 'And' criteria [virtualMachineLogicalElement.vmMemory gteq 0] }
[2022-01-20 16:04:09] DEBUG com.normation.rudder.services.queries.AcceptedNodesLDAPQueryProcessor - [post-filter:rudderNode] Found 0 nodes when filtering for info service existence and properties (1 ms)

This is because we use in the ldap schema

attributetype ( InventoryAttributes:400.16
  NAME 'vmMemory'
  DESC 'process virtual memory'
  EQUALITY caseIgnoreMatch
  SUBSTR caseIgnoreSubstringsMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )

which is a string, and not a number

• XenCitrixServer.pm: MEMORY is a number
• Virtuozzo.pm: MEMORY is a number (painfully converted from output with M, G, K)
• Parralels.pm: MEMORY is a number
• Xen.pm: MEMORY is a number
• HyperV.pm: MEMORY is a number
• Qemu.pm: MEMORY is a number
• VmWareESX.pm: depends on the output of vmware-cmd -l, for line memsize; which seems to always be a number (if command still exists)
• Vserver.pm: no MEMORY field findable
• Jails.pm: no MEMORY field findable
• Lxd.pm: depends on the output of lxc config show, key limits.memory, which can be a string like 256MB
• VmWareDesktop.pm: depends on the output of vmrun list, entry memsize, which doesn't seem there...
• Docker.pm: no MEMORY field
• VirtualBox.pm: depends on the output of VBoxManage -nologo list --long vms, entry Memory Size, which can be a string like 256MB
• SolarisZones.pm: MEMORY is a number
• Lxc.pm: MEMORY is a number
• Libvirt.pm: MEMORY is a number
• Hpvm.pm: MEMORY is a number

So we have Lxd.pm and VirtualBox.pm which can return non number

I tried to switch to a MemoryComparator in Rudder - it improves for the regex, but still no < nor >
I'm afraid we ought to sanitize this field, change the attribute, and change the comparator

After testing, we have a simpler migration path: it appears that openldap is lenient with already existing stored value, and changing the schema to "integer" will not cause damage for values that are not integer if the case happens.

So we can:

In 6.2.x:

- 1/ change the schema to integer for vmMemory
- 2/ change the vmMemory object to "Memory" type
- 3/ change the parsing of new inventory to check for integer or string and parse accordingly.

These three step ensure new inventories are leading to correct data in ldap.
For existing one:
- 4/ write a special parsing rule when mapping from LDAP vmMemory with a catch in case the value was not an integer, and parsing the string in that case (it will be corrected at next inventory)
- 4bis/ alternatively, we can had a boot check option that sanitize data if needed

Both option can be removed in next minor version without a direct migration path from 6.2 (so, 7.1).

I think I prefer the special parsing rule (4), it didn't change perf in the happy path and it is very simple.

Lars Koenen : in your case, there is a simple workaround, since you have only integer values in your ldap for vmMemory:

- in rudder root server, edit /opt/rudder/etc/openldap/schema/inventory.

- replace:

attributetype ( InventoryAttributes:400.16
  NAME 'vmMemory'
  DESC 'virtual machine memory'
  EQUALITY caseIgnoreMatch
  SUBSTR caseIgnoreSubstringsMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )

By (on the last three lines change - be careful of spaces at the begining of line after the first, they are mandatory):

attributetype ( InventoryAttributes:400.16
  NAME 'vmMemory'
  DESC 'virtual machine memory'
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.27
  EQUALITY integerMatch
  ORDERING integerOrderingMatch )

- restart rudder-slapd: systemctl restart rudder-slapd