Project

General

Profile

User story #8919

Remove modifications made to the inventory by the agent

Added by Matthieu CERDA over 2 years ago. Updated 8 months ago.

Status:
Released
Priority:
3
Category:
Web - Nodes & inventories
Target version:
Suggestion strength:
User visibility:
Effort required:

Description

We currently do three things when running an inventory update in Rudder:
  • Trigger a fusioninventory-agent run and store the generated inventory XML
  • Modify it using the agent to add custom entries
  • Send it to the Rudder server

It is a fact that as of now, most of the additions we do using the agent are obsolete, since they are added by FusionInventory itself as part of standard inventory tasks.

We should clean up our custom additions so we may someday be able to eat a vanilla fusioninventory report !

The addition is done in the system technique "techniques/system/inventory/1.0/fusionAgent.st"

The added fields are:

  • on android, <OPERATINGSYSTEM> and <ACCESSLOG>. We don't care of that, because Android support must be upgraded in all case. So we should directly patch fusion if still needed in place of adding something fusion is supposed to do (existing tag)
  • <VMS><VM></VM></VMS>: not mandatory for Rudder, superseeded by <VIRTUALMACHINES> tag, much more complete and supported since BEFORE Rudder 2.11 / fusionagent 2.3.6 (they are their since at least 2.1.0)
  • <USERSLIST><USER></USER></USERLIST>: not mandatory, superseeded by fusion LOCAL_USERS (same as VMS)
  • UUID, USER, AGENTSNAME, CFKEY, POLICY_SERVER, HOSTNAME: there are superseeded by the <RUDDER> tag provided in fusion in no more used for real since BEFORE 2.11
  • PROCESSORS: it is NO MORE USED since AT LEAST Rudder 2.11
  • MACHINEID: It is the same think that in <HARDWARE><UUID>. It used to be buggy with VM, but our is, too, and since 2.11 AT LEAST, if the MACHINEID field does not exists or is empty, we are derivating one from node UUID. So it may lead to some regeneration of machines, but that shouldn't break anything
  • SERVER_ROLES: exists only in added tag by CFEngine It was done and in 4.1, it is both under <RUDDER> and at the end in added tags.

So, given all the above, the only actions to do are:

  • add SERVER_ROLES to fusioninventory <RUDDER> tag in Rudder 3.1. It is a bug that it wasn't added in that place but in a deprecated one. Add the corresponding parsing in ldap-inventory. On the server, parse that and fall-back with parsing the one outside <RUDDER>
  • add support for HARDWARE/UUID, LOCAL_USERS and VIRTUALMACHINES in ldap-inventory with fallback to MACHINEID / USERLIST / VMS in Rudder 3.1

With that, we are able to safely remove ALL added things for nodes up-to-date in 3.1, and as soon as 3.1 - but of course, we won't do it before 4.0.

So, removing addition in 4.0, with the corresponding update in fusion/ldap-inventory in 3.1, covers the cases:
  • "last Rudder 3.1 or Rudder 4.0 with nodes in last 3.1, 3.2, or 4.0": as explained, all is good in that case.
  • "Rudder 4.0 in other case": as we continue fall-back parsing old place for SERVER_ROLE, LOCAL_USERS and VIRTUALMACHINES, it should just work as before.
  • "non last Rudder 3.1 with nodes 4.0": none of the missing tag is mandatory and as soon as the node get its system technique, it will add them (so at worst, first inventory is missing information). We could also keep the old addition only in initial promises for the first time (now that they are merged :)

Subtasks

User story #8920: Use FusionInventory system attributes instead of Rudder's custom onesReleasedFrançois ARMAND
User story #8922: Remove custom inventory modifications from the inventory promisesReleasedAlexis MOUSSET
Bug #12242: Error at node installation - Can't stat file '/var/rudder/cfengine-community/inputs/inventory/1.0/virtualMachines.cf' for parsing. (stat: No such file or directory)ReleasedAlexis MOUSSET
User story #12231: Remove parsing of MachineID in ldap-inventoryRejectedFrançois ARMAND

Related issues

Related to Rudder - User story #8127: On agent, check inventory before sending it to Rudder serverReleased2016-10-17
Related to Rudder - Bug #7092: Include Timezone in Node InfoReleased
Related to Rudder - User story #7093: Include Timezone in InventoryReleased2016-10-18
Related to Rudder - Bug #8921: The FusionInventory Rudder task does not work on Windows and will not return server rolesReleased2016-09-15

Associated revisions

Revision 88df5ccb (diff)
Added by François ARMAND 10 months ago

Fixes #8919: Remove modifications made to the inventory by the agent

History

#1 Updated by François ARMAND over 2 years ago

  • Related to User story #8127: On agent, check inventory before sending it to Rudder server added

#2 Updated by François ARMAND over 2 years ago

  • Description updated (diff)

We have in #8127 all the mandatory fields. We can see than NONE of the added fields are mandatory.
Also, explained what is exactly going on for each tags, in each version.

#3 Updated by François ARMAND over 2 years ago

  • Description updated (diff)

#4 Updated by François ARMAND about 2 years ago

  • Description updated (diff)

The proposed solution does not handle the case where a node was on 3.1-pre-fusion-updated, and is updated to system technique post-removal of the added option, then the node won't be able to provide SERVER_ROLES. That mean that we won't be able to know the role at all in such a case.

The only case where it is a problem is on relay servers installed before the patch, with a root server after the patch, or in a multi-version multi-instance rudder installation (which seems to be asking for problem)

So, we have three possibilities:

1/ keep the added extension in 4.0 for server_roles only. We will remove them in a long future away (until we don't support anymore relay on a pre-patch version, at least until end of 3.1 support)
2/ impose user to upgrade relay servers before upgrading Rudder server, and rudder-multi-instance install in one go;
3/ don't care for relay server, as server role for relay server is just an information, and users which rely on that information are able to switch first relay, then Rudder.

I really don't like 1/, because it means supporting a complexe addition for just one missing case. But at least, we won't have to support it on non cfengine agent, as it is relevant only for server with rudder roles.

3/ seem a little to surprising for people with group based on server_roles and who don't carefully read upgrade information. 2/ may not be ok because relay servers may be in difficult place to reach.

Thougts ?

#5 Updated by François ARMAND about 2 years ago

We also need to addd the agent timezone in Rudder extension, because it is blocking for #7092.

#6 Updated by François ARMAND about 2 years ago

  • Related to Bug #7092: Include Timezone in Node Info added

#7 Updated by François ARMAND about 2 years ago

#8 Updated by François ARMAND about 2 years ago

  • Target version changed from 4.0.0~rc2 to 4.1.0~beta1

#9 Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 4.1.0~beta1 to 4.1.0~beta2

#10 Updated by Vincent MEMBRÉ almost 2 years ago

  • Related to Bug #8921: The FusionInventory Rudder task does not work on Windows and will not return server roles added

#11 Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 4.1.0~beta2 to 4.1.0~beta3

#12 Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 4.1.0~beta3 to 4.1.0~rc1

#13 Updated by François ARMAND almost 2 years ago

  • Target version changed from 4.1.0~rc1 to 4.2.0~beta1

#14 Updated by Alexis MOUSSET over 1 year ago

  • Target version changed from 4.2.0~beta1 to 4.2.0~beta2

#15 Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 4.2.0~beta2 to 4.2.0~beta3

#16 Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 4.2.0~beta3 to 4.2.0~rc1

#17 Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 4.2.0~rc1 to 4.2.0~rc2

#18 Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 4.2.0~rc2 to 4.2.0

#19 Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 4.2.0 to 4.2.1

#20 Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 4.2.1 to 4.2.2

#21 Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 4.2.2 to 4.2.3

#22 Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 4.2.3 to 4.2.4

#23 Updated by François ARMAND 10 months ago

  • Target version changed from 4.2.4 to 4.3.0~rc1

It can be done when #8920 is merged, safe for server_role) but that will simplify a lot of things.

Note for MACHINEID: we don't use it for the machine id (i.e the node container link) at all since at least 3.1. So actually, MACHINEID is really the motherboarduuid, it is set in that place, but it is less reliable than newer fusion HARDWARE/UUID.

#24 Updated by François ARMAND 10 months ago

  • Description updated (diff)

So, in 4.3 we can:

- only parse <RUDDER> tag (and its children)
- remove all added tags at the end of the inventory file \o/

#25 Updated by François ARMAND 10 months ago

  • Status changed from New to In progress

#27 Updated by François ARMAND 10 months ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Vincent MEMBRÉ
  • Pull Request set to https://github.com/Normation/ldap-inventory/pull/129

#28 Updated by Normation Quality Assistant 10 months ago

  • Assignee changed from Vincent MEMBRÉ to François ARMAND

#29 Updated by Vincent MEMBRÉ 9 months ago

  • Target version changed from 4.3.0~rc1 to 4.3.0~rc2

#30 Updated by François ARMAND 9 months ago

  • Status changed from Pending technical review to Pending release

#31 Updated by Vincent MEMBRÉ 8 months ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 4.3.0~rc2 which was released today.

Also available in: Atom PDF