Bug #19650
closedNeed a migration script about changes in system directives, groups and rules
Description
It needs to take into account all changes in system techniques and it also need to remove all_servers_with_role and all_servers_without_role
otherwise we have error in logs about ruleTargets:
/var/log/rudder/webapp/2021_08_02.stderrout.log:[2021-08-02 11:50:48+0200] WARN com.normation.rudder.repository.ldap.RoLDAPNodeGroupRepository - Error when mapping entry with DN 'ruleTarget=special:all_servers_with_role,groupCategoryId=SystemGroups,groupCategoryId=GroupRoot,ou=Rudder,cn=rudder-configuration' from node groups library, that entry will be ignored; cause was: UnexpectedObject: Can not unserialize target, 'special:all_servers_with_role' does not match any known target format /var/log/rudder/webapp/2021_08_02.stderrout.log:[2021-08-02 11:50:47+0200] WARN com.normation.rudder.repository.ldap.RoLDAPNodeGroupRepository - Error when mapping entry with DN 'ruleTarget=special:all_nodes_without_role,groupCategoryId=SystemGroups,groupCategoryId=GroupRoot,ou=Rudder,cn=rudder-configuration' from node groups library, that entry will be ignored; cause was: UnexpectedObject: Can not unserialize target, 'special:all_nodes_without_role' does not match any known target format
The migration will be handled by the webapp.
Things to migrate:
- allowed networks from directive to their setting
- all system techniques & related objects
- remove old group linked to roles
For each one, we do it in a "create new config, validate it's ok, delete old config" fashion.
The first two step are linked because we can't delete allowed network in common directive before and of step 2.
The third is simpler (just deletion).
An error in that migration must lead to a coredump and a big error message and actionnable info for the poor ops.
Here comes a summary of all renaming and related migration for config objects:
Techniques: we don't migrate them, but we need to check that we have these one loaded:
===========
- common (applies on all nodes, be it root, a relay, or a simple node. Manage inventory, agent config and runs, etc)
- server-common (applies on policy servers, same naming scheme than dsc-common. Manage policy distribution, etc)
- rudder-service-apache
- rudder-service-postgresql
- rudder-service-relayd
- rudder-service-slapd
- rudder-service-webapp
If dsc plugin present:
- dsc-common
Directives: this is the target result:
===========
Conventions:
- postfix by all if applied to all nodes
- each time the directive is specific to a policy server, postfix with its id
- commons depend upon the policy server of the node it is applied to, so postfix with "hasPolicyServer-${policyserverid}"
Example for root:
inventory/inventory-all
common/common-hasPolicyServer-root
server-common/server-common-root
rudder-service-apache/rudder-service-apache-root
rudder-service-postgresql/rudder-service-postgresql-root
rudder-service-relayd/rudder-service-relayd-root
rudder-service-slapd/rudder-service-slapd-root
rudder-service-webapp/rudder-service-webapp-root
And for relays:
server-common/server-common-$relayid
rudder-service-apache/rudder-service-apache-$relayid
rudder-service-relayd/rudder-service-relayd-$relayid
For DSC:
dsc-common/dsc-common-all => ok
We will get information from the following directives:
- common
- distributPolicy
- inventory
- dsc-common
Groups: target is:
=======
- nodeGroupId=all-nodes-with-cfengine-agent
- nodeGroupId=all-nodes-with-dsc-agent
- nodeGroupId=hasPolicyServer-root
- ruleTarget=policyServer:root
(and for relay, same with $relayid)
So here, there is nothing to change.
Rules:
======
(convention: "-" for directives, "*" for groups)
inventory-all
- inventory-all
* group:all-nodes-with-cfengine-agent
=> nothing to change
hasPolicyServer-root
- common-hasPolicyServer-root
* group:hasPolicyServer-root
=> change directive from common-root to common-hasPolicyServer-root
root-DP => rename to policy-server-root + change all directives to match:
- server-common-root
- rudder-service-apache-root
- rudder-service-postgresql-root
- rudder-service-relayd-root
- rudder-service-slapd-root
- rudder-service-webapp-root
* policyServer:root
For relays:
${relayId}-distributePolicy => policy-server-$relayid and change directives
- server-common-$relayid
- rudder-service-apache-$relayid
- rudder-service-relayd-$relayid
* policyServer:$relayid
DSC (keep it like in 6.2):
dsc-agent-all
- dsc-common-all
* group:all-nodes-with-dsc-agent
Server roles
============
Remove active techniques + directives "server-roles"
Remove rule "server-roles"
Updated by Nicolas CHARLES almost 5 years ago
- Status changed from New to In progress
- Assignee set to Nicolas CHARLES
Updated by Nicolas CHARLES almost 5 years ago
- Description updated (diff)
Updated by Nicolas CHARLES almost 5 years ago
- Subject changed from Error in logs about rule target to Need a migration script about ruletarget all_servers_with_role and all_servers_without_role
- Description updated (diff)
- Category changed from Server components to Packaging
Updated by Nicolas CHARLES almost 5 years ago
- Status changed from In progress to New
Updated by François ARMAND almost 5 years ago
- Subject changed from Need a migration script about ruletarget all_servers_with_role and all_servers_without_role to Need a migration script about changes in system directives, groups and rules
- Description updated (diff)
- Category changed from Packaging to System integration
- Assignee deleted (
Nicolas CHARLES) - Target version changed from 7.0.0~beta1 to 900
Updated by Vincent MEMBRÉ almost 5 years ago
- Target version changed from 900 to 7.0.0~beta2
Updated by Vincent MEMBRÉ almost 5 years ago
- Parent task deleted (
#19625)
Updated by Vincent MEMBRÉ almost 5 years ago
- Related to User story #19625: Remove server roles in webapp and add support for remote postgres added
Updated by François ARMAND over 4 years ago
- Status changed from New to In progress
- Assignee set to François ARMAND
Updated by François ARMAND over 4 years ago
- Description updated (diff)
Updated by François ARMAND over 4 years ago
Work in progess here: https://github.com/fanf/rudder/commit/7b8d112b03faba358133d85113ff46574ac517fd
Updated by François ARMAND over 4 years ago
Work in progess here: https://github.com/fanf/rudder/commit/c85d55c22102eecb064465fd71793a693b1edd79
Updated by François ARMAND over 4 years ago
Work in progess here: https://github.com/fanf/rudder/commit/548d8ee3178d45a24e1b56ec38df256e13d0019f
Updated by François ARMAND over 4 years ago
- Status changed from In progress to Pending technical review
- Assignee changed from François ARMAND to Vincent MEMBRÉ
- Pull Request set to https://github.com/Normation/rudder/pull/3898
Updated by François ARMAND over 4 years ago
- Assignee changed from Vincent MEMBRÉ to Nicolas CHARLES
Updated by François ARMAND over 4 years ago
- Assignee changed from Nicolas CHARLES to Vincent MEMBRÉ
Updated by François ARMAND over 4 years ago
- Status changed from Pending technical review to Pending release
Applied in changeset rudder|f4918924913b454c616dfefe7acc429207e292da.
Updated by Vincent MEMBRÉ over 4 years ago
- Status changed from Pending release to Released
This bug has been fixed in Rudder 7.0.0~beta2 which was released today.
Updated by François ARMAND over 4 years ago
- Related to Bug #20790: Groups based on server roles are not modified during upgrade added