Project

General

Profile

Actions

Bug #19650

closed

Need a migration script about changes in system directives, groups and rules

Added by Nicolas CHARLES over 3 years ago. Updated about 3 years ago.

Status:
Released
Priority:
N/A
Category:
System integration
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
To do
Fix check:
To do
Regression:

Description

It needs to take into account all changes in system techniques and it also need to remove all_servers_with_role and all_servers_without_role
otherwise we have error in logs about ruleTargets:

/var/log/rudder/webapp/2021_08_02.stderrout.log:[2021-08-02 11:50:48+0200] WARN  com.normation.rudder.repository.ldap.RoLDAPNodeGroupRepository - Error when mapping entry with DN 'ruleTarget=special:all_servers_with_role,groupCategoryId=SystemGroups,groupCategoryId=GroupRoot,ou=Rudder,cn=rudder-configuration' from node groups library, that entry will be ignored; cause was: UnexpectedObject: Can not unserialize target, 'special:all_servers_with_role' does not match any known target format
/var/log/rudder/webapp/2021_08_02.stderrout.log:[2021-08-02 11:50:47+0200] WARN  com.normation.rudder.repository.ldap.RoLDAPNodeGroupRepository - Error when mapping entry with DN 'ruleTarget=special:all_nodes_without_role,groupCategoryId=SystemGroups,groupCategoryId=GroupRoot,ou=Rudder,cn=rudder-configuration' from node groups library, that entry will be ignored; cause was: UnexpectedObject: Can not unserialize target, 'special:all_nodes_without_role' does not match any known target format

The migration will be handled by the webapp.
Things to migrate:

- allowed networks from directive to their setting
- all system techniques & related objects
- remove old group linked to roles

For each one, we do it in a "create new config, validate it's ok, delete old config" fashion.
The first two step are linked because we can't delete allowed network in common directive before and of step 2.

The third is simpler (just deletion).

An error in that migration must lead to a coredump and a big error message and actionnable info for the poor ops.

Here comes a summary of all renaming and related migration for config objects:

Techniques: we don't migrate them, but we need to check that we have these one loaded:
===========
- common (applies on all nodes, be it root, a relay, or a simple node. Manage inventory, agent config and runs, etc)
- server-common (applies on policy servers, same naming scheme than dsc-common. Manage policy distribution, etc)
- rudder-service-apache
- rudder-service-postgresql
- rudder-service-relayd
- rudder-service-slapd
- rudder-service-webapp

If dsc plugin present:
- dsc-common 

Directives: this is the target result:
===========
Conventions:
- postfix by all if applied to all nodes
- each time the directive is specific to a policy server, postfix with its id
- commons depend upon the policy server of the node it is applied to, so postfix with "hasPolicyServer-${policyserverid}" 

Example for root: 
inventory/inventory-all
common/common-hasPolicyServer-root
server-common/server-common-root
rudder-service-apache/rudder-service-apache-root
rudder-service-postgresql/rudder-service-postgresql-root
rudder-service-relayd/rudder-service-relayd-root
rudder-service-slapd/rudder-service-slapd-root
rudder-service-webapp/rudder-service-webapp-root

And for relays:
server-common/server-common-$relayid
rudder-service-apache/rudder-service-apache-$relayid
rudder-service-relayd/rudder-service-relayd-$relayid

For DSC:
dsc-common/dsc-common-all => ok

We will get information from the following directives: 
- common
- distributPolicy
- inventory
- dsc-common

Groups: target is:
=======

- nodeGroupId=all-nodes-with-cfengine-agent 
- nodeGroupId=all-nodes-with-dsc-agent      
- nodeGroupId=hasPolicyServer-root          
- ruleTarget=policyServer:root            

(and for relay, same with $relayid)

So here, there is nothing to change.

Rules:
======
(convention: "-" for directives, "*" for groups)

inventory-all
- inventory-all
* group:all-nodes-with-cfengine-agent

=> nothing to change

hasPolicyServer-root
- common-hasPolicyServer-root
* group:hasPolicyServer-root

=> change directive from common-root to common-hasPolicyServer-root

root-DP => rename to policy-server-root + change all directives to match:
- server-common-root
- rudder-service-apache-root
- rudder-service-postgresql-root
- rudder-service-relayd-root
- rudder-service-slapd-root
- rudder-service-webapp-root
* policyServer:root

For relays: 
${relayId}-distributePolicy => policy-server-$relayid and change directives
- server-common-$relayid
- rudder-service-apache-$relayid
- rudder-service-relayd-$relayid
* policyServer:$relayid

DSC (keep it like in 6.2):

dsc-agent-all
- dsc-common-all
* group:all-nodes-with-dsc-agent

Server roles
============
Remove active techniques + directives "server-roles" 
Remove rule "server-roles" 


Subtasks 2 (0 open2 closed)

Bug #20018: Remove no more used POLICYSERVER variableReleasedVincent MEMBRÉActions
Bug #20091: System technique are not added as system and should not be disabledReleasedNicolas CHARLESActions

Related issues 2 (1 open1 closed)

Related to Rudder - User story #19625: Remove server roles in webapp and add support for remote postgresReleasedAlexis MoussetActions
Related to Rudder - Bug #20790: Groups based on server roles are not modified during upgradeNewActions
Actions #1

Updated by Nicolas CHARLES over 3 years ago

  • Status changed from New to In progress
  • Assignee set to Nicolas CHARLES
Actions #2

Updated by Nicolas CHARLES over 3 years ago

  • Description updated (diff)
Actions #3

Updated by Nicolas CHARLES over 3 years ago

  • Subject changed from Error in logs about rule target to Need a migration script about ruletarget all_servers_with_role and all_servers_without_role
  • Description updated (diff)
  • Category changed from Server components to Packaging
Actions #4

Updated by Nicolas CHARLES over 3 years ago

  • Status changed from In progress to New
Actions #5

Updated by François ARMAND over 3 years ago

  • Subject changed from Need a migration script about ruletarget all_servers_with_role and all_servers_without_role to Need a migration script about changes in system directives, groups and rules
  • Description updated (diff)
  • Category changed from Packaging to System integration
  • Assignee deleted (Nicolas CHARLES)
  • Target version changed from 7.0.0~beta1 to 900
Actions #6

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 900 to 7.0.0~beta2
Actions #7

Updated by Vincent MEMBRÉ over 3 years ago

  • Parent task deleted (#19625)
Actions #8

Updated by Vincent MEMBRÉ over 3 years ago

  • Related to User story #19625: Remove server roles in webapp and add support for remote postgres added
Actions #9

Updated by François ARMAND over 3 years ago

  • Status changed from New to In progress
  • Assignee set to François ARMAND
Actions #10

Updated by François ARMAND over 3 years ago

  • Description updated (diff)
Actions #14

Updated by François ARMAND about 3 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Vincent MEMBRÉ
  • Pull Request set to https://github.com/Normation/rudder/pull/3898
Actions #15

Updated by François ARMAND about 3 years ago

  • Assignee changed from Vincent MEMBRÉ to Nicolas CHARLES
Actions #16

Updated by François ARMAND about 3 years ago

  • Assignee changed from Nicolas CHARLES to Vincent MEMBRÉ
Actions #17

Updated by François ARMAND about 3 years ago

  • Status changed from Pending technical review to Pending release
Actions #18

Updated by Vincent MEMBRÉ about 3 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 7.0.0~beta2 which was released today.

Actions #19

Updated by François ARMAND almost 3 years ago

  • Related to Bug #20790: Groups based on server roles are not modified during upgrade added
Actions

Also available in: Atom PDF