Project

General

Profile

Actions

Architecture #15864

open

Add an option to remove the backup of node policy at the end of policy generation

Added by Nicolas CHARLES over 4 years ago. Updated 9 days ago.

Status:
New
Priority:
N/A
Category:
Performance and scalability
Target version:
Effort required:
Name check:
Fix check:
To do
Regression:

Description

When we generate policies, we are deleting previous backup, moving previous policies to backup folder, and moving new policies to final position. This can be really long (if it's on a different mount point, it can take up to a 1/3 of the policy generation time, for instance with 10 000 nodes, it took 1200 seconds), but we need to move the previous policies out of the way to have unitary changes - ticket #16072 takes care of documenting that they should be on the same FS.
Ultimately, these backup are rarely used, so we could simply drop them async at the end of generation, so that we don't have to delete them during next policy generation, and save some disk space, time and IO - this could be done via a post generation hook

  1. remove the backup of previous policies at the end generation
    curl -k -H "X-API-Token: xxx" -X POST 'https://.../rudder/api/latest/settings/rudder_remove_previous_policies' -d "value=true"

Related issues 1 (0 open1 closed)

Related to Rudder - Bug #16072: Improve documentation in rudder-web.properties about backup folderReleasedFrançois ARMANDActions
Actions #1

Updated by Nicolas CHARLES over 4 years ago

  • Status changed from New to In progress
  • Assignee set to Nicolas CHARLES
Actions #2

Updated by Nicolas CHARLES over 4 years ago

  • Description updated (diff)
  • Category deleted (Performance and scalability)
  • Target version deleted (5.0.15)
  • Name check deleted (To do)
Actions #3

Updated by Nicolas CHARLES over 4 years ago

  • Category set to Performance and scalability
  • Target version set to 5.0.15
Actions #4

Updated by Nicolas CHARLES over 4 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to François ARMAND
  • Pull Request set to https://github.com/Normation/rudder/pull/2561
Actions #5

Updated by Nicolas CHARLES over 4 years ago

  • Related to Bug #16072: Improve documentation in rudder-web.properties about backup folder added
Actions #6

Updated by Nicolas CHARLES over 4 years ago

  • Status changed from Pending technical review to In progress

it's a bit more complex than that - deleting rather than moving the old policies is not atomic, so can end up with broken policies on the node/relay if shared at the wrong time (the error windows is too large).
Note that having /var/rudder/backup on a different fs than /var/rudder/share result also in non atomic change (the move is a copy+delete)

Actions #7

Updated by Nicolas CHARLES over 4 years ago

  • Subject changed from add an option to prevent backup of node policy at the end of policy generation to Add an option to remove the backup of node policy at the end of policy generation
  • Description updated (diff)
Actions #8

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 5.0.15 to 5.0.16
Actions #9

Updated by Alexis Mousset about 4 years ago

  • Target version changed from 5.0.16 to 5.0.17
Actions #10

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 5.0.17 to 5.0.18
Actions #11

Updated by François ARMAND almost 4 years ago

  • Tracker changed from Bug to Architecture
  • Target version changed from 5.0.18 to 6.2.0~beta1
  • Priority deleted (0)
Actions #12

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.2.0~beta1 to 6.2.0~rc1
Actions #13

Updated by François ARMAND over 3 years ago

  • Target version changed from 6.2.0~rc1 to 6.1.7
Actions #14

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.1.7 to 6.1.8
Actions #15

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.1.8 to 6.1.9
Actions #16

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.9 to 6.1.10
Actions #17

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.10 to 6.1.11
Actions #18

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.11 to 6.1.12
Actions #19

Updated by Vincent MEMBRÉ about 3 years ago

  • Target version changed from 6.1.12 to 6.1.13
Actions #20

Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 6.1.13 to 6.1.14
Actions #21

Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 6.1.14 to 6.1.15
Actions #22

Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 6.1.15 to 6.1.16
Actions #23

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.16 to 6.1.17
Actions #24

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.17 to 6.1.18
Actions #25

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.18 to 6.1.19
Actions #26

Updated by Vincent MEMBRÉ about 2 years ago

  • Target version changed from 6.1.19 to 6.1.20
Actions #27

Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 6.1.20 to 6.1.21
Actions #28

Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 6.1.21 to old 6.1 issues to relocate
Actions #29

Updated by Alexis Mousset about 1 year ago

  • Status changed from In progress to New
Actions #30

Updated by Alexis Mousset about 1 year ago

  • Target version changed from old 6.1 issues to relocate to 8.0.0~alpha1
Actions #31

Updated by Vincent MEMBRÉ 9 months ago

  • Target version changed from 8.0.0~alpha1 to 8.0.0~beta1
Actions #32

Updated by Alexis Mousset 9 months ago

  • Target version changed from 8.0.0~beta1 to 8.1.0~alpha1
Actions #33

Updated by Vincent MEMBRÉ 3 months ago

  • Target version changed from 8.1.0~alpha1 to 8.1.0~beta1
Actions #34

Updated by Vincent MEMBRÉ about 2 months ago

  • Target version changed from 8.1.0~beta1 to 8.1.0~beta2
Actions #35

Updated by Vincent MEMBRÉ 30 days ago

  • Target version changed from 8.1.0~beta2 to 8.1.0~rc1
Actions #36

Updated by Vincent MEMBRÉ 15 days ago

  • Target version changed from 8.1.0~rc1 to 8.1.0
Actions #37

Updated by Vincent MEMBRÉ 9 days ago

  • Target version changed from 8.1.0 to 8.1.1
Actions

Also available in: Atom PDF