Project

General

Profile

Actions

User story #14144

open

When pgsql is shutdown because partition is full, agent should also shutdown rudder web app

Added by François ARMAND almost 6 years ago. Updated about 4 years ago.

Status:
New
Priority:
N/A
Assignee:
-
Category:
System integration
Target version:
-
UX impact:
Suggestion strength:
User visibility:
Effort required:
Name check:
Fix check:
Regression:

Description

To prevent grave data corruption on postgresql database, we have a policy on Rudder server that looks at partition size on DB and shutdown postgres if the partition is full.

The problem is that Rudder web app is still available, but seems broken / not up to date. We want to also shutdown jetty so that it is really apparent that something is not going as expected and that the user must take a look.

(not: this must be documented with a big warning to help people be aware that that behavior may happen).


Related issues 1 (0 open1 closed)

Related to Rudder - Bug #11739: When we have a /var full, Rudder LDAP or Postgres can be corruptedReleasedAlexis MoussetActions
Actions #1

Updated by Nicolas CHARLES almost 6 years ago

We could/should also have a script that would clean database (purging indexes and recreating them concurrently), that would stop web interface, get current maintenace_mem ( select current_setting('maintenance_work_mem'); ), increase maintenance_work_mem (SET maintenance_work_mem TO '32 MB';), recreat indexes, restore old value of maintenance_work_mem, check free space, restart web interface

and we could either run it automatically when out of space (and not more often than once per X day), or strongly suggest to user to run it if rudder is stopped (change the loading page in case of malfunction?)

Actions #2

Updated by Nicolas CHARLES almost 6 years ago

  • Related to Bug #11739: When we have a /var full, Rudder LDAP or Postgres can be corrupted added
Actions #3

Updated by Nicolas CHARLES almost 6 years ago

Shutdown doesn't work anymore in Rudder 5.0, as rudder is not a service anymore there, but rudder-server is
In 5.1 postgresql is not a dependency anymore of rudder, so shuting down rudder doesn't stop postgres anymore

Plus, we hsould definitively NOT stop postgresql automatically if rsyslog is in TCP, the risk is too high for the infrastructure

Actions #4

Updated by Alexis Mousset almost 6 years ago

  • Target version changed from 5.0.5 to 5.0.6
Actions #5

Updated by Vincent MEMBRÉ almost 6 years ago

  • Target version changed from 5.0.6 to 5.0.7
Actions #6

Updated by François ARMAND almost 6 years ago

  • Target version changed from 5.0.7 to 5.0.9
Actions #7

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 5.0.9 to 5.0.10
Actions #8

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 5.0.10 to 5.0.11
Actions #9

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 5.0.11 to 5.0.12
Actions #10

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 5.0.12 to 5.0.13
Actions #11

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 5.0.13 to 5.0.14
Actions #12

Updated by Vincent MEMBRÉ about 5 years ago

  • Target version changed from 5.0.14 to 5.0.15
Actions #13

Updated by Vincent MEMBRÉ about 5 years ago

  • Target version changed from 5.0.15 to 5.0.16
Actions #14

Updated by Alexis Mousset almost 5 years ago

  • Target version changed from 5.0.16 to 5.0.17
Actions #15

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 5.0.17 to 5.0.18
Actions #16

Updated by Benoît PECCATTE over 4 years ago

  • Target version changed from 5.0.18 to 6.2.0~beta1
Actions #17

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 6.2.0~beta1 to 6.2.0~rc1
Actions #18

Updated by François ARMAND about 4 years ago

  • Target version deleted (6.2.0~rc1)
Actions

Also available in: Atom PDF