Project

General

Profile

Actions

Architecture #15000

open

While Rudder is fetching changes from the database, it cannot display the compliance for node list

Added by Nicolas CHARLES almost 5 years ago. Updated 12 days ago.

Status:
New
Priority:
N/A
Assignee:
-
Category:
Performance and scalability
Target version:
Effort required:
Name check:
Fix check:
Regression:

Description

With a large install, and a lot of repairs, fetching the changes list is fairly long (several minutes)
during this time, the interface is unable to show compliance in node list

web interface CPU usage is some percent, so it's clearly blocked


Related issues 1 (1 open0 closed)

Related to Rudder - Architecture #17921: improve searching in ruddersysevents for the reports in store run agentsNewActions
Actions #1

Updated by Vincent MEMBRÉ almost 5 years ago

  • Target version changed from 5.0.12 to 5.0.13
Actions #2

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 5.0.13 to 5.0.14
Actions #3

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 5.0.14 to 5.0.15
Actions #4

Updated by Nicolas CHARLES over 4 years ago

some metrics

rudder=> SELECT
   relname AS objectname,
   relkind AS objecttype,
   reltuples AS "#entries", pg_size_pretty(relpages::bigint*8*1024) AS size
   FROM pg_class
   WHERE relpages >= 8
   ORDER BY relpages DESC;
                objectname                | objecttype |  #entries   |  size   
------------------------------------------+------------+-------------+---------
 ruddersysevents                          | r          | 2.56554e+08 | 86 GB
 composite_node_execution_idx             | i          | 2.56554e+08 | 42 GB
 ruleid_idx                               | i          | 2.56554e+08 | 34 GB
 keyvalue_idx                             | i          | 2.56554e+08 | 19 GB
 executiontimestamp_idx                   | i          | 2.56554e+08 | 15 GB
 ruddersysevents_pkey                     | i          | 2.56554e+08 | 11 GB
 changes_executiontimestamp_idx           | i          | 1.27593e+07 | 751 MB
 pg_toast_16522                           | t          |      262164 | 510 MB
 reportsexecution_nodeid_nodeconfigid_idx | i          |      691807 | 281 MB

creating a table with all the repairs of ruddersysevents:


CREATE SEQUENCE changeserial START 101;

CREATE TABLE ChangeSysEvents (
  id                 bigint PRIMARY KEY default nextval('changeserial')
, executionDate      timestamp with time zone NOT NULL
, nodeId             text NOT NULL CHECK (nodeId <> '')
, directiveId        text NOT NULL CHECK (directiveId <> '')
, ruleId             text NOT NULL CHECK (ruleId <> '')
, serial             integer NOT NULL
, component          text NOT NULL CHECK (component <> '')
, keyValue           text
, executionTimeStamp timestamp with time zone NOT NULL
, eventType          text
, policy             text
, msg                text
);

CREATE INDEX changes_executionTimeStamp_changes_idx ON ChangeSysEvents (executionTimeStamp) ;

insert into ChangeSysEvents select * from ruddersysevents where eventType = 'result_repaired';
INSERT 0 12997914

SELECT
   relname AS objectname,
   relkind AS objecttype,
   reltuples AS "#entries", pg_size_pretty(relpages::bigint*8*1024) AS size
   FROM pg_class
   WHERE relpages >= 8
   ORDER BY relpages DESC;
                objectname                | objecttype |  #entries   |  size   
------------------------------------------+------------+-------------+---------
 ruddersysevents                          | r          | 2.56554e+08 | 86 GB
 composite_node_execution_idx             | i          | 2.56554e+08 | 42 GB
 ruleid_idx                               | i          | 2.56554e+08 | 34 GB
 keyvalue_idx                             | i          | 2.56554e+08 | 19 GB
 executiontimestamp_idx                   | i          | 2.56554e+08 | 15 GB
 ruddersysevents_pkey                     | i          | 2.56554e+08 | 11 GB
 changesysevents                          | r          | 1.29989e+07 | 4491 MB
 changes_executiontimestamp_idx           | i          | 1.27593e+07 | 751 MB
 pg_toast_16522                           | t          |      262164 | 510 MB
 changesysevents_pkey                     | i          | 1.29989e+07 | 373 MB
 changes_executiontimestamp_changes_idx   | i          | 1.29989e+07 | 284 MB

so the table is much larger than the previous index

Stripping useless content

alter table ChangeSysEvents drop column eventType;
alter table ChangeSysEvents drop column policy;

saves only 300 MB on the table

So, as a result, having an external table "costs" 4,1GB+278MB+278MB (compared to the 751MB of the index in ruddersysevents)
Perf might be better, but this needs disks space

Actions #5

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 5.0.15 to 5.0.16
Actions #6

Updated by Alexis Mousset about 4 years ago

  • Target version changed from 5.0.16 to 5.0.17
Actions #7

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 5.0.17 to 5.0.18
Actions #8

Updated by François ARMAND almost 4 years ago

  • Severity set to Major - prevents use of part of Rudder | no simple workaround
  • User visibility set to Operational - other Techniques | Rudder settings | Plugins
  • Priority changed from 0 to 43
Actions #9

Updated by Nicolas CHARLES almost 4 years ago

  • Related to Architecture #17921: improve searching in ruddersysevents for the reports in store run agents added
Actions #10

Updated by Vincent MEMBRÉ almost 4 years ago

  • Target version changed from 5.0.18 to 5.0.19
Actions #11

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 5.0.19 to 5.0.20
Actions #12

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 5.0.20 to 797
Actions #13

Updated by Benoît PECCATTE almost 3 years ago

  • Target version changed from 797 to 6.1.14
Actions #14

Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 6.1.14 to 6.1.15
Actions #15

Updated by Vincent MEMBRÉ almost 3 years ago

  • Target version changed from 6.1.15 to 6.1.16
Actions #16

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.16 to 6.1.17
Actions #17

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.17 to 6.1.18
Actions #18

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 6.1.18 to 6.1.19
Actions #19

Updated by François ARMAND about 2 years ago

  • Tracker changed from Bug to Architecture
  • Severity deleted (Major - prevents use of part of Rudder | no simple workaround)
  • User visibility deleted (Operational - other Techniques | Rudder settings | Plugins)
  • Priority deleted (43)
Actions #20

Updated by Vincent MEMBRÉ about 2 years ago

  • Target version changed from 6.1.19 to 6.1.20
Actions #21

Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 6.1.20 to 6.1.21
Actions #22

Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 6.1.21 to old 6.1 issues to relocate
Actions #23

Updated by Alexis Mousset about 1 year ago

  • Target version changed from old 6.1 issues to relocate to 8.0.0~alpha1
Actions #24

Updated by Vincent MEMBRÉ 10 months ago

  • Target version changed from 8.0.0~alpha1 to 8.0.0~beta1
Actions #25

Updated by Alexis Mousset 9 months ago

  • Target version changed from 8.0.0~beta1 to 8.1.0~alpha1
Actions #26

Updated by Vincent MEMBRÉ 3 months ago

  • Target version changed from 8.1.0~alpha1 to 8.1.0~beta1
Actions #27

Updated by Vincent MEMBRÉ about 2 months ago

  • Target version changed from 8.1.0~beta1 to 8.1.0~beta2
Actions #28

Updated by Vincent MEMBRÉ about 1 month ago

  • Target version changed from 8.1.0~beta2 to 8.1.0~rc1
Actions #29

Updated by Vincent MEMBRÉ 18 days ago

  • Target version changed from 8.1.0~rc1 to 8.1.0
Actions #30

Updated by Vincent MEMBRÉ 12 days ago

  • Target version changed from 8.1.0 to 8.1.1
Actions

Also available in: Atom PDF