Actions
Architecture #15000
openWhile Rudder is fetching changes from the database, it cannot display the compliance for node list
Pull Request:
Effort required:
Name check:
Fix check:
Regression:
Description
With a large install, and a lot of repairs, fetching the changes list is fairly long (several minutes)
during this time, the interface is unable to show compliance in node list
web interface CPU usage is some percent, so it's clearly blocked
Updated by Vincent MEMBRÉ over 5 years ago
- Target version changed from 5.0.12 to 5.0.13
Updated by Vincent MEMBRÉ about 5 years ago
- Target version changed from 5.0.13 to 5.0.14
Updated by Vincent MEMBRÉ about 5 years ago
- Target version changed from 5.0.14 to 5.0.15
Updated by Nicolas CHARLES about 5 years ago
some metrics
rudder=> SELECT relname AS objectname, relkind AS objecttype, reltuples AS "#entries", pg_size_pretty(relpages::bigint*8*1024) AS size FROM pg_class WHERE relpages >= 8 ORDER BY relpages DESC; objectname | objecttype | #entries | size ------------------------------------------+------------+-------------+--------- ruddersysevents | r | 2.56554e+08 | 86 GB composite_node_execution_idx | i | 2.56554e+08 | 42 GB ruleid_idx | i | 2.56554e+08 | 34 GB keyvalue_idx | i | 2.56554e+08 | 19 GB executiontimestamp_idx | i | 2.56554e+08 | 15 GB ruddersysevents_pkey | i | 2.56554e+08 | 11 GB changes_executiontimestamp_idx | i | 1.27593e+07 | 751 MB pg_toast_16522 | t | 262164 | 510 MB reportsexecution_nodeid_nodeconfigid_idx | i | 691807 | 281 MB
creating a table with all the repairs of ruddersysevents:
CREATE SEQUENCE changeserial START 101; CREATE TABLE ChangeSysEvents ( id bigint PRIMARY KEY default nextval('changeserial') , executionDate timestamp with time zone NOT NULL , nodeId text NOT NULL CHECK (nodeId <> '') , directiveId text NOT NULL CHECK (directiveId <> '') , ruleId text NOT NULL CHECK (ruleId <> '') , serial integer NOT NULL , component text NOT NULL CHECK (component <> '') , keyValue text , executionTimeStamp timestamp with time zone NOT NULL , eventType text , policy text , msg text ); CREATE INDEX changes_executionTimeStamp_changes_idx ON ChangeSysEvents (executionTimeStamp) ; insert into ChangeSysEvents select * from ruddersysevents where eventType = 'result_repaired'; INSERT 0 12997914 SELECT relname AS objectname, relkind AS objecttype, reltuples AS "#entries", pg_size_pretty(relpages::bigint*8*1024) AS size FROM pg_class WHERE relpages >= 8 ORDER BY relpages DESC; objectname | objecttype | #entries | size ------------------------------------------+------------+-------------+--------- ruddersysevents | r | 2.56554e+08 | 86 GB composite_node_execution_idx | i | 2.56554e+08 | 42 GB ruleid_idx | i | 2.56554e+08 | 34 GB keyvalue_idx | i | 2.56554e+08 | 19 GB executiontimestamp_idx | i | 2.56554e+08 | 15 GB ruddersysevents_pkey | i | 2.56554e+08 | 11 GB changesysevents | r | 1.29989e+07 | 4491 MB changes_executiontimestamp_idx | i | 1.27593e+07 | 751 MB pg_toast_16522 | t | 262164 | 510 MB changesysevents_pkey | i | 1.29989e+07 | 373 MB changes_executiontimestamp_changes_idx | i | 1.29989e+07 | 284 MB
so the table is much larger than the previous index
Stripping useless content
alter table ChangeSysEvents drop column eventType; alter table ChangeSysEvents drop column policy;
saves only 300 MB on the table
So, as a result, having an external table "costs" 4,1GB+278MB+278MB (compared to the 751MB of the index in ruddersysevents)
Perf might be better, but this needs disks space
Updated by Vincent MEMBRÉ almost 5 years ago
- Target version changed from 5.0.15 to 5.0.16
Updated by Alexis Mousset almost 5 years ago
- Target version changed from 5.0.16 to 5.0.17
Updated by Vincent MEMBRÉ over 4 years ago
- Target version changed from 5.0.17 to 5.0.18
Updated by François ARMAND over 4 years ago
- Severity set to Major - prevents use of part of Rudder | no simple workaround
- User visibility set to Operational - other Techniques | Rudder settings | Plugins
- Priority changed from 0 to 43
Updated by Nicolas CHARLES over 4 years ago
- Related to Architecture #17921: improve searching in ruddersysevents for the reports in store run agents added
Updated by Vincent MEMBRÉ over 4 years ago
- Target version changed from 5.0.18 to 5.0.19
Updated by Vincent MEMBRÉ about 4 years ago
- Target version changed from 5.0.19 to 5.0.20
Updated by Vincent MEMBRÉ about 4 years ago
- Target version changed from 5.0.20 to 797
Updated by Benoît PECCATTE over 3 years ago
- Target version changed from 797 to 6.1.14
Updated by Vincent MEMBRÉ over 3 years ago
- Target version changed from 6.1.14 to 6.1.15
Updated by Vincent MEMBRÉ over 3 years ago
- Target version changed from 6.1.15 to 6.1.16
Updated by Vincent MEMBRÉ about 3 years ago
- Target version changed from 6.1.16 to 6.1.17
Updated by Vincent MEMBRÉ about 3 years ago
- Target version changed from 6.1.17 to 6.1.18
Updated by Vincent MEMBRÉ almost 3 years ago
- Target version changed from 6.1.18 to 6.1.19
Updated by François ARMAND almost 3 years ago
- Tracker changed from Bug to Architecture
- Severity deleted (
Major - prevents use of part of Rudder | no simple workaround) - User visibility deleted (
Operational - other Techniques | Rudder settings | Plugins) - Priority deleted (
43)
Updated by Vincent MEMBRÉ over 2 years ago
- Target version changed from 6.1.19 to 6.1.20
Updated by Vincent MEMBRÉ over 2 years ago
- Target version changed from 6.1.20 to 6.1.21
Updated by Vincent MEMBRÉ over 2 years ago
- Target version changed from 6.1.21 to old 6.1 issues to relocate
Updated by Alexis Mousset over 1 year ago
- Target version changed from old 6.1 issues to relocate to 8.0.0~alpha1
Updated by Vincent MEMBRÉ over 1 year ago
- Target version changed from 8.0.0~alpha1 to 8.0.0~beta1
Updated by Alexis Mousset over 1 year ago
- Target version changed from 8.0.0~beta1 to 8.1.0~alpha1
Updated by Vincent MEMBRÉ 10 months ago
- Target version changed from 8.1.0~alpha1 to 8.1.0~beta1
Updated by Vincent MEMBRÉ 9 months ago
- Target version changed from 8.1.0~beta1 to 8.1.0~beta2
Updated by Vincent MEMBRÉ 8 months ago
- Target version changed from 8.1.0~beta2 to 8.1.0~rc1
Updated by Vincent MEMBRÉ 8 months ago
- Target version changed from 8.1.0~rc1 to 8.1.0
Updated by Vincent MEMBRÉ 8 months ago
- Target version changed from 8.1.0 to 8.1.1
Updated by Vincent MEMBRÉ 7 months ago
- Target version changed from 8.1.1 to 8.1.2
Updated by Vincent MEMBRÉ 7 months ago
- Target version changed from 8.1.2 to 8.1.3
Updated by Vincent MEMBRÉ 6 months ago
- Target version changed from 8.1.3 to 8.1.4
Updated by Vincent MEMBRÉ 5 months ago
- Target version changed from 8.1.4 to 8.1.5
Updated by Vincent MEMBRÉ 5 months ago
- Target version changed from 8.1.5 to 8.1.6
Updated by Vincent MEMBRÉ 4 months ago
- Target version changed from 8.1.6 to 8.1.7
Updated by Vincent MEMBRÉ 2 months ago
- Target version changed from 8.1.7 to 8.1.8
Updated by Vincent MEMBRÉ 14 days ago
- Target version changed from 8.1.8 to 8.1.9
Actions