Project

General

Profile

Actions

Bug #15041

closed

The computation of changes by rules, at start of web interface, is quite slow

Added by Nicolas CHARLES almost 5 years ago. Updated almost 5 years ago.

Status:
Released
Priority:
N/A
Category:
Performance and scalability
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Reviewed
Fix check:
Checked
Regression:

Description

at the start of the application, we fetch all changes, in the last three days, with the following query

select ruleid, count(*) as number, ( extract('epoch' from executiontimestamp)::bigint - ${start})/21600 as interval
          from ruddersysevents
          where eventtype = 'result_repaired' and executionTimeStamp > '${new Timestamp(startTime.getMillis)}'::timestamp
          group by ruleid, interval;

this is quite inefficient, as:
  • postgres needs to to mathematical operation for every lines in ruddersysevents, to get an interval with an index
  • postgres goes over 3 days worth of reports

we should split this in 12 queries (1 per 6 hours), without any mathematical operation, like

select ruleid, count(*) as member  from ruddersysevents where eventtype = 'result_repaired' and executionTimeStamp > '2019-06-04 00:00:00.0'::timestamp and executionTimeStamp < '2019-06-04 06:00:00.0' group by ruleid

on super loaded instance, it goes from several hourshours to 12*2 minutes


Related issues 1 (1 open0 closed)

Related to Rudder - Bug #15092: Queries to fetch changes are too bigNewActions
Actions #1

Updated by Nicolas CHARLES almost 5 years ago

  • Status changed from New to In progress
  • Assignee set to Nicolas CHARLES
Actions #2

Updated by Nicolas CHARLES almost 5 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to François ARMAND
  • Pull Request set to https://github.com/Normation/rudder/pull/2246
Actions #3

Updated by Nicolas CHARLES almost 5 years ago

On a very partial database (6 hours of history), i have 122 061ms with this PR and 166 013 ms without

Actions #4

Updated by Nicolas CHARLES almost 5 years ago

with 17 hours:

618 245 without this PR
379 010 with this PR

Actions #5

Updated by Rudder Quality Assistant almost 5 years ago

  • Assignee changed from François ARMAND to Nicolas CHARLES
Actions #6

Updated by Nicolas CHARLES almost 5 years ago

  • Status changed from Pending technical review to Pending release
Actions #7

Updated by Nicolas CHARLES almost 5 years ago

  • Related to Bug #15092: Queries to fetch changes are too big added
Actions #8

Updated by Vincent MEMBRÉ almost 5 years ago

  • Name check set to To do
Actions #9

Updated by Vincent MEMBRÉ almost 5 years ago

  • Fix check set to To do
Actions #10

Updated by François ARMAND almost 5 years ago

  • Fix check changed from To do to Checked
Actions #11

Updated by Alexis Mousset almost 5 years ago

  • Name check changed from To do to Reviewed
Actions #12

Updated by Vincent MEMBRÉ almost 5 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 5.0.12 which was released today.

Actions

Also available in: Atom PDF