Project

General

Profile

Bug #15041

The computation of changes by rules, at start of web interface, is quite slow

Added by Nicolas CHARLES 4 months ago. Updated 3 months ago.

Status:
Released
Priority:
N/A
Category:
Performance and scalability
Target version:
Severity:
User visibility:
Effort required:
Priority:
0

Description

at the start of the application, we fetch all changes, in the last three days, with the following query

select ruleid, count(*) as number, ( extract('epoch' from executiontimestamp)::bigint - ${start})/21600 as interval
          from ruddersysevents
          where eventtype = 'result_repaired' and executionTimeStamp > '${new Timestamp(startTime.getMillis)}'::timestamp
          group by ruleid, interval;

this is quite inefficient, as:
  • postgres needs to to mathematical operation for every lines in ruddersysevents, to get an interval with an index
  • postgres goes over 3 days worth of reports

we should split this in 12 queries (1 per 6 hours), without any mathematical operation, like

select ruleid, count(*) as member  from ruddersysevents where eventtype = 'result_repaired' and executionTimeStamp > '2019-06-04 00:00:00.0'::timestamp and executionTimeStamp < '2019-06-04 06:00:00.0' group by ruleid

on super loaded instance, it goes from several hourshours to 12*2 minutes


Related issues

Related to Rudder - Bug #15092: Queries to fetch changes are too bigNewActions

Associated revisions

Revision e6aaea38 (diff)
Added by Nicolas CHARLES 4 months ago

Fixes #15041: The computation of changes by rules, at start of web interface, is quite slow

History

#1

Updated by Nicolas CHARLES 4 months ago

  • Status changed from New to In progress
  • Assignee set to Nicolas CHARLES
#2

Updated by Nicolas CHARLES 4 months ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to François ARMAND
  • Pull Request set to https://github.com/Normation/rudder/pull/2246
#3

Updated by Nicolas CHARLES 4 months ago

On a very partial database (6 hours of history), i have 122 061ms with this PR and 166 013 ms without

#4

Updated by Nicolas CHARLES 4 months ago

with 17 hours:

618 245 without this PR
379 010 with this PR

#5

Updated by Rudder Quality Assistant 4 months ago

  • Assignee changed from François ARMAND to Nicolas CHARLES
#6

Updated by Nicolas CHARLES 4 months ago

  • Status changed from Pending technical review to Pending release
#7

Updated by Nicolas CHARLES 4 months ago

  • Related to Bug #15092: Queries to fetch changes are too big added
#12

Updated by Vincent MEMBRÉ 3 months ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 5.0.12 which was released today.

Also available in: Atom PDF