Actions
Bug #16228
closed
Bug #16083: Big memory usage when fetching/writing node configuration and expected reports
Fetching nodes last run is not batched
Status:
Released
Priority:
N/A
Assignee:
Category:
Performance and scalability
Target version:
Pull Request:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Reviewed
Fix check:
Checked
Regression:
Description
we need to do the same as the parent ticket for getNodesLastRun
Query time is 25s with 5500 nodes, which is clearly too much (and too large)
Updated by Nicolas CHARLES over 4 years ago
- Status changed from New to In progress
- Assignee set to Nicolas CHARLES
Updated by Nicolas CHARLES over 4 years ago
- Status changed from In progress to Pending technical review
- Assignee changed from Nicolas CHARLES to François ARMAND
- Pull Request set to https://github.com/Normation/rudder/pull/2623
Updated by Nicolas CHARLES over 4 years ago
ok, the query is really inefficient
it's much more efficient to do
select r.nodeid, r.date, r.nodeconfigid, r.complete, r.insertionid,
c.nodeid, c.nodeconfigid, c.begindate, c.enddate, c.configuration from
(select reportsexecution.nodeid, reportsexecution.date, reportsexecution.nodeconfigid, reportsexecution.complete, reportsexecution.insertionid from
reportsexecution where nodeid in (values ('cfc8fe3d-09e6-4680-aea9-e347385753a2'),('540bc906-c672-49ce-acab-240d6e65676b'),('9b295838-6e8e-481b-927d-1e1891c317b2')) and (nodeid, insertionid) in
(select nodeid, max(insertionid) as insertionid
from reportsexecution
where complete = true
GROUP BY nodeid)) as r
left outer join nodeconfigurations as c
on r.nodeId = c.nodeid and r.nodeconfigid = c.nodeconfigid;
or
select r.nodeid, r.date, r.nodeconfigid, r.complete, r.insertionid,
c.nodeid, c.nodeconfigid, c.begindate, c.enddate, c.configuration from
(select reportsexecution.nodeid, reportsexecution.date, reportsexecution.nodeconfigid, reportsexecution.complete, reportsexecution.insertionid from
reportsexecution where (nodeid, insertionid) in
(select nodeid, max(insertionid) as insertionid
from reportsexecution
where complete = true
and nodeid in ((values ('cfc8fe3d-09e6-4680-aea9-e347385753a2'),('540bc906-c672-49ce-acab-240d6e65676b'),('9b295838-6e8e-481b-927d-1e1891c317b2')))
GROUP BY nodeid)) as r
left outer join nodeconfigurations as c
on r.nodeId = c.nodeid and r.nodeconfigid = c.nodeconfigid;
Updated by Nicolas CHARLES over 4 years ago
latest query shows an order of magnitude of perfs improvement
Updated by Nicolas CHARLES over 4 years ago
- Status changed from Pending technical review to Pending release
Applied in changeset rudder|719f84138a780a8780ff7f42e2bd1a54a7cc2513.
Updated by Alexis Mousset over 4 years ago
- Name check changed from To do to Reviewed
Updated by Nicolas CHARLES over 4 years ago
- Fix check changed from To do to Checked
Updated by Vincent MEMBRÉ over 4 years ago
- Status changed from Pending release to Released
This bug has been fixed in Rudder 5.0.15 which was released today.
Actions