Actions
Bug #26611
closed
invalidateComplianceRequest get saturated with UpdateCompliance messages
Status:
Released
Priority:
1 (highest)
Assignee:
-
Category:
Performance and scalability
Target version:
Pull Request:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
To do
Fix check:
Checked
Regression:
No
Description
The queue for updating compliance cache get saturated by `UpdateCompliance` and `ExpiredCompliance` messages.
This cause RAM exaustion, since `UpdateCompliance` contains a whole `NodeStatusReport`.
There is also an accumulation of `NodeStatusReport` with something updating postgresql, likely "lastCompliance" table (even if we're not sure).
The accumulation is massive, going to several hundred of thousands of update message for an instance with only 10k nodes, no change in configuration, and 10 reports received per second.
This leads to rudder crashing.
Updated by François ARMAND about 1 month ago
- Priority changed from N/A to 1 (highest)
Updated by François ARMAND 15 days ago
- Status changed from New to Pending release
All subsequent tickets are now solved.
Updated by Vincent MEMBRÉ 2 days ago
- Status changed from Pending release to Released
This bug has been fixed in Rudder 8.2.6 and 8.3.1 which were released today.
Actions