Project

General

Profile

Bug #16256

Deadlock on compliance computing

Added by Nicolas CHARLES 6 months ago. Updated 6 months ago.

Status:
Released
Priority:
N/A
Category:
Web - Compliance & node report
Target version:
Severity:
User visibility:
Effort required:
Priority:
0

Description

I generated policies, accepted nodes and the same time, and went to home page several time
after a while, compliance didn't show up...

running jstack show that a lot of thread are BLOCKED (see attachement)


Files

thread_dump (212 KB) thread_dump Nicolas CHARLES, 2019-11-22 23:48
API_casse (497 KB) API_casse Nicolas CHARLES, 2019-11-25 11:08

Related issues

Related to Rudder - Bug #16200: Missing timing info in compliance computation logsReleasedFrançois ARMANDActions
#1

Updated by Nicolas CHARLES 6 months ago

I had a query 'idle in transaction' in postgresql: select nodeid, nodeconfigid, begindate, configuration from nodeconfigurations where enddate is null;
this was because of the load script

#2

Updated by Nicolas CHARLES 6 months ago

There is also an issue on the policy generation: calling policy/regenerate every 30 minutes causes it to lock after first generation after midnight: every subsequent call results in

<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em><a href="/rudder/api/latest/system/regenerate/policies">POST&nbsp;/rudder/api/latest/system/regenerate/policies</a></em>.<p>
Reason: <strong>Error reading from remote server</strong></p></p>
</body></html>

compliance doesn't show up either

thread dump is attached

#3

Updated by Nicolas CHARLES 6 months ago

  • Related to Bug #16200: Missing timing info in compliance computation logs added
#4

Updated by Nicolas CHARLES 6 months ago

  • Status changed from New to In progress
  • Assignee set to Nicolas CHARLES
#5

Updated by Nicolas CHARLES 6 months ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to François ARMAND
  • Pull Request set to https://github.com/Normation/rudder/pull/2632
#6

Updated by Nicolas CHARLES 6 months ago

this does have a negative impact on perfs, but make it works.

#7

Updated by Nicolas CHARLES 6 months ago

  • Status changed from Pending technical review to Pending release
#8

Updated by Nicolas CHARLES 6 months ago

tested, after 2 days of non stop regeneration and group update, there is no deadlock

#9

Updated by Alexis MOUSSET 6 months ago

  • Subject changed from Deadlock on compliance on Rudder 5.0 to Deadlock on compliance computing
#10

Updated by Vincent MEMBRÉ 6 months ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 5.0.15 which was released today.

Also available in: Atom PDF