Project

General

Profile

Actions
Copy link

Bug #16256

closed

Deadlock on compliance computing

Added by Nicolas CHARLES over 4 years ago. Updated over 4 years ago.

Status:
Released
Priority:
N/A
Assignee:
François ARMAND
Category:
Web - Compliance & node report
Target version:
5.0.15
Pull Request:
https://github.com/Normation/rudder/p...
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Reviewed
Fix check:
Checked
Regression:

Description

I generated policies, accepted nodes and the same time, and went to home page several time
after a while, compliance didn't show up...

running jstack show that a lot of thread are BLOCKED (see attachement)

Files

Download all files
thread_dump (212 KB) thread_dump Nicolas CHARLES, 2019-11-22 23:48
API_casse (497 KB) API_casse Nicolas CHARLES, 2019-11-25 11:08

Related issues 1 (0 open — 1 closed)

Related to Rudder - Bug #16200: Missing timing info in compliance computation logsReleasedFrançois ARMANDActions
Actions
Copy link
 #1

Updated by Nicolas CHARLES over 4 years ago

I had a query 'idle in transaction' in postgresql: select nodeid, nodeconfigid, begindate, configuration from nodeconfigurations where enddate is null;
this was because of the load script

Actions
Copy link
 #2

Updated by Nicolas CHARLES over 4 years ago

There is also an issue on the policy generation: calling policy/regenerate every 30 minutes causes it to lock after first generation after midnight: every subsequent call results in

<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em><a href="/rudder/api/latest/system/regenerate/policies">POST&nbsp;/rudder/api/latest/system/regenerate/policies</a></em>.<p>
Reason: <strong>Error reading from remote server</strong></p></p>
</body></html>

compliance doesn't show up either

thread dump is attached

Actions
Copy link
 #3

Updated by Nicolas CHARLES over 4 years ago

  • Related to Bug #16200: Missing timing info in compliance computation logs added
Actions
Copy link
 #4

Updated by Nicolas CHARLES over 4 years ago

  • Status changed from New to In progress
  • Assignee set to Nicolas CHARLES
Actions
Copy link
 #5

Updated by Nicolas CHARLES over 4 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to François ARMAND
  • Pull Request set to https://github.com/Normation/rudder/pull/2632
Actions
Copy link
 #6

Updated by Nicolas CHARLES over 4 years ago

this does have a negative impact on perfs, but make it works.

Actions
Copy link
 #7

Updated by Nicolas CHARLES over 4 years ago

  • Status changed from Pending technical review to Pending release
Actions
Copy link
 #8

Updated by Nicolas CHARLES over 4 years ago

  • Fix check changed from To do to Checked

tested, after 2 days of non stop regeneration and group update, there is no deadlock

Actions
Copy link
 #9

Updated by Alexis Mousset over 4 years ago

  • Subject changed from Deadlock on compliance on Rudder 5.0 to Deadlock on compliance computing
  • Name check changed from To do to Reviewed
Actions
Copy link
 #10

Updated by Vincent MEMBRÉ over 4 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 5.0.15 which was released today.

Actions
Copy link

Also available in: Atom PDF