Project

General

Profile

Actions
Copy link

Bug #18437

closed

Too many open files in relayd when disk is full

Added by Alexis Mousset over 3 years ago. Updated over 3 years ago.

Status:
Released
Priority:
N/A
Assignee:
Benoît PECCATTE
Category:
Relay server or API
Target version:
6.1.7
Pull Request:
https://github.com/Normation/rudder/p...
Severity:
Minor - inconvenience | misleading | easy workaround
UX impact:
User visibility:
Infrequent - complex configurations | third party integrations
Effort required:
Priority:
21
Name check:
To do
Fix check:
Checked
Regression:

Description

After several weeks it seems relayds accumulates too many open files.

Actions
Copy link
 #1

Updated by Alexis Mousset over 3 years ago

  • User visibility set to Operational - other Techniques | Rudder settings | Plugins
Actions
Copy link
 #2

Updated by Alexis Mousset over 3 years ago

Reproduced with only 4 nodes.

Actions
Copy link
 #3

Updated by Alexis Mousset over 3 years ago

  • Assignee set to Alexis Mousset
Actions
Copy link
 #4

Updated by Alexis Mousset over 3 years ago

  • Severity set to Minor - inconvenience | misleading | easy workaround
  • User visibility changed from Operational - other Techniques | Rudder settings | Plugins to Infrequent - complex configurations | third party integrations
  • Priority changed from 0 to 22

This happens when the /var partition gets full, setting visibility to infrequent.

A service restart once their is enough free space fixes the problem, setting severity to minor.

We can try to handle the problem better and avoid file descriptor exhaustion.

Actions
Copy link
 #5

Updated by Alexis Mousset over 3 years ago

  • Subject changed from Too many open files in relayd to Too many open files in relayd when disk is full
Actions
Copy link
 #6

Updated by Alexis Mousset over 3 years ago

The bug comes from diesel https://github.com/diesel-rs/diesel/blob/master/CHANGELOG.md#145---2020-06-09:

Fix a bug that result in leaking sockets/file descriptors on failed connection attempts for postgresql
Actions
Copy link
 #7

Updated by Alexis Mousset over 3 years ago

  • Status changed from New to In progress
Actions
Copy link
 #8

Updated by Alexis Mousset over 3 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Alexis Mousset to Benoît PECCATTE
  • Pull Request set to https://github.com/Normation/rudder/pull/3353
Actions
Copy link
 #9

Updated by Alexis Mousset over 3 years ago

  • Status changed from Pending technical review to Pending release
Actions
Copy link
 #10

Updated by Alexis Mousset over 3 years ago

  • Fix check changed from To do to Checked
Actions
Copy link
 #11

Updated by Vincent MEMBRÉ over 3 years ago

  • Status changed from Pending release to Released
  • Priority changed from 22 to 21

This bug has been fixed in Rudder 6.1.7 which was released today.

Actions
Copy link

Also available in: Atom PDF