Project

General

Profile

Actions

Bug #18198

closed

Webapp logs are not always rotated in Rudder 6.0 and 6.1

Added by Nicolas CHARLES over 3 years ago. Updated over 3 years ago.

Status:
Rejected
Priority:
N/A
Assignee:
-
Category:
System integration
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
To do
Fix check:
To do
Regression:

Description

Some systems, sometimes don't have their webapp log rotated
Seen on centos and sles, but it's not all systems

Actions #1

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.0.9 to 6.0.10
Actions #2

Updated by François ARMAND over 3 years ago

  • Status changed from New to Rejected

I tried to reproduced that one with several tactic, and never succeeded. I will close it and hope for more details if it happen again.

I learned some things along the way, noting them for post-mortem:
- there used to be a bug in old jetty: https://github.com/eclipse/jetty.project/issues/1357
- this bug was corrected in https://github.com/eclipse/jetty.project/commit/0d4fcd0cb4330e1895def6f5daa8996f8e23a6de, ie jetty 9.2.22 and 9.3.21 ,
- tests were latter added in https://github.com/eclipse/jetty.project/commit/b0bead2bc0de39c57eb8e531d72bd3f201ce7018 (jetty 9.4.7)
- rudder is in jetty 9.4.11 or up since at least 5.0 branche

So it is unlikely that the problem is due to that bug, and that jetty bug lead to torought testing.

Nonetheless, the behavior of the rotation is VERY surprising. It is not linked to the actual hour of the system, but to a latency since last rotation (or app start):

- when the app start or a rotation happens, jetty looks at current time and schedule a rotation in (24h-current time).
- that means that is time change, rotation will happen not at 00:00 but after computed latency is finished,
- that also may mean that IF a rotation can't happen, none will happen ever again.

That last option is what could explain the observed bug, but I can't see how it could happen without a big problem (like thread killed or something). And we didn't saw anything like that in logs.

Any clue on that problem would be much appreciated.

Actions

Also available in: Atom PDF