Project

General

Profile

Actions

Bug #3916

closed

Jetty should compress the files rotated (/var/log/rudder/webapp/*.log)

Added by Nicolas PERRON almost 9 years ago. Updated over 7 years ago.

Status:
Released
Priority:
1
Category:
Web - Maintenance
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:

Description

Rudder logs most of its messages in /var/log/rudder/webapp/ and Jetty rotate these logs but without compression.
The problem is that with a high verbosity (too many errors or in a debug mode) these files could fill all the space of the partition.


Related issues 1 (0 open1 closed)

Related to Rudder - Bug #4433: Log rotation problems due to bad post-rotate scriptRejectedBenoît PECCATTE2014-01-31Actions
Actions #1

Updated by Nicolas PERRON almost 9 years ago

  • Tracker changed from User story to Architecture
Actions #2

Updated by Nicolas PERRON almost 9 years ago

  • Tracker changed from Architecture to User story
  • Status changed from New to Discussion
  • Assignee set to François ARMAND

I don't know if this is an architecture ticket or a bug. To not have it could be harmful on a system with a lot of errors and/or in a debug mode (i.e: oops, i've forgot to remove the debug mode) in a limited partition.

What's your opinion?

Actions #3

Updated by François ARMAND almost 9 years ago

  • Status changed from Discussion to 8

Well, seems like a bug. We should compress old logs.

Our log rotation are handled by Jetty, not directly by Rudder web app. We can configure it to do compression by adding ".gz" at the end of the target file name in /opt/rudder/jetty7/etc/jetty-logging.xml

See: http://stackoverflow.com/questions/3329385/compress-log4j-files
(it's the same solution for more recent Jetty version that uses SLF4J).

Actions #4

Updated by François ARMAND almost 9 years ago

  • Target version changed from Ideas (not version specific) to 2.8.0~beta1

Some more explanation on that one:

- log4j need "copytruncate"
- "copytruncate" can loose logs, see: http://serverfault.com/questions/292513/when-to-use-delaycompress-option-in-logrotate

So we will manage both log rotation and compression on log4j.

See also #3922 and #3923 for more information about delaycompress.

Actions #5

Updated by Nicolas CHARLES over 8 years ago

With my limited understanding , i tried to edit the /opt/rudder/jetty7/etc/jetty-logging.xml by adding .gz at the end, without success

Actions #6

Updated by Nicolas CHARLES over 8 years ago

Fanf, could you have a look at this ?
i figure we should do what is there : http://logback.qos.ch/access.html , but couldn't find how to squeeze that in our config files

Actions #7

Updated by Vincent MEMBRÉ over 8 years ago

  • Target version changed from 2.8.0~beta1 to 2.8.0~rc1
Actions #8

Updated by François ARMAND over 8 years ago

Nicolas CHARLES wrote:

With my limited understanding , i tried to edit the /opt/rudder/jetty7/etc/jetty-logging.xml by adding .gz at the end, without success

It's what I understood we had to do, but if it does not work... Well, pehaps Jetty does not use slf4j/logback for that? That would be not really nice, but the class name seems really jetty-implementation related...
If so, pehaps we have a lot more log configuration to do, and it seems a rather big part.

Actions #9

Updated by Vincent MEMBRÉ over 8 years ago

  • Target version changed from 2.8.0~rc1 to Ideas (not version specific)

This will not be done in 2.8.0, and is postponed to later version

Actions #10

Updated by Benoît PECCATTE almost 8 years ago

This bug is still present in 2.11

Actions #11

Updated by Jonathan CLARKE over 7 years ago

  • Assignee deleted (François ARMAND)
  • Target version changed from Ideas (not version specific) to 3.0.0~beta2

OK, another approach is needed: none of the log rotation tools seem useful here. All we want to do is gzip the files that have already been rotated. Let's just add a CFEngine bundle to find the files logname.log.[0-9]+$ (and not .gz) and run gzip on them, using CFEngine's "transformer" action on files.

See https://github.com/Normation/rudder-techniques/blob/master/techniques/system/inventory/1.0/fusionAgent.st#L537 for an example.

Actions #12

Updated by Matthieu CERDA over 7 years ago

  • Status changed from 8 to Discussion
  • Assignee set to Jonathan CLARKE

Problem is, Jetty does not actually "rotate" the files in its log directory, it just changes the log filename everyday as the log name contains the date, and thus opens a new file to log in it instead. (It's more like jumping from file to file instead of always logging to the same place and putting old logs aside).

It poses a problem:
  • The current log file name is not fixed
  • There is no specific pattern to identify old logs (no .1, .2, ...)

Basically, we just have a bunch of files with the same pattern, and even as humans have to guess the good name for the current log file when we need to access it...

I see two solutions:

What do ? :D

Actions #13

Updated by Jonathan CLARKE over 7 years ago

  • Assignee changed from Jonathan CLARKE to Matthieu CERDA

Well, as my previous comment explained it, I'm just going to copy/paste:

Let's just add a CFEngine bundle to find the files logname.log.[0-9]+$ (and not .gz) and run gzip on them, using CFEngine's "transformer" action on files.

See https://github.com/Normation/rudder-techniques/blob/master/techniques/system/inventory/1.0/fusionAgent.st#L537 for an example.

I understand that the filenames don't match the ones described, but if they contain dates, I'm sure we can somehow select any files older than today.

Actions #14

Updated by François ARMAND over 7 years ago

  • Target version changed from 3.0.0~beta2 to 3.0.0~rc1
Actions #15

Updated by Benoît PECCATTE over 7 years ago

  • Tracker changed from User story to Bug
  • Assignee changed from Matthieu CERDA to Benoît PECCATTE
  • Target version changed from 3.0.0~rc1 to 2.10.9
Actions #16

Updated by Benoît PECCATTE over 7 years ago

  • Status changed from Discussion to Pending technical review
  • Assignee changed from Benoît PECCATTE to Nicolas CHARLES
  • Pull Request set to https://github.com/Normation/rudder-techniques/pull/623
Actions #17

Updated by Benoît PECCATTE over 7 years ago

  • Status changed from Pending technical review to Pending release
  • % Done changed from 0 to 100

Applied in changeset policy-templates:commit:541cdf0885e4415f187d23ea938de04ce4b0c3b0.

Actions #18

Updated by Nicolas CHARLES over 7 years ago

Applied in changeset policy-templates:commit:0d33a5350e8c862610a2d7ee1bb261e98f0161af.

Actions #19

Updated by Vincent MEMBRÉ over 7 years ago

  • Target version changed from 2.10.9 to 2.10.10
Actions #20

Updated by Vincent MEMBRÉ over 7 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 2.10.10 and 2.11.7, which were these days.

Actions

Also available in: Atom PDF