Project

General

Profile

Actions

Bug #6381

closed

File /var/log/rudder/{compliance/non-compliant-reports.log, core/rudder-webapp.log} are not correctly rotated

Added by Nicolas CHARLES over 7 years ago. Updated over 6 years ago.

Status:
Released
Priority:
N/A
Category:
System integration
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:

Description

This file is not correctly rotated, as on the systems, it is empty, and the file /var/log/rudder/compliance/non-compliant-reports.log.1 contains the most up to date reports (but is 250Mo big, with 2 months of logs)

Oddly, on my test system, it was correctly rotated 26, 27, 28 January,and rotation broke on 29 January.

Happens on Debian and SLES and Centos.

It is the same thing for /var/log/rudder/core/rudder-webapp.log.


Subtasks 1 (0 open1 closed)

Bug #6385: Remove /var/log/rudder/compliance/non-compliant-reports.log rotation from logrotateReleasedNicolas CHARLES2015-03-11Actions

Related issues 5 (2 open3 closed)

Related to Rudder - Bug #5453: Incomplete logrotate configurationRejectedMatthieu CERDA2014-09-01Actions
Related to Rudder - User story #6384: Clean log rotation & compression letting Java aloneNew2015-03-11Actions
Related to Rudder - Bug #4433: Log rotation problems due to bad post-rotate scriptRejectedBenoît PECCATTE2014-01-31Actions
Related to Rudder - User story #6719: Suppress webapp logs after some timeRejectedFrançois ARMANDActions
Related to Rudder - Bug #8418: Configuration files do not get updated when user has modified the stock onesNewActions
Actions #1

Updated by Benoît PECCATTE over 7 years ago

  • Category changed from 39 to System integration
Actions #2

Updated by François ARMAND over 7 years ago

This is heavely rooted to http://www.rudder-project.org/redmine/issues/6384. Each time we try to manage logs produced by a Java source with logrotate, it ends badly.

The solution for that one is simple:

- remove configuration for that log in logrotate => #6385
- use http://logback.qos.ch/manual/appenders.html#onRollingPolicies to manage the rotation and compression of the logs (this ticket)

Actions #3

Updated by François ARMAND over 7 years ago

So, today we have the following configuration for /var/log/rudder/compliance/non-compliant-reports.log logrotation:

/var/log/rudder/compliance/non-compliant-reports.log {
        daily
        missingok
        rotate 365
        compress
        notifempty
        create 640 root adm
        delaycompress
}

We can't get exactly the same configuration, because the windows (number of past report) is limited to 20 in logback (it can be change, be require code that we certainly don't want to do for futur-proofing, see: https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/rolling/FixedWindowRollingPolicy.java#L43 ).

So the corresponding logback appender with fixe windows configuration would be:

<appender name="XXXX" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>/var/log/rudder/compliance/non-compliant-reports.log</file>
    <append>true</append>

    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>non-compliant-reports.%i.gz</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>365</maxIndex>
    </rollingPolicy>

    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
    </triggeringPolicy>

    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>

And based on time:

<appender name="XXXX" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>/var/log/rudder/compliance/non-compliant-reports.log</file>
    <append>true</append>

    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <!-- daily rollover -->
      <fileNamePattern>non-compliant-reports.%d{yyyy-MM-dd}.log.gz</fileNamePattern>
      <maxHistory>365</maxHistory>
    </rollingPolicy>

    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>
Actions #4

Updated by François ARMAND over 7 years ago

  • Reproduced set to No

OK, with more invesigation, the previous comment is wrong:

- with FixedWindowRollingPolicy, we actually have the xxx.1.log, xxx.2.log, etc naming scheme, but we can't rotate on a daily basis,
- with RollingFileAppender, we will have AT LEAST FOR ROTATED FILE a naming scheme contening the date like xxx.2015_03_11.log, etc.

I ask more information here: http://stackoverflow.com/questions/28986907/logback-configuration-to-mimic-standard-logrotate-naming-scheme

So I see the following option:

- 1/ code a logback "triggering policy" based on time for FixedWindowRollingPolicy,
- 2/ accept the broke the rotated log scheme in the middle of a release (which seems to be a very bad idea).
- 3/ I don't see any 3 :(

Actions #5

Updated by François ARMAND over 7 years ago

The chosen solution is to use the default time rolling pattern, so that we are switching from a non working solution to a working one - even if we change the naming scheme.

Actions #6

Updated by François ARMAND over 7 years ago

  • Status changed from New to Pending technical review
  • Assignee changed from Benoît PECCATTE to Nicolas CHARLES
Actions #7

Updated by François ARMAND over 7 years ago

  • Subject changed from File /var/log/rudder/compliance/non-compliant-reports.log is not correctly rotated to File /var/log/rudder/{compliance/non-compliant-reports.log, core/rudder-webapp.log} are not correctly rotated
  • Description updated (diff)

Adding /var/log/rudder/core/rudder-webapp.log which is exactly in the same position as .../compliance/non-compliant-reports.log.

Actions #8

Updated by François ARMAND over 7 years ago

  • Pull Request set to https://github.com/Normation/rudder/pull/836
  • How to reproduce updated (diff)
Actions #10

Updated by François ARMAND over 7 years ago

  • Status changed from Pending technical review to Pending release
  • % Done changed from 0 to 100

Applied in changeset rudder|commit:b1f71497395d5c5b9de2d4f435e7cda305bb7ed8.

Actions #11

Updated by Nicolas CHARLES over 7 years ago

Applied in changeset rudder|commit:11a7bb1d9b7bf13a55d0e5c3d56ec9c598a630f5.

Actions #12

Updated by Vincent MEMBRÉ over 7 years ago

This bug has been fixed in Rudder 2.10.12, 2.11.9 and 3.0.3 which were released today.

Actions #13

Updated by Vincent MEMBRÉ over 7 years ago

  • Status changed from Pending release to Released
Actions #14

Updated by François ARMAND about 7 years ago

Actions #15

Updated by Benoît PECCATTE over 6 years ago

  • Pull Request changed from https://github.com/Normation/rudder/pull/836 to https://github.com/Normation/rudder/pull/837
Actions #16

Updated by Nicolas CHARLES about 6 years ago

  • Related to Bug #8418: Configuration files do not get updated when user has modified the stock ones added
Actions

Also available in: Atom PDF