Project

General

Profile

Actions

User story #2427

closed

Make Rudder log usable by sysadmin (i.e: looking like syslog)

Added by François ARMAND about 12 years ago. Updated over 11 years ago.

Status:
Released
Priority:
2
Category:
Web - Maintenance
Target version:
UX impact:
Suggestion strength:
User visibility:
Effort required:
Name check:
Fix check:
Regression:

Description

Logger identity is set to 36 char max.

Most of the time, logger identity are fully qualified class name, and if the name is longer than that limit, only the first letter of package is kept, like:

com.normation.rudder.domain.Foo => c.n.r.d.Foo.

That is even less meaningful for chosen name, like "the-logger-1.0-for-that-task" => t.0-for-that-task.

So, we should either completly remove the limit, or at least make it much bigger (60 char ?)


Related issues 2 (0 open2 closed)

Related to Rudder - Bug #2799: Change path of admin Rudder syslogReleasedNicolas PERRON2012-08-10Actions
Related to Rudder - User story #2800: Log-rotate /var/log/rudder/core/rudder-webapp.logReleasedNicolas PERRON2012-08-10Actions
Actions #1

Updated by François ARMAND about 12 years ago

  • Status changed from 8 to Discussion
  • Assignee changed from François ARMAND to Matthieu CERDA
  • % Done changed from 0 to 100

I just remove the identifier length limit for logger.

Some admin could look to the new log and say:

  • if the no-limit parameter is OK
  • if not, what could be a good length ?

Note that Logbak has a tone of things that can be set in log lines, see: http://logback.qos.ch/manual/layouts.html#ClassicPatternLayout

Actions #2

Updated by Jonathan CLARKE almost 12 years ago

  • Target version changed from 2.4.0~alpha7 to 18
Actions #3

Updated by Jonathan CLARKE almost 12 years ago

  • Target version changed from 18 to 24
Actions #4

Updated by François ARMAND almost 12 years ago

  • Target version changed from 24 to 47
Actions #5

Updated by François ARMAND almost 12 years ago

  • Subject changed from Allow logger identity to be longer to Make Rudder log usable by sysadmin (i.e: looking like syslog)

It has been decided to add a specific kind of logger that will look like syslog output, and only logs operational events, with their own levels.

Example:

  • when application start / stop,
  • when a plugin is detected
  • when a modification happens in the configuration
  • when a (operation) problem occured

It is also accepted that logs in event logs don't need to appear in the syslog-kind of logs.

Actions #6

Updated by François ARMAND almost 12 years ago

  • Status changed from Discussion to In progress

The first use case will be for report log.

Actions #7

Updated by François ARMAND almost 12 years ago

The syslog layout pattern is:

Jun  5 13:00:53 orchestrateur-3 logrotate: Reload syslog service..done
Jun  5 13:00:56 orchestrateur-3 cf-serverd[28198]:  Accepting connection from "192.168.110.21" 
Jun 27 13:02:53 orchestrateur-3 slapd[9274]: conn=1002 op=6001 SRCH attr=1.1

Apache logs:

[Fri Oct 07 12:48:55 2011] [warn] module rewrite_module is already loaded, skipping
[Fri Oct 07 12:48:55 2011] [warn] module dav_module is already loaded, skipping
[Fri Oct 07 12:48:55 2011] [warn] module dav_fs_module is already loaded, skipping

So we could go for something like:

Jun 27 13:02:53 orchestrateur-3 rudder[report]: [warn] receive a report for directive 'bla' for node 'node' with component value to FOO but was expecting None

(I can't understand why such a date format is more convenient than '2012-06-29 17:23:59', but if it has always been like that, it should be ok... and that's certainly why ops need their own logs :)

Actions #8

Updated by François ARMAND almost 12 years ago

  • Assignee changed from Matthieu CERDA to François ARMAND

So,

There is now a Scala ReportLogger, which must be used to report information to ops.

The logger behaviour is defined in logback.xml in the following XML directives:

  <property name="OPSLOG_DIR" value="/var/log/rudder/webapp-opslog" />

That property defined the directory into which the logs will be written.

  <appender name="OPSLOG" class="ch.qos.logback.core.FileAppender">
    <file>${OPSLOG_DIR}/rudder-webapp.log</file>
    <append>true</append>
    <encoder>
      <pattern>%d{MMM dd HH:mm:ss} ${HOSTNAME} rudder[%logger]: [%level] %msg%n</pattern>
    </encoder>
  </appender>

This is the new appender, with an almost syslog format.
The filename is "rudder-webapp.log", but perhaps an other name would be better ?
The date patter zero-pad days, a space-based padding is not available.

Last, the file are not rotated by logback, but could be if needed. The assumption was that log file rotation is more an system property than an application one.

Finally, we configure the logger for the first business case, reports:

   <logger name="report" level="info" additivity="false">
     <appender-ref ref="OPSLOG" />
     <!-- comment the following appender if you don't want to have logs about report in both stdout and opslog -->
     <appender-ref ref="STDOUT" />
   </logger>

As written, the current configuration output "report" log message in both OPSLOG and STDOUT appender (so that STDOUT get all logs, but it can changed).

Note that for each ops logger that we will want to do, we will have to also create a Scala ****Logger object with the corresponding name.

Actions #9

Updated by François ARMAND almost 12 years ago

  • Status changed from In progress to Discussion
  • Assignee changed from François ARMAND to Nicolas CHARLES

I pass the relay to NCH to implement actual log.

Actions #10

Updated by Jonathan CLARKE almost 12 years ago

François ARMAND wrote:

There is now a Scala ReportLogger, which must be used to report information to ops.

This looks like a fantastic start! Thanks François.

Some comments follow:

The logger behaviour is defined in logback.xml in the following XML directives:

<property name="OPSLOG_DIR" value="/var/log/rudder/webapp-opslog" />

I don't like this name "webapp-opslog". Can we just have "core" instead?

Will this directory be created by logback if it doesn't exist, or do we need to add it in packaging?

The filename is "rudder-webapp.log", but perhaps an other name would be better ?

That looks fine to me. This means the full log path would be /var/log/rudder/core/rudder-webapp.log, which makes sense. Even though "rudder" is duplicated, I think that's a good idea because if someone sends you just the file, it means you can tell which log it is easily.

The date patter zero-pad days, a space-based padding is not available.

That's fine.

Last, the file are not rotated by logback, but could be if needed. The assumption was that log file rotation is more an system property than an application one.

I agree. Please don't forget to create a ticket to add this log rotation to packaging.

Actions #11

Updated by François ARMAND almost 12 years ago

That bug should also unify Jetty logging system with our logger, see:

And so, all the logging will be configured at one point.

Actions #12

Updated by Jonathan CLARKE almost 12 years ago

  • Target version changed from 47 to 50
Actions #13

Updated by Jonathan CLARKE almost 12 years ago

  • Target version changed from 50 to 2.4.0~beta3
Actions #14

Updated by Nicolas PERRON over 11 years ago

I wonder if I should move it to the next run with a technical review to do or if I should close this ticket and open another one for further advancement ?

Actions #15

Updated by François ARMAND over 11 years ago

  • Status changed from Discussion to Pending technical review

I think that bug should be closed: it's main goal, allowing to build log for the aim of operators using Rudder, is done.

Other ticket should be open to follow other queries, like #2799 (change log directory) and #2800 (rotate the log file).

Actions #16

Updated by Vincent MEMBRÉ over 11 years ago

This looks nice, thank you François and Nicolas!

Actions #17

Updated by Nicolas CHARLES over 11 years ago

I reviewed Francois's code, which seems valid. Francois, could you do the same for mine?

Actions #18

Updated by François ARMAND over 11 years ago

  • Status changed from Pending technical review to 10

It's OK for me !

Actions #19

Updated by Jonathan CLARKE over 11 years ago

  • Status changed from 10 to Released
Actions

Also available in: Atom PDF