Rudder

I have a server in CET and a node in CDT. These two timezones have 6 hours difference.

The logs generated on my node look like this:

Mar 30 09:26:12 Message forwarded from sovma136: rudder[512100]: R: @@Common@@log_info@@hasPolicyServer-root@@common-root@@11@@common@@EndRun@@2014-03-30 14:26:00+00:00##f5b7b567-de02-0000-02c0-a8fd88000000@#End execution

As you can see, the time inside the Rudder report is 5 hours ahead of the time in the syslog timestamp. That timestamp comes from the node (it is in CDT).

Our rsyslog config uses that timestamp (from the node!) to insert a line into our PSQL DB:

$template RudderReportsFormat,"insert into RudderSysEvents (executionDate, ...) values ('%timereported:::date-rfc3339%',... )",stdsql

The rsyslog property timereported is defined as follows:

timestamp from the message. Resolution depends on what was provided in the message (in most cases, only seconds)

However, I don't think we should be trusting the time reported by the node (it could be completely out of sync, or worse, lying!), and I don't think we should be using it because we can't know what timezone it's in.

This does not seem to affect reporting, because I think reporting uses the middle field (the time inside the report).

So, I see two solutions:

1. We change rsyslog to insert the timegenerated (defined as "timestamp when the message was RECEIVED"), and then the display will always be in our local timezone. I don't like this solution, because it means that if logs were delayed (due to unavailable netork or whatever), they will all suddenly appear with the same timestamp, which can be very misleading.
2. We change the webapp to display the time from the reports field, rather than the syslog time.

I prefer solution 2, but it begs the question: why would be storing the other timestamp too? Maybe we should rename that a "time report received" information that could be useful in advanced debugging or future use?

Thoughts and opinions?