Project

General

Profile

Actions

Bug #3906

closed

Migration of eventlogs v1 does not work if eventlogs v2 exists

Added by Nicolas PERRON over 9 years ago. Updated almost 8 years ago.

Status:
Released
Priority:
2
Category:
Packaging
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
Regression:

Description

It seems that the logic of the migration of eventlogs contains some inconsistency between the call from rudder-upgrade script and the code in Scala.

The result is that eventlogs from version is never made and the PostgreSQL could contains this:

rudder=# \d migrationeventlog
 id                  | integer                  | non NULL Par défaut, nextval('migrationeventlogid'::regclass)
 detectiontime       | timestamp with time zone | non NULL
 detectedfileformat  | integer                  | 
 migrationstarttime  | timestamp with time zone | 
 migrationendtime    | timestamp with time zone | 
 migrationfileformat | integer                  | 
 description         | text                     |

rudder=# SELECT * FROM migrationeventlog;
  1 | 2013-08-01 14:01:21.003138+02 |                  1 |                            |                            |                     | 
  2 | 2013-08-01 14:01:21.245159+02 |                  2 |                            |                            |                     | 
  3 | 2013-08-01 14:10:31.346582+02 |                  1 |                            |                            |                     | 
  4 | 2013-08-01 14:10:31.558145+02 |                  2 |                            |                            |                     | 
  5 | 2013-08-01 14:33:01.578001+02 |                  1 |                            |                            |                     | 
  6 | 2013-08-01 14:33:02.058985+02 |                  2 | 2013-08-01 14:45:23.896+02 | 2013-08-01 14:45:25.961+02 |                   3 | 
  7 | 2013-08-08 10:06:55.36604+02  |                  2 |                            |                            |                     | 
  8 | 2013-08-08 10:06:55.762059+02 |                  3 | 2013-08-08 10:07:02.453+02 | 2013-08-08 10:07:06.033+02 |                   4 |

That's a server which contains a lot of eventlogs v1:

rudder=# select count(*) from (select xpath('/entry/*[@fileFormat=1]',data) AS x from eventlog) as Y where array_upper(x, 1) > 0;
   146

The logic in the script is:
  • If I find a eventlog v1, I add a flag 1
  • If I find a eventlog v2, I add a flag 2
  • etc...
The logic in Scala is:
  • I search for the last flag and if there is no date of migration:
    • I migrate from the version of the flag to the next version until I have reached the more recent on (3 on 2.6 and 4 on 2.7)
    • I write the date of the begining and the end of the migration

Finally, the logic to is to have a flag from the oldest version existing in PostgreSQL to let Rudder make a migration from it but the rudder-upgrade script add all the versions found and write them successively into PostgreSQL.

The fix will be to change rudder-upgrade script to only write the oldest eventlog version.

Actions #1

Updated by Nicolas PERRON over 9 years ago

  • Status changed from New to Discussion
  • Assignee set to François ARMAND

François, do you agree with the details of the issue ?

Actions #2

Updated by François ARMAND over 9 years ago

Seems perfectly accurate (I just checked in Rudder).

So the migration script logic should be something like:


OLDEST_FORMAT=""

if(...... file format 1 ...) {
OLDEST_FORMAT="1"
}

if(...... file format 1 ...) {
OLDEST_FORMAT="1"
}

if( OLDEST_FORMAT non empty ) { update database with the value }

Actions #3

Updated by Nicolas PERRON over 9 years ago

  • Status changed from Discussion to Pending technical review
  • Assignee changed from François ARMAND to Jonathan CLARKE
  • % Done changed from 0 to 100
  • Pull Request set to https://github.com/Normation/rudder-packages/pull/116

Pull Request URL added: https://github.com/Normation/rudder-packages/pull/116

Jon, could you review it please ?

Actions #4

Updated by Jonathan CLARKE over 9 years ago

  • Assignee changed from Jonathan CLARKE to François ARMAND

This makes sense to me, and looks valid from a shell scripting point of view, but I don't understand the logic behind it. I'd rather François did the Technical Review.

Actions #5

Updated by François ARMAND over 9 years ago

  • Assignee changed from François ARMAND to Nicolas PERRON

Comments on the pull request.

Actions #6

Updated by Nicolas PERRON over 9 years ago

  • Assignee changed from Nicolas PERRON to François ARMAND

Pull request updated. François, could you review it please ?

Actions #7

Updated by Nicolas PERRON over 9 years ago

  • Status changed from Pending technical review to Pending release

Applied in changeset commit:ba827e169949830be0a94e18eeaea2e78ebb6dd7.

Actions #8

Updated by Anonymous over 9 years ago

Applied in changeset commit:2ee83c8d1eb29602a977b335609baa4e3b72aa30.

Actions #9

Updated by Nicolas PERRON over 9 years ago

This bug has been fixed in Rudder 2.4.9, which was released today.
Check out:

Actions #10

Updated by Nicolas PERRON over 9 years ago

  • Status changed from Pending release to Released
Actions #11

Updated by Benoît PECCATTE almost 8 years ago

  • Project changed from 34 to Rudder
  • Category set to Packaging
Actions

Also available in: Atom PDF