Actions
Bug #6007
closedError in the reports archiving process cause log explosion
Status:
Released
Priority:
1 (highest)
Assignee:
Category:
Performance and scalability
Target version:
Pull Request:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
Name check:
Fix check:
Regression:
Description
If there is an error during the archiving of reports, the process is restarted immediatelly, causing log size explosion.
Ex:
12-16 00:19:04] INFO report - Reports database: Automatic archive started for all reports before 2014-11-16 00:19 [2014-12-16 00:19:04] ERROR com.normation.rudder.repository.jdbc.ReportsJdbcRepository - Could not delete entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921 [2014-12-16 00:19:04] ERROR report - Reports database: Error while processing database deleting, cause is: Failure(Could not delete entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921,Full(org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921),Empty) [2014-12-16 00:19:04] ERROR report - Reports database: Relaunching automatic deleting process for all reports before 2014-09-17 00:19 [2014-12-16 00:19:04] INFO report - Reports database: Automatic delete started for all reports before 2014-09-17 00:19 [2014-12-16 00:19:04] ERROR com.normation.rudder.repository.jdbc.ReportsJdbcRepository - Could not archive entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921 [2014-12-16 00:19:04] ERROR report - Reports database: Error while processing database archiving, cause is: Failure(Could not archive entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921,Full(org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921),Empty) [2014-12-16 00:19:04] ERROR report - Reports database: Relaunching automatic archiving process for all reports before 2014-11-16 00:19 [2014-12-16 00:19:04] INFO report - Reports database: Automatic archive started for all reports before 2014-11-16 00:19 [2014-12-16 00:19:04] ERROR com.normation.rudder.repository.jdbc.ReportsJdbcRepository - Could not delete entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921 [2014-12-16 00:19:04] ERROR report - Reports database: Error while processing database deleting, cause is: Failure(Could not delete entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921,Full(org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921),Empty) [2014-12-16 00:19:04] ERROR report - Reports database: Relaunching automatic deleting process for all reports before 2014-09-17 00:19 [2014-12-16 00:19:04] INFO report - Reports database: Automatic delete started for all reports before 2014-09-17 00:19 [2014-12-16 00:19:04] ERROR com.normation.rudder.repository.jdbc.ReportsJdbcRepository - Could not archive entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921 [2014-12-16 00:19:04] ERROR report - Reports database: Error while processing database archiving, cause is: Failure(Could not archive entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921,Full(org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921),Empty) [2014-12-16 00:19:04] ERROR report - Reports database: Relaunching automatic archiving process for all reports before 2014-11-16 00:19 [2014-12-16 00:19:04] INFO report - Reports database: Automatic archive started for all reports before 2014-11-16 00:19 [2014-12-16 00:19:04] ERROR com.normation.rudder.repository.jdbc.ReportsJdbcRepository - Could not delete entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921 [2014-12-16 00:19:04] ERROR report - Reports database: Error while processing database deleting, cause is: Failure(Could not delete entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921,Full(org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921),Empty) [2014-12-16 00:19:04] ERROR report - Reports database: Relaunching automatic deleting process for all reports before 2014-09-17 00:19 [2014-12-16 00:19:04] INFO report - Reports database: Automatic delete started for all reports before 2014-09-17 00:19 [2014-12-16 00:19:04] ERROR com.normation.rudder.repository.jdbc.ReportsJdbcRepository - Could not archive entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921 [2014-12-16 00:19:04] ERROR report - Reports database: Error while processing database archiving, cause is: Failure(Could not archive entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921,Full(org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921),Empty) [2014-12-16 00:19:04] ERROR report - Reports database: Relaunching automatic archiving process for all reports before 2014-11-16 00:19 [2014-12-16 00:19:04] INFO report - Reports database: Automatic archive started for all reports before 2014-11-16 00:19 [2014-12-16 00:19:04] ERROR com.normation.rudder.repository.jdbc.ReportsJdbcRepository - Could not delete entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921 [2014-12-16 00:19:04] ERROR report - Reports database: Error while processing database deleting, cause is: Failure(Could not delete entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921,Full(org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921),Empty) [2014-12-16 00:19:04] ERROR report - Reports database: Relaunching automatic deleting process for all reports before 2014-09-17 00:19 [2014-12-16 00:19:04] INFO report - Reports database: Automatic delete started for all reports before 2014-09-17 00:19 [2014-12-16 00:19:04] ERROR com.normation.rudder.repository.jdbc.ReportsJdbcRepository - Could not archive entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921 [2014-12-16 00:19:04] ERROR report - Reports database: Error while processing database archiving, cause is: Failure(Could not archive entries in the database, cause is StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921,Full(org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [vacuum ruddersysevents]; SQL state [XX001]; error code [0]; ERROR: invalid page header in block 8779 of relation base/16385/24921; nested exception is org.postgresql.util.PSQLException: ERROR: invalid page header in block 8779 of relation base/16385/24921),Empty) [2014-12-16 00:19:04] ERROR report - Reports database: Relaunching automatic archiving process for all reports before 2014-11-16 00:19 [2014-12-16 00:19:04] INFO report - Reports database: Automatic archive started for all reports before 2014-11-16 00:19 .... .... ....
I'm not sure restarting the process is a good idea, since it's almost sure that without an human intervention, it will just continue to fails miserably.
I propose to simply log an error and not do anything else.
Actions