Project

General

Profile

Actions

Bug #12719

closed

Some reports are duplicated between agent and postgres leading to "unexpected" compliance

Added by François ARMAND about 3 years ago. Updated almost 3 years ago.

Status:
Released
Priority:
N/A
Category:
Web - Compliance & node report
Target version:
Severity:
Critical - prevents main use of Rudder | no workaround | data loss | security
User visibility:
Operational - other Techniques | Technique editor | Rudder settings
Effort required:
Priority:
107
Tags:

Description

We have users reporting a lot of "unexpected", for example in copy-file or enforce file content post-hook components. After analysis, there seems to be a pattern where:

- on the node side, `/var/rudder/cfengine-community/outputs/the_run_output` contains the correct reports
- in database, the corresponding reports is duplicated for that run (with non consecutive insert id)

So something happen between the agent and postgres that duplicates the messages. All our investigation let us believe it's a duplicated syslog message, which could be correlated by the fact that the network is bad on these installations.

The long terme solution is to process messages with a run atomicity, and send them on the network with an "at least once" messaging semantic (it's easy to detect a duplicated run report server side). That need major architectural changes on Rudder.

A short term workaround is to had an option to let the compliance ignores unexpected when we have exactly the same reports. The compliance detail would still display both messages, but we would use the report compliance level.

We don't forsee a lot of cases where a fully duplicated message can be produced, so in most cases it seems preferable to use that option (and remove a lot of false positives bad compliance which hurts actionnability). So we believe that the option should be on by default, but still present for people who want to be strict on non compliance level.

That option would be removed once the long term solution is available.


Related issues

Related to Rudder - Bug #7758: When several directives value have a CFEngine var, we get "unexpected" due to bad regex matchingReleasedNicolas CHARLES2016-01-13Actions
Related to ncf - Bug #10769: variable_iterator use breaks reportingRejectedActions
Has duplicate Rudder - Bug #12859: Reports from multivalued variable lead to unexpected and should notRejectedFrançois ARMANDActions
Actions #1

Updated by François ARMAND about 3 years ago

  • Status changed from New to In progress
Actions #4

Updated by François ARMAND about 3 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Nicolas CHARLES
  • Pull Request set to https://github.com/Normation/rudder/pull/1969
Actions #5

Updated by François ARMAND about 3 years ago

  • Related to Bug #7758: When several directives value have a CFEngine var, we get "unexpected" due to bad regex matching added
Actions #6

Updated by Rudder Quality Assistant almost 3 years ago

  • Status changed from Pending technical review to Discussion
  • Assignee changed from Nicolas CHARLES to François ARMAND
Actions #7

Updated by François ARMAND almost 3 years ago

  • Assignee changed from François ARMAND to Nicolas CHARLES
Actions #8

Updated by Rudder Quality Assistant almost 3 years ago

  • Assignee changed from Nicolas CHARLES to François ARMAND
Actions #9

Updated by François ARMAND almost 3 years ago

  • Assignee changed from François ARMAND to Nicolas CHARLES
Actions #10

Updated by Rudder Quality Assistant almost 3 years ago

  • Assignee changed from Nicolas CHARLES to François ARMAND
Actions #11

Updated by François ARMAND almost 3 years ago

  • Status changed from Discussion to Pending release
Actions #12

Updated by François ARMAND almost 3 years ago

  • Has duplicate Bug #12859: Reports from multivalued variable lead to unexpected and should not added
Actions #13

Updated by François ARMAND almost 3 years ago

  • Related to Bug #10769: variable_iterator use breaks reporting added
Actions #14

Updated by Vincent MEMBRÉ almost 3 years ago

  • Status changed from Pending release to Released
  • Priority changed from 108 to 107

This bug has been fixed in Rudder 4.1.13, 4.2.7 and 4.3.3 which were released today.

Actions

Also available in: Atom PDF