Bug #2582
closedWhen using a \ in a report key, the value is not correctly inserted in the database
Description
For some reason, when we add a \ in a report key, it get interpreted when the report is added in the RudderSysEvent, and lead to inconsistency in the comparision of expected reports (where the \ is \ed), and the entry inserted (where the \ is consummed)
It is probably a rsyslog-postgres parameter to change
Updated by Nicolas CHARLES over 12 years ago
Several attempt to correct this issue.
On the scala side, 'base becomes 'base in the SQL, and \base becomes \\base
- the current configuration is 'base and \x08ase
- with the SQL encoding (rather than stdsql), it does Error and \\base
- with the SQL encoding (rather than stdsql), string escaped with E, it does Error and \base
- with the string surrounded by $$ $$, it becomes \base and ''base
- with the stdsql encoding, string escaped with E, it does 'base and \x08ase
- with the stdsql and Alter database Rudder set standard_conforming_strings=true; it does \base and 'base (which is better)
- with the stdsql and Alter database Rudder set standard_conforming_strings=true; and the string escaped with E it does \x08ase and 'base
Altering the standart conforming strings doesn't seems to have any bad side effect (http://www.postgresql.org/docs/8.2/static/runtime-config-compatible.html)
Updated by Nicolas CHARLES over 12 years ago
One last attempt :
Alter database Rudder set standard_conforming_strings=false;
$$ $$ string
with an SQL encoding leads to
\\base and \'base
Updated by Nicolas CHARLES over 12 years ago
Every modification attemps fails one way or another, with more or less severity (and using sql encoding rather than stdsql leads to security issues)
CFEngine interprets paths on windows with either / or \ the same way, so I guess the best way, for now, is to tell user not to use \, until we can figure out something more robust in the given time
Updated by Jonathan CLARKE over 12 years ago
- Target version changed from 2.3.8 to 2.3.9
Updated by Nicolas CHARLES over 12 years ago
- Status changed from New to In progress
I'm trying again with
alter database rudder set standard_conforming_strings=true;
I allows for proper inserting within the database (proper meaning no line lost). However, when the reports are read from scala, they need to have escaped escape.
So it does not do has expected, but it is still an improvement (no lines dropped)
Updated by Nicolas CHARLES over 12 years ago
Interestingly enough, given the screen, rudder can or cannot read the \ :
On the technical log page, it does show \tmp\one2 as the key value, but on the reports page, it shows mp\ne2
There is something strange there, that need to be looked at
Updated by Nicolas CHARLES over 12 years ago
Nicolas CHARLES wrote:
Interestingly enough, given the screen, rudder can or cannot read the \ :
On the technical log page, it does show \tmp\one2 as the key value, but on the reports page, it shows mp\ne2
There is something strange there, that need to be looked at
I couldn't reproduce this issue.
Apparently, this alone :
alter database rudder set standard_conforming_strings=true;
solves everything. But the database and rudder need to be restarted afterward for it to be taken into account.
Updated by Nicolas CHARLES over 12 years ago
I have trouble writing the condition on when to do this update, so I'll need an admin for that
The only test I found that seems reliable is :
su - postgres -c "psql -t -d rudder -c \"select '\\foo';\"" | grep "foo"
On a system with the standard_conforming_strings=true, it will output
\foo
on a system without the standard_conforming_strings=false, it will output nothing, but the stderr will be filled with warning :
ATTENTION: utilisation non standard d'un échappement dans une chaîne littérale LIGNE 1 : select '\foo'; ^ ASTUCE : Utilisez la syntaxe de la chaîne d'échappement pour les échappements, c'est-à-dire E'\r\n'. \x0Coo
The return code is 0 thought ...
Updated by Nicolas CHARLES over 12 years ago
- Assignee changed from Nicolas CHARLES to Nicolas PERRON
This is a job for a super packager !
Updated by Nicolas PERRON over 12 years ago
- Status changed from In progress to Pending technical review
- % Done changed from 0 to 100
Applied in changeset commit:7f81d50a7905b268a970669b54360a7a065727e5.
Updated by Jonathan CLARKE over 12 years ago
- Status changed from Pending technical review to Released
Looks good to me!