Bug #2768
closed
reporting doesn't work on ubuntu server
Added by Michael Gliwinski over 11 years ago. Updated over 11 years ago.
Description
This is on rudder server running on ubuntu server 12.04.
rudder-server-root 2.4.0~beta2-precise0
rudder-reports 2.4.0~beta2-precise0
Basically the problem is that /etc/rsyslog.d/rudder.conf configures rsyslog to listen on port 514, but ubuntu config (/etc/rsyslog.conf) configures rsyslog to drop privileges to 'syslog' user. Due to a bug in rsyslog, privs are dropped before the ports are bound. Since 514 is a privileged port, 'syslog' user doesn't have permissions to bind to it. See:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/789174
It appears some of the solutions/workarounds are:
- change port to >= 1025
- reconfigure rsyslog not to drop privileges
- change port to >= 1025 but use an iptables hack to redirect traffic from port 514 (so reconfiguring clients is not necessary)
Apparently fixing rsyslog is not trivial and would require a bit of redesign.
For reference here's a log of rsyslog's startup:
rsyslogd: [origin software="rsyslogd" swVersion="5.8.6" x-pid="803" x-info="http://www.rsyslog.com"] start rsyslogd: rsyslogd's groupid changed to 103 rsyslogd: rsyslogd's userid changed to 101 rsyslogd-2077: Could not create tcp listener, ignoring port 514. [try http://www.rsyslog.com/e/2077 ]
Updated by Jonathan CLARKE over 11 years ago
- Category changed from Web - Compliance & node report to System techniques
- Status changed from New to Discussion
- Assignee changed from Nicolas CHARLES to Jonathan CLARKE
- Target version set to 2.4.0~beta4
Michael,
Thank you very much for this bug report and the analysis you have done. This is an unfortunate bug...
It seems to me that our only option here is to use a different port number for rsyslog on Ubuntu (and possibly other OSes if they start using the same approach...), and configure Rudder to read the port number from a variable and update node's configurations accordingly. This is possible, since the configs are read via CFEngine, so will still be updated even if no reports are ever received.
Do you think this would be an appropriate solution? Any opinions on this from the dev team?
Updated by Michael Gliwinski over 11 years ago
Jonathan CLARKE wrote:
It seems to me that our only option here is to use a different port number for rsyslog on Ubuntu (and possibly other OSes if they start using the same approach...), and configure Rudder to read the port number from a variable and update node's configurations accordingly. This is possible, since the configs are read via CFEngine, so will still be updated even if no reports are ever received.
Do you think this would be an appropriate solution? Any opinions on this from the dev team?
This would definitely work, yes.
In the meantime I've disabled $PrivDropToUser syslog and $PrivDropToGroup syslog lines from /etc/rsyslog.conf to keep rsyslog running as root and it worked too. This is probably less secure though, so whichever you think is more appropriate.
Thanks for looking into this!
Updated by Jonathan CLARKE over 11 years ago
- Target version changed from 2.4.0~beta4 to 2.4.0~beta5
Updated by Jonathan CLARKE over 11 years ago
- Target version changed from 2.4.0~beta5 to 2.4.0~rc1
Updated by Jonathan CLARKE over 11 years ago
- Target version changed from 2.4.0~rc1 to 2.4.0~rc2
Updated by Nicolas PERRON over 11 years ago
Jonathan CLARKE wrote:
Michael,
Thank you very much for this bug report and the analysis you have done. This is an unfortunate bug...
It seems to me that our only option here is to use a different port number for rsyslog on Ubuntu (and possibly other OSes if they start using the same approach...), and configure Rudder to read the port number from a variable and update node's configurations accordingly. This is possible, since the configs are read via CFEngine, so will still be updated even if no reports are ever received.
Do you think this would be an appropriate solution? Any opinions on this from the dev team?
Jon, I'm not sure about the implementation.
Should we use a variable in rudder-web.properties which define a port number of syslog ? This imply that the port could be modified easily and not automatically.
Updated by François ARMAND over 11 years ago
After a discussion with Nicolas P., it seems that the work to do is more at CF-Clerk and Rudder Scala level than in init promises:
- 1/ we have to define a new system variable, "rsyslog port";
- 2/ we have to read its value from somewhere
- 3/ finally, we have to modify promises so that they use that system variable (that the only point related to cfengine).
For 2/, we believe that the simplest and more futur-proof solution is to simply have a property in Rudder configuration.properties file for that port, defaulted to default Rsyslog port, and with a comment specifying to value to set to be compliant with Ubuntu default choice.
In that case, an other task will be to add to migration scripts that new parameter.
Jon, what do you think about that ?
Updated by Nicolas PERRON over 11 years ago
A first attempt could be this:
- Create a new property : rudder.syslog.port=514 (which could be modified manually of automatically by packaging [Thank Vincent for the idea])
- Usage of a system variable (like SYSLOGPORT) from Rudder which value is set from the property above
- Add the new system variable in the metadata.xml of common and distributePolicy
- Change every occurrence of syslog port by this system variable in the Techniques
- Pray for the sake of Rudder
Updated by Jonathan CLARKE over 11 years ago
- Assignee changed from Jonathan CLARKE to Nicolas PERRON
François ARMAND wrote:
After a discussion with Nicolas P., it seems that the work to do is more at CF-Clerk and Rudder Scala level than in init promises:
- 1/ we have to define a new system variable, "rsyslog port";
- 2/ we have to read its value from somewhere
- 3/ finally, we have to modify promises so that they use that system variable (that the only point related to cfengine).For 2/, we believe that the simplest and more futur-proof solution is to simply have a property in Rudder configuration.properties file for that port, defaulted to default Rsyslog port, and with a comment specifying to value to set to be compliant with Ubuntu default choice.
In that case, an other task will be to add to migration scripts that new parameter.Jon, what do you think about that ?
Agreed.
A couple of precisions:- The logic to edit the port number should be in a postinst, not a migration script (this is not an upgrade issue)
- That logic should have a very simple and limited test along the lines of "if this is an Ubuntu machine with version >= to the one where the problem occurs (did this start in 12.04 or before?), and the syslog property in rudder-web.properties is not defined OR defined to a port < 1024, then change the port number". Any other conditions, don't touch that.
- The initial promises installed on a Rudder server need to know about this too, but they won't have the info from the system variable. I think they could read in the port number from rudder-web.properties directly.
Updated by François ARMAND over 11 years ago
- Status changed from Discussion to In progress
- Assignee changed from Nicolas PERRON to François ARMAND
Updated by François ARMAND over 11 years ago
- Status changed from In progress to Discussion
- Assignee changed from François ARMAND to Nicolas PERRON
A new system variable is available, with name "SYSLOGPORT".
It takes its value from the configuration.properties file, from the "rudder.syslog.port" parameter.
Nico, I reassign that task to you to finish the techniques and packaging stuff.
Updated by Nicolas PERRON over 11 years ago
François ARMAND wrote:
A new system variable is available, with name "SYSLOGPORT".
It takes its value from the configuration.properties file, from the "rudder.syslog.port" parameter.Nico, I reassign that task to you to finish the techniques and packaging stuff.
Thanks !
Could you document the way you use to add a new variable, please ? Somewhere like dev part in foskwiki ?
Updated by Nicolas PERRON over 11 years ago
- Status changed from Discussion to In progress
Updated by Nicolas PERRON over 11 years ago
- % Done changed from 0 to 80
Jonathan CLARKE wrote:
- The initial promises installed on a Rudder server need to know about this too, but they won't have the info from the system variable. I think they could read in the port number from rudder-web.properties directly.
I was wondering if we should really read from a file which doesn't exist on nodes because it could lead to errors like #2848 . But After checking this promise isn't call on nodes then it make senses.
Updated by Nicolas PERRON over 11 years ago
- Status changed from In progress to Pending technical review
- % Done changed from 80 to 100
Updated by Nicolas PERRON over 11 years ago
- Status changed from Pending technical review to In progress
- % Done changed from 100 to 90
http://www.rudder-project.org/redmine/projects/policy-templates/repository/revisions/bc295da23e2b232e3d5d52cfb0663922e0bce13d is not good. function readfile() is not properly used.
Updated by Nicolas PERRON over 11 years ago
- Status changed from In progress to Pending technical review
- % Done changed from 90 to 100
Applied in changeset commit:e8ed674a834b5e41baca2a3242aa2ce6c13408a4.
Updated by Nicolas CHARLES over 11 years ago
- Status changed from Pending technical review to Released
This looks valid, thank you Francois, Nicolas and Jon
Updated by Nicolas PERRON over 11 years ago
- Status changed from Released to New
- % Done changed from 100 to 90
Just need to add a default value if the rudder.syslog.port attribute is not present in rudder-web.properties
Updated by Nicolas PERRON over 11 years ago
- Status changed from New to Pending technical review
- % Done changed from 90 to 100
Applied in changeset commit:a68a90fd8a53fe6d73a143b37e8fb77c03753fe9.
Updated by Jonathan CLARKE over 11 years ago
- The CHECK_DIST can be done more easily using the "-s" option to lsb_release
- If you use lsb_release, you should depend on it
- The regexps for reading the existing value in rudder-web.properties and updating it were not flexible enough: you could have spaces around the "=" sign, they have to be included in the regexps
- The filename for rudder-web.properties was wrong (/tmp/rudder-web.properties)
- The check doesn't have to be "==5514", just "! <1024"
- The description text for the variable added to rudder-web.properties was not up to date
Updated by Jonathan CLARKE over 11 years ago
- There were some missing ";" in the CFEngine code!
- The regexp to read from the rudder-web.properties file was again too precise, not flexible enough
- Some comments would have helped
- Our convention is to always have promise types in the same order as normal ordering
Commiting a fix now.
Updated by Jonathan CLARKE over 11 years ago
Jonathan CLARKE wrote:
I've reviewed the packaging side of this, and commited a few changes:
- The CHECK_DIST can be done more easily using the "-s" option to lsb_release
- If you use lsb_release, you should depend on it
On the same subject, we shouldn't just check for Ubuntu 12.04, but for Ubuntu >= 12.04. Fixing.
Updated by Jonathan CLARKE over 11 years ago
- Status changed from Pending technical review to Released
Technical review done (but it took me some time!)
Updated by Nicolas PERRON over 11 years ago
- Target version changed from 2.4.0~rc2 to 2.4.0~beta5