Project

General

Profile

Actions

Bug #3383

closed

Rudder status check fails if a proxy is defined in the environment

Added by Dennis Cabooter almost 10 years ago. Updated almost 10 years ago.

Status:
Released
Priority:
1
Category:
System techniques
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
Regression:

Description

I see the following error in /var/log/rudder/webapp/XXXX_XX_XX.stderrout.log and Jetty restarts all the time (so the web interface is unuseable for a little while):

[2013-04-02 16:01:24] ERROR non-compliant-reports - [2013-04-02 16:00:17+0200] N: root [rudder.example.com] S: [result_error] R: root-DP [distributePolicy] D: root-distributePolicy [Distribute Policy] T: distributePolicy/1.0 C: [Check rudder status] V: [None] The http://localhost:8080/rudder/api/status web application failed to respond for the second time. Restarting jetty NOW !
[2013-04-02 16:01:24] WARN non-compliant-reports - [2013-04-02 16:00:49+0200] N: root [rudder.example.com] S: [result_repaired] R: root-DP [distributePolicy] D: root-distributePolicy [Distribute Policy] T: distributePolicy/1.0 C: [Check jetty process] V: [None] The jetty process has been restarted
[2013-04-02 16:01:24] ERROR non-compliant-reports - [2013-04-02 16:00:49+0200] N: root [rudder.example.com] S: [result_error] R: root-DP [distributePolicy] D: root-distributePolicy [Distribute Policy] T: distributePolicy/1.0 C: [Check endpoint status] V: [None] The http://localhost:8080/endpoint/api/status web application failed to respond for the second time. Restarting jetty NOW !

When I invoke the command that has been used directly on the CLI, I get a strange answer:

root@winzts021:/tmp# curl -s http://localhost/rudder/api/status
<HTML>
<HEAD><TITLE>Redirection</TITLE></HEAD>
<BODY><H1>Redirect</H1></BODY>
</HTML>


Related issues 2 (0 open2 closed)

Related to Rudder - User story #3384: We should allow the user to specify a proxy to be used with the Rudder cURL callsRejectedFran├žois ARMANDActions
Related to Rudder - Bug #3401: The send-clean.sh script fails to send inventoriesReleasedMatthieu CERDA2013-04-09Actions
Actions #1

Updated by Nicolas CHARLES almost 10 years ago

Hi Dennis,

Did you change the context name of the application within Jetty ?

What is the output of the following command ? (note the :8080)

curl -s http://localhost:8080/endpoint/api/status

Actions #2

Updated by Dennis Cabooter almost 10 years ago

What do you mean with context name?

root@winzts021:/tmp# curl -s http://localhost:8080/endpoint/api/status
<HTML>
<HEAD><TITLE>Redirection</TITLE></HEAD>
<BODY><H1>Redirect</H1></BODY>
</HTML>

Also Apache has errors in /var/log/rudder/apache2/error.log:

[Tue Apr 02 15:01:27 2013] [error] [client 10.122.245.14] (20014)Internal error: proxy: error reading status line from remote server localhost:8080, referer: http://rudder.wintershall.nl/rudder/secure/nodeManager/searchNodes
[Tue Apr 02 15:01:27 2013] [error] (111)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:8080 (localhost) failed
[Tue Apr 02 15:01:27 2013] [error] ap_proxy_connect_backend disabling worker for (localhost)
[Tue Apr 02 15:01:37 2013] [error] proxy: HTTP: disabled connection for (localhost)
[Tue Apr 02 15:01:47 2013] [error] proxy: HTTP: disabled connection for (localhost)
[Tue Apr 02 15:01:57 2013] [error] proxy: HTTP: disabled connection for (localhost)
[Tue Apr 02 15:02:07 2013] [error] proxy: HTTP: disabled connection for (localhost)
[Tue Apr 02 15:02:17 2013] [error] proxy: HTTP: disabled connection for (localhost)
[Tue Apr 02 15:02:20 2013] [error] proxy: HTTP: disabled connection for (localhost)

Actions #3

Updated by Dennis Cabooter almost 10 years ago

Also from /var/log/rudder/apache2/error.log:

[Tue Apr 02 16:04:34 2013] [error] [client 10.122.245.14] client denied by server configuration: proxy:http://localhost:8080/rudder/api/status

Actions #4

Updated by Dennis Cabooter almost 10 years ago

When running cf-agent on the server I see this:

-> Transformer /var/rudder/inventories/accepted-nodes-updates/winzts022-2013-03-27-14-41-31.ocs => /var/rudder/tools/send-clean.sh http://localhost:8080/endpoint/upload/ /var/rudder/inventories/accepted-nodes-updates/winzts022-2013-03-27-14-41-31.ocs /var/rudder/inventories/received/ returned error
Transforming: /var/rudder/tools/send-clean.sh http://localhost:8080/endpoint/upload/ /var/rudder/inventories/accepted-nodes-updates/winzts021-2013-03-26-11-25-05.ocs /var/rudder/inventories/received/
  % Total    % Received 10ferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  196k    0     0  100  196k      0   608k --:--:-- --:--:-- --:--:--  660k
curl: (22) The requested URL returned error: 404
Actions #5

Updated by Matthieu CERDA almost 10 years ago

Can you retry the exact same command, but with a:

unset http_proxy
unset https_proxy
unset ftp_proxy

You might have some proxy env variables interfering with cURL.

Actions #6

Updated by Dennis Cabooter almost 10 years ago

Now it works! The proxy was in the way:

root@winzts021:/tmp# unset http_proxy
root@winzts021:/tmp# unset https_proxy
root@winzts021:/tmp# unset ftp_proxy
root@winzts021:/tmp# echo $http_proxy
root@winzts021:/tmp# curl -s http://localhost:8080/endpoint/api/status
OK

I've implemented the following workaround for now:

root@winzts021:/tmp# . /etc/bash.bashrc
root@winzts021:/tmp# echo $http_proxy
http://proxy.example.com:8080
root@winzts021:/tmp# echo "noproxy = *" > ~/.curlrc
root@winzts021:/tmp# curl -s http://localhost:8080/endpoint/api/status
OKroot@winzts021:/tmp#

However, it would be better to add "--noproxy \*" to all curl commands. I think the best thing to do for now is adding the workaround in the documentation.

Thank you for your help! :)

Actions #7

Updated by Jonathan CLARKE almost 10 years ago

I'm glad this is working for you. I think we should build this into the base Rudder Techniques, and use curl's "--no-proxy" option, for the hostname that we're downloading from.

I see no reason why the Rudder agent should contact it's root server via a HTTP proxy... This wouldn't work for syslog or CFEngine network flows, so let's consider that we don't need it for HTTP either. Unless I'm really mistaken here?

Actions #8

Updated by Dennis Cabooter almost 10 years ago

I think you can safely build this into the base Rudder Techniques. I see no reason for using a proxy to communicate with localhost. :)

Actions #9

Updated by Matthieu CERDA almost 10 years ago

  • Subject changed from Rudder status check to Rudder status check fails if a proxy is defined in the environment
  • Category set to System techniques
  • Status changed from New to 8
  • Assignee set to Matthieu CERDA
  • Priority changed from N/A to 1
  • Target version set to 2.3.12

So, this is a confirmed bug, and the workaround has been tested successfully (thanks Dennis), I'm on it.

I will document a way of using proxies with Rudder nodes too in the FaQ.

Actions #10

Updated by Matthieu CERDA almost 10 years ago

  • Status changed from 8 to Pending technical review
  • Assignee changed from Matthieu CERDA to Jonathan CLARKE
  • Pull Request set to https://github.com/Normation/rudder-techniques/pull/66
Actions #11

Updated by Matthieu CERDA almost 10 years ago

  • % Done changed from 0 to 100
Actions #12

Updated by Jonathan CLARKE almost 10 years ago

  • Status changed from Pending technical review to Pending release

Applied in changeset policy-templates:commit:0b9f245e0d83b1c49802ea817af35562b71b1904.

Actions #13

Updated by Jonathan CLARKE almost 10 years ago

Just commited this fix. It'll be available in tonight's nightly builds (tomorrow morning). All you need to use it is to upgrade the rudder-techniques package, and copy the Techniques from /opt/rudder/share/techniques to /var/rudder/configuration-repository/techniques as shown in the docs.

Actions #14

Updated by Nicolas PERRON almost 10 years ago

  • Status changed from Pending release to Released
Actions

Also available in: Atom PDF