Bug #11416: remote execution is cut at 30 secondes - Rudder - Issue Tracker

Actions

Copy link

Bug #11416

closed

remote execution is cut at 30 secondes

Added by Nicolas CHARLES over 6 years ago. Updated about 6 years ago.

Status:

Released

Priority:

N/A

Assignee:

Alexis Mousset

Category:

System techniques

Target version:

4.1.10

Pull Request:

https://github.com/Normation/rudder-a...

Severity:

Major - prevents use of part of Rudder | no simple workaround

UX impact:

User visibility:

Operational - other Techniques | Technique editor | Rudder settings

Effort required:

Small

Priority:

Name check:

Fix check:

Regression:

Description

We do not define a default timeout for remote execution, and it cuts execution at execution time: 30.61s
Timeout is defined in https://docs.cfengine.com/docs/master/reference-components-cf-runagent.html, but if it is not defined, then default_timeout apply, which default to 30s (see https://docs.cfengine.com/docs/master/reference-components-cf-agent.html#default_timeout )

we should set a default timeout for remote execution

Actions

Copy link

Updated by Janos Mattyasovszky over 6 years ago

Timeout manifests as "was interrupted" error from agent run:

policy_server # rudder remote run ruddernode
error    Rudder agent was interrupted during execution by a fatal error
         Run with -i to see log messages.

## Summary #####################################################################
0 components verified in 1 directives
execution time: 30.61s
################################################################################

Actions

Copy link

Updated by François ARMAND over 6 years ago

Severity set to Major - prevents use of part of Rudder | no simple workaround
User visibility set to Operational - other Techniques | Technique editor | Rudder settings
Priority changed from 0 to 52

Actions

Copy link

Updated by Vincent MEMBRÉ over 6 years ago

Target version changed from 4.1.8 to 4.1.9

Actions

Copy link

Updated by Benoît PECCATTE over 6 years ago

Priority changed from 52 to 51

Actions

Copy link

Updated by Benoît PECCATTE over 6 years ago

Effort required set to Small
Priority changed from 51 to 67

Actions

Copy link

Updated by Alexis Mousset over 6 years ago

Priority changed from 67 to 66

In fact the documentation (which states it is a connection timeout) is wrong, it is applied as SO_RCVTIMEO on the socket, and hence closes the connection when we see no output for 30 seconds, which may happen in changes only mode for example.

Actions

Copy link

Updated by Vincent MEMBRÉ over 6 years ago

Target version changed from 4.1.9 to 4.1.10

Actions

Copy link

Updated by Nicolas CHARLES over 6 years ago

Priority changed from 66 to 65

Interestingly, it's the agent that closes the connection.
Agent side

   error: SSL_read: receive timeout
rudder  verbose: Connection to 192.168.42.3 is closed

then server side

rudder  verbose: 192.168.42.2> MD5=918083b6dad780baf4e93367aa1cfe7f: Client is TRUSTED, public key MATCHES stored one.
rudder  verbose: 192.168.42.2>      Received:    EXEC 
rudder     info: 192.168.42.2> cf-serverd executing cfruncommand: /opt/rudder/bin/rudder agent run -uR
   error: 192.168.42.2> SSL_write: underlying network error (Broken pipe)
rudder     info: 192.168.42.2> Sending failed, aborting EXEC (send: Broken pipe)
   error: 192.168.42.2> SSL_write: (-1 SSL_ERROR_SSL) bad write retry 
rudder  verbose: 192.168.42.2> Unable to reply with terminator. (send: Success)
rudder  verbose: 192.168.42.2> Remote peer terminated TLS session (SSL_read)
rudder     info: 192.168.42.2> Closing connection, terminating thread

I configured the timeout everywhere

Actions

Copy link