Project

General

Profile

Actions

Bug #16448

closed

Trigger remote run in node details says it timeouts

Added by François ARMAND over 4 years ago. Updated about 4 years ago.

Status:
Released
Priority:
N/A
Category:
Web - Compliance & node report
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Reviewed
Fix check:
Checked
Regression:

Description

Trigger remote run in node details always says that the request time while the request is correctly done, the run started, and we can even have the result when the compliance table is refreshed.


Files


Related issues 1 (0 open1 closed)

Related to Rudder - Bug #16760: Remote run always timeout for nodes behing relaysReleasedBenoît PECCATTEActions
Actions #1

Updated by Elaad FURREEDAN about 4 years ago

  • Status changed from New to In progress
Actions #2

Updated by François ARMAND about 4 years ago

  • Subject changed from Trigger remote run in node details timeeout to Trigger remote run in node details says it timeeout

It happens on centos 7 or 8 but not on debian 10 => selinux problem ?
It also happens in debian 9

Actions #3

Updated by François ARMAND about 4 years ago

I get on output with some errors about lmdb which don't appear if I interactivelly run agent from node with rudder agent run -v (see screenshot)

Actions #4

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 6.0.2 to 6.0.3
Actions #5

Updated by Elaad FURREEDAN about 4 years ago

On Debian 9 the first run failed with : "Error when trying to contact internal remote-run API: Read timed out" and succeed on the second run.
On CentOS 7.6 it's not working on every try.

Actions #6

Updated by Elaad FURREEDAN about 4 years ago

  • Assignee changed from Elaad FURREEDAN to Alexis Mousset
Actions #7

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 6.0.3 to 6.0.4
Actions #8

Updated by François ARMAND about 4 years ago

Update: on centos 7.6, it does not work for the first run(s? - perhaps several?) but as for Debian 9 in https://issues.rudder.io/issues/16448#note-5, it works after some time - both on root server and on other nodes.

Actions #9

Updated by François ARMAND about 4 years ago

And it stopped working - no idea why or how.

Actions #10

Updated by François ARMAND about 4 years ago

When the problem happens, I get that log trace:

[2020-02-19 12:07:15+0000] DEBUG nodes - Executing remote run call: HttpRequest(https://localhost/rudder/relay-api/remote-run/nodes/root,POST,FormBodyConnectFunc,List((classes,), (keep_output,true), (asynchronous,false)),List((User-Agent,rudder/remote run query for node root), (content-type,application/x-www-form-urlencoded)),List(scalaj.http.HttpOptions$$$Lambda$2085/304503000@5c6990e1, scalaj.http.HttpOptions$$$Lambda$2086/805996536@51b20348, scalaj.http.HttpOptions$$$Lambda$8733/1911143550@24a94956, scalaj.http.HttpOptions$$$Lambda$2085/304503000@b77b0e, scalaj.http.HttpOptions$$$Lambda$2086/805996536@10c91d61, scalaj.http.HttpOptions$$$Lambda$2087/1983137208@5120f473),None,UTF-8,4096,PlainUrlFunc,true,None)
[2020-02-19 12:07:15+0000] DEBUG nodes - Processing remore-run on root: HTTP status 200
[2020-02-19 12:07:20+0000] DEBUG nodes - === done processing request !
[2020-02-19 12:07:20+0000] ERROR nodes - Error occured when contacting internal remote-run API to apply classes on Node 'root': request timed out after 5 s
[2020-02-19 12:07:20+0000] DEBUG nodes - === I'm going to close output stream!
[2020-02-19 12:07:20+0000] DEBUG nodes - === closed!
[2020-02-19 12:07:20+0000] DEBUG nodes - === fiber joined!
[2020-02-19 12:07:20+0000] DEBUG nodes - === out closed!
Actions #11

Updated by François ARMAND about 4 years ago

  • Assignee changed from Alexis Mousset to François ARMAND
Actions #12

Updated by François ARMAND about 4 years ago

  • Assignee changed from François ARMAND to Vincent MEMBRÉ
  • Pull Request set to https://github.com/Normation/rudder/pull/2777
Actions #13

Updated by François ARMAND about 4 years ago

  • Assignee changed from Vincent MEMBRÉ to Alexis Mousset

The "timeout" problem was due to pipe being not closed at the correct place, and so leading to other stream never ending.

But there is still a problem with remote run behind relays (see linked ticket).

Actions #14

Updated by François ARMAND about 4 years ago

  • Related to Bug #16760: Remote run always timeout for nodes behing relays added
Actions #15

Updated by François ARMAND about 4 years ago

  • Status changed from In progress to Pending release
Actions #16

Updated by François ARMAND about 4 years ago

  • Target version changed from 6.0.4 to 6.0.3
Actions #17

Updated by François ARMAND about 4 years ago

  • Subject changed from Trigger remote run in node details says it timeeout to Trigger remote run in node details says it timeouts
Actions #18

Updated by François ARMAND about 4 years ago

  • Fix check changed from To do to Checked
Actions #19

Updated by Alexis Mousset about 4 years ago

  • Name check changed from To do to Reviewed
Actions #20

Updated by Vincent MEMBRÉ about 4 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 6.0.3 which was released today.

Actions

Also available in: Atom PDF