Project

General

Profile

Actions

Bug #21053

open

Rudder-jetty doesn't start in 7.0

Added by Nicolas CHARLES 3 months ago. Updated 9 days ago.

Status:
New
Priority:
N/A
Assignee:
-
Category:
System integration
Target version:
Severity:
UX impact:
It bothers me each time
User visibility:
Effort required:
Priority:
0
Regression:
No

Description

I don't really know how it happened, but I got to a state where rudder-jetty wouldn't start
Nothing happens when running

service rudder-jetty start
Job for rudder-jetty.service failed because the control process exited with error code.
See "systemctl status rudder-jetty.service" and "journalctl -xe" for details.

No logs in /var/log/rudder nor systemctl

running

bash /opt/rudder/bin/rudder-jetty.sh start
/opt/rudder/etc/rudder-jetty.conf: line 72: warning: setlocale: LC_NUMERIC: cannot change locale ()
/opt/rudder/etc/rudder-jetty.conf: line 72: warning: setlocale: LC_NUMERIC: cannot change locale ()
/opt/rudder/etc/rudder-jetty.conf: line 73: warning: setlocale: LC_TIME: cannot change locale ()
/opt/rudder/etc/rudder-jetty.conf: line 73: warning: setlocale: LC_TIME: cannot change locale ()
Setting umask to 0007
Starting Jetty: process already running.

shows that it thinks it is running, but it is not

command

start-stop-daemon -k 0007 -S -p/var/rudder/run/rudder-jetty.pid -d/opt/rudder/etc/rudder-jetty-base -b -m -a /bin/java -- -server -Xms1024m -Xmx1024m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -Dfile.encoding=UTF-8 -Drudder.configFile=/opt/rudder/etc/rudder-web.properties -Drudder.authFile=/opt/rudder/etc/rudder-users.xml -Dlogback.configurationFile=/opt/rudder/etc/logback.xml -Drun.mode=production -XX:+CrashOnOutOfMemoryError -Djetty.home=/opt/rudder/jetty -Djetty.base=/opt/rudder/etc/rudder-jetty-base -Djava.io.tmpdir=/var/rudder/tmp/jetty -jar /opt/rudder/jetty/start.jar OPTIONS=Server jetty.state=/opt/rudder/etc/rudder-jetty-base/rudder-jetty.state jetty-started.xml start-log-file=/var/log/rudder/webapp/java.log
process already running.

so that's the culprit

file /var/rudder/run/rudder-jetty.pid does exist, with PID 877
but no process with that ID

moving the pid file and running does

 service rudder-jetty start
Job for rudder-jetty.service failed because the control process exited with error code.
See "systemctl status rudder-jetty.service" and "journalctl -xe" for details.

but service is started ...


Related issues 1 (1 open0 closed)

Related to Rudder - Bug #21240: Rudder-jetty service can not be startedNewActions
Actions #1

Updated by Nicolas CHARLES 3 months ago

Interstingly, file ./opt/rudder/etc/rudder-jetty-base/rudder-jetty.state exists in 6.2 but not in 7.0, even thought files /opt/rudder/bin/rudder-jetty.sh are identical

Actions #2

Updated by Nicolas CHARLES 3 months ago

Killing rudder-jetty and doing a status returns:
In 6.2

root@server:/# systemctl status rudder-jetty
● rudder-jetty.service - Jetty Web Application Server
   Loaded: loaded (/lib/systemd/system/rudder-jetty.service; enabled; vendor preset: enabled)
   Active: failed (Result: signal) since Thu 2022-05-05 10:11:06 CEST; 57s ago
  Process: 6423 ExecStart=/opt/rudder/bin/rudder-jetty.sh start (code=exited, status=0/SUCCESS)
 Main PID: 6487 (code=killed, signal=KILL)

May 05 10:11:06 server systemd[1]: rudder-jetty.service: Main process exited, code=killed, status=9/KILL
May 05 10:11:06 server systemd[1]: rudder-jetty.service: Failed with result 'signal'.
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

in 7.0

root@server:/var/rudder/configuration-repository/techniques/system# systemctl status rudder-jetty
● rudder-jetty.service - Jetty Web Application Server
     Loaded: loaded (/lib/systemd/system/rudder-jetty.service; enabled; vendor preset: enabled)
     Active: failed (Result: signal) since Wed 2022-05-04 15:54:26 UTC; 30s ago
    Process: 19265 ExecStart=/opt/rudder/bin/rudder-jetty.sh start (code=exited, status=0/SUCCESS)
   Main PID: 19329 (code=killed, signal=KILL)
        CPU: 1min 50.807s

May 04 14:01:10 server systemd[1]: rudder-jetty.service: Failed with result 'exit-code'.
May 04 14:01:10 server systemd[1]: Stopped Jetty Web Application Server.
May 04 14:01:10 server systemd[1]: rudder-jetty.service: Consumed 40.332s CPU time.
May 04 14:01:10 server systemd[1]: Starting Jetty Web Application Server...
May 04 14:01:10 server rudder-jetty.sh[19265]: Setting umask to 0007
May 04 14:01:30 server rudder-jetty.sh[19265]: Starting Jetty: . . . . OK Wed May  4 14:01:30 UTC 2022
May 04 14:01:30 server systemd[1]: Started Jetty Web Application Server.
May 04 15:54:26 server systemd[1]: rudder-jetty.service: Main process exited, code=killed, status=9/KILL
May 04 15:54:26 server systemd[1]: rudder-jetty.service: Failed with result 'signal'.
May 04 15:54:26 server systemd[1]: rudder-jetty.service: Consumed 1min 50.807s CPU time.

running the start does work in this case though

There's clearly a difference of behaviour between 6.2 and 7.0, but I cannot reliably recreate a state where rudder-jetty is not starting

Actions #3

Updated by Vincent MEMBRÉ 3 months ago

  • Target version changed from 7.0.3 to 7.0.4
Actions #4

Updated by Félix DALLIDET 2 months ago

  • Related to Bug #21240: Rudder-jetty service can not be started added
Actions #5

Updated by Vincent MEMBRÉ about 2 months ago

  • Target version changed from 7.0.4 to 7.0.5
Actions #6

Updated by Vincent MEMBRÉ 24 days ago

  • Target version changed from 7.0.5 to 7.0.6
Actions #7

Updated by François ARMAND 9 days ago

  • UX impact set to I hate Rudder for that
  • Regression set to No
Actions #8

Updated by François ARMAND 9 days ago

  • UX impact changed from I hate Rudder for that to It bothers me each time
Actions

Also available in: Atom PDF