Project

General

Profile

Actions
Copy link

Bug #22246

open

Webapp stopped to watch for inventories

Added by Félix DALLIDET almost 2 years ago. Updated 5 months ago.

Status:
New
Priority:
N/A
Assignee:
-
Category:
Web - Nodes & inventories
Target version:
7.3.17
Pull Request:
Severity:
Major - prevents use of part of Rudder | no simple workaround
UX impact:
I hate Rudder for that
User visibility:
Infrequent - complex configurations | third party integrations
Effort required:
Priority:
74
Name check:
To do
Fix check:
To do
Regression:
No

Description

In a 7.3.0~alpha, up for several days, the webapp kept working but was not watching anymore the incoming inventories. I had to restart the rudder-jetty process to have my inventories
processed.

See the webapp log before the restart, the exception occurred when running the jetty restart:

[2023-01-10 00:00:27+0000] INFO  report - Reports database: Automatic delete started for all reports before 2023-01-06 00:00
[2023-01-10 00:00:27+0000] INFO  report - Compliance level database: Automatic delete started for all compliance levels reports before 2023-01-02 00:00
[2023-01-10 00:00:27+0000] INFO  report - Reports database: automatic delete completed for all reports before 2023-01-06 00:00, 1 reports deleted
[2023-01-10 02:56:28+0000] WARN  healthcheck - CPU cores: Only one core, recommended value is at least 2
[2023-01-10 03:42:00+0000] INFO  git-repository - git-gc performed on fact-repository in PT0.180S
[2023-01-10 03:42:01+0000] INFO  git-repository - git-gc performed on configuration-repository in PT1.403S
[2023-01-10 04:50:29+0000] INFO  inventory-processing - Received new inventory file 'server-root.ocs' with signature available: process.
[2023-01-10 04:50:30+0000] INFO  inventory-processing - Inventory 'server-root.ocs' for node 'server.rudder.local' [root] (signature:certified) processed in 388 milliseconds
[2023-01-10 04:50:42+0000] INFO  scheduled.job - Update in node inventories main information detected: triggering a policy generation
[2023-01-10 04:50:42+0000] INFO  policy.generation - Start policy generation, checking updated rules
[2023-01-10 04:50:42+0000] INFO  policy.generation - [metrics] Xmx:1 GB nodes:1 (cached:1) rules:5 (enabled:4) techniques:9 (enabled:9) directives:9 (enabled:9) groups:3 (dynamic:3) parameters:2
[2023-01-10 04:50:42+0000] INFO  policy.generation - No node configuration was updated, no policies to write
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Timing summary:
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Run pre-gen scripts hooks     :          1 ms
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Run pre-gen modules hooks     :          0 ms
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Fetch all information         :         27 ms
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Build current rule values     :          1 ms
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Build target configuration    :          8 ms
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Write node configurations     :        203 ms
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Save expected reports         :          0 ms
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Run post generation hooks     :         63 ms
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Number of nodes updated       :          0   
[2023-01-10 04:50:42+0000] INFO  policy.generation.timing - Policy generation succeeded in:           
[2023-01-10 04:50:42+0000] INFO  policy.generation.manager - Successful policy update '11' [started 2023-01-10 04:50:42 - ended 2023-01-10 04:50:42]
[2023-01-10 06:10:31+0000] WARN  application - [Store Agent Run Times] Task frequency is set too low! Last task took 270121 ms but tasks are scheduled every 5000 ms. Adjust rudder.batch.storeAgentRunTimes.updateInterval if this problem persists.
[2023-01-10 06:25:32+0000] WARN  application - [Store Agent Run Times] Task frequency is set too low! Last task took 885914 ms but tasks are scheduled every 5000 ms. Adjust rudder.batch.storeAgentRunTimes.updateInterval if this problem persists.
[2023-01-10 06:40:33+0000] WARN  application - [Store Agent Run Times] Task frequency is set too low! Last task took 855907 ms but tasks are scheduled every 5000 ms. Adjust rudder.batch.storeAgentRunTimes.updateInterval if this problem persists.
[2023-01-10 06:55:34+0000] WARN  application - [Store Agent Run Times] Task frequency is set too low! Last task took 890957 ms but tasks are scheduled every 5000 ms. Adjust rudder.batch.storeAgentRunTimes.updateInterval if this problem persists.
[2023-01-10 09:48:55+0000] WARN  application - [Store Agent Run Times] Task frequency is set too low! Last task took 10386107 ms but tasks are scheduled every 5000 ms. Adjust rudder.batch.storeAgentRunTimes.updateInterval if this problem persists.
[2023-01-10 09:48:55+0000] WARN  healthcheck - CPU cores: Only one core, recommended value is at least 2
2023-01-10 09:53:34.580:INFO :oejs.Server:JettyShutdownThread: Stopped Server@692f203f{STOPPING}[10.0.12,sto=5000]
2023-01-10 09:53:34.582:INFO :oejs.Server:JettyShutdownThread: Shutdown Server@692f203f{STOPPING}[10.0.12,sto=5000]
2023-01-10 09:53:39.589:INFO :oejs.AbstractConnector:JettyShutdownThread: Stopped ServerConnector@14999fd5{HTTP/1.1, (http/1.1)}{127.0.0.1:8080}
2023-01-10 09:53:39.598:WARN :oejss.SessionHandler:qtp902919927-17: Unable to release Session Session@65d62ff2{id=node0v0xyy37egl4wpw2y4wcsmcvq238,x=node0v0xyy37egl4wpw2y4wcsmcvq238.node0,req=0,res=false}
java.lang.IllegalStateException: Not started
    at org.eclipse.jetty.server.session.AbstractSessionDataStore.store(AbstractSessionDataStore.java:164)
    at org.eclipse.jetty.server.session.AbstractSessionCache.release(AbstractSessionCache.java:541)
    at org.eclipse.jetty.server.session.SessionHandler.complete(SessionHandler.java:358)
    at org.eclipse.jetty.server.Request.lambda$leaveSession$0(Request.java:424)
    at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1468)
    at org.eclipse.jetty.server.Request.leaveSession(Request.java:424)
    at org.eclipse.jetty.server.Request.onCompleted(Request.java:1463)
    at org.eclipse.jetty.server.HttpChannel.onCompleted(HttpChannel.java:966)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:485)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
    at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:139)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:933)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1077)
    at java.base/java.lang.Thread.run(Thread.java:829)
2023-01-10 09:53:39.610:INFO :oejshC.rudder_xml:JettyShutdownThread: Closing Spring root WebApplicationContext
2023-01-10 09:53:39.664:INFO :oejsh.ContextHandler:JettyShutdownThread: Stopped o.e.j.w.WebAppContext@62230c58{rudder.xml,/rudder,null,STOPPED}{/opt/rudder/share/webapps/rudder.war}

The new log after restart:

2023-01-10 09:53:42.346:INFO :oejs.SetUIDListener:main: Setting umask=07
2023-01-10 09:53:42.353:INFO :oejs.Server:main: jetty-10.0.12; built: 2022-09-14T01:54:40.076Z; git: 408d0139887e27a57b54ed52e2d92a36731a7e88; jvm 11.0.16+8-post-Debian-1deb11u1
2023-01-10 09:53:42.432:INFO :oejdp.ScanningAppProvider:main: Deployment monitor [file:///opt/rudder/share/webapps/]
2023-01-10 09:53:44.188:INFO :oejw.StandardDescriptorProcessor:main: NO JSP Support for /rudder, did not find org.eclipse.jetty.jsp.JettyJspServlet
Use configuration file defined by JVM property -Dlogback.configurationFile : /opt/rudder/etc/logback.xml
2023-01-10 09:53:44.537:INFO :oejss.DefaultSessionIdManager:main: Session workerName=node0
[2023-01-10 09:53:44+0000] INFO  application - Rudder starts with PID 196319 on 1 cores
[2023-01-10 09:53:45+0000] INFO  application - Rudder application parameters are read from file defined by JVM property -Drudder.configFile: /opt/rudder/etc/rudder-web.properties
[2023-01-10 09:53:45+0000] INFO  application - -> files for overriding configuration parameters are read from directory /opt/rudder/etc/rudder-web.properties.d (that path can be overridden with JVM property -Drudder.configDir)
[2023-01-10 09:53:45+0000] INFO  application - Global exception handler configured to stop Rudder on: java.lang.Error
[2023-01-10 09:53:46+0000] INFO  application - Property 'rudder.batch.delete.software.interval' is missing or empty in rudder.configFile. Default to 24 hours.
[2023-01-10 09:53:46+0000] INFO  application - Property 'metrics.healthcheck.scheduler.period' is missing or empty in rudder.configFile. Default to 6 hours.
[2023-01-10 09:53:46+0000] INFO  application - Using 'erase' behavior when a node is deleted
[2023-01-10 09:53:46+0000] INFO  application - Starting Rudder 7.3.0~alpha1-SNAPSHOT web application [build timestamp: 2023-01-09T01:31:43Z]
[2023-01-10 09:53:52+0000] INFO  report - Disable automatic database archive sinces property rudder.batch.reportsCleaner.archive.TTL is 0 or negative
[2023-01-10 09:53:52+0000] WARN  healthcheck - CPU cores: Only one core, recommended value is at least 2
[2023-01-10 09:53:52+0000] INFO  git-repository - Automatic git-gc starts on fact-repository (schedule 'sec min h dayMonth month DayWeek': '0 42 3 * * ?')
[2023-01-10 09:53:52+0000] INFO  inventory-processing - Automatic cleaning of file older than PT720H starts for directories: '/var/rudder/inventories/failed', '/var/rudder/inventories/received' (schedule 'sec min h dayMonth month DayWeek': '0 32 3 * * ?')
[2023-01-10 09:53:52+0000] INFO  git-repository - Automatic git-gc starts on configuration-repository (schedule 'sec min h dayMonth month DayWeek': '0 42 3 * * ?')
[2023-01-10 09:53:53+0000] INFO  campaign - Starting campaign scheduler
[2023-01-10 09:53:53+0000] INFO  scheduled.job - Starting [Store Agent Run Times] scheduler with a period of 5000000000 nanoseconds s
[2023-01-10 09:53:53+0000] INFO  application - List of registered properties:
[2023-01-10 09:53:53+0000] INFO  application - registered property: awt.toolkit="sun.awt.X11.XToolkit" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: file.encoding="UTF-8" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: file.separator="/" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: history.inventories.rootdir="/var/rudder/inventories/historical" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: inventories.root.directory="/var/rudder/inventories" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: inventories.watcher.max.age.before.deletion="3 days" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: inventories.watcher.period.garbage.old="5 minutes" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: inventory.parse.parallelization="2" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.awt.graphicsenv="sun.awt.X11GraphicsEnvironment" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.awt.printerjob="sun.print.PSPrinterJob" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.class.path="/opt/rudder/etc/rudder-jetty-base/resources:/opt/rudder/jetty/lib/logging/slf4j-api-2.0.0.jar:/opt/rudder/jetty/lib/logging/jetty-slf4j-impl-10.0.12.jar:/opt/rudder/jetty/lib/jetty-servlet-api-4.0.6.jar:/opt/rudder/jetty/lib/jetty-http-10.0.12.jar:/opt/rudder/jetty/lib/jetty-server-10.0.12.jar:/opt/rudder/jetty/lib/jetty-xml-10.0.12.jar:/opt/rudder/jetty/lib/jetty-util-10.0.12.jar:/opt/rudder/jetty/lib/jetty-io-10.0.12.jar:/opt/rudder/jetty/lib/jetty-security-10.0.12.jar:/opt/rudder/jetty/lib/jetty-servlet-10.0.12.jar:/opt/rudder/jetty/lib/jetty-webapp-10.0.12.jar:/opt/rudder/jetty/lib/jetty-deploy-10.0.12.jar:/opt/rudder/jetty/lib/setuid/jetty-setuid-java-1.0.4.jar" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.class.version="55.0" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.home="/usr/lib/jvm/java-11-openjdk-amd64" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.io.tmpdir="/tmp" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.library.path="/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.runtime.name="OpenJDK Runtime Environment" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.runtime.version="11.0.16+8-post-Debian-1deb11u1" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.specification.name="Java Platform API Specification" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.specification.vendor="Oracle Corporation" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.specification.version="11" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vendor.url.bug="https://bugs.debian.org/openjdk-11" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.version="11.0.16" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vm.compressedOopsMode="32-bit" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vm.info="mixed mode, sharing" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vm.name="OpenJDK 64-Bit Server VM" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vm.specification.name="Java Virtual Machine Specification" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vm.specification.vendor="Oracle Corporation" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vm.specification.version="11" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vm.vendor="Debian" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: java.vm.version="11.0.16+8-post-Debian-1deb11u1" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: jdk.debug="release" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: jetty.base="/opt/rudder/etc/rudder-jetty-base" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: jetty.git.hash="408d0139887e27a57b54ed52e2d92a36731a7e88" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: jetty.home="/opt/rudder/jetty" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: ldap.authdn="cn=manager,cn=rudder-configuration" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: ldap.authpw=**********
[2023-01-10 09:53:53+0000] INFO  application - registered property: ldap.host="localhost" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: ldap.maxPoolSize="2" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: ldap.port="389" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: ldif.tracelog.rootdir="/var/rudder/inventories/debug" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: line.separator="\n" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: logback.configurationFile="/opt/rudder/etc/logback.xml" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: os.arch="amd64" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: os.name="Linux" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: os.version="5.10.0-10-amd64" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: path.separator=":" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.auth.idle-timeout="30 minutes" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.auth.provider="file" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.authFile="/opt/rudder/etc/rudder-users.xml" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.autoArchiveItems="true" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.check.node.cache.interval="15 s" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.databasecleaner.runtime.day="sunday" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.databasecleaner.runtime.hour="0" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.databasecleaner.runtime.minute="0" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.dyngroup.updateInterval="5" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.purge.inventories.delete.TTL="7" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.purge.inventories.delete.interval="24" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.reports.logInterval="1" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.reportsCleaner.deleteLogReport.TTL="2x" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.reportscleaner.archive.TTL="0" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.reportscleaner.compliancelevels.delete.TTL="8" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.reportscleaner.delete.TTL="4" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.reportscleaner.frequency="daily" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.storeAgentRunTimes.maxBatchSize="5" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.storeAgentRunTimes.maxDays="0" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.storeAgentRunTimes.maxMinutes="30" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.storeAgentRunTimes.updateInterval="5" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.batch.techniqueLibrary.updateInterval="5" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.config.repo.new.file.group.owner="rudder" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.configFile="/opt/rudder/etc/rudder-web.properties" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.debug.nodeconfiguration.path="/var/log/rudder/nodeConfigurations" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.dir.backup="/var/rudder/share/backup/" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.dir.dependencies="/var/rudder/tools" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.dir.gitRoot="/var/rudder/configuration-repository" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.dir.lock="/var/rudder/lock/" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.dir.shared.files.folder="/var/rudder/configuration-repository/shared-files" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.generated.policies.group.owner="rudder-policy-reader" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.git.gc="0 42 3 * * ?" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.hooks.ignore-suffixes=".swp, ~, .bak, .cfnew   , .cfsaved  , .cfedited, .cfdisabled, .cfmoved,.dpkg-old, .dpkg-dist, .dpkg-new, .dpkg-tmp,.disable , .disabled , _disable , _disabled,.ucf-old , .ucf-dist , .ucf-new ,.rpmnew  , .rpmsave  , .rpmorig" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.inventories.cleanup.old.files.age="30 days" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.inventories.cleanup.old.files.cron="0 32 3 * * ?" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.jdbc.batch.max.size="500" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.jdbc.driver="org.postgresql.Driver" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.jdbc.maxPoolSize="25" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.jdbc.password=**********
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.jdbc.url="jdbc:postgresql://localhost:5432/rudder" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.jdbc.username="rudder" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.jvm.fatal.exceptions="" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.nodes.delete.defaultMode="erase" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.policy.distribution.port.cfengine="5309" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.policy.distribution.port.https="443" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.postgresql.local="true" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.relayd.reload="/opt/rudder/bin/rudder relay reload -p" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.server.hsts="false" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.server.relay.api="https://localhost/rudder/relay-api" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.webdav.password=**********
[2023-01-10 09:53:53+0000] INFO  application - registered property: rudder.webdav.user="rudder" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: run.mode="production" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.arch.data.model="64" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.boot.library.path="/usr/lib/jvm/java-11-openjdk-amd64/lib" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.cpu.endian="little" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.cpu.isalist="" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.io.unicode.encoding="UnicodeLittle" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.java.command="org.eclipse.jetty.xml.XmlConfiguration file.encoding=UTF-8 java.version=11.0.16 java.version.major=11 java.version.micro=16 java.version.minor=0 java.version.platform=11 jetty.base=/opt/rudder/etc/rudder-jetty-base jetty.base.uri=file:///opt/rudder/etc/rudder-jetty-base jetty.deploy.monitoredDir=/opt/rudder/share/webapps jetty.deploy.scanInterval=0 jetty.home=/opt/rudder/jetty jetty.home.uri=file:///opt/rudder/jetty jetty.http.host=127.0.0.1 jetty.http.port=8080 jetty.httpConfig.requestHeaderSize=8192 jetty.httpConfig.sendDateHeader=false jetty.httpConfig.sendServerVersion=false jetty.setuid.groupName=root jetty.setuid.startServerAsPrivileged=true jetty.setuid.umask=0007 jetty.setuid.userName=root jetty.state=/opt/rudder/etc/rudder-jetty-base/rudder-jetty.state jetty.webapp.addServerClasses=org.eclipse.jetty.logging.,${jetty.home.uri}/lib/logging/,org.slf4j. logback.configurationFile=/opt/rudder/etc/logback.xml OPTIONS=Server rudder.authFile=/opt/rudder/etc/rudder-users.xml rudder.configFile=/opt/rudder/etc/rudder-web.properties run.mode=production runtime.feature.alpn=true slf4j.version=2.0.0 /opt/rudder/jetty/etc/jetty-bytebufferpool.xml /opt/rudder/etc/rudder-jetty-base/etc/console-capture.xml /opt/rudder/jetty/etc/jetty-threadpool.xml /opt/rudder/jetty/etc/jetty.xml /opt/rudder/jetty/etc/jetty-webapp.xml /opt/rudder/jetty/etc/jetty-deploy.xml /opt/rudder/jetty/etc/jetty-http.xml /opt/rudder/jetty/etc/jetty-setuid.xml /opt/rudder/jetty/etc/jetty-started.xml start-log-file=/var/log/rudder/webapp/java.log" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.java.launcher="SUN_STANDARD" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.jnu.encoding="ANSI_X3.4-1968" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.management.compiler="HotSpot 64-Bit Tiered Compilers" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: sun.os.patch.level="unknown" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: user.country="US" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: user.dir="/opt/rudder/etc/rudder-jetty-base" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: user.home="/root" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: user.language="en" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: user.name="root" 
[2023-01-10 09:53:53+0000] INFO  application - registered property: user.timezone="Etc/UTC" 
[2023-01-10 09:53:53+0000] INFO  application - Plugin's license directory: '/opt/rudder/etc/plugins/licenses/'
[2023-01-10 09:53:53+0000] INFO  campaign - Campaign Scheduler initialized with 0 events
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check LDAP and PostgreSQL connection
[2023-01-10 09:53:53+0000] INFO  bootchecks - LDAP and PostgreSQL connection are OK
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check for force reload of Techniques library
[2023-01-10 09:53:53+0000] INFO  bootchecks - Flag file '/opt/rudder/etc/force_technique_reload' does not exist, do not Technique library will not be reloaded
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check if special target all_policyServers from Rudder 7.0 is present
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check if system groups hasPolicyServer-root and all-nodes-with-cfengine-agent have the correct description and name
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check if migration of system configuration object and allowed networks to Rudder 7.0 format is needed
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check if 'rudder_generation_rudderc_enabled_targets' is present and must be deleted
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check mandatory DIT entries
[2023-01-10 09:53:53+0000] INFO  bootchecks - All the required DIT entries are present in the LDAP directory
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check initialization of User Technique Library
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check that `rudder` global parameter matches default value
[2023-01-10 09:53:53+0000] INFO  bootchecks - Check existence of at least one archive of the configuration
[2023-01-10 09:53:53+0000] INFO  bootchecks - First full archive of configuration-repository items done
[2023-01-10 09:53:53+0000] INFO  bootchecks - Regenerate all ncf techniques
[2023-01-10 09:53:54+0000] INFO  bootchecks - Flag file '/opt/rudder/etc/force_ncf_technique_update' does not exist, do not regenerate ncf Techniques
[2023-01-10 09:53:54+0000] INFO  bootchecks - Trigger policy update automatically at start
[2023-01-10 09:53:54+0000] INFO  bootchecks - Create system api token
[2023-01-10 09:53:54+0000] INFO  bootchecks - System api token file created in /var/rudder/run/api-token
[2023-01-10 09:53:54+0000] INFO  bootchecks - Load node compliance cache
2023-01-10 09:53:54.716:INFO :oejshC.rudder_xml:main: Initializing Spring root WebApplicationContext
[2023-01-10 09:53:55+0000] INFO  application - Configured authentication provider(s): [rootAdmin, file]
[2023-01-10 09:53:56+0000] INFO  application - No master admin account is defined. You can define one with 'rudder.auth.admin.login' and 'rudder.auth.admin.password' properties in the configuration file
[2023-01-10 09:53:56+0000] INFO  application - Add backend providers 'Default authentication backends provider: 'file','rootAdmin'
[2023-01-10 09:53:56+0000] INFO  application - Using configuration file defined by JVM property -Drudder.authFile : /opt/rudder/etc/rudder-users.xml
[2023-01-10 09:53:56+0000] INFO  bootstrap.liftweb.Boot - classpath
[2023-01-10 09:53:56+0000] INFO  org.reflections.Reflections - Reflections took 148 ms to scan 4 urls, producing 50 keys and 258 values
[2023-01-10 09:53:56+0000] INFO  campaign - Cleaned 0 events
[2023-01-10 09:53:56+0000] INFO  campaign - Cleaned 0 Rules/directives that are not linked to active campaign events
[2023-01-10 09:53:56+0000] INFO  campaign - Campaign Scheduler initialized with 0 events
[2023-01-10 09:53:57+0000] INFO  application.plugin - Initializing plugin 'rudder-plugin-system-updates': 7.3.0~alpha1-1.11.0
[2023-01-10 09:53:57+0000] INFO  application - Configure inventory processing with parallelism of '2'
[2023-01-10 09:53:57+0000] INFO  application.plugin - Initializing plugin 'rudder-plugin-dsc': 7.3.0~alpha1-2.3.0
[2023-01-10 09:53:57+0000] INFO  application - Using configuration file defined by JVM property -Drudder.authFile : /opt/rudder/etc/rudder-users.xml
[2023-01-10 09:53:57+0000] INFO  scheduled.job.metrics - Starting node count historization batch (min:15 m; max:4 h)
[2023-01-10 09:53:57+0000] INFO  inventory-processing - Incoming inventory garbage collection started - process existing inventories
[2023-01-10 09:53:57+0000] INFO  inventory-processing - Incoming inventory watcher started
[2023-01-10 09:53:57+0000] INFO  application - Application Rudder started
2023-01-10 09:53:57.820:INFO :oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@62230c58{rudder.xml,/rudder,file:///var/rudder/tmp/jetty/jetty-rudder.war.dir/webapp/,AVAILABLE}{/opt/rudder/share/webapps/rudder.war}
2023-01-10 09:53:57.835:INFO :oejs.AbstractConnector:main: Started ServerConnector@145f3f76{HTTP/1.1, (http/1.1)}{127.0.0.1:8080}
2023-01-10 09:53:57.843:INFO :oejs.Server:main: Started Server@692f203f{STARTING}[10.0.12,sto=5000] @16534ms
[2023-01-10 09:53:58+0000] INFO  policy.generation - Start policy generation, checking updated rules
[2023-01-10 09:53:58+0000] INFO  policy.generation - [metrics] Xmx:1 GB nodes:1 (cached:1) rules:5 (enabled:4) techniques:9 (enabled:9) directives:9 (enabled:9) groups:3 (dynamic:3) parameters:2
[2023-01-10 09:53:59+0000] INFO  inventory-processing - Received new inventory file 'AGENT1-be54b732-1604-4a46-8fc1-330eaa1f6455.ocs' with signature available: process.
[2023-01-10 09:54:00+0000] INFO  policy.generation - No node configuration was updated, no policies to write
[2023-01-10 09:54:00+0000] INFO  inventory-processing - Inventory 'AGENT1-be54b732-1604-4a46-8fc1-330eaa1f6455.ocs' for node 'agent1' [be54b732-1604-4a46-8fc1-330eaa1f6455] (signature:certified) processed in 633 milliseconds
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Timing summary:
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Run pre-gen scripts hooks     :         58 ms
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Run pre-gen modules hooks     :          2 ms
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Fetch all information         :        302 ms
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Build current rule values     :         60 ms
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Build target configuration    :        933 ms
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Write node configurations     :        410 ms
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Save expected reports         :          1 ms
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Run post generation hooks     :        211 ms
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Number of nodes updated       :          0   
[2023-01-10 09:54:00+0000] INFO  policy.generation.timing - Policy generation succeeded in:        2 s
[2023-01-10 09:54:00+0000] INFO  policy.generation.manager - Successful policy update '12' [started 2023-01-10 09:53:54 - ended 2023-01-10 09:54:00]
[2023-01-10 09:54:05+0000] INFO  application.plugin - Plugin 'rudder-plugin-dsc' has a license and the license signature is valid.
[2023-01-10 09:54:22+0000] INFO  scheduled.job - Update in node inventories main information detected: triggering a policy generation
[2023-01-10 09:54:22+0000] INFO  policy.generation - Start policy generation, checking updated rules
[2023-01-10 09:54:22+0000] INFO  policy.generation - [metrics] Xmx:1 GB nodes:1 (cached:1) rules:5 (enabled:4) techniques:9 (enabled:9) directives:9 (enabled:9) groups:3 (dynamic:3) parameters:2
[2023-01-10 09:54:22+0000] INFO  policy.generation - No node configuration was updated, no policies to write
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Timing summary:
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Run pre-gen scripts hooks     :          2 ms
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Run pre-gen modules hooks     :          0 ms
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Fetch all information         :        106 ms
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Build current rule values     :         10 ms
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Build target configuration    :         22 ms
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Write node configurations     :        203 ms
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Save expected reports         :          0 ms
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Run post generation hooks     :        192 ms
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Number of nodes updated       :          0   
[2023-01-10 09:54:22+0000] INFO  policy.generation.timing - Policy generation succeeded in:           
[2023-01-10 09:54:22+0000] INFO  policy.generation.manager - Successful policy update '13' [started 2023-01-10 09:54:22 - ended 2023-01-10 09:54:22]

A tree taken some minutes before the jetty restart:

root@server:/var/rudder/inventories# tree
.
├── accepted-nodes-updates
│   └── uuid.hive
├── debug
├── failed
├── incoming
│   ├── AGENT1-be54b732-1604-4a46-8fc1-330eaa1f6455.ocs.gz
│   └── AGENT1-be54b732-1604-4a46-8fc1-330eaa1f6455.ocs.sign
└── received
    ├── server-root.ocs
    └── server-root.ocs.sign

5 directories, 5 files

Related issues 2 (0 open — 2 closed)

Related to Rudder - Bug #20994: Some inventories are stuck in the incoming folderResolvedActions
Related to Rudder - Enhancement #23045: Make commiting nodes to fact-repo optionnalReleasedVincent MEMBRÉActions
Actions
Copy link
 #1

Updated by Félix DALLIDET almost 2 years ago

  • Related to Bug #20994: Some inventories are stuck in the incoming folder added
Actions
Copy link
 #2

Updated by Vincent MEMBRÉ almost 2 years ago

  • Target version changed from 7.3.0~beta1 to 7.3.0~rc1
Actions
Copy link
 #3

Updated by Benoît PECCATTE almost 2 years ago

  • Severity set to Major - prevents use of part of Rudder | no simple workaround
  • Priority changed from 0 to 88
Actions
Copy link
 #4

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 7.3.0~rc1 to 7.3.0
  • Priority changed from 88 to 87
Actions
Copy link
 #5

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 7.3.0 to 7.3.1
  • Priority changed from 87 to 86
Actions
Copy link
 #6

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 7.3.1 to 7.3.2
  • Priority changed from 86 to 85
Actions
Copy link
 #7

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 7.3.2 to 7.3.3
  • Priority changed from 85 to 84
Actions
Copy link
 #8

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 7.3.3 to 7.3.4
  • Priority changed from 84 to 83
Actions
Copy link
 #9

Updated by François ARMAND over 1 year ago

Actions
Copy link
 #10

Updated by Vincent MEMBRÉ over 1 year ago

  • Target version changed from 7.3.4 to 7.3.5
  • Priority changed from 83 to 82
Actions
Copy link
 #11

Updated by Alexis Mousset over 1 year ago

  • Target version changed from 7.3.5 to 7.3.6
  • Priority changed from 82 to 81
Actions
Copy link
 #12

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 7.3.6 to 7.3.7
  • Priority changed from 81 to 79
Actions
Copy link
 #13

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 7.3.7 to 7.3.8
Actions
Copy link
 #14

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 7.3.8 to 7.3.9
  • Priority changed from 79 to 77
Actions
Copy link
 #15

Updated by Vincent MEMBRÉ about 1 year ago

  • Target version changed from 7.3.9 to 7.3.10
Actions
Copy link
 #16

Updated by Vincent MEMBRÉ 12 months ago

  • Target version changed from 7.3.10 to 7.3.11
  • Priority changed from 77 to 75
Actions
Copy link
 #17

Updated by Vincent MEMBRÉ 10 months ago

  • Target version changed from 7.3.11 to 7.3.12
  • Priority changed from 75 to 74
Actions
Copy link
 #18

Updated by Vincent MEMBRÉ 9 months ago

  • Target version changed from 7.3.12 to 7.3.13
Actions
Copy link
 #19

Updated by Vincent MEMBRÉ 9 months ago

  • Target version changed from 7.3.13 to 7.3.14
Actions
Copy link
 #20

Updated by Vincent MEMBRÉ 7 months ago

  • Target version changed from 7.3.14 to 7.3.15
Actions
Copy link
 #21

Updated by Vincent MEMBRÉ 6 months ago

  • Target version changed from 7.3.15 to 7.3.16
Actions
Copy link
 #22

Updated by Vincent MEMBRÉ 5 months ago

  • Target version changed from 7.3.16 to 7.3.17
Actions
Copy link

Also available in: Atom PDF