Project

General

Profile

Actions
Copy link

Bug #15012

closed

When receving pleinty of inventories at the same time, the web interface starts to parse them all at once

Added by Nicolas CHARLES over 5 years ago. Updated about 5 years ago.

Status:
Released
Priority:
N/A
Assignee:
Nicolas CHARLES
Category:
Web - Nodes & inventories
Target version:
6.0.0~beta1
Pull Request:
https://github.com/Normation/rudder/p...
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Fix check:
Regression:

Description

I've put around 350 inventories in incoming
Result, I have 350 lines

[2019-06-01 22:30:38] DEBUG inventory-processing - Start parsing inventory 'RDR-LOAD2-RandHost275.rudder.local275.ocs'
[2019-06-01 22:30:38] DEBUG inventory-processing - Start parsing inventory 'RDR-LOAD2-RandHost276.rudder.local276.ocs'
[2019-06-01 22:30:38] DEBUG inventory-processing - Start parsing inventory 'RDR-LOAD2-RandHost274.rudder.local274.ocs'
[2019-06-01 22:30:38] DEBUG inventory-processing - Start parsing inventory 'RDR-LOAD2-RandHost273.rudder.local273.ocs'
[2019-06-01 22:30:38] DEBUG inventory-processing - Start parsing inventory 'RDR-LOAD2-RandHost272.rudder.local272.ocs'
[2019-06-01 22:30:38] DEBUG inventory-processing - Start parsing inventory 'RDR-LOAD2-RandHost270.rudder.local270.ocs'

A full GC

then 

[2019-06-01 22:31:15] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost349.rudder.local349.ocs' parsed in 170 milliseconds ms, now saving
[2019-06-01 22:31:15] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost310.rudder.local310.ocs' parsed in 0 milliseconds ms, now saving
[2019-06-01 22:31:15] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost294.rudder.local294.ocs' parsed in 0 milliseconds ms, now saving
[2019-06-01 22:31:15] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost27.rudder.local27.ocs' parsed in 0 milliseconds ms, now saving
[2019-06-01 22:31:15] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost268.rudder.local268.ocs' parsed in 0 milliseconds ms, now saving

(not all files were parsed)

[2019-06-01 22:31:17] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost368.rudder.local368.ocs' for node '19ab73c0-214a-423a-94f6-a2f86dff3d56' pre-processed in 39 seconds and 466 milliseconds ms
[2019-06-01 22:31:17] WARN  inventory-processing - Rejecting Inventory 'RDR-LOAD2-RandHost368.rudder.local368.ocs' for Node '19ab73c0-214a-423a-94f6-a2f86dff3d56' because processing queue is full.
[2019-06-01 22:31:17] WARN  inventory-processing - Rejecting Inventory 'RDR-LOAD2-RandHost48.rudder.local48.ocs' for Node '8b66ada8-49b5-491e-a7fd-65ee3118db5a' because processing queue is full.
[2019-06-01 22:31:17] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost79.rudder.local79.ocs' for node 'aa2a4e0e-79c0-49b4-aa47-48c23bbd6fcd' pre-processed in 31 seconds and 647 milliseconds ms
[2019-06-01 22:31:17] WARN  inventory-processing - Rejecting Inventory 'RDR-LOAD2-RandHost79.rudder.local79.ocs' for Node 'aa2a4e0e-79c0-49b4-aa47-48c23bbd6fcd' because processing queue is full.
[2019-06-01 22:31:17] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost164.rudder.local164.ocs' for node '24089e7c-f7b2-4ea3-b330-84475afffe0f' pre-processed in 40 seconds and 850 milliseconds ms
[2019-06-01 22:31:17] WARN  inventory-processing - Rejecting Inventory 'RDR-LOAD2-RandHost164.rudder.local164.ocs' for Node '24089e7c-f7b2-4ea3-b330-84475afffe0f' because processing queue is full.
[2019-06-01 22:31:17] DEBUG inventory-processing - Inventory 'RDR-LOAD2-RandHost339.rudder.local339.ocs' for node '1acbaf70-5be7-4d4a-8889-e37c23d328b3' pre-processed in 39 seconds and 461 milliseconds ms

the parsing should probably be less agressive

note: those who were rejected because a a full queue went to failed

Actions
Copy link
 #1

Updated by Vincent MEMBRÉ over 5 years ago

  • Target version changed from 5.0.12 to 5.0.13
Actions
Copy link
 #2

Updated by François ARMAND over 5 years ago

  • Status changed from New to In progress
  • Assignee set to François ARMAND
Actions
Copy link
 #3

Updated by François ARMAND over 5 years ago

  • Target version changed from 5.0.13 to 6.0.0~beta1

A correction is easely done in 5.1. We will already address it in that version.

Actions
Copy link
 #4

Updated by François ARMAND over 5 years ago

Now we have a new config property (upgrade get a default value).
Processing looks like:

INFO Configure inventory processing with parallelism of '4' and queue size of '1000'
.... send 7 inventories....
[2019-08-02 11:29:09] INFO  inventory-processing - Watch new inventory file 'centos-7-64-0c846655-cb06-486f-ace4-eaeb11372097.ocs' with signature available: process.
[2019-08-02 11:29:09] INFO  inventory-processing - Watch new inventory file 'debian-8-64-717b63d1-01fe-4d4f-a7e5-cfb7c0d47b4f.ocs' with signature available: process.
[2019-08-02 11:29:09] INFO  inventory-processing - Watch new inventory file 'ubuntu-16-04-64-94b6d33d-a23e-46b9-b5f8-971751bebcbb.ocs' with signature available: process.
[2019-08-02 11:29:09] INFO  inventory-processing - Watch new inventory file 'centos-7-64-0d396c08-2f18-4c38-ac1c-114f278357a3.ocs' with signature available: process.
[2019-08-02 11:29:09] INFO  inventory-processing - Watch new inventory file 'centos-5-32-e308dc89-bbf3-416f-8d2e-95da78fdce3b.ocs' with signature available: process.
[2019-08-02 11:29:09] INFO  inventory-processing - Watch new inventory file 'raspberry-2-2fd55361-a19c-46d6-b1a0-5aac3cf7223b.ocs' with signature available: process.
[2019-08-02 11:29:09] INFO  inventory-processing - Watch new inventory file 'ubuntu-18-04-64-cdf721e6-3024-4240-bd24-f88a0ed35006.ocs' with signature available: process.
[2019-08-02 11:29:09] DEBUG inventory-processing - Start parsing inventory 'debian-8-64-717b63d1-01fe-4d4f-a7e5-cfb7c0d47b4f.ocs'
[2019-08-02 11:29:09] DEBUG inventory-processing - Start parsing inventory 'ubuntu-18-04-64-cdf721e6-3024-4240-bd24-f88a0ed35006.ocs'
[2019-08-02 11:29:09] DEBUG inventory-processing - Start parsing inventory 'centos-5-32-e308dc89-bbf3-416f-8d2e-95da78fdce3b.ocs'
[2019-08-02 11:29:09] DEBUG inventory-processing - Start parsing inventory 'centos-7-64-0d396c08-2f18-4c38-ac1c-114f278357a3.ocs'
[2019-08-02 11:29:09] WARN  inventory-logger - Error when parsing date for last user loggin. Awaited format is EEE MMM dd HH:mm, found: Sep  4 11:07 -
[2019-08-02 11:29:10] TRACE inventory-processing - Parsing done for inventory 'centos-5-32-e308dc89-bbf3-416f-8d2e-95da78fdce3b.ocs'
[2019-08-02 11:29:10] TRACE inventory-processing - Parsing done for inventory 'ubuntu-18-04-64-cdf721e6-3024-4240-bd24-f88a0ed35006.ocs'
[2019-08-02 11:29:10] TRACE inventory-processing - Parsing done for inventory 'centos-7-64-0d396c08-2f18-4c38-ac1c-114f278357a3.ocs'
[2019-08-02 11:29:10] TRACE inventory-processing - Parsing done for inventory 'debian-8-64-717b63d1-01fe-4d4f-a7e5-cfb7c0d47b4f.ocs'
[2019-08-02 11:29:10] DEBUG inventory-processing - Start parsing inventory 'raspberry-2-2fd55361-a19c-46d6-b1a0-5aac3cf7223b.ocs'
[2019-08-02 11:29:10] DEBUG inventory-processing - Start parsing inventory 'ubuntu-16-04-64-94b6d33d-a23e-46b9-b5f8-971751bebcbb.ocs'
[2019-08-02 11:29:10] DEBUG inventory-processing - Start parsing inventory 'centos-7-64-0c846655-cb06-486f-ace4-eaeb11372097.ocs'
...
Actions
Copy link
 #5

Updated by François ARMAND over 5 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Nicolas CHARLES
  • Pull Request set to https://github.com/Normation/rudder/pull/2367
Actions
Copy link
 #6

Updated by François ARMAND over 5 years ago

  • Status changed from Pending technical review to Pending release
Actions
Copy link
 #7

Updated by Vincent MEMBRÉ about 5 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 6.0.0~beta1 which was released today.

Actions
Copy link

Also available in: Atom PDF