Project

General

Profile

Actions
Copy link

Bug #7285

closed

Only run inventory collection when no other is running

Added by Janos Mattyasovszky about 9 years ago. Updated almost 8 years ago.

Status:
Rejected
Priority:
N/A
Assignee:
Nicolas CHARLES
Category:
System techniques
Target version:
3.1.18
Pull Request:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
Name check:
Fix check:
Regression:

Description

Please make a lock for running an inventory, so it only runs once, so any hanging / long running inventory will not overlap with a new one.

Otherwise you'd end up with something like this:

/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null
      `- lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count
/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null
      `- lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count
/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null
      `- lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count
/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory --scan-homedirs
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory --scan-homedirs
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null
      `- lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count
/bin/sh /opt/rudder/bin/run-inventory --local=/var/rudder/tmp/inventory
`- /opt/rudder/bin/perl -I /opt/rudder/lib/perl5 /opt/rudder/bin/fusioninventory-agent --config=none --local=/var/rudder/tmp/inventory
   `- sh -c lvs -a --noheading --nosuffix --units M -o lv_name,vg_uuid,lv_attr,lv_size,lv_uuid,seg_count 2>/dev/null

Times 1500...

Related issues 2 (0 open — 2 closed)

Related to Rudder - Bug #9626: The locks are removed every 10 minutes by check-rudder-agent when there is no promise updateReleasedNicolas CHARLES2016-11-04Actions
Is duplicate of Rudder - Bug #9622: Fusioninventory is not tracked by check-rudder-healthReleasedNicolas CHARLESActions
Actions
Copy link
 #1

Updated by Vincent MEMBRÉ about 9 years ago

  • Category changed from Agent to System techniques
  • Assignee set to Nicolas CHARLES

That would be great indeed to prevent those multiples runs :(

Nicolas, is it possible to check before launching the inventory, that a run-inventory process is not already running ?

Actions
Copy link
 #2

Updated by Benoît PECCATTE over 8 years ago

  • Target version set to 2.11.21
Actions
Copy link
 #3

Updated by Vincent MEMBRÉ over 8 years ago

  • Target version changed from 2.11.21 to 2.11.22
Actions
Copy link
 #4

Updated by Vincent MEMBRÉ over 8 years ago

  • Target version changed from 2.11.22 to 2.11.23
Actions
Copy link
 #5

Updated by Vincent MEMBRÉ over 8 years ago

  • Target version changed from 2.11.23 to 2.11.24
Actions
Copy link
 #6

Updated by Vincent MEMBRÉ about 8 years ago

  • Target version changed from 2.11.24 to 308
Actions
Copy link
 #7

Updated by Vincent MEMBRÉ about 8 years ago

  • Target version changed from 308 to 3.1.14
Actions
Copy link
 #8

Updated by Vincent MEMBRÉ about 8 years ago

  • Target version changed from 3.1.14 to 3.1.15
Actions
Copy link
 #9

Updated by Vincent MEMBRÉ about 8 years ago

  • Target version changed from 3.1.15 to 3.1.16
Actions
Copy link
 #10

Updated by Nicolas CHARLES about 8 years ago

Investigations on this topic show that:
  1. locks are correctly working: if an inventory is running, then following concurrent runs won't try to run the inventory command (tests with sleep 10000 in the run-inventory command)
  2. BUT purging the cf_lock.lmdb file lift this locking: execution can be concurrents

Promises timeout by default after 2 hours - we need to check if the commands are automatically stopped after 2 hours, even if lock file is removed

Note: cf_lock.lmdb is purged as part of the check_rudder_agent command, when file gets bigger than 10 MB, which apparently can cause bad behaviour :/

Actions
Copy link
 #11

Updated by Nicolas CHARLES about 8 years ago

Okn the hypothesis is correct: when we purge the cf_lock.lmdb database, agent looses all track of when was launched something. So things can pile up :(

Actions
Copy link
 #12

Updated by Nicolas CHARLES about 8 years ago

workaround could be to check with service_check_running that there are no inventory running - but that will not solve underlying issue that promises don't expire properly when we kill cf_lock, and that we need to purge cf_lock because it grows out of bound

Actions
Copy link
 #13

Updated by Vincent MEMBRÉ about 8 years ago

  • Target version changed from 3.1.16 to 3.1.17
Actions
Copy link
 #14

Updated by Vincent MEMBRÉ almost 8 years ago

  • Target version changed from 3.1.17 to 3.1.18
Actions
Copy link
 #15

Updated by Nicolas CHARLES almost 8 years ago

  • Is duplicate of Bug #9622: Fusioninventory is not tracked by check-rudder-health added
Actions
Copy link
 #16

Updated by Nicolas CHARLES almost 8 years ago

  • Status changed from New to Rejected

Issue should have been fixed in #9622 : when we kill the agent, we didn't kill subtasks, so everything was building up, with no chance of being garbage collected.
With change in related ticket, it should not happen anymore
#9226 fixed also the killing of locks, so agent properly tracks which promise is in progress
Rejected as fixed by others tickets

Actions
Copy link
 #17

Updated by Nicolas CHARLES almost 8 years ago

  • Related to Bug #9626: The locks are removed every 10 minutes by check-rudder-agent when there is no promise update added
Actions
Copy link

Also available in: Atom PDF