Bug #20589: git repository is full of pack_tmp and idx_tmp files in 7.0 - Rudder - Issue Tracker

Actions

Copy link

Bug #20589

open

git repository is full of pack_tmp and idx_tmp files in 7.0

Added by Nicolas CHARLES almost 3 years ago. Updated 6 months ago.

Status:

New

Priority:

N/A

Assignee:

Category:

System integration

Target version:

7.3.17

Pull Request:

Severity:

UX impact:

User visibility:

Operational - other Techniques | Rudder settings | Plugins

Effort required:

Priority:

Name check:

To do

Fix check:

To do

Regression:

Description

I have a configuration-repository git with 3.4 Go of file slikes

gc_3024324805558075120.pack_tmp  gc_8963748319320174561.idx_tmp
gc_4065393425363431946.idx_tmp     gc_8963748319320174561.pack_tmp
gc_4065393425363431946.pack_tmp  gc_9032135440682501564.idx_tmp
gc_4735211254661606721.idx_tmp     gc_9032135440682501564.pack_tmp

dating from the 1st december
git gc doesn't remove them

#/var/rudder/configuration-repository/.git/objects# git count-objects -v -H
warning: garbage found: /var/rudder/configuration-repository/.git/objects/pack/gc_2050152969671646544.pack_tmp
warning: garbage found: /var/rudder/configuration-repository/.git/objects/pack/gc_7939984065452016776.pack_tmp
count: 0
size: 0 bytes
in-pack: 682628
packs: 1
size-pack: 108.82 MiB
prune-packable: 0
garbage: 68
size-garbage: 3.58 GiB

Actions

Copy link

Updated by Nicolas CHARLES almost 3 years ago

procedure in https://stackoverflow.com/questions/27257399/git-garbage-collection-doesnt-seem-to-fully-work doesn't solves the issue

Actions

Copy link

Updated by Nicolas CHARLES almost 3 years ago

Description updated (diff)

Actions

Copy link

Updated by Nicolas CHARLES almost 3 years ago

git reflog expire --expire-unreachable=now --all
git gc --prune=now

doesn't work either

Actions

Copy link

Updated by Nicolas CHARLES almost 3 years ago

 git repack && git prune-packed && git reflog expire --expire=1.month.ago && git gc --aggressive

neither

Actions

Copy link

Updated by François ARMAND almost 3 years ago

Target version changed from 7.0.0~rc3 to 7.1.0~beta1
User visibility set to Operational - other Techniques | Rudder settings | Plugins

Root cause and analysis here: https://gitlab.com/gitlab-org/gitaly/-/issues/1656

These files appears when a git index-pack (during a git gc) is interrupted (likely by interactive user action like KILL -9, or oomkiller, or some other guard).

Once the git gc has ended (correctly or not), they are never used anymore (nor cleaned-up by git, which is bad).
They can be removed by hand.

> in rudder, we need to had that to the git maintenance batch.
> we also need to be sure to not start a git gc if another is already running

Actions

Copy link