Question #8912: Incorrect packaging in apt repo - Rudder - Issue Tracker

Custom queries

8.1 beta
Bugs triage
My TR
Need technical review
Open bugs (all)
Open bugs (UI - UX)
Open UX defects (all)
Opened by users

Actions

Copy link

Question #8912

closed

Incorrect packaging in apt repo

Added by Dmitry Svyatogorov over 7 years ago. Updated over 7 years ago.

Status:

Resolved

Priority:

Assignee:

Category:

Packaging

Target version:

3.2.9

Regression:

Description

Using rudder repo in Ubuntu 14.04:

# cat /etc/apt/sources.list.d/rudder.list
deb http://www.rudder-project.org/apt-3.2/ trusty main

Running update:

# apt-get update
… All previous sources was updated ok …
W: Failed to fetch http://www.rudder-project.org/apt-3.2/dists/trusty/main/binary-amd64/Packages  Hash Sum mismatch
W: Failed to fetch http://www.rudder-project.org/apt-3.2/dists/trusty/main/binary-i386/Packages  Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.

Running "apt-get -o Debug::pkgAcquire::Auth=true update" shows hashes are really different for rudder at this time.
Running "rm -Rf /var/lib/apt/lists/*" fixes the problem until next repo update.

M.b. http://www.rudder-project.org/apt-3.2/dists/trusty/Release is missing update for some reason?

Most of our hosts with (ubuntu + rudder) stays behind apt-cacher-ng proxy. I'll try to debug this moment deeply in few days.

History
Notes
Property changes

Actions

Copy link

Updated by Vincent MEMBRÉ over 7 years ago

Priority changed from N/A to 1

Thank you Dmitry!

If you don't apply your fix (rm -Rf /var/lib/apt/lists/*), can packages be updated ?

I'll try to make an update on an ubuntu to understand this issue better ...

Actions

Copy link

Updated by Benoît PECCATTE over 7 years ago

I cannot reproduce.
Is it still a problem ?

Actions

Copy link

Updated by Dmitry Svyatogorov over 7 years ago

# apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 474A19E8
# echo "deb http://www.rudder-project.org/apt-3.2/ $(lsb_release -cs) main" > /etc/apt/sources.list.d/rudder.list
# apt-get update
Get:1 http://www.rudder-project.org trusty InRelease [9,597 B]
…
Get:3 http://www.rudder-project.org trusty/main amd64 Packages [5,554 B]       
…
Get:5 http://www.rudder-project.org trusty/main i386 Packages [5,552 B]        
…
Ign http://www.rudder-project.org trusty/main Translation-en_US                
Ign http://www.rudder-project.org trusty/main Translation-en                   
…
Fetched 52.4 kB in 9s (5,325 B/s)                                              
W: Failed to fetch http://www.rudder-project.org/apt-3.2/dists/trusty/main/binary-amd64/Packages  Hash Sum mismatch

W: Failed to fetch http://www.rudder-project.org/apt-3.2/dists/trusty/main/binary-i386/Packages  Hash Sum mismatch

E: Some index files failed to download. They have been ignored, or old ones used instead.

The same error after "rm -Rf /var/lib/apt/lists/*".

# cat /etc/apt/apt.conf.d/01proxy 
Acquire::http { Proxy "http://apt-proxy:3142"; };

So, the trouble appears at apt-cacher-ng, installed on host "apt-proxy".

# egrep -v '^\s*(#|$)' /etc/apt-cacher-ng/acng.conf
CacheDir: /var/cache/apt-cacher-ng
LogDir: /var/log/apt-cacher-ng
Port:3142
ReportPage: acng-report.html
ExTreshold: 4
LocalDirs: acng-doc /usr/share/doc/apt-cacher-ng

# ll /var/cache/apt-cacher-ng/www.rudder-project.org/apt-3.2/dists/trusty/
total 52
drwxr-sr-x 3 apt-cacher-ng apt-cacher-ng 4096 Jul  6 23:34 ./
drwxr-sr-x 3 apt-cacher-ng apt-cacher-ng 4096 Jul  6 13:41 ../
-rw-r--r-- 1 apt-cacher-ng apt-cacher-ng 9597 Jul  6 23:30 InRelease
-rw-r--r-- 1 apt-cacher-ng apt-cacher-ng  224 Jul  6 23:30 InRelease.head
drwxr-sr-x 4 apt-cacher-ng apt-cacher-ng 4096 Jul  6 13:41 main/
-rw-r--r-- 1 apt-cacher-ng apt-cacher-ng 8729 Jul  6 23:34 Release
-rw-r--r-- 1 apt-cacher-ng apt-cacher-ng  819 Sep 14 07:24 Release.gpg
-rw-r--r-- 1 apt-cacher-ng apt-cacher-ng  225 Sep 14 07:24 Release.gpg.head
-rw-r--r-- 1 apt-cacher-ng apt-cacher-ng  222 Jul  6 23:34 Release.head

"Release" is appreciable out-of-dated. But, wat's the reason (ubuntu repos etc. are renewing well)?

< HTTP/1.1 200 OK
< Last-Modified: Tue, 30 Aug 2016 14:51:49 GMT
< ETag: "1bee021-4ae1-53b4b1eceab40" 
< Accept-Ranges: bytes
< Content-Length: 19169
< Accept-Ranges: bytes
< Date: Tue, 20 Sep 2016 14:21:48 GMT
< Age: 0
< Connection: keep-alive
< X-Cache: uncached

My current suspicion is that apt-cacher-ng implements some weak http parser, so it expects order like apache/nginx produces

< HTTP/1.1 200 OK
< Date: Tue, 20 Sep 2016 14:44:03 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Last-Modified: Thu, 08 May 2014 14:19:03 GMT
< ETag: "60-4f8e429b803c0" 
< Accept-Ranges: bytes
< Content-Length: 96
< Cache-Control: max-age=0, proxy-revalidate
< Expires: Tue, 20 Sep 2016 14:44:03 GMT

< HTTP/1.1 200 OK
< Server: nginx/1.4.6 (Ubuntu)
< Date: Tue, 20 Sep 2016 14:53:25 GMT
< Content-Type: application/octet-stream
< Content-Length: 58512
< Last-Modified: Thu, 08 May 2014 14:19:26 GMT
< Connection: keep-alive
< ETag: "536b926e-e490" 
< Accept-Ranges: bytes

where "Date:"/"Server:" comes first.

Actions

Copy link

Updated by Dmitry Svyatogorov over 7 years ago

# dpkg -l | grep apt-cacher
ii  apt-cacher-ng                        0.7.26-1                         amd64        caching proxy server for software repositories

Actions

Copy link

Updated by Dmitry Svyatogorov over 7 years ago

Work-around for apt-cacher-ng, from https://www.unix-ag.uni-kl.de/~bloch/acng/html/howtos.html#howto-dontcache1
Use "DontCache:" directive. No examles, but works so:

echo "DontCache: www.rudder-project.org" >> /etc/apt-cacher-ng/acng.conf

Actions

Copy link

Updated by Benoît PECCATTE over 7 years ago

I see no reason a tool would not parse properly out of order HTTP headers, since there has never been any order in http headers.
Maybe aptcacher-ng has a broken etags database. I checked and our etags are based on the default apache one: inode+mtime+size which is correct.

Anyway, this looks like an aptcacher-ng bug.
I don't see what I can do to help and would like to close this issue, what do you think ?

Actions

Copy link