Bug #3830: Technique "Package management for APT systems": packages with suffix :amd64 are not correctly detected - Rudder - Issue Tracker

Actions

#1

Updated by Nicolas PERRON over 10 years ago

Status changed from New to Discussion
Assignee set to Fabrice FLORE-THÉBAULT

Thank you for your report. Could you add the output of the command ?:

grep 'openjdk' /var/rudder/cfengine-community/state/software_packages.csv

It seems to me that this is the problem.

Actions

Copy link

#2

Updated by Fabrice FLORE-THÉBAULT over 10 years ago

openjdk-6-jre-lib,6b27-1.12.6-1~deb7u1,default,dpkg
openjdk-6-jre-headless:amd64,6b27-1.12.6-1~deb7u1,default,dpkg
openjdk-6-jre:amd64,6b27-1.12.6-1~deb7u1,default,dpkg

Actions

Copy link

#3

Updated by Nicolas PERRON over 10 years ago

Assignee changed from Fabrice FLORE-THÉBAULT to Matthieu CERDA

I was working on this bug and...

First, I was able to reproduce it:

On a Squeeze, the installation of the openjdk-6-jre will only be considered as a success:

root@debian-6-64:~# /var/rudder/cfengine-community/bin/cf-agent -KIb check_apt_package_installation
 >> Using command line specified bundlesequence
R: @@packageInstallation@@result_success@@49173e84-0818-4082-84e1-a00d8b6f2fbe@@78922455-116b-41ad-ae11-237abb4b4608@@2@@Debian/Ubuntu packages@@openjdk-6-jre@@2013-08-13 15:41:44+02:00##b9a71482-5030-4699-984d-b03d28bbbf36@#No action required for deb package openjdk-6-jre with policy: add

on a Wheezy, with multi arch activated (dpkg --add-architecture i386), the package installed will need the use of aptitude each time:

root@debian-7-64:~# /var/rudder/cfengine-community/bin/cf-agent -KIb check_apt_package_installation
[...]
Installing openjdk-6-jre...
Q:env DEBIAN_FRONTEND= ...:Reading package lists...
Q:env DEBIAN_FRONTEND= ...:Building dependency tree...
Q:env DEBIAN_FRONTEND= ...:Reading state information...
Q:env DEBIAN_FRONTEND= ...:Reading extended state information...
Q:env DEBIAN_FRONTEND= ...:Initializing package states...
Q:env DEBIAN_FRONTEND= ...:Reading task descriptions...
Q:env DEBIAN_FRONTEND= ...:No packages will be installed, upgraded, or removed.
Q:env DEBIAN_FRONTEND= ...:0 packages upgraded, 0 newly installed, 0 to remove and 35 not upgraded.
Q:env DEBIAN_FRONTEND= ...:Need to get 0 B of archives. After unpacking 0 B will be used.
Q:env DEBIAN_FRONTEND= ...:Writing extended state information...
Q:env DEBIAN_FRONTEND= ...:Reading package lists...
Q:env DEBIAN_FRONTEND= ...:Building dependency tree...
Q:env DEBIAN_FRONTEND= ...:Reading state information...
Q:env DEBIAN_FRONTEND= ...:Reading extended state information...
Q:env DEBIAN_FRONTEND= ...:Initializing package states...
Q:env DEBIAN_FRONTEND= ...:Reading task descriptions...
Q:env DEBIAN_FRONTEND= ...:
-> Command related to promiser "openjdk-6-jre" returned code defined as promise kept (0)
R: @@packageInstallation@@result_success@@b3d33690-89f6-46ab-982b-8365bfd6cfd7@@bacb72ee-6a0a-4228-b32d-040d172251de@@1@@Debian/Ubuntu packages@@openjdk-6-jre@@2013-08-13 15:57:12+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#No action required for deb package openjdk-6-jre with policy: add

And the content in the software_packages.csv file is different on the first and the second case:

On Squeeze:

root@debian-6-64:~# grep openjdk-6 /var/rudder/cfengine-community/state/software_packages.csv 
openjdk-6-jre-lib,6b27-1.12.6-1~deb6u1,default,dpkg
openjdk-6-jre-headless,6b27-1.12.6-1~deb6u1,default,dpkg
openjdk-6-jre,6b27-1.12.6-1~deb6u1,default,dpkg

On Wheezy multi arch:

root@debian-7-64:~# grep openjdk-6 /var/rudder/cfengine-community/state/software_packages.csv 
openjdk-6-jre-lib,6b27-1.12.6-1~deb7u1,default,dpkg
openjdk-6-jre-headless:amd64,6b27-1.12.6-1~deb7u1,default,dpkg
openjdk-6-jre:amd64,6b27-1.12.6-1~deb7u1,default,dpkg

Secondly, i've found that an option was missing into the package_method body used into the promise: package_default_arch_command (cf https://cfengine.com/archive/manuals/cf3-Reference#package_005fmethod-in-packages)

This command allows CFEngine to detect default architecture of packages managed by package manager. As an example, multiarch-enabled dpkg only lists architectures explicitly for multiarch-enabled packages.

In case this command is not provided, CFEngine treats all packages without explicit architecture set as belonging to implicit “default” architecture.

Then I've added it to the generated promise but the problem was still there. With the verbose mode I was able to know that the problem is really from the file software_packages.csv (which contains the packages installed on the machine...this is a cache file):

On Squeeze:

[...]
rudder>    =========================================================
rudder>    packages in bundle check_apt_package_installation (1)
rudder>    =========================================================
rudder>
rudder>
rudder>     .........................................................
rudder>     Promise's handle:
rudder>     Promise made by: "openjdk-6-jre" 
rudder>
rudder>     Comment:  Handling openjdk-6-jre using apt_nobulk, policy : add
rudder>     .........................................................
rudder>
rudder> Obtaining default architecture for package manager: /usr/bin/dpkg --print-architecture
rudder> Default architecture for package manager is 'amd64'
rudder>  -> Cache file exists and is sufficiently fresh according to (package_list_update_ifelapsed)
rudder>  -> Package (zlib1g-dev,1:1.2.3.4.dfsg-3,amd64) found
[...]
rudder>  -> Package (openjdk-6-jre-lib,6b27-1.12.6-1~deb6u1,amd64) found
rudder>  -> Package (openjdk-6-jre-headless,6b27-1.12.6-1~deb6u1,amd64) found
rudder>  -> Package (openjdk-6-jre,6b27-1.12.6-1~deb6u1,amd64) found
[...]
rudder>  -> Package version was not specified
rudder>  -> Looking for (openjdk-6-jre,*,*)
rudder>  -> Matched name openjdk-6-jre
rudder>  -> Matched version *
rudder>  -> Looking for (openjdk-6-jre,*,*)
rudder>  -> Matched name openjdk-6-jre
rudder>  -> Matched version *
rudder> Checking if package (openjdk-6-jre,*,*) is at the desired state (installed=1,matched=1)
rudder>  -> Package promises to refer to itself as "openjdk-6-jre" to the manager
rudder>  -> Package version seems to match criteria
rudder>  -> Package "openjdk-6-jre" already installed, so we never add it again
rudder>  ?> defining promise result class debian_install_kept_openjdk_6_jre
[...]

On Wheezy multiarch:

[...]
rudder>    =========================================================
rudder>    packages in bundle check_apt_package_installation (1)
rudder>    =========================================================
rudder>
rudder>
rudder>     .........................................................
rudder>     Promise's handle:
rudder>     Promise made by: "openjdk-6-jre" 
rudder>
rudder>     Comment:  Handling openjdk-6-jre using apt_nobulk, policy : add
rudder>     .........................................................
rudder>
rudder> Obtaining default architecture for package manager: /usr/bin/dpkg --print-architecture
rudder> Default architecture for package manager is 'amd64'
rudder>  -> Cache file exists and is sufficiently fresh according to (package_list_update_ifelapsed)
rudder>  -> Package (zlib1g:amd64,1:1.2.7.dfsg-13,amd64) found
[...]
rudder>  -> Package (openjdk-6-jre-lib,6b27-1.12.6-1~deb7u1,amd64) found
rudder>  -> Package (openjdk-6-jre-headless:amd64,6b27-1.12.6-1~deb7u1,amd64) found
rudder>  -> Package (openjdk-6-jre:amd64,6b27-1.12.6-1~deb7u1,amd64) found
[...]
rudder>  -> Package version was not specified
rudder>  -> Looking for (openjdk-6-jre,*,*)
rudder> No installed packages matched (openjdk-6-jre,*,*)
rudder>  -> Looking for (openjdk-6-jre,*,*)
rudder> No installed packages matched (openjdk-6-jre,*,*)
rudder> Checking if package (openjdk-6-jre,*,*) is at the desired state (installed=0,matched=0)
rudder>  -> Package promises to refer to itself as "openjdk-6-jre" to the manager
rudder>  -> Package version seems to match criteria
rudder>  -> Schedule package for addition
rudder>  -> Package (openjdk-6-jre,any,any) found
[...]

The most important part is that to know if the package is installed, the regex used is wrong:

rudder>  -> Looking for (openjdk-6-jre,*,*)
rudder> No installed packages matched (openjdk-6-jre,*,*)

Because since the multiarch is activated the installed package is described as:

Package (openjdk-6-jre:amd64,6b27-1.12.6-1~deb7u1,amd64) found

Finally, I suppose this is a CFEngine bug. Do you agree with me Matthieu ?

Actions

Copy link

#4

Updated by Nicolas PERRON over 10 years ago

Status changed from Discussion to In progress
Assignee changed from Matthieu CERDA to Nicolas PERRON

This is really a CFEngine bug, and this is not specific to the activation of multiarch

Actions

Copy link

#5

Updated by Nicolas PERRON over 10 years ago

Status changed from In progress to 8

I've opened a bug into CFEngine bug tracker here: https://cfengine.com/dev/issues/3280

I hope they will work on it since it concerns CFEngine 3.5.1-3...

Actions

Copy link

#6

Updated by Nicolas PERRON over 10 years ago

Status changed from 8 to In progress

I've found the problem. It was in the definition of the package_method. It should just not match the ':' which correspond to the architecture. I will make a Pull Request ASAP :)

Actions

Copy link

#7

Updated by Nicolas PERRON over 10 years ago

Target version changed from 2.6.4 to 2.4.8

This issue concerns all versions of Rudder, of course

Actions

Copy link

#8

Updated by Nicolas PERRON over 10 years ago

Project changed from Rudder to 24
Category deleted (~~Techniques~~)

Actions

Copy link

#9

Updated by Nicolas PERRON over 10 years ago

Status changed from In progress to Pending technical review
Assignee changed from Nicolas PERRON to Matthieu CERDA
% Done changed from 0 to 100

Pull Request URL added: https://github.com/Normation/rudder-techniques/pull/165

Matthieu, since Jonathan and Nicolas are absent, could you review it please ?

Actions

Copy link

#10

Updated by Matthieu CERDA over 10 years ago

Assignee changed from Matthieu CERDA to Nicolas PERRON

I approve this PR. Having multiarch support is good !

Actions

Copy link

#11

Updated by Nicolas PERRON over 10 years ago

Assignee changed from Nicolas PERRON to Matthieu CERDA

Matthieu, I have updated the PR. Could you review it please ?

Actions

Copy link

#12

Updated by Jonathan CLARKE over 10 years ago

Status changed from Pending technical review to Discussion
Assignee changed from Matthieu CERDA to Nicolas PERRON

I'm sorry but I'm not keen on this approach. If there are bugs upstream (ie, in CFEngine, or it's stdlib as in this case), we must strive to fix them upstream. Otherwise, this is equivalent to forking their code, with all the inconvenvience that brings: we have to maintain it, we won't benefit from other bug fixes, etc...

So I think a better way to go about this would be to send a PR to CFEngine to change the buggy package_method on their stdlib, and patch our copy of cfengine_stdlib.cf until the PR is merged upstream.

Nicolas, can you work on this please?

Actions

Copy link

#13

Updated by Nicolas PERRON over 10 years ago

Assignee changed from Nicolas PERRON to Jonathan CLARKE

Jonathan CLARKE wrote:

I'm sorry but I'm not keen on this approach. If there are bugs upstream (ie, in CFEngine, or it's stdlib as in this case), we must strive to fix them upstream. Otherwise, this is equivalent to forking their code, with all the inconvenvience that brings: we have to maintain it, we won't benefit from other bug fixes, etc...

So I think a better way to go about this would be to send a PR to CFEngine to change the buggy package_method on their stdlib, and patch our copy of cfengine_stdlib.cf until the PR is merged upstream.

Nicolas, can you work on this please?

Ok, now i've made a Pull Request what should we do for our Techniques ? Should we patch our CFEngine standard Library and continue to use the body from it ?

Actions

Copy link

#14

Updated by Jonathan CLARKE over 10 years ago

Nicolas PERRON wrote:

Jonathan CLARKE wrote:

I'm sorry but I'm not keen on this approach. If there are bugs upstream (ie, in CFEngine, or it's stdlib as in this case), we must strive to fix them upstream. Otherwise, this is equivalent to forking their code, with all the inconvenvience that brings: we have to maintain it, we won't benefit from other bug fixes, etc...

So I think a better way to go about this would be to send a PR to CFEngine to change the buggy package_method on their stdlib, and patch our copy of cfengine_stdlib.cf until the PR is merged upstream.

Ok, now i've made a Pull Request what should we do for our Techniques ? Should we patch our CFEngine standard Library and continue to use the body from it ?

We should patch all package_methods in our promises that need it. That includes, but is not limited to, the cfengine_stdlib.cf. That way, we avoid modifying our Techniques, and we ensure that things will keep running smoothly when the stdlib is update, and we continue to benefit from other improvements in the stdlib.

Actions

Copy link

#15

Updated by Jonathan CLARKE over 10 years ago

Assignee changed from Jonathan CLARKE to Nicolas PERRON

Actions

Copy link

#16

Updated by Nicolas PERRON over 10 years ago

Target version changed from 2.4.8 to 2.4.9

Actions

Copy link

#17

Updated by Nicolas PERRON over 10 years ago

Assignee changed from Nicolas PERRON to Jonathan CLARKE

Jon, patch has been accepted and merged but for CFEngine 3.5.*. What should we do now ? merging the PR here ? wait for a Rudder release including CFEngine 3.5.* ?

Actions

Copy link

#18

Updated by Jonathan CLARKE over 10 years ago

Assignee changed from Jonathan CLARKE to Nicolas PERRON
Pull Request set to https://github.com/Normation/rudder-techniques/pull/165

Nicolas PERRON wrote:

Jon, patch has been accepted and merged but for CFEngine 3.5.*. What should we do now ? merging the PR here ? wait for a Rudder release including CFEngine 3.5.* ?

Nicolas, I already explained the approach to take in comment #14 above.

The current Pull Request attached to this issue will not be merged as is, as it constitues a fork of one of CFEngine's most complex bodies. We have provided upstream with a patch, we must apply this to our copy too and move on.

Actions

Copy link

#19

Updated by Nicolas PERRON over 10 years ago

Target version changed from 2.4.9 to 2.4.10

Actions

Copy link

#20

Updated by Nicolas PERRON over 10 years ago

Target version changed from 2.4.10 to 2.4.11

Actions

Copy link

#21

Updated by Nicolas PERRON over 10 years ago

Target version changed from 2.4.11 to 2.4.12

Actions

Copy link

#22

Updated by Nicolas PERRON over 10 years ago

Target version changed from 2.4.12 to 2.4.13

Actions

Copy link

#23

Updated by Nicolas CHARLES over 10 years ago

Category set to Techniques
Status changed from Discussion to 8
Assignee changed from Nicolas PERRON to Nicolas CHARLES
Target version changed from 2.4.13 to 2.6.11

Bug has been fixed upstream \o/
https://github.com/cfengine/core/pull/1294/files

Going to import the fix from CFEngine

Actions

Copy link

#24

Updated by Nicolas CHARLES about 10 years ago

ha, i completly missed that one, as it was qualified
yeah for fix from upstream !

Actions

Copy link

#25

Updated by Nicolas CHARLES about 10 years ago

Pull Request changed from https://github.com/Normation/rudder-techniques/pull/165 to https://github.com/Normation/rudder-techniques/pull/298

PR is there:
https://github.com/Normation/rudder-techniques/pull/298

Actions

Copy link

#26

Updated by Nicolas CHARLES about 10 years ago

Status changed from 8 to Pending technical review
Assignee changed from Nicolas CHARLES to Jonathan CLARKE

Actions

Copy link

#27

Updated by Nicolas CHARLES about 10 years ago

Status changed from Pending technical review to Pending release

Applied in changeset commit:168e2fe0bc1b63fc651d8d347b68a8d46282ee06.

Actions

Copy link

#28

Updated by Jonathan CLARKE about 10 years ago

Applied in changeset commit:a538bd3518afad3e150f9b183c2bfc205f031576.

Actions

Copy link

#29

Updated by Vincent MEMBRÉ about 10 years ago

Subject changed from some inconsistencies in detection of packages installed on debian when package has the :amd64 suffix to Technique "Package management for APT systems": packages with suffix :amd64 are not correctly detected

Actions

Copy link

#30

Updated by Vincent MEMBRÉ about 10 years ago

Status changed from Pending release to Released

This bug has been fixed in Rudder 2.6.11, which was released today.
Check out:

The release announcement: http://www.rudder-project.org/pipermail/rudder-announce/2014-March/000077.html
The full ChangeLog: http://www.rudder-project.org/foswiki/bin/view/System/Documentation:ChangeLog26
Download information: https://www.rudder-project.org/site/get-rudder/downloads/

Actions

Copy link

#31

Updated by Benoît PECCATTE about 9 years ago

Project changed from 24 to Rudder
Category changed from Techniques to Techniques

Project

General

Profile

Rudder

Custom queries

Bug #3830

Technique "Package management for APT systems": packages with suffix :amd64 are not correctly detected

Updated by Nicolas PERRON over 10 years ago

Updated by Fabrice FLORE-THÉBAULT over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Matthieu CERDA over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Jonathan CLARKE over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Jonathan CLARKE over 10 years ago

Updated by Jonathan CLARKE over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Jonathan CLARKE over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas PERRON over 10 years ago

Updated by Nicolas CHARLES over 10 years ago

Updated by Nicolas CHARLES about 10 years ago

Updated by Nicolas CHARLES about 10 years ago

Updated by Nicolas CHARLES about 10 years ago

Updated by Nicolas CHARLES about 10 years ago

Updated by Jonathan CLARKE about 10 years ago

Updated by Vincent MEMBRÉ about 10 years ago

Updated by Vincent MEMBRÉ about 10 years ago

Updated by Benoît PECCATTE about 9 years ago