Rudder

Thank you for your report. Could you add the output of the command ?:

grep 'openjdk' /var/rudder/cfengine-community/state/software_packages.csv

It seems to me that this is the problem.

I was working on this bug and...

First, I was able to reproduce it:

• On a Squeeze, the installation of the openjdk-6-jre will only be considered as a success:
  

  root@debian-6-64:~# /var/rudder/cfengine-community/bin/cf-agent -KIb check_apt_package_installation
   >> Using command line specified bundlesequence
  R: @@packageInstallation@@result_success@@49173e84-0818-4082-84e1-a00d8b6f2fbe@@78922455-116b-41ad-ae11-237abb4b4608@@2@@Debian/Ubuntu packages@@openjdk-6-jre@@2013-08-13 15:41:44+02:00##b9a71482-5030-4699-984d-b03d28bbbf36@#No action required for deb package openjdk-6-jre with policy: add
  

• on a Wheezy, with multi arch activated (dpkg --add-architecture i386), the package installed will need the use of aptitude each time:
  

  root@debian-7-64:~# /var/rudder/cfengine-community/bin/cf-agent -KIb check_apt_package_installation
  [...]
  Installing openjdk-6-jre...
  Q:env DEBIAN_FRONTEND= ...:Reading package lists...
  Q:env DEBIAN_FRONTEND= ...:Building dependency tree...
  Q:env DEBIAN_FRONTEND= ...:Reading state information...
  Q:env DEBIAN_FRONTEND= ...:Reading extended state information...
  Q:env DEBIAN_FRONTEND= ...:Initializing package states...
  Q:env DEBIAN_FRONTEND= ...:Reading task descriptions...
  Q:env DEBIAN_FRONTEND= ...:No packages will be installed, upgraded, or removed.
  Q:env DEBIAN_FRONTEND= ...:0 packages upgraded, 0 newly installed, 0 to remove and 35 not upgraded.
  Q:env DEBIAN_FRONTEND= ...:Need to get 0 B of archives. After unpacking 0 B will be used.
  Q:env DEBIAN_FRONTEND= ...:Writing extended state information...
  Q:env DEBIAN_FRONTEND= ...:Reading package lists...
  Q:env DEBIAN_FRONTEND= ...:Building dependency tree...
  Q:env DEBIAN_FRONTEND= ...:Reading state information...
  Q:env DEBIAN_FRONTEND= ...:Reading extended state information...
  Q:env DEBIAN_FRONTEND= ...:Initializing package states...
  Q:env DEBIAN_FRONTEND= ...:Reading task descriptions...
  Q:env DEBIAN_FRONTEND= ...:
  -> Command related to promiser "openjdk-6-jre" returned code defined as promise kept (0)
  R: @@packageInstallation@@result_success@@b3d33690-89f6-46ab-982b-8365bfd6cfd7@@bacb72ee-6a0a-4228-b32d-040d172251de@@1@@Debian/Ubuntu packages@@openjdk-6-jre@@2013-08-13 15:57:12+02:00##5ee53fbe-ff96-4606-a78b-5e408cbf77c9@#No action required for deb package openjdk-6-jre with policy: add
  

• On Squeeze:
  

  root@debian-6-64:~# grep openjdk-6 /var/rudder/cfengine-community/state/software_packages.csv 
  openjdk-6-jre-lib,6b27-1.12.6-1~deb6u1,default,dpkg
  openjdk-6-jre-headless,6b27-1.12.6-1~deb6u1,default,dpkg
  openjdk-6-jre,6b27-1.12.6-1~deb6u1,default,dpkg
  

• On Wheezy multi arch:
  

  root@debian-7-64:~# grep openjdk-6 /var/rudder/cfengine-community/state/software_packages.csv 
  openjdk-6-jre-lib,6b27-1.12.6-1~deb7u1,default,dpkg
  openjdk-6-jre-headless:amd64,6b27-1.12.6-1~deb7u1,default,dpkg
  openjdk-6-jre:amd64,6b27-1.12.6-1~deb7u1,default,dpkg
  

Secondly, i've found that an option was missing into the package_method body used into the promise: package_default_arch_command (cf https://cfengine.com/archive/manuals/cf3-Reference#package_005fmethod-in-packages)

This command allows CFEngine to detect default architecture of packages managed by package manager. As an example, multiarch-enabled dpkg only lists architectures explicitly for multiarch-enabled packages.

In case this command is not provided, CFEngine treats all packages without explicit architecture set as belonging to implicit “default” architecture. 

Then I've added it to the generated promise but the problem was still there. With the verbose mode I was able to know that the problem is really from the file software_packages.csv (which contains the packages installed on the machine...this is a cache file):

• On Squeeze:
  

  [...]
  rudder>    =========================================================
  rudder>    packages in bundle check_apt_package_installation (1)
  rudder>    =========================================================
  rudder>
  rudder>
  rudder>     .........................................................
  rudder>     Promise's handle:
  rudder>     Promise made by: "openjdk-6-jre" 
  rudder>
  rudder>     Comment:  Handling openjdk-6-jre using apt_nobulk, policy : add
  rudder>     .........................................................
  rudder>
  rudder> Obtaining default architecture for package manager: /usr/bin/dpkg --print-architecture
  rudder> Default architecture for package manager is 'amd64'
  rudder>  -> Cache file exists and is sufficiently fresh according to (package_list_update_ifelapsed)
  rudder>  -> Package (zlib1g-dev,1:1.2.3.4.dfsg-3,amd64) found
  [...]
  rudder>  -> Package (openjdk-6-jre-lib,6b27-1.12.6-1~deb6u1,amd64) found
  rudder>  -> Package (openjdk-6-jre-headless,6b27-1.12.6-1~deb6u1,amd64) found
  rudder>  -> Package (openjdk-6-jre,6b27-1.12.6-1~deb6u1,amd64) found
  [...]
  rudder>  -> Package version was not specified
  rudder>  -> Looking for (openjdk-6-jre,*,*)
  rudder>  -> Matched name openjdk-6-jre
  rudder>  -> Matched version *
  rudder>  -> Looking for (openjdk-6-jre,*,*)
  rudder>  -> Matched name openjdk-6-jre
  rudder>  -> Matched version *
  rudder> Checking if package (openjdk-6-jre,*,*) is at the desired state (installed=1,matched=1)
  rudder>  -> Package promises to refer to itself as "openjdk-6-jre" to the manager
  rudder>  -> Package version seems to match criteria
  rudder>  -> Package "openjdk-6-jre" already installed, so we never add it again
  rudder>  ?> defining promise result class debian_install_kept_openjdk_6_jre
  [...]
  

• On Wheezy multiarch:
  

  [...]
  rudder>    =========================================================
  rudder>    packages in bundle check_apt_package_installation (1)
  rudder>    =========================================================
  rudder>
  rudder>
  rudder>     .........................................................
  rudder>     Promise's handle:
  rudder>     Promise made by: "openjdk-6-jre" 
  rudder>
  rudder>     Comment:  Handling openjdk-6-jre using apt_nobulk, policy : add
  rudder>     .........................................................
  rudder>
  rudder> Obtaining default architecture for package manager: /usr/bin/dpkg --print-architecture
  rudder> Default architecture for package manager is 'amd64'
  rudder>  -> Cache file exists and is sufficiently fresh according to (package_list_update_ifelapsed)
  rudder>  -> Package (zlib1g:amd64,1:1.2.7.dfsg-13,amd64) found
  [...]
  rudder>  -> Package (openjdk-6-jre-lib,6b27-1.12.6-1~deb7u1,amd64) found
  rudder>  -> Package (openjdk-6-jre-headless:amd64,6b27-1.12.6-1~deb7u1,amd64) found
  rudder>  -> Package (openjdk-6-jre:amd64,6b27-1.12.6-1~deb7u1,amd64) found
  [...]
  rudder>  -> Package version was not specified
  rudder>  -> Looking for (openjdk-6-jre,*,*)
  rudder> No installed packages matched (openjdk-6-jre,*,*)
  rudder>  -> Looking for (openjdk-6-jre,*,*)
  rudder> No installed packages matched (openjdk-6-jre,*,*)
  rudder> Checking if package (openjdk-6-jre,*,*) is at the desired state (installed=0,matched=0)
  rudder>  -> Package promises to refer to itself as "openjdk-6-jre" to the manager
  rudder>  -> Package version seems to match criteria
  rudder>  -> Schedule package for addition
  rudder>  -> Package (openjdk-6-jre,any,any) found
  [...]
  

The most important part is that to know if the package is installed, the regex used is wrong:

rudder>  -> Looking for (openjdk-6-jre,*,*)
rudder> No installed packages matched (openjdk-6-jre,*,*)

Because since the multiarch is activated the installed package is described as:

Package (openjdk-6-jre:amd64,6b27-1.12.6-1~deb7u1,amd64) found

Finally, I suppose this is a CFEngine bug. Do you agree with me Matthieu ?

This is really a CFEngine bug, and this is not specific to the activation of multiarch

I've opened a bug into CFEngine bug tracker here: https://cfengine.com/dev/issues/3280

I hope they will work on it since it concerns CFEngine 3.5.1-3...

I've found the problem. It was in the definition of the package_method. It should just not match the ':' which correspond to the architecture. I will make a Pull Request ASAP :)

This issue concerns all versions of Rudder, of course

Pull Request URL added: https://github.com/Normation/rudder-techniques/pull/165

Matthieu, since Jonathan and Nicolas are absent, could you review it please ?

I approve this PR. Having multiarch support is good !

Matthieu, I have updated the PR. Could you review it please ?

I'm sorry but I'm not keen on this approach. If there are bugs upstream (ie, in CFEngine, or it's stdlib as in this case), we must strive to fix them upstream. Otherwise, this is equivalent to forking their code, with all the inconvenvience that brings: we have to maintain it, we won't benefit from other bug fixes, etc...

So I think a better way to go about this would be to send a PR to CFEngine to change the buggy package_method on their stdlib, and patch our copy of cfengine_stdlib.cf until the PR is merged upstream.

Nicolas, can you work on this please?

Jonathan CLARKE wrote:

I'm sorry but I'm not keen on this approach. If there are bugs upstream (ie, in CFEngine, or it's stdlib as in this case), we must strive to fix them upstream. Otherwise, this is equivalent to forking their code, with all the inconvenvience that brings: we have to maintain it, we won't benefit from other bug fixes, etc...

So I think a better way to go about this would be to send a PR to CFEngine to change the buggy package_method on their stdlib, and patch our copy of cfengine_stdlib.cf until the PR is merged upstream.

Nicolas, can you work on this please?

Ok, now i've made a Pull Request what should we do for our Techniques ? Should we patch our CFEngine standard Library and continue to use the body from it ?

Jon, patch has been accepted and merged but for CFEngine 3.5.*. What should we do now ? merging the PR here ? wait for a Rudder release including CFEngine 3.5.* ?

Nicolas PERRON wrote:

Jon, patch has been accepted and merged but for CFEngine 3.5.*. What should we do now ? merging the PR here ? wait for a Rudder release including CFEngine 3.5.* ?

Nicolas, I already explained the approach to take in comment #14 above.

The current Pull Request attached to this issue will not be merged as is, as it constitues a fork of one of CFEngine's most complex bodies. We have provided upstream with a patch, we must apply this to our copy too and move on.

Bug has been fixed upstream \o/
https://github.com/cfengine/core/pull/1294/files

Going to import the fix from CFEngine

PR is there:
https://github.com/Normation/rudder-techniques/pull/298

Applied in changeset commit:168e2fe0bc1b63fc651d8d347b68a8d46282ee06.

This bug has been fixed in Rudder 2.6.11, which was released today.
Check out:

• The release announcement: http://www.rudder-project.org/pipermail/rudder-announce/2014-March/000077.html
• The full ChangeLog: http://www.rudder-project.org/foswiki/bin/view/System/Documentation:ChangeLog26
• Download information: https://www.rudder-project.org/site/get-rudder/downloads/