Architecture #2668
closedReplace rudder-cfengine-community by rudder-agent for the packages required with rudder-server-root
Added by Nicolas PERRON about 12 years ago. Updated over 9 years ago.
Description
Since we're trying to let the server be managed by itself (#1971), we need to use rudder-agent instead of rudder-cfengine-community. Nevertheless, it seems that somes differences are present:
- Not the same versions of BerkeleyDB for SLES 11 between rudder-agent and rudder-cfengine-community
- db43-devel for rudder-agent
- libdb-4_5-devel for rudder-cfengine-community
- Not the same requirement for build
- make db4-devel byacc for rudder-agent
- db4-devel openssl-devel for rudder-cfengine-community
- A patch about header (fix-missing-headers) is applied only on rudder-agent
- Options about BerkeleyDB are used in build command only in rudder-agent (that's normal because rudder-cfengine-community is not used on SLES10 and options are differents from SLES 11)
- All logic about fusionInventory are only in rudder-agent package
- Initial promises are installed only with rudder-agent packages and only initial promises for nodes
- node's initial promises are installed in rudder-agent
- root's initial promises are installed in rudder-server-root
- Init daemon and default files are differents
- the names are differents: rudder-agent and cfengine-community
- the default file use the name of the init daemon, which are differents like said in the point above
- UUID are generated only in rudder-agent
- UUID is checked and created if needed in rudder-agent
- UUID is checked and created if needed in rudder-server-root
Updated by Nicolas PERRON about 12 years ago
Before changing anything in the packaging dependencies, we have to ensure that initial promises are merged correctly (#2669)
Updated by Nicolas PERRON about 12 years ago
The merged initial promises are located in rudder-techniques repository at rudder-techniques/initial-promises/initialPromises/.
We have to change all the packaging in order to use it instead of rudder-techniques/initial-promises/nodeInitialPromises or rudder-techniques/initial-promises/rootServerInitialPromises.
Updated by Jonathan CLARKE about 12 years ago
Nicolas PERRON wrote:
The merged initial promises are located in rudder-techniques repository at rudder-techniques/initial-promises/initialPromises/.
We have to change all the packaging in order to use it instead of rudder-techniques/initial-promises/nodeInitialPromises or rudder-techniques/initial-promises/rootServerInitialPromises.
Please use a different path - this is the most confusing name I've seen! A directory with the same name inside itself is silly, and especially is you use CamelCase one and dashed the next. How about initial-promises/node-server/ ?
Updated by Nicolas PERRON about 12 years ago
Jonathan CLARKE wrote:
Nicolas PERRON wrote:
The merged initial promises are located in rudder-techniques repository at rudder-techniques/initial-promises/initialPromises/.
We have to change all the packaging in order to use it instead of rudder-techniques/initial-promises/nodeInitialPromises or rudder-techniques/initial-promises/rootServerInitialPromises.Please use a different path - this is the most confusing name I've seen! A directory with the same name inside itself is silly, and especially is you use CamelCase one and dashed the next. How about initial-promises/node-server/ ?
I agree with you, it is confusing and initial-promises/node-server/ will be better. But at the end, there will be only one folder: initial-promises/ which will contain all the promises for nodes, servers built on CFEngine Community and CFEngine Nova.
Updated by Jonathan CLARKE about 12 years ago
- Status changed from New to In progress
Nicolas, your last commit on the rudder-agent package seems to have a logic flaw: the conflicts and replaces items in debian/control should be <= 2.4.0~beta3, surely, not >= ? And maybe Provides too, but I'm not sure about that one.
Updated by Jonathan CLARKE about 12 years ago
Jonathan CLARKE wrote:
Nicolas, your last commit on the rudder-agent package seems to have a logic flaw: the conflicts and replaces items in debian/control should be <= 2.4.0~beta3, surely, not >= ? And maybe Provides too, but I'm not sure about that one.
Ah, a clarification on Provides, from http://www.debian.org/doc/debian-policy/ch-relationships.html :
All of the fields except for Provides may restrict their applicability to particular versions of each named package.
So you may not specify a version with Provides.
Updated by Jonathan CLARKE about 12 years ago
Actually, I think the Conflicts: and Replaces: attributes should not have version specifications. Whatever the version of rudder-cfengine-community, if we're installing this version of rudder-agent, we want to replace/conflict with it.
Updated by Nicolas PERRON about 12 years ago
Jonathan CLARKE wrote:
Actually, I think the Conflicts: and Replaces: attributes should not have version specifications. Whatever the version of rudder-cfengine-community, if we're installing this version of rudder-agent, we want to replace/conflict with it.
You're right, it seems that we were doing it wrong with rudder-webapp/rudder-policy-templates as i explained in #1971 . Indeed, here is the explanation of provides (http://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual):
If a relationship field has a version number attached, only real packages will be considered to see whether the relationship is satisfied (or the prohibition violated, for a conflict or breakage). In other words, if a version number is specified, this is a request to ignore all Provides for that package name and consider only real packages.[...]
Updated by Nicolas PERRON about 12 years ago
- rudder-agent generate an random uuid in /opt/rudder/etc/uuid.hive and rudder-server-root check if this uuid is root. If this is not root, the packaging stop and ask what to do. This is not good
- rudder-agent install initial promises and rudder-server-root too.
We have deal with these problems, except that it doesn't work for the first one. We have decided to force 'root' in /opt/rudder/etc/uuid.hive at rudder-server-root postint and it didn't work.
I suggest to force it in the preinst instead, it should work.
Updated by Nicolas PERRON about 12 years ago
- % Done changed from 0 to 80
We have made a large part of work
Updated by Jonathan CLARKE about 12 years ago
Nicolas PERRON wrote:
There is a problem now:
- rudder-agent generate an random uuid in /opt/rudder/etc/uuid.hive and rudder-server-root check if this uuid is root. If this is not root, the packaging stop and ask what to do. This is not good
- rudder-agent install initial promises and rudder-server-root too.
We have deal with these problems, except that it doesn't work for the first one. We have decided to force 'root' in /opt/rudder/etc/uuid.hive at rudder-server-root postint and it didn't work.
How does it "not work"? Please explain, this is very unclear.
Updated by Nicolas PERRON about 12 years ago
Jonathan CLARKE wrote:
Nicolas PERRON wrote:
There is a problem now:
- rudder-agent generate an random uuid in /opt/rudder/etc/uuid.hive and rudder-server-root check if this uuid is root. If this is not root, the packaging stop and ask what to do. This is not good
- rudder-agent install initial promises and rudder-server-root too.
We have deal with these problems, except that it doesn't work for the first one. We have decided to force 'root' in /opt/rudder/etc/uuid.hive at rudder-server-root postint and it didn't work.
How does it "not work"? Please explain, this is very unclear.
The problem as not been fixed. At the installation stage of rudder-server-root, dpkg ask us which file we want to keep: local or installation file:
Errors were encountered while processing: rudder-server-root E: Sub-process /usr/bin/dpkg returned an error code (1) A package failed to install. Trying to recover: Setting up rudder-server-root (2.4.0~beta3~git201207240019-squeeze0) ... Configuration file `/opt/rudder/etc/uuid.hive' ==> File on system created by you or by a script. ==> File also in package provided by package maintainer. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** uuid.hive (Y/I/N/O/D/Z) [default=N] ?
It think that's force 'root' to the /opt/rudder/etc/uuid.hive file before the installation (preinst) is a solution. (rudder-server-root doesn't have to be anything else than 'root')
Updated by Jonathan CLARKE about 12 years ago
I think you're misunderstanding something fundamental here: there is a significant difference in packaging between "installing" a file, and "adding content" to a file. In the first case, it means that that file is always managed by one, and only one, package. In the second, anything can happen to the file.
Looking at these packages, I see that:- rudder-agent does not install /opt/rudder/etc/uuid.hive (there is no mention of it in debian/rules or debian/rudder-agent.install)
- rudder-agent does concatenate the result of the "uuidgen" command into that file (see https://github.com/Normation/rudder-packages/blob/master/rudder-agent/debian/postinst#L52)
- rudder-server-root does install /opt/rudder/etc/uuid.hive (see https://github.com/Normation/rudder-packages/blob/master/rudder-server-root/debian/rules#L60)
- rudder-server-root does also concatenate the string "root" into that file (see https://github.com/Normation/rudder-packages/blob/branches/rudder/2.4/rudder-server-root/debian/postinst#L23)
So the problem here is that rudder-agent is being installed, generating the content of /opt/rudder/etc/uuid.hive as expected, and then rudder-server-root gets installed but complains because it knows that /opt/rudder/etc/uuid.hive belongs to itself.
The solution here is simple: adopt one or the other method. I suggest that :- rudder-agent should own the /opt/rudder/etc/uuid.hive file. That means:
- rudder-agent should install it, maybe with a file that just has a comment in saying "# The content of this file should be replaced with a valid UUID for Rudder")
- rudder-agent should have it liste as a configuration file (this ensures that the file will be deleted if a "aptitude purge rudder-*" is run)
- rudder-server-root should no longer install it (or list it as a config file)
- both rudder-agent and rudder-server-root should overwrite it's contents in their postinst scripts (as is already the case)
What do you think? Is this clearer?
Updated by Nicolas PERRON about 12 years ago
Jonathan CLARKE wrote:
I think you're misunderstanding something fundamental here: there is a significant difference in packaging between "installing" a file, and "adding content" to a file. In the first case, it means that that file is always managed by one, and only one, package. In the second, anything can happen to the file.
Looking at these packages, I see that:
- rudder-agent does not install /opt/rudder/etc/uuid.hive (there is no mention of it in debian/rules or debian/rudder-agent.install)
- rudder-agent does concatenate the result of the "uuidgen" command into that file (see https://github.com/Normation/rudder-packages/blob/master/rudder-agent/debian/postinst#L52)
- rudder-server-root does install /opt/rudder/etc/uuid.hive (see https://github.com/Normation/rudder-packages/blob/master/rudder-server-root/debian/rules#L60)
- rudder-server-root does also concatenate the string "root" into that file (see https://github.com/Normation/rudder-packages/blob/branches/rudder/2.4/rudder-server-root/debian/postinst#L23)
So the problem here is that rudder-agent is being installed, generating the content of /opt/rudder/etc/uuid.hive as expected, and then rudder-server-root gets installed but complains because it knows that /opt/rudder/etc/uuid.hive belongs to itself.
The solution here is simple: adopt one or the other method. I suggest that :
- rudder-agent should own the /opt/rudder/etc/uuid.hive file. That means:
- rudder-agent should install it, maybe with a file that just has a comment in saying "# The content of this file should be replaced with a valid UUID for Rudder")
- rudder-agent should have it liste as a configuration file (this ensures that the file will be deleted if a "aptitude purge rudder-*" is run)
- rudder-server-root should no longer install it (or list it as a config file)
- both rudder-agent and rudder-server-root should overwrite it's contents in their postinst scripts (as is already the case)
What do you think? Is this clearer?
Ok, this is clearer, now. So the file uuid.hive will be managed only by rudder-agent ad I prefer the first method.
Updated by Nicolas PERRON about 12 years ago
- its packaging considers that uuid.hive is a file to install (with just a comment)
- its postinst check if the file exist or not. If not the file is created with a generated uuid, else nothing is made.
After a rudder-agent installation, uuid.hive contains no uuid but a comment and CFEngine consider this as a valid uuid, which is not !
I wonder what correction I have to do. Should I force uuid to be generated by postinst ? I don't think so. Should I use a real check of uuid in this file in postinst and/or CFEngine promises ? I think that it's a better approach.
Updated by Nicolas PERRON about 12 years ago
- The uuid.hive file must be random, so it can't be installed from a reference file and as it said in the debian package maintainer guide (http://www.debian.org/doc/manuals/maint-guide/dother.en.html#conffiles) :
If your program uses configuration files but also rewrites them on its own, it's best not to make them conffiles because dpkg will then prompt users to verify the changes all the time.
- We have a logic in the postinst which correspond to our needs: If a uuid.hive exists, do nothing, if it doesn't exist create it with a generated uuid.
Maybe should we have to stop to consider /opt/rudder/etc/uuid.hive to be a configuration file then ?
Updated by Nicolas PERRON about 12 years ago
Nicolas PERRON wrote:
I'm not sure that rudder-agent should consider /opt/rudder/etc/uuid.hive to be a configuration file for some reasons:
- The uuid.hive file must be random, so it can't be installed from a reference file and as it said in the debian package maintainer guide (http://www.debian.org/doc/manuals/maint-guide/dother.en.html#conffiles) :
[...]- We have a logic in the postinst which correspond to our needs: If a uuid.hive exists, do nothing, if it doesn't exist create it with a generated uuid.
Maybe should we have to stop to consider /opt/rudder/etc/uuid.hive to be a configuration file then ?
In the meantime, I've added a check of uuid in the rudder-agent postinst
Updated by Nicolas PERRON about 12 years ago
If we keep uuid.hive as a configuration file, we can consider this issue as over.
Updated by Jonathan CLARKE about 12 years ago
- Status changed from In progress to Pending technical review
Nicolas PERRON wrote:
If we keep uuid.hive as a configuration file, we can consider this issue as over.
Yes we must most certainly keep uuid.hive as a config file, because it is a config file, that is it contains information specific to one installation of rudder-agent, that cannot be regenerated (changing a node's UUID causes it to appear as a new node in Rudder, losing all reporting).
I'll put this into Technical Review then.
Updated by Jonathan CLARKE about 12 years ago
- Status changed from Pending technical review to 2
Looking at your last commit here, I'm bothered that yuo check validity of the UUID only as matching a real UUID regex. This should also be considered valid if the UUID is the special value "root" (for root servers). Please amend.
Updated by Jonathan CLARKE about 12 years ago
Jonathan CLARKE wrote:
Looking at your last commit here, I'm bothered that yuo check validity of the UUID only as matching a real UUID regex. This should also be considered valid if the UUID is the special value "root" (for root servers). Please amend.
Aside from this issue, all other commits look valid to me.
Updated by Nicolas PERRON about 12 years ago
- Status changed from 2 to Pending technical review
- % Done changed from 80 to 100
Jonathan CLARKE wrote:
Looking at your last commit here, I'm bothered that yuo check validity of the UUID only as matching a real UUID regex. This should also be considered valid if the UUID is the special value "root" (for root servers). Please amend.
Done.
Updated by Jonathan CLARKE about 12 years ago
- Status changed from Pending technical review to Released
Looks good to me, thanks for this huge task, Nico!
Updated by Nicolas PERRON over 11 years ago
- Project changed from Rudder to 34
- Category deleted (
11)
Updated by Benoît PECCATTE over 9 years ago
- Project changed from 34 to Rudder
- Category set to Packaging