Project

General

Profile

Actions

User story #7402

closed

Add a <FILE> tag in metadata.xml to allow simple file copy into techniques

Added by François ARMAND over 5 years ago. Updated over 5 years ago.

Status:
Released
Priority:
1
Category:
Web - Config management
Target version:
Suggestion strength:
User visibility:
Effort required:

Description

We need to be able to directly copy files into a techniques, like for templates, but without using stringtemplate parser on them.

The exact spec will be:

<FILES>
<FILE name="file.txt"><OUTPATH>foo/bar/other-name.txt</OUTPATH></FILE>
<FILE name="RUDDER_CONFIGURATION_REPOSITORY/some/absolute/file.txt"><OUTPATH>foo/bar/some-name.txt</OUTPATH></FILE>
</FILES>

- name is mandatory. It's the path to file to copy, either relative to the technique directory (i.e, at the same level as metadata.xml) or absolute from the configuration repository directory if it starts with RUDDER_CONFIGURATION_REPOSITORY (and yes, if fobids the use case where you want to have a sub-directory named RUDDER_CONFIGURATION_REPOSITORY under the technique directory - I'm sure one will find other way to do it if really needed :). The file will be taken from git, at the same git revision as other tehniques files.

- OUTPATH is optionnal. If not specified, the file will be copied into the target node promises at the same place than other files for the technique, with the same name. If specified, you have to give a path+name, where path is relative to the directory for agent promises on the node (i.e, if you want to put the file in the technique directory, you need to use "techniqueName/new-file-name.txt")


Subtasks 1 (0 open1 closed)

Bug #7429: Files added with the <FILE> tag in rudderify are not included in the input listReleasedFrançois ARMAND2015-11-22Actions

Related issues

Related to ncf - User story #7377: Adapt rudderify script to use <FILE> in the generated metadata.xmlReleasedBenoît PECCATTE2015-11-23Actions
Related to Rudder - User story #7376: Authorize both path relative to technique and to config-repos in technique metadata.xml descriptorReleasedNicolas CHARLES2015-11-24Actions
Related to Rudder - User story #7432: Don't copy files of local/50_techniques on the nodesReleasedBenoît PECCATTE2015-11-24Actions
Actions #1

Updated by Vincent MEMBRÉ over 5 years ago

Should OUTPTAH follow the same logic as TML ? (nothing specified put the file in techniqueName directory) ?

Actions #2

Updated by Vincent MEMBRÉ over 5 years ago

My bad I thought there was an OUTPATH in TML

Actions #3

Updated by François ARMAND over 5 years ago

  • Description updated (diff)
Actions #4

Updated by Vincent MEMBRÉ over 5 years ago

  • Related to User story #7377: Adapt rudderify script to use <FILE> in the generated metadata.xml added
Actions #5

Updated by François ARMAND over 5 years ago

Here, we have a decision to take: either files in <FILES> are "tracked" by Rudder (or part of Rudder internal boudaries) or not.

So, what does I mean here? I mean does updating a file is observed by Rudder (and MUST be observed, always), or does such a file is external and can't be observed (or even if it can, it is not mandatory). (this comment is also linled to http://www.rudder-project.org/redmine/issues/7376#note-5)

The main practical difference is about who is responsible to see that the file changed and that every node copy of the file should be updated. If the file is external, Rudder can't do it (not always, at least), and if internal, it MUST do it.

My analysis is that as long as the file copy is part of the promise generation process, it must be internal, and basically it is a part of what defined the technique and must follows the same requirement as a template.
If so, it's quite easy to know how the file is in Rudder system boundaries: it's commited in our configuration-repository. And an update is always observable via a git diff.

Of course, there is drawbacks: for one, files must be in the git, with all implied by tha (binaries are going to waste space in the repo, user must git add/git commit, etc).

Note that is would be mandatory for a "API" kind of resource, if such one existed, to be external (and so, it must be the nodes which have the update logic of such a resource).

What do you think ?

Actions #6

Updated by François ARMAND over 5 years ago

François ARMAND wrote:

If so, it's quite easy to know how the file is in Rudder system boundaries: it's commited in our configuration-repository. And an update is always observable via a git diff.

This part can be relaxed and we can just look at the last know file modification time to observe an update. That take care of the binaries problem, but on the other hand give us less control about what changed. That can be what is expected by a user.

An other point: it can be very suprising for an user to have a file versionned on git, but to use the file content available on the FS and not on the last commit. I'm thinking at that especially for ncf techniques, which are expected to use the <FILE> tag, are commited on "configuration-repository" git, but that content won't be used.
On the other hand, if we have a different for file under coniguration-repository than on other parts of the FS, it's complete madness. So that would mean to tags, one for "file in git", and other for "file in fs"... Which is barelly clearer...

So, not sure about what is the best.

Actions #7

Updated by François ARMAND over 5 years ago

  • Description updated (diff)

So, we choose to only authorise file in git configuration-repository for now. Big binary files should be avoid and use the share-files folder for now. In the future, we may had an other tag to handle FS-based files.

Actions #8

Updated by François ARMAND over 5 years ago

  • Description updated (diff)
Actions #9

Updated by François ARMAND over 5 years ago

  • Related to User story #7376: Authorize both path relative to technique and to config-repos in technique metadata.xml descriptor added
Actions #10

Updated by François ARMAND over 5 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Nicolas CHARLES
  • Pull Request set to https://github.com/Normation/rudder/pull/967
Actions #11

Updated by François ARMAND over 5 years ago

  • Status changed from Pending technical review to Pending release
  • % Done changed from 0 to 100
Actions #13

Updated by Nicolas CHARLES over 5 years ago

we are missing an option to add it in the input list, so files are not taken into account

Actions #14

Updated by Benoît PECCATTE over 5 years ago

  • Related to User story #7432: Don't copy files of local/50_techniques on the nodes added
Actions #16

Updated by Vincent MEMBRÉ over 5 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 3.2.0~beta1 which was released today.

Actions

Also available in: Atom PDF