Project

General

Profile

Actions

User story #2738

closed

Handle big files payloads in the Enforce a file content Technique

Added by Matthieu CERDA over 12 years ago. Updated over 5 years ago.

Status:
Rejected
Priority:
2
Category:
Techniques
Target version:
UX impact:
Suggestion strength:
User visibility:
Effort required:
Name check:
Fix check:
Regression:

Description

CFengine is unable to handle too big files payloads in the Enforce a file content Technique.

In Rudder 3.0, the limit is at 16ko and the error is at the promise validation time
The error message is:

input buffer overflow, can't enlarge buffer because scanner uses REJECT

Before, the limit is at 4ko ou 8ko, and for example, when trying to apply a backup-manager configuration using it, there is a deployment error, outputting this:

Expansion overflow constructing string. Increase CF_EXPANDSIZE macro. Tried to add " 

# Should we purge only archives built with BM_ARCHIVE_PREFIX
export BM_ARCHIVE_STRICTPURGE="true" 

# You may want to nice the commands run for archive-creation
# (Recommanded for desktop users.)
# Choose a nice level from -20 (most favorable scheduling) to 19 (least favorable).
export BM_ARCHIVE_NICE_LEVEL="10" 

# The backup method to use.
# Available methods are:
# - tarball
# - tarball-incremental
# - mysql
# - svn
# - pipe
# - none
# If you don't want to use any backup method (you don't want to
# build archives) then choose "none" 
export BM_ARCHIVE_METHOD="tarball-incremental" 

##############################################################
# Encryption - because you cannot trust the place your 
#              archives are
##############################################################

# If you want to encrypt your archives locally, Backup Manager 
# can use GPG while building the archive (so the archive is never
# written to the disk without being encrypted.

# Note: this feature is only possible with the following archive types:
# tar, tar.gz, tar.bz2

# Uncomment the following line if you want to enable encryption
# available method: gpg
export BM_ENCRYPTION_METHOD="gpg" 

# The encryption will be made using a GPG ID
# Examples:
# export BM_ENCRYPTION_RECIPIENT="" 
# export BM_ENCRYPTION_RECIPIENT="" 
export BM_ENCRYPTION_RECIPIENT="0C60B2BA" 

##############################################################
# Section "TARBALL" 
# - Backup method: tarball
#############################################################

# Archive filename format
#     long  : host-full-path-to-folder.tar.gz
#     short : parentfolder.tar.gz
export BM_TARBALL_NAMEFORMAT="long" 

# Type of archives
# Available types are:
#     tar, tar.gz, tar.bz2, tar.lz, dar, zip.
# Make sure to satisfy the appropriate dependencies 
# (bzip2, dar, lzma, ...).
export BM_TARBALL_FILETYPE="tar.bz2" 

# You can choose to build archives remotely over SSH.
# You will then need to fill the BM_UPLOAD_SSH variables 
# (BM_UPLOAD_SSH_HOSTS, BM_UPLOAD_SSH_USER, BM_UPLOAD_SSH_KEY).
# If this boolean is set to true, archive will be saved locally (in 
# BM_REPOSITORY_ROOT but will be built by the remote host).
# Thus, BM_TARBALL_DIRECTORIES will be used to backup remote directories.
# Those archive will be prefixed with the remote host name.
export BM_TARBALL_OVER_SSH="false" 

# Do you want to dereference the files pointed by symlinks ? 
# enter true or false (true can lead to huge archives, be careful).
export BM_TARBALL_DUMPSYMLINKS="false" 

# Targets to backup

# You can use two different variables for defining the targets of 
# your backups, either a simple space-separated list (BM_TARBALL_DIRECTORIES)
# or an array (BM_TARBALL_TARGETS[]).
# Use the first one for simple path that doesn't contain spaces in their name.
# Use the former if you want to specify paths to backups with spaces.

# It's recommanded to use BM_TARBALL_TARGETS[] though.
# Warning! You *must not* use both variables at the same time.
# NOTE: The Debian package will only update BM_TARBALL_DIRECTORIES 

# Paths without spaces in their name:
export BM_TARBALL_DIRECTORIES="/etc /home/* /root /var/log" 

# If one or more of the targets contain a space, use the array:
# declare -a BM_TARBALL_TARGETS
# BM_TARBALL_TARGETS[0]="/etc" 
# BM_TARBALL_TARGETS[1]="/boot" 
# export BM_TARBALL_TARGETS

# Files to exclude when generating tarballs, you can put absolute 
# or relative paths, Bash wildcards are possible.
export BM_TARBALL_BLACKLIST="/var/archives" 

# With the "dar" filetype, you can choose a maximum slice limit.
export BM_TARBALL_SLICESIZE="1000M" 

# Extra options to append to the tarball generation 
# (take care to what you do; this will be silently added to the 
# command line.)
export BM_TARBALL_EXTRA_OPTIONS="" 

##############################################################
# The tarball-incremental method uses the same keys as the 
# tarball method, plus two others.
########################################################
Fatal cfengine error: Can't expand varstring
[2012-07-31 12:15:21] ERROR com.normation.rudder.services.policies.RudderCf3PromisesFileWriterServiceImpl - The generated promises at /var/rudder/share/713c52d2-78d8-4dbe-87be-21457496de32.new/rules/cfengine-community are invalid

I do think this is because CFengine is unable to handle such big variables unless compiled with a larger CF_EXPANDSIZE. But in that case, it is better to use a file template, I think.


Related issues 1 (0 open1 closed)

Related to Rudder - Bug #3574: Using checkgenericfilecontent Technique with 16Ko content leads to errorRejectedActions
Actions

Also available in: Atom PDF