Project

General

Profile

Actions
Copy link

Bug #20359

closed

techniques are seens as deleted even if there are there in Rudder 7.0

Added by Nicolas CHARLES almost 3 years ago. Updated almost 3 years ago.

Status:
Released
Priority:
N/A
Assignee:
François ARMAND
Category:
Web - Config management
Target version:
7.0.0~rc2
Pull Request:
https://github.com/Normation/rudder/p...
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
To do
Fix check:
To do
Regression:

Description

[2021-12-02 13:36:36+0000] INFO  bootchecks - Check for force reload of Techniques library
[2021-12-02 13:36:36+0000] INFO  bootchecks - Flag file '/opt/rudder/etc/force_technique_reload' found, reload Techniqu
e library now
[2021-12-02 13:36:38+0000] INFO  techniques.reader - Reloading technique library, found modified technique(s): ['rudder
-service-postgresql': deleted (1.0)], ['server-roles': deleted (1.0)], ['server-common': deleted (1.0)], ['common': del
eted (1.0)], ['rudder-service-relayd': deleted (1.0)], ['rudder-service-webapp': deleted (1.0)], ['rudder-service-apach
e': deleted (1.0)], ['distributePolicy': deleted (1.0)], ['rudder-service-slapd': deleted (1.0)]
[2021-12-02 13:36:38+0000] ERROR application - System technique 'rudder-service-postgresql' (1.0)' is deleted in git ba
se. This will likely cause grave problem. You should investigate.
[2021-12-02 13:36:38+0000] ERROR application - System technique 'server-common' (1.0)' is deleted in git base. This wil
l likely cause grave problem. You should investigate.
[2021-12-02 13:36:38+0000] ERROR application - System technique 'common' (1.0)' is deleted in git base. This will likel
y cause grave problem. You should investigate.
[2021-12-02 13:36:38+0000] ERROR application - System technique 'rudder-service-relayd' (1.0)' is deleted in git base. 
This will likely cause grave problem. You should investigate.
[2021-12-02 13:36:38+0000] ERROR application - System technique 'rudder-service-webapp' (1.0)' is deleted in git base. 
This will likely cause grave problem. You should investigate.
[2021-12-02 13:36:38+0000] ERROR application - System technique 'rudder-service-apache' (1.0)' is deleted in git base. 
This will likely cause grave problem. You should investigate.
[2021-12-02 13:36:38+0000] ERROR application - System technique 'rudder-service-slapd' (1.0)' is deleted in git base. T
his will likely cause grave problem. You should investigate.
[2021-12-02 13:36:39+0000] INFO  com.normation.rudder.services.policies.DeployOnTechniqueCallback - Reload Technique li
brary at start up

but there are there

[root@server system]# cd /var/rudder/configuration-repository/techniques/system
[root@server system]# ls
category.xml  inventory              rudder-service-postgresql  rudder-service-slapd   server-common
common        rudder-service-apache  rudder-service-relayd      rudder-service-webapp

Files

Download all files
7.0.ldif (908 KB) 7.0.ldif export of ldap Nicolas CHARLES, 2021-12-02 15:03
post-upgrade.ldif (908 KB) post-upgrade.ldif Nicolas CHARLES, 2021-12-02 16:30

Related issues 5 (0 open — 5 closed)

Related to Rudder - Bug #20351: System technique migration fails on old serverReleasedAlexis MoussetActions
Related to Rudder - Bug #20356: When upgrading from 6.2 -> 7.0 on centos8, old system technique are still present, and break policy generationReleasedAlexis MoussetActions
Related to Rudder - Bug #20390: webapp doesn't start after upgrade from 6.2 to 7.0ReleasedFrançois ARMANDActions
Related to Rudder - User story #20462: Delay policy generation until rudder app is fully bootReleasedNicolas CHARLESActions
Related to Rudder - Bug #20355: rudder-webap: Could not lock HEAD. RefUpdate return code was: LOCK_FAILUREResolvedActions
Actions
Copy link
 #1

Updated by Nicolas CHARLES almost 3 years ago

touching & commiting every technique & restarting the webapp doesn't solve the issue

[2021-12-02 13:53:01+0000] INFO  policy.generation.timing - Policy generation failed after:           
[2021-12-02 13:53:01+0000] ERROR policy.generation.manager - Error when updating policy, reason was: Cannot build Rule vals <- Could not find configuration vals <- Cannot find directive with id 'common-root' when building rule 'Rule for policy server root' (policy-server-root)

Actions
Copy link
 #2

Updated by Nicolas CHARLES almost 3 years ago

Actions
Copy link
 #3

Updated by Nicolas CHARLES almost 3 years ago

  • Related to Bug #20351: System technique migration fails on old server added
Actions
Copy link
 #4

Updated by Nicolas CHARLES almost 3 years ago

  • Related to Bug #20356: When upgrading from 6.2 -> 7.0 on centos8, old system technique are still present, and break policy generation added
Actions
Copy link
 #5

Updated by Nicolas CHARLES almost 3 years ago

after fixing #20356, there are no technique issue, but generation fails, still with the same error
see log

[2021-12-02 15:23:51+0000] INFO  techniques.reader - Reloading technique library, found modified technique(s): ['rudder
-service-postgresql': updated (1.0: updated)], ['server-common': updated (1.0: updated)], ['common': updated (1.0: upda
ted)], ['rudder-service-relayd': updated (1.0: updated)], ['rudder-service-webapp': updated (1.0: updated)], ['rudder-s
ervice-apache': updated (1.0: updated)], ['rudder-service-slapd': updated (1.0: updated)]
[2021-12-02 15:23:52+0000] INFO  com.normation.rudder.services.policies.DeployOnTechniqueCallback - Reload Technique li
brary to allows migration to 7.x system techniques
[2021-12-02 15:23:52+0000] INFO  migration - System technique v7.0 present: 'rudder-service-postgresql'
[2021-12-02 15:23:52+0000] INFO  migration - System technique v7.0 present: 'rudder-service-slapd'
[2021-12-02 15:23:52+0000] INFO  migration - System technique v7.0 present: 'rudder-service-webapp'
[2021-12-02 15:23:52+0000] INFO  migration - System technique v7.0 present: 'common'
[2021-12-02 15:23:52+0000] INFO  migration - System technique v7.0 present: 'server-common'
[2021-12-02 15:23:52+0000] INFO  migration - System technique v7.0 present: 'rudder-service-apache'
[2021-12-02 15:23:52+0000] INFO  migration - System technique v7.0 present: 'rudder-service-relayd'
[2021-12-02 15:23:52+0000] INFO  migration - Creating new system directives for policy server 'root'
[2021-12-02 15:23:53+0000] INFO  policy.generation - Start policy generation, checking updated rules
[2021-12-02 15:23:53+0000] ERROR rudder-ldap-entity-mapper - Invalid attribute 'special:all_servers_with_role' for entr
y ruleTarget.
[2021-12-02 15:23:53+0000] INFO  migration - Creating new system rules for policy server 'root'
[2021-12-02 15:23:53+0000] WARN  com.normation.rudder.repository.ldap.RoLDAPNodeGroupRepository - Error when mapping en
try with DN 'ruleTarget=special:all_servers_with_role,groupCategoryId=SystemGroups,groupCategoryId=GroupRoot,ou=Rudder,
cn=rudder-configuration' from node groups library, that entry will be ignored; cause was: UnexpectedObject: Can not uns
erialize target, 'special:all_servers_with_role' does not match any known target format
[2021-12-02 15:23:53+0000] ERROR com.normation.rudder.domain.policies.RuleTarget - Error when trying to read the follow
ing serialized Rule target as a composite target: 'special:all_nodes_without_role'. Reported parsing error cause was: u
nknown token s
Near: sp

[2021-12-02 15:23:53+0000] WARN  com.normation.rudder.repository.ldap.RoLDAPNodeGroupRepository - Error when mapping en
try with DN 'ruleTarget=special:all_nodes_without_role,groupCategoryId=SystemGroups,groupCategoryId=GroupRoot,ou=Rudder
,cn=rudder-configuration' from node groups library, that entry will be ignored; cause was: UnexpectedObject: Can not un
serialize target, 'special:all_nodes_without_role' does not match any known target format
[2021-12-02 15:23:53+0000] INFO  migration - Starting migration for system rules for policy server 'root''
[2021-12-02 15:23:53+0000] INFO  migration - [root] move old 6.2 rule 'hasPolicyServer-root' to 'old_hasPolicyServer-ro
ot'
[2021-12-02 15:23:53+0000] INFO  policy.generation - [metrics] Xmx:1015 MB nodes:2 (cached:2) rules:5 (enabled:5) techn
iques:50 (enabled:50) directives:12 (enabled:12) groups:2 (dynamic:2) parameters:2
[2021-12-02 15:23:53+0000] INFO  migration - [root] move old 6.2 rule 'root-DP' to 'old_root-DP'
[2021-12-02 15:23:53+0000] INFO  migration - [root] move new 7.0 rule 'new_hasPolicyServer-root' to 'hasPolicyServer-ro
ot'
[2021-12-02 15:23:53+0000] INFO  policy.generation.timing - Policy generation failed after:           
[2021-12-02 15:23:53+0000] ERROR policy.generation.manager - Error when updating policy, reason was: Cannot build Rule 
vals <- Could not find configuration vals <- Version '1.0' of technique 'distributePolicy' is not available for directi
ve 'Distribute Policy' [root-distributePolicy]
[2021-12-02 15:23:53+0000] ERROR policy.generation.manager - Policy update error for process '10' at 2021-12-02 15:23:5
3: Cannot build Rule vals

[2021-12-02 15:23:53+0000] INFO  migration - [root] move new 7.0 rule 'new_policy-server-root' to 'policy-server-root'
[2021-12-02 15:23:53+0000] INFO  migration - [root techniques] delete old rule 'hasPolicyServer-root'
[2021-12-02 15:23:53+0000] INFO  migration - [root techniques] delete old rule 'root-DP'
[2021-12-02 15:23:53+0000] INFO  migration - [root techniques] delete old directive 'common-root'
[2021-12-02 15:23:53+0000] INFO  migration - Migration of system techniques, directives and rules done for 'root'
[2021-12-02 15:23:53+0000] INFO  migration - Deleting old technique 'distributePolicy' information
[2021-12-02 15:23:53+0000] INFO  migration - Deleting old system object for deprecated server-roles: activeTechniqueId=
server-roles,techniqueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudder,cn=rudder-configuratio
n
[2021-12-02 15:23:53+0000] INFO  migration - Deleting old system object for deprecated server-roles: ruleId=server-role
s,ou=Rules,ou=Rudder,cn=rudder-configuration
[2021-12-02 15:23:53+0000] INFO  migration - Deleting old system object for deprecated server-roles: ruleTarget=special
:all_nodes_without_role,groupCategoryId=SystemGroups,groupCategoryId=GroupRoot,ou=Rudder,cn=rudder-configuration
[2021-12-02 15:23:53+0000] INFO  migration - Deleting old system object for deprecated server-roles: ruleTarget=special
:all_servers_with_role,groupCategoryId=SystemGroups,groupCategoryId=GroupRoot,ou=Rudder,cn=rudder-configuration
[2021-12-02 15:23:53+0000] INFO  migration - Migration of all system configuration to Rudder 7.0: done
[2021-12-02 15:23:54+0000] INFO  bootchecks - Check mandatory DIT entries
[2021-12-02 15:23:54+0000] INFO  bootchecks - All the required DIT entries are present in the LDAP directory
[2021-12-02 15:23:54+0000] INFO  bootchecks - Check initialization of User Technique Library
[2021-12-02 15:23:54+0000] INFO  bootchecks - Check that `rudder` global parameter matches default value
[2021-12-02 15:23:54+0000] INFO  bootchecks - Check event log migration format 5 -> 6
[2021-12-02 15:23:54+0000] INFO  bootchecks - Check existence of at least one archive of the configuration
[2021-12-02 15:23:54+0000] INFO  migration - No migration detected by migration script (table 'migrationeventlog' is em
pty or does not exist)
[2021-12-02 15:23:54+0000] INFO  bootchecks - First full archive of configuration-repository items done
[2021-12-02 15:23:54+0000] INFO  bootchecks - Check for force reload of Techniques library
[2021-12-02 15:23:54+0000] INFO  bootchecks - Flag file '/opt/rudder/etc/force_technique_reload' found, reload Technique library now

[2021-12-02 15:23:54+0000] INFO  bootchecks - Successfully reloaded Technique library on start up. now deleting flag fi
le '/opt/rudder/etc/force_technique_reload'
[2021-12-02 15:23:54+0000] INFO  bootchecks - Flag file '/opt/rudder/etc/force_technique_reload' successfully removed
[2021-12-02 15:23:54+0000] INFO  bootchecks - Regenerate all ncf techniques
[2021-12-02 15:23:54+0000] INFO  bootchecks - started
[2021-12-02 15:23:54+0000] INFO  bootchecks - techniques - update
[2021-12-02 15:23:55+0000] INFO  bootchecks - methods
[2021-12-02 15:23:55+0000] INFO  bootchecks - techniques - read
[2021-12-02 15:23:56+0000] INFO  techniques.reader - Reloading technique library, no modified techniques found
[2021-12-02 15:23:56+0000] INFO  com.normation.rudder.services.policies.DeployOnTechniqueCallback - Update Technique li
brary after updating all techniques at start up
[2021-12-02 15:23:56+0000] INFO  bootchecks - Trigger policy update automatically at start
[2021-12-02 15:23:56+0000] INFO  bootchecks - Create system api token
[2021-12-02 15:23:56+0000] INFO  bootchecks - System api token file created in /var/rudder/run/api-token
[2021-12-02 15:23:56+0000] INFO  bootchecks - Load node compliance cache
2021-12-02 15:23:56.473:INFO:oejshC.rudder:main: Initializing Spring root WebApplicationContext
[2021-12-02 15:23:56+0000] INFO  policy.generation - Start policy generation, checking updated rules
[2021-12-02 15:23:56+0000] INFO  policy.generation - [metrics] Xmx:1015 MB nodes:2 (cached:2) rules:4 (enabled:4) techn
iques:48 (enabled:48) directives:9 (enabled:9) groups:2 (dynamic:2) parameters:2
[2021-12-02 15:23:56+0000] INFO  policy.generation.timing - Policy generation failed after:           
[2021-12-02 15:23:56+0000] ERROR policy.generation.manager - Error when updating policy, reason was: Cannot build Rule 
vals <- Could not find configuration vals <- Cannot find directive with id 'common-root' when building rule 'Rule for p
olicy server root' (policy-server-root)
[2021-12-02 15:23:56+0000] ERROR policy.generation.manager - Policy update error for process '11' at 2021-12-02 15:23:5
6: Cannot build Rule vals
[2021-12-02 15:23:57+0000] INFO  application - Configured authentication provider(s): [rootAdmin, file]
[2021-12-02 15:23:57+0000] INFO  application - No master admin account is defined. You can define one with 'rudder.auth
.admin.login' and 'rudder.auth.admin.password' properties in the configuration file

[2021-12-02 15:23:58+0000] INFO  application - Add backend providers 'Default authentication backends provider: 'file','rootAdmin'
[2021-12-02 15:23:58+0000] INFO  application - Using configuration file defined by JVM property -Drudder.authFile : /opt/rudder/etc/rudder-users.xml
[2021-12-02 15:23:58+0000] INFO  bootstrap.liftweb.Boot - classpath
[2021-12-02 15:23:58+0000] INFO  org.reflections.Reflections - Reflections took 200 ms to scan 2 urls, producing 15 keys and 53 values 
[2021-12-02 15:23:59+0000] INFO  application - INFO Configure inventory processing with parallelism of '1' and queue size of '5'
[2021-12-02 15:23:59+0000] INFO  scheduled.job.metrics - Starting node count historization batch (min:15 m; max:4 h)
[2021-12-02 15:23:59+0000] INFO  inventory-processing - Incoming inventory watcher started - process existing inventories
[2021-12-02 15:23:59+0000] INFO  application - Application Rudder started
2021-12-02 15:23:59.602:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@7a1ebcd8{/rudder,file:///var/rudder/tmp/jetty/jetty-rudder.war.dir/webapp/,AVAILABLE}{/opt/rudder/share/webapps/rudder.war}
2021-12-02 15:23:59.745:INFO:oejs.AbstractConnector:main: Started ServerConnector@5cdf221a{HTTP/1.1, (http/1.1)}{127.0.0.1:8080}
2021-12-02 15:23:59.746:INFO:oejs.Server:main: Started @22651ms
[2021-12-02 15:24:18+0000] INFO  scheduled.job - Update in node inventories main information detected: triggering a policy generation
[2021-12-02 15:24:18+0000] INFO  policy.generation - Start policy generation, checking updated rules
[2021-12-02 15:24:18+0000] INFO  policy.generation - [metrics] Xmx:1015 MB nodes:2 (cached:2) rules:4 (enabled:4) techniques:48 (enabled:48) directives:9 (enabled:9) groups:2 (dynamic:2) parameters:2
[2021-12-02 15:24:18+0000] INFO  policy.generation.timing - Policy generation failed after:           
[2021-12-02 15:24:18+0000] ERROR policy.generation.manager - Error when updating policy, reason was: Cannot build Rule vals <- Could not find configuration vals <- Cannot find directive with id 'common-root' when building rule 'Rule for policy server root' (policy-server-root)
[2021-12-02 15:24:18+0000] ERROR policy.generation.manager - Policy update error for process '12' at 2021-12-02 15:24:18: Cannot build Rule vals
Actions
Copy link
 #6

Updated by Nicolas CHARLES almost 3 years ago

ldif post (working) upgrade
Rule dn: ruleId=policy-server-root,ou=Rules,ou=Rudder,cn=rudder-configuration need directives
directiveId: server-common-root
directiveId: rudder-service-apache-root
directiveId: rudder-service-postgresql-root
directiveId: rudder-service-slapd-root
directiveId: rudder-service-webapp-root
directiveId: common-root
directiveId: rudder-service-relayd-root

but I don't have a "common-root", I do have a
dn: directiveId=common-hasPolicyServer-root,activeTechniqueId=common,techniq
ueCategoryId=Rudder Internal,techniqueCategoryId=Active Techniques,ou=Rudde
r,cn=rudder-configuration

Actions
Copy link
 #7

Updated by Nicolas CHARLES almost 3 years ago

  • Status changed from New to In progress
  • Assignee set to Nicolas CHARLES
Actions
Copy link
 #8

Updated by Nicolas CHARLES almost 3 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from Nicolas CHARLES to François ARMAND
  • Pull Request set to https://github.com/Normation/rudder/pull/4038
Actions
Copy link
 #9

Updated by Nicolas CHARLES almost 3 years ago

  • Status changed from Pending technical review to Pending release
Actions
Copy link
 #10

Updated by François ARMAND almost 3 years ago

  • Related to Bug #20390: webapp doesn't start after upgrade from 6.2 to 7.0 added
Actions
Copy link
 #11

Updated by Vincent MEMBRÉ almost 3 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 7.0.0~rc2 which was released today.

Actions
Copy link
 #12

Updated by François ARMAND almost 3 years ago

  • Related to User story #20462: Delay policy generation until rudder app is fully boot added
Actions
Copy link
 #13

Updated by François ARMAND almost 3 years ago

  • Related to Bug #20355: rudder-webap: Could not lock HEAD. RefUpdate return code was: LOCK_FAILURE added
Actions
Copy link

Also available in: Atom PDF