Bug #26125
open
After removing techniques, they still appear in the techniques tree and filesystem
Description
Rudder server 8.2.2
After removing techniques by using the GUI “Archive / Restore everything” to restore a previous GIT snapshot at a point in time when these techniques didn't yet exist :
- They are removed from the “Techniques” GUI page.
- But they still appear in the “techniques tree” in the “Reference technique library” (left pane)
- They are still present on the filesystem in /var/rudder/configuration-repository/techniques/ncf_techniques
- A subsequent attempt to import these techniques again from an archive using the API silently fails :
- It shows a success output :
root@server:/tmp# curl --insecure --header "X-API-Token: $(cat /var/rudder/run/api-token)" -X POST https://localhost/rudder/api/latest/archives/import --form "archive=@tech-ONLY_240502aa.zip" --form "merge=keep-rule-groups"
{"action":"import","result":"success","data":{"success":true}}root@server:/tmp#
root@server:/tmp# rudder server reload-techniques
ok: reload techniques from files.
- The webapp log shows an import success :
2025-01-02 10:15:44+0000 INFO application.archive - Received a new policy archive 'tech-ONLY_240502aa.zip', processing 2025-01-02 10:15:48+0000 INFO application.archive - Uploaded archive 'tech-ONLY_240502aa.zip' processed successfully 2025-01-02 10:15:48+0000 INFO policy.generation - Start policy generation, checking updated rules 2025-01-02 10:15:49+0000 INFO policy.generation - [metrics] Xmx:1 GB nodes:2 (cached:2) rules:4 (enabled:3) techniques:8 (enabled:8) directives:8 (enabled:8) groups:2 (dynamic:2) parameters:2 2025-01-02 10:15:49+0000 INFO policy.generation - No node configuration was updated, no policies to write
- Still the imported techniques do not show up on the GUI “Techniques page”
- But they do still appear on the “Technique tree” page
- AND IT IS POSSIBLE to reactivate them by dragging them from the left pane to the right pane (Although the drag-drop process does not function quite well).
Files
| Empty_techniques_250102a.png (13.4 KB) Empty_techniques_250102a.png | No techniques on techniques page | ||
| Still_present_techniques_250102a.png (169 KB) Still_present_techniques_250102a.png | Techniques present in techniques tree | ||
| Import_silent_fail_250102a.png (20.8 KB) Import_silent_fail_250102a.png | Import silently fails | ||
| Import_webapp_log_250102a.png (30.4 KB) Import_webapp_log_250102a.png | Import Webapp log | ||
| Some_techniques_dragged_back_250102a.png (82.9 KB) Some_techniques_dragged_back_250102a.png | Techniques dragged back | ||
| Dragged_techniques_reappeared_250102a.png (26.3 KB) Dragged_techniques_reappeared_250102a.png | Dragged techniques reappeared | ||
| clipboard-202501301647-2txst.png (39.6 KB) clipboard-202501301647-2txst.png |
Updated by François ARMAND about 2 months ago
- Assignee set to François ARMAND
- Priority changed from To review to 1 (highest)
Updated by François ARMAND about 2 months ago
- Related to Bug #24294: API Import does not reload imported techniques before importing elements that rely on them added
Updated by Vincent MEMBRÉ about 1 month ago
- Target version changed from 8.2.4 to 8.2.5
Updated by François ARMAND 24 days ago
· Edited
I can confirm that "restoring a git snapshot" does not delete technique files created since that snapshot.
I think it was the intented behavior, the "restore git" was intented to restore user change, and technique were particular. But it's a least strange and a no-sense today.
But there's more questions.
I did:
- create a snapshot (archives/full/2025-01-30_15-24-46.415)
- create a technique test1
At that point, I have the following files in FS:
- techniques/ncf_techniques
├── category.xml
└── test1
└── 1.0
├── metadata.xml
├── technique.cf
├── technique.ps1
└── technique.yml
- directives/ncf_techniques
├── ...
└── test1
└── 1.0
├── activeTechniqueSettings.xml
...
If I restore the snapshot, it only delete activeTechniqueSettings.xml, which doesn't make much sense.
The active tree correctly contains nothing 
And git is consistent with fs:
[root@server ncf_techniques]# git ls-tree -r HEAD 100644 blob db1a001b1c427863eda5f83dc64abf5550c25140 category.xml 100644 blob 9398f60228c71c2aaec2967b573ea34068d0044d test1/1.0/metadata.xml 100644 blob bf348718fa0ceaeb3570a2126026c708ab324852 test1/1.0/technique.cf 100644 blob 055276fe195162d780755b1cb74362f5b43aee2e test1/1.0/technique.ps1 100644 blob 79ba6d3347f21a1ea0a3d9ead2512abde121fee0 test1/1.0/technique.yml [root@server ncf_techniques]# git status On branch master nothing to commit, working tree clean
What makes even less sense is that the technique is removed from technique editor UI while there is a yaml file for it. This persists even in case of a "technique reload" or "full regeneration" which are almost the only user action to do things here.
So there are two bugs I think:
- the technique editor doesn't correctly see what is in the file system, and since it works with techniques (and not active techniques), it must be aware of changes there,
- the snapshot restore doesn't change the techniques directory. It might be the sequel of a time where the techniques directory was only touch by admin, and didn't dare restore that. But today it's just confusing.
Updated by François ARMAND 24 days ago
· Edited
Even stranger:
If I copy test1 technique files into a test2 (and change content accordingly), and then reload techniques, test2 is correctly seen, but still no test1.
Even restarting rudder does nothing. Test1 is blacklisted.
We need to change it, commit the change, and reload technique to have it appear again in the technique editor.
Updated by François ARMAND 17 days ago
So, the technique editor filter out techniques that are not "activated" (ie in directive library). So the whole thing is somehow self-consistant. When we restore a previous git snapshot, we restore "active techniques" at that point. That may need change, but given that as work hypothesis :
- the new technique is not "activate" anymore, so it does not appear in directive tree,
- it does not appear in technique editor either, since not activated techniques are filtered out,
- but it is in the "technique tree" page, since that page shows technique that are in Git but not in LDAP (not "activated").
I think we should NOT filter out technique from git in the technique editor, but make them appear greyed-out and perhaps add a "activate" button for them.
It remains the question about why an archive import fails for not activated technique.