Project

General

Profile

Actions

Bug #26867

open

Repeated transient / random errors with the sshKeyDistribution standard technique

Added by Michel BOUISSOU 14 days ago. Updated 3 days ago.

Status:
New
Priority:
To review
Assignee:
-
Category:
Techniques
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
To do
Fix check:
To do
Regression:
No

Description

On many of my nodes using a single, simple directive that uses the standard sshKeyDistribution technique :
- Setting SSH authoriszed keys for 2 users : myself and root
- With the option “Remove other keys“ = Yes

Nothing else AFAIK changing any thing with the authorized_keys on those machines.

- Most of the times the Rudder agent runs good with everything compliant.
- But every so often, the Rudder agents reports that :
-- The SSH key for each user was “repaired”
-- The SSH keys for each user “could not be flushed”

I have no explanation for such behaviour.

See attached screenshots.


Files

sshKeyDistrbution_errors_250504d.png (228 KB) sshKeyDistrbution_errors_250504d.png Used directive parameters Michel BOUISSOU, 2025-05-04 19:57
sshKeyDistrbution_errors_250504a.png (465 KB) sshKeyDistrbution_errors_250504a.png Agent run with errors Michel BOUISSOU, 2025-05-04 19:57
sshKeyDistrbution_errors_250504b.png (185 KB) sshKeyDistrbution_errors_250504b.png Compliance report with errors Michel BOUISSOU, 2025-05-04 19:57
sshKeyDistrbution_errors_250504c.png (489 KB) sshKeyDistrbution_errors_250504c.png Agent run without errors Michel BOUISSOU, 2025-05-04 19:57
Actions #1

Updated by Vincent MEMBRÉ 12 days ago

  • Target version changed from 8.3.1 to 8.3.2
Actions #2

Updated by François ARMAND 3 days ago

  • Assignee set to Michel BOUISSOU
  • Target version changed from 8.3.2 to 8.2.7

We need more info with rudder agent history

Actions #3

Updated by Michel BOUISSOU 3 days ago

  • Assignee changed from Michel BOUISSOU to François ARMAND

“rudder agent history” doesn't tell much.

Most of the times the agent runs with “Compliant” for SSH keys, but every so often :

It shows 2 repairs for “sshKeyDistribution” where it seems to have modified the files contents (where it shouldn't as nothing has changed) then the 2 following errors :

E| error         sshKeyDistribution        Flush SSH file            michel michel RSA  The keys for user michel could not be flushed
E| error         sshKeyDistribution        Flush SSH file            root michel RSA    The keys for user root could not be flushed

The relevant parts of cfengine-community/outputs/cf_vpn1__1747286851_Thu_May_15_07_27_31_2025_0 (on considered node) seems to be :

Close to the beginning :

rudder     info: Deleted the promised line 1 'ssh-rsa <some_SSH_public_key>' from /var/rudder/tmp/check_ssh_key_distribution//michel.authorized_keys.tmp
rudder     info: delete_lines promise '.*' repaired
rudder     info: Moved '/var/rudder/tmp/check_ssh_key_distribution//michel.authorized_keys.tmp.cf-before-edit' to repository location '/var/rudder/modified-files/_var_rudder_tmp_check_ssh_key_distribution__michel_authorized_keys_tmp_cf_before_edit'
rudder     info: Edited file '/var/rudder/tmp/check_ssh_key_distribution//michel.authorized_keys.tmp'
rudder     info: Deleted the promised line 1 'ssh-rsa <some_SSH_public_key>' from /var/rudder/tmp/check_ssh_key_distribution//root.authorized_keys.tmp
rudder     info: delete_lines promise '.*' repaired
rudder     info: Moved '/var/rudder/tmp/check_ssh_key_distribution//root.authorized_keys.tmp.cf-before-edit' to repository location '/var/rudder/modified-files/_var_rudder_tmp_check_ssh_key_distribution__root_authorized_keys_tmp_cf_before_edit'
rudder     info: Edited file '/var/rudder/tmp/check_ssh_key_distribution//root.authorized_keys.tmp'

Then :

rudder     info: insert_lines promise 'ssh-rsa <some_SSH_public_key>' repaired
rudder     info: Moved '/home/michel/.ssh/authorized_keys.cf-before-edit' to repository location '/var/rudder/modified-files/_home_michel__ssh_authorized_keys_cf_before_edit'
rudder     info: Edited file '/home/michel/.ssh/authorized_keys'
rudder     info: Inserted the promised line 'ssh-rsa <some_SSH_public_key>' into '/root/.ssh/authorized_keys' after locator
rudder     info: insert_lines promise 'ssh-rsa <some_SSH_public_key>' repaired
rudder     info: Moved '/root/.ssh/authorized_keys.cf-before-edit' to repository location '/var/rudder/modified-files/_root__ssh_authorized_keys_cf_before_edit'
rudder     info: Edited file '/root/.ssh/authorized_keys'
   error: File '/var/rudder/tmp/check_ssh_key_distribution//michel.authorized_keys.tmp' was marked for editing but could not be opened
   error: Errors encountered when actuating files promise '/var/rudder/tmp/check_ssh_key_distribution//michel.authorized_keys.tmp'
   error: File '/var/rudder/tmp/check_ssh_key_distribution//root.authorized_keys.tmp' was marked for editing but could not be opened
   error: Errors encountered when actuating files promise '/var/rudder/tmp/check_ssh_key_distribution//root.authorized_keys.tmp'
R: @@sshKeyDistribution@@result_repaired@@0caa2ab2-67a0-441e-80b5-be02e602b678@@2df3bf3a-d2d5-4c3d-97df-d8df68d2bcf2@@0@@SSH key@@michel michel RSA@@2025-05-15 05:27:33+00:00##ddb6665c-35d4-48c4-af44-f7d9cc0a6f29@#SSH key "michel michel RSA" for user michel was repaired
R: @@sshKeyDistribution@@result_repaired@@0caa2ab2-67a0-441e-80b5-be02e602b678@@2df3bf3a-d2d5-4c3d-97df-d8df68d2bcf2@@0@@SSH key@@root michel RSA@@2025-05-15 05:27:33+00:00##ddb6665c-35d4-48c4-af44-f7d9cc0a6f29@#SSH key "root michel RSA" for user root was repaired

Then close to the end:

   error: Can't stat file '/var/rudder/tmp/check_ssh_key_distribution//michel.authorized_keys.tmp' on 'localhost' in files.copy_from promise, it may be missing or access may not be authorized
   error: Errors encountered when actuating files promise '/home/michel/.ssh/authorized_keys'
   error: Can't stat file '/var/rudder/tmp/check_ssh_key_distribution//root.authorized_keys.tmp' on 'localhost' in files.copy_from promise, it may be missing or access may not be authorized
   error: Errors encountered when actuating files promise '/root/.ssh/authorized_keys'
R: @@sshKeyDistribution@@result_error@@0caa2ab2-67a0-441e-80b5-be02e602b678@@2df3bf3a-d2d5-4c3d-97df-d8df68d2bcf2@@0@@Flush SSH file@@michel michel RSA@@2025-05-15 05:27:33+00:00##ddb6665c-35d4-48c4-af44-f7d9cc0a6f29@#The keys for user michel could not be flushed
R: @@sshKeyDistribution@@result_error@@0caa2ab2-67a0-441e-80b5-be02e602b678@@2df3bf3a-d2d5-4c3d-97df-d8df68d2bcf2@@0@@Flush SSH file@@root michel RSA@@2025-05-15 05:27:33+00:00##ddb6665c-35d4-48c4-af44-f7d9cc0a6f29@#The keys for user root could not be flushed

Actions #4

Updated by François ARMAND 3 days ago

  • Assignee deleted (François ARMAND)

It will need qualification

Actions

Also available in: Atom PDF