Rudder

So, I'm able to reproduce that and to the point where "how the hell it's not working, I'm seeing the code that is checking that". So next, it should be "how the hell it has ever worked".

For information, the code is just here: https://github.com/Normation/rudder/blob/branches/rudder/2.11/rudder-core/src/main/scala/com/normation/rudder/services/policies/nodeconfig/NodeConfigurationCacheRepository.scala#L116

Well, if you say the code itself is good, then from what I see is that either it has never really worked, and it was just not found until now, or it broke sometime by having one of the variables not being populated, so the hash never actually changes even the real policy server being reported by the inventory does. I'd have a look at how the inventory and the policy server of a relay is being processed, but I do not speak java/scala :-)

Sorry I misread the first comment on link it to the issue #7288 :/

@Janos Matya: the code is good for a developper not understanding the problem yet, as in the six stage of debugging ("1/ That can't happen. 2/ That doesn't happen on my machine. 3/That shouldn't happen. 4/Why does that happen? 5/Oh, I see. 6/How did that ever work?)

OK, I'm making some advance: now, I do have found a bug in an error message. And the behavior is "OK" is the policy server is not know.

"ok" meaning that we get an error message saying "[2016-05-06 12:03:26] ERROR com.normation.rudder.services.policies.DeploymentServiceImpl - Error while building target configuration node for node 009e0af1-1a3c-4ee0-a91d-10b68a008f5d which is one of the target of rules. Ignoring it for the rest of the process <- Policy server with ID 'some-other-uuid-2' was not found (mandatory for building promises for the node)"

(that message was previously: """"Error while building target configuration node for node 009e0af1-1a3c-4ee0-a91d-10b68a008f5d which is one of the target of rules. Ignoring it for the rest of the process <- Node with ID 009e0af1-1a3c-4ee0-a91d-10b68a008f5d was not found"""" which is false.)

So now, I have to understand why when an existing policy server is set, it does not regenerate policies as expected.

So, the problem is likelly due to a previous optimisation done to avoid to write promises on some cases (typically when a new node is accepted, for other nodes).

The problems seems to be that at some point, the nodeConfigCache for the node with the modified policy server id get erased (why?). So the node is considered like a new node (with have cache for other, but not it) - which seems to be a dubious assumption: what about a partial clear cache, only for that node ?

So, I'm in a test case where I have a valid policy server (root) and changing to a valid nodeId, not a policy server. So the rules for the root policy server are correctly updated to not accpet the node anymore. But nothing tell the node that it should have its promises written.

So, two things to check/change:
- an updated node should not have its cache removed;
- a node without a configuration cache should alway have ALL the promises written

After a day working on it and findings funny behavior ( #8246, #8247), but NOT what is described in the bug, I reread the ticket and I believe I completly mis-understood it at first.

So, does the problem is 1/ that the inventory upload does not TRIGGER a promise regeneration (but when triggered by hand, the promises are correctly generated for the new relay)? Or is it that 2/ even if the generation is triggered by hand, the new relay server don't get the policies for the node?

If it is 1/, it is a known (bad) limitation: change in inventories don't trigger regeneration http://www.rudder-project.org/redmine/issues/1411

If it's 2/, as I thought, I'm not able to reproduce it. But I think it's 1/, and I just mis-understood at first.

So could you please confirm if the problem is 1/ or 2/ ? Or something else ?

Thanks!

Hi.

It is 1/, and because it does not regenerate on it self, the changed node never receives a new policy from them new policy server until a manual trigger is done, which is not always the case, so nodes simply become cut off the system without having a real clue why.

And if generation takes about 20+ minutes, you usually don't push the button too often on your own. .

OK, so I propose to close that one, mark it as a duplicate of #1411, change #1411 as a bug (and not a user story, because it actually broke your production), solve 1411. Is it OK ?

Well, it's not a direct duplicate of #1411, since not the node hostname-change did not start to regenerate the rules, but the policyserver-change.

However, it basically boils down to the same: An important change on the node was reported by an inventory upload, but Rudder did not react to it, so the node got "disconnected" from further policy updates.

This is also true for hostname changes, as cf-serverd would not allow the node by the new name to fetch the policy until one manually clicks on the "Update policy" Button, which is not so nice and could be avoided...

I corrected the description of 1411 to reflect the notion of "important properties". Is it OK now to close that one as duplicate ?

sure, go ahead.