Project

General

Profile

Bug #9299

Extend groups search capabilities for complex JSON Properties / JsonPath

Added by Janos Mattyasovszky about 2 years ago. Updated 5 months ago.

Status:
Pending release
Priority:
N/A
Category:
Web - Nodes & inventories
Target version:
Severity:
Major - prevents use of part of Rudder | no simple workaround
User visibility:
Operational - other Techniques | Technique editor | Rudder settings
Effort required:
Medium
Priority:
29

Description

Currently when the feature of extended JSON-Based node properties is enabled, there is no real optimal way to define dynamic groups based on that.

The current option to match a Key=Value pair is as following:
- [ Properties ▼] [ Name=Value ▼] [ Name= ▼] <field_of_key>
- [ Properties ▼] [ Name=Value ▼] [ = ▼] <field_of_value>

My proposal is that you create a new dropdown-type field that says like:
- [ Properties ▼] [ JSON Data ▼] <field_of_key> [< match_operator> ▼] <field_ov_matched_value>

This could be used to create a new type of dynamic group, that would match on a specific value inside the JSON value of the property.

(Legend: [ X ▼] is a dropdown menu, <field> is an input-type field.)


Subtasks

User story #12763: Document "JSON select" operator for node PropertiesReleasedAlexis MOUSSET

Related issues

Related to Rudder - User story #9551: Group criteria operator for propertiesRejected
Related to Rudder - Bug #10599: Impossible to search or build groups based on JSON values in node propertiesReleased
Related to Rudder - Bug #10743: Dynamic group search not available on boolean node property valueRejected
Related to Rudder - Bug #12572: Custom node properties from inventory are not found in group searchReleased
Related to Rudder - User story #4670: Allows inventories to be augmented by the user with custom propertiesReleased

Associated revisions

Revision 0b8a0e2e (diff)
Added by François ARMAND 6 months ago

Fixes #9299: Extend groups search capabilities for complex JSON Properties / JsonPath

History

#1 Updated by François ARMAND about 2 years ago

  • Category set to Web - Nodes & inventories
  • Target version set to Ideas (not version specific)

This is an excellent idea, and it seems to be somehow the natural way of building dyngroup from properties given the way node properties are used elsewhere.

We are going to work on that kind of stuff in 4.1.

Thanks for reporting and sharing the idea.

#2 Updated by François ARMAND about 2 years ago

#3 Updated by François ARMAND about 2 years ago

So, it seems that it won't go into 4.1 but 4.2. The idea and use case it still really clear.

#4 Updated by Benoît PECCATTE over 1 year ago

  • Tracker changed from Bug to User story

#5 Updated by François ARMAND over 1 year ago

  • Related to Bug #10599: Impossible to search or build groups based on JSON values in node properties added

#6 Updated by François ARMAND over 1 year ago

  • Related to Bug #10743: Dynamic group search not available on boolean node property value added

#7 Updated by Benoît PECCATTE over 1 year ago

  • Tracker changed from User story to Bug
  • Priority set to 0

#8 Updated by Benoît PECCATTE over 1 year ago

  • Severity set to Major - prevents use of part of Rudder | no simple workaround
  • User visibility set to Operational - other Techniques | Technique editor | Rudder settings
  • Effort required set to Medium
  • Priority changed from 0 to 31

#9 Updated by Benoît PECCATTE about 1 year ago

  • Priority changed from 31 to 29

#10 Updated by François ARMAND 6 months ago

  • Subject changed from Extend dynamic groups search capabilities for complex JSON Properties to Extend groups search capabilities for complex JSON Properties / JsonPath
  • Target version changed from Ideas (not version specific) to 4.3.2

#11 Updated by François ARMAND 6 months ago

  • Status changed from New to In progress

#12 Updated by François ARMAND 6 months ago

Now that #12572 is added, we have the possibility to do that ticket in a simple way IF we accept to not modify the select/input layout, which is a big change.

I propose to add a new operator, JSON Path ("Name:JsonPath"), which would allows to use JsonPath in the right part of the ":". JsonPath is a kind of XPath for JSON: https://github.com/json-path/JsonPath.
The semantic would be "as long as the JsonPath selector returns a non-empty list, the search is ok and the node selected".

For example, let say you have a node with a property whose name is "user" and value is:

{
  "id": "xxxxxx",
  "accepted": true,
  "personal": {
    "name": "Smith Jones",
      "address": {
          "streetaddress": "7 24th Street",
          "city": "New York",
          "state": "NY",
          "postalcode": 10038
      },
      "phones": [
        {"type":"home","number":"(541) 754-3010"},
        {"type":"mobile","number":"(541) 754-9999"}
      ]
    }
  }
}

The following query would all select the node (I'm only displaying what would go in the input text field):

//select based on the value of a JSON leaf - boolean type
user:$.[?(@.accepted==true)]" }

//select based on the value of a JSON leaf - string type
user:$.personal.address[?(@.city=='New York')]

//select because "city" exists (whatever its value)
user:$.personal.address.city

//select based on a regex
user:$.personal.phones[?(@.number=~/.*123.*/)]

//select based on belonging to a set
user:$.personal.address[?(@.state in ['NY', 'CA', 'LA', 'TX'])]

It's just a glimpse of what is possible, JsonPath allows a lot of other operations. See examples in https://github.com/json-path/JsonPath.

#13 Updated by François ARMAND 6 months ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Vincent MEMBRÉ
  • Pull Request set to https://github.com/Normation/rudder/pull/1961

#14 Updated by François ARMAND 6 months ago

  • Related to Bug #12572: Custom node properties from inventory are not found in group search added

#15 Updated by François ARMAND 6 months ago

  • Related to User story #4670: Allows inventories to be augmented by the user with custom properties added

#16 Updated by François ARMAND 6 months ago

  • Status changed from Pending technical review to Pending release

#17 Updated by Vincent MEMBRÉ 6 months ago

This bug has been fixed in Rudder 4.3.2 which was released today.

Also available in: Atom PDF