Project

General

Profile

Actions

Architecture #16197

closed

Serialization of NodeConfigurations is fairly expensive

Added by Nicolas CHARLES over 4 years ago. Updated about 2 years ago.

Status:
Released
Priority:
N/A
Category:
Performance and scalability
Target version:
Effort required:
Name check:
To do
Fix check:
To do
Regression:

Description

Node configuration serialization is quite large, using a lot of space & memory

{"modes":{"globalPolicyMode":{"mode":"enforce","overridable":true},"globalComplianceMode":"full-compliance","globalHeartbeatPeriod":1,"globalAgentRunInterval":{"interval":15,"startMinute":0,"splayHour":0,"splaytime":4}},"rules":[{"ruleId":"7a57d357-0821-46aa-86b8-b20806915ebf","directives":[{"directiveId":"3be9f8d1-c7ee-4766-8a44-b795762c5899","isSystem":false,"components":[{"componentName":"Variable definition","values":["load148"],"unexpanded":["load148"]}]},{"directiveId":"8d0a6374-baf3-4006-bba9-d2587698199e","isSystem":false,"components":[{"componentName":"File","values":["/tmp/test60"],"unexpanded":["/tmp/test60"]},{"componentName":"Enforce content by section","values":["/tmp/test60"],"unexpanded":["/tmp/test60"]},{"componentName":"Line deletion regular expressions","values":["/tmp/test60"],"unexpanded":["/tmp/test60"]},{"componentName":"Line replacement regular expressions","values":["/tmp/test60"],"unexpanded":["/tmp/test60"]},{"componentName":"Permission adjustment","values":["/tmp/test60"],"unexpanded":["/tmp/test60"]},{"componentName":"Post-modification hook","values":["/tmp/test60"],"unexpanded":["/tmp/test60"]}]},{"directiveId":"77ec2114-8c9b-4839-8cb2-7dd55e665265","isSystem":false,"components":[{"componentName":"File or directory to monitor","values":["/etc/passwd"],"unexpanded":["/etc/passwd"]}]}]},{"ruleId":"inventory-all","directives":[{"directiveId":"inventory-all","isSystem":true,"components":[{"componentName":"inventory","values":["None"],"unexpanded":[]}]}]},{"ruleId":"hasPolicyServer-root","directives":[{"directiveId":"common-root","isSystem":true,"components":[{"componentName":"Update","values":["None"],"unexpanded":["None"]},{"componentName":"ncf Initialization","values":["None"],"unexpanded":["None"]},{"componentName":"Security parameters","values":["None"],"unexpanded":["None"]},{"componentName":"Log system for reports","values":["None"],"unexpanded":["None"]},{"componentName":"CRON Daemon","values":["None"],"unexpanded":["None"]},{"componentName":"Binaries update","values":["None"],"unexpanded":["None"]},{"componentName":"Process checking","values":["None"],"unexpanded":["None"]},{"componentName":"Monitoring","values":["None"],"unexpanded":["None"]}]}]},{"ruleId":"025fc9a2-6185-4f28-ae2e-850e9936a5fb","directives":[{"directiveId":"f894d24b-5646-4fd1-9c6b-c1c7a62dd810","isSystem":false,"components":[{"componentName":"File","values":["/tmp/test102"],"unexpanded":["/tmp/test102"]},{"componentName":"Enforce content by section","values":["/tmp/test102"],"unexpanded":["/tmp/test102"]},{"componentName":"Line deletion regular expressions","values":["/tmp/test102"],"unexpanded":["/tmp/test102"]},{"componentName":"Line replacement regular expressions","values":["/tmp/test102"],"unexpanded":["/tmp/test102"]},{"componentName":"Permission adjustment","values":["/tmp/test102"],"unexpanded":["/tmp/test102"]},{"componentName":"Post-modification hook","values":["/tmp/test102"],"unexpanded":["/tmp/test102"]}]},{"directiveId":"16b7fade-0019-4409-8cde-3c8cc0c5bb19","isSystem":false,"components":[{"componentName":"File","values":["/tmp/test29"],"unexpanded":["/tmp/test29"]},{"componentName":"Enforce content by section","values":["/tmp/test29"],"unexpanded":["/tmp/test29"]},{"componentName":"Line deletion regular expressions","values":["/tmp/test29"],"unexpanded":["/tmp/test29"]},{"componentName":"Line replacement regular expressions","values":["/tmp/test29"],"unexpanded":["/tmp/test29"]},{"componentName":"Permission adjustment","values":["/tmp/test29"],"unexpanded":["/tmp/test29"]},{"componentName":"Post-modification hook","values":["/tmp/test29"],"unexpanded":["/tmp/test29"]}]}]},{"ruleId":"ff6435cf-51fb-4166-add4-1c7b09450498","directives":[{"directiveId":"d64879eb-f338-4dad-9aa2-ac1d20607f74","isSystem":false,"components":[{"componentName":"File","values":["/tmp/test136"],"unexpanded":["/tmp/test136"]},{"componentName":"Enforce content by section","values":["/tmp/test136"],"unexpanded":["/tmp/test136"]},{"componentName":"Line deletion regular expressions","values":["/tmp/test136"],"unexpanded":["/tmp/test136"]},{"componentName":"Line replacement regular expressions","values":["/tmp/test136"],"unexpanded":["/tmp/test136"]},{"componentName":"Permission adjustment","values":["/tmp/test136"],"unexpanded":["/tmp/test136"]},{"componentName":"Post-modification hook","values":["/tmp/test136"],"unexpanded":["/tmp/test136"]}]},{"directiveId":"cf6f12cb-a615-4c58-a387-353645f4755b","isSystem":false,"components":[{"componentName":"File","values":["/tmp/test36"],"unexpanded":["/tmp/test36"]},{"componentName":"Enforce content by section","values":["/tmp/test36"],"unexpanded":["/tmp/test36"]},{"componentName":"Line deletion regular expressions","values":["/tmp/test36"],"unexpanded":["/tmp/test36"]},{"componentName":"Line replacement regular expressions","values":["/tmp/test36"],"unexpanded":["/tmp/test36"]},{"componentName":"Permission adjustment","values":["/tmp/test36"],"unexpanded":["/tmp/test36"]},{"componentName":"Post-modification hook","values":["/tmp/test36"],"unexpanded":["/tmp/test36"]}]}]},{"ruleId":"da89882e-8e4c-4094-aa71-20513217ee24","directives":[{"directiveId":"40b89eb7-aab5-4efa-be47-48c14a395722","isSystem":false,"components":[{"componentName":"Variable definition","values":["load106"],"unexpanded":["load106"]}]},{"directiveId":"cfab773b-e330-445a-bb5a-d97aaaee4a83","isSystem":false,"components":[{"componentName":"File","values":["/tmp/test41"],"unexpanded":["/tmp/test41"]},{"componentName":"Enforce content by section","values":["/tmp/test41"],"unexpanded":["/tmp/test41"]},{"componentName":"Line deletion regular expressions","values":["/tmp/test41"],"unexpanded":["/tmp/test41"]},{"componentName":"Line replacement regular expressions","values":["/tmp/test41"],"unexpanded":["/tmp/test41"]},{"componentName":"Permission adjustment","values":["/tmp/test41"],"unexpanded":["/tmp/test41"]},{"componentName":"Post-modification hook","values":["/tmp/test41"],"unexpanded":["/tmp/test41"]}]}]},{"ruleId":"201ca349-6b49-45a5-a585-751d537dcb62","directives":[{"directiveId":"bad998ed-af31-4692-86a8-233c7b219913","isSystem":false,"components":[{"componentName":"Variable definition","values":["load10"],"unexpanded":["load10"]}]},{"directiveId":"3137011e-534b-4758-81c1-9a3dd63b6893","isSystem":false,"components":[{"componentName":"Variable definition","values":["load50"],"unexpanded":["load50"]}]}]}],"overrides":[]}' )

We could do better than that
We need to change SerializeNodeExpectedReportsComposite and parseJsonNodeExpectedReports , while keeping the parsing compatible with previous versions

Also, we would want to use an efficient json parsing/serialisation lib to avoid inneficiencies.

Actions #1

Updated by Nicolas CHARLES over 4 years ago

  • Target version changed from 5.0.15 to 6.0.0
Actions #2

Updated by Alexis Mousset over 4 years ago

  • Target version changed from 6.0.0 to 6.0.1
Actions #3

Updated by Vincent MEMBRÉ over 4 years ago

  • Target version changed from 6.0.1 to 6.0.2
Actions #4

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 6.0.2 to 6.0.3
Actions #5

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 6.0.3 to 6.0.4
Actions #6

Updated by Vincent MEMBRÉ about 4 years ago

  • Target version changed from 6.0.4 to 6.0.5
Actions #7

Updated by Vincent MEMBRÉ almost 4 years ago

  • Target version changed from 6.0.5 to 6.0.6
Actions #8

Updated by Vincent MEMBRÉ almost 4 years ago

  • Target version changed from 6.0.6 to 6.0.7
Actions #9

Updated by Vincent MEMBRÉ almost 4 years ago

  • Target version changed from 6.0.7 to 6.0.8
Actions #10

Updated by François ARMAND over 3 years ago

  • Tracker changed from Bug to Architecture
  • Target version changed from 6.0.8 to 6.2.0~beta1
  • Priority deleted (0)
Actions #11

Updated by Vincent MEMBRÉ over 3 years ago

  • Target version changed from 6.2.0~beta1 to 6.2.0~rc1
Actions #12

Updated by François ARMAND over 3 years ago

  • Target version changed from 6.2.0~rc1 to 7.0.0~beta1
Actions #13

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 7.0.0~beta1 to 7.0.0~beta2
Actions #14

Updated by Vincent MEMBRÉ over 2 years ago

  • Target version changed from 7.0.0~beta2 to 7.0.0~beta3
Actions #15

Updated by Alexis Mousset over 2 years ago

  • Target version changed from 7.0.0~beta3 to 7.1.0~beta1
Actions #16

Updated by François ARMAND about 2 years ago

  • Description updated (diff)
Actions #17

Updated by François ARMAND about 2 years ago

  • Status changed from New to In progress
  • Assignee set to François ARMAND
Actions #18

Updated by François ARMAND about 2 years ago

  • Status changed from In progress to Pending technical review
  • Assignee changed from François ARMAND to Nicolas CHARLES
  • Pull Request set to https://github.com/Normation/rudder/pull/4193
Actions #19

Updated by François ARMAND about 2 years ago

  • Status changed from Pending technical review to Pending release
Actions #20

Updated by Vincent MEMBRÉ about 2 years ago

  • Status changed from Pending release to Released

This bug has been fixed in Rudder 7.1.0~beta1 which was released today.

Actions

Also available in: Atom PDF