Architecture #3007: Update String Template to version 4 - Rudder - Issue Tracker

Actions

Copy link

Architecture #3007

closed

Update String Template to version 4

Added by François ARMAND about 12 years ago. Updated almost 6 years ago.

Status:

Rejected

Priority:

N/A

Assignee:

Nicolas CHARLES

Category:

Architecture - Dependencies

Target version:

Ideas (not version specific)

Pull Request:

Effort required:

Large

Name check:

Fix check:

Regression:

Description

Faster, lighter, better: http://www.antlr.org/wiki/display/ST4/Differences+between+v3+and+v4

As explained in the given link, that update is some work, and so it can not be considered as a simple "library update".

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by François ARMAND about 12 years ago

After looking a little better, it seems that the last version (4.0.6) is not available in maven repos. In fact, the last available version is 4.0.2, almost two year old.

So I'm not sure it is wise to try to update to 4 for now. Or perhaps we could add that in our nexus ?

Actions

Copy link

Updated by François ARMAND about 12 years ago

Description updated (diff)
Parent task deleted (~~#3003~~)

Actions

Copy link

Updated by Nicolas CHARLES about 12 years ago

The 4.0.4 is available : http://mvnrepository.com/artifact/org.antlr/ST4/4.0.4
They changed name of the artefact from stringtemplate to ST
However the git refers to 4.0.4-SNAPSHOT in the pom, hence the unpublished version 4.0.6 ...

Actions

Copy link

Updated by Jonathan CLARKE almost 12 years ago

Status changed from 2 to New

Actions

Copy link

Updated by François ARMAND almost 12 years ago

Status changed from New to 8
Target version set to Ideas (not version specific)

Actions

Copy link

Updated by Benoît PECCATTE over 9 years ago

Status changed from 8 to New

Actions

Copy link

Updated by Nicolas CHARLES over 7 years ago

Has duplicate User story #10061: Upgrade to StringTemplate 4 to improve policy generation time added

Actions

Copy link

Updated by Nicolas CHARLES over 6 years ago

Ok, I've made some measures
With current implementation

[2018-06-04 12:28:42] DEBUG policy.generation - Promises written in 39543 ms
[2018-06-04 12:28:42] DEBUG policy.generation - Properties written in 138 ms
[2018-06-04 12:28:42] DEBUG policy.generation - Parameters written in 59 ms
[2018-06-04 12:28:42] DEBUG policy.generation - Licenses copied in 0 ms
[2018-06-04 12:28:50] DEBUG policy.generation - Policies moved to their final position in 7514 ms
[2018-06-04 12:28:50] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written on filesystem in 48106 ms
[2018-06-04 12:28:51] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration cached in LDAP in 1174 ms
[2018-06-04 12:28:51] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written in 49282 ms, start to update expected reports.
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node expected reports saved in base in 49282 ms.
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Post-policy-generation hooks ran in 2 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Timing summary:
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen scripts hooks :          0 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen modules hooks :          0 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Fetch all information     :        311 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Historize names           :        973 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build current rule values :         73 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build target configuration:       8180 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Write node configurations :      49282 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Save expected reports     :        904 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run post generation hooks :          2 ms
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Number of nodes updated   :       1597   
[2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Policy generation completed in 75630 ms

With StringTemplate4

[2018-06-04 13:56:04] DEBUG policy.generation - Promises written in 22994 ms
[2018-06-04 13:56:04] DEBUG policy.generation - Properties written in 44 ms
[2018-06-04 13:56:04] DEBUG policy.generation - Parameters written in 46 ms
[2018-06-04 13:56:04] DEBUG policy.generation - Licenses copied in 0 ms
[2018-06-04 13:56:13] DEBUG policy.generation - Policies moved to their final position in 9383 ms
[2018-06-04 13:56:13] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written on filesystem in 33251 ms
[2018-06-04 13:56:14] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration cached in LDAP in 1026 ms
[2018-06-04 13:56:14] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written in 34279 ms, start to update expected reports.
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node expected reports saved in base in 34279 ms.
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Post-policy-generation hooks ran in 2 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Timing summary:
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen scripts hooks :          0 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen modules hooks :          0 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Fetch all information     :        319 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Historize names           :        845 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build current rule values :         46 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build target configuration:       7852 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Write node configurations :      34279 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Save expected reports     :        940 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run post generation hooks :          2 ms
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Number of nodes updated   :       1597   
[2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Policy generation completed in 57238 ms

So the step "Promises written" that uses a lot of stringtemplate goes from 39s to 23s - a significant improvement !

Alas, StringTemplate 4 is not maintained any more, adapting it to make it work with our template is a pretty signigicant change (we cannot use & as a seperator in stringtemplate), it needs rewritting of techniques, due to change of syntax, plus there are some minor breaking change, especially around }, iteration, and so on.

Actions

Copy link

Updated by Nicolas CHARLES over 6 years ago

Some possible replacement:

http://software.clapper.org/scalasti/ : Scala interface to StringTemplate - no idea of the performance
https://github.com/playframework/twirl : seems to be for html only
scalatags : http://www.lihaoyi.com/scalatags/ : as it is also for css, may work for plain text

Bottom line: they will all require Technique rewrite, so we'll need both engine at the same time for a period of time

Actions

Copy link

#10

Updated by Nicolas CHARLES over 6 years ago

Effort required set to Large

Actions

Copy link

#11

Updated by Alexis Mousset about 6 years ago

Status changed from New to Discussion
Assignee set to Nicolas CHARLES

Nicolas, IIRC you said it won't be possible?

Actions

Copy link

#12

Updated by Nicolas CHARLES almost 6 years ago

Status changed from Discussion to Rejected

I'm rejecting this issue - migrating to stringtemplate 4 is not possible (incompatible syntax, and the project is not rellay maintained), and #13987 did improve a lot performance.

Actions

Copy link

Also available in: Atom PDF

Project

General

Profile

Rudder

Custom queries