Architecture #3007
closed
Update String Template to version 4
Added by François ARMAND over 11 years ago. Updated over 5 years ago.
Description
Faster, lighter, better: http://www.antlr.org/wiki/display/ST4/Differences+between+v3+and+v4
As explained in the given link, that update is some work, and so it can not be considered as a simple "library update".
Updated by François ARMAND over 11 years ago
After looking a little better, it seems that the last version (4.0.6) is not available in maven repos. In fact, the last available version is 4.0.2, almost two year old.
So I'm not sure it is wise to try to update to 4 for now. Or perhaps we could add that in our nexus ?
Updated by François ARMAND over 11 years ago
- Description updated (diff)
- Parent task deleted (
#3003)
Updated by Nicolas CHARLES over 11 years ago
The 4.0.4 is available : http://mvnrepository.com/artifact/org.antlr/ST4/4.0.4
They changed name of the artefact from stringtemplate to ST
However the git refers to 4.0.4-SNAPSHOT in the pom, hence the unpublished version 4.0.6 ...
Updated by
Updated by François ARMAND over 11 years ago
- Status changed from New to 8
- Target version set to Ideas (not version specific)
Updated by
Updated by Nicolas CHARLES about 7 years ago
- Has duplicate User story #10061: Upgrade to StringTemplate 4 to improve policy generation time added
Updated by Nicolas CHARLES almost 6 years ago
Ok, I've made some measures
With current implementation
[2018-06-04 12:28:42] DEBUG policy.generation - Promises written in 39543 ms [2018-06-04 12:28:42] DEBUG policy.generation - Properties written in 138 ms [2018-06-04 12:28:42] DEBUG policy.generation - Parameters written in 59 ms [2018-06-04 12:28:42] DEBUG policy.generation - Licenses copied in 0 ms [2018-06-04 12:28:50] DEBUG policy.generation - Policies moved to their final position in 7514 ms [2018-06-04 12:28:50] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written on filesystem in 48106 ms [2018-06-04 12:28:51] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration cached in LDAP in 1174 ms [2018-06-04 12:28:51] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written in 49282 ms, start to update expected reports. [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node expected reports saved in base in 49282 ms. [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Post-policy-generation hooks ran in 2 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Timing summary: [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen scripts hooks : 0 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen modules hooks : 0 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Fetch all information : 311 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Historize names : 973 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build current rule values : 73 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build target configuration: 8180 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Write node configurations : 49282 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Save expected reports : 904 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run post generation hooks : 2 ms [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Number of nodes updated : 1597 [2018-06-04 12:29:04] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Policy generation completed in 75630 ms
With StringTemplate4
[2018-06-04 13:56:04] DEBUG policy.generation - Promises written in 22994 ms [2018-06-04 13:56:04] DEBUG policy.generation - Properties written in 44 ms [2018-06-04 13:56:04] DEBUG policy.generation - Parameters written in 46 ms [2018-06-04 13:56:04] DEBUG policy.generation - Licenses copied in 0 ms [2018-06-04 13:56:13] DEBUG policy.generation - Policies moved to their final position in 9383 ms [2018-06-04 13:56:13] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written on filesystem in 33251 ms [2018-06-04 13:56:14] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration cached in LDAP in 1026 ms [2018-06-04 13:56:14] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node configuration written in 34279 ms, start to update expected reports. [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Node expected reports saved in base in 34279 ms. [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Post-policy-generation hooks ran in 2 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Timing summary: [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen scripts hooks : 0 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run pre-gen modules hooks : 0 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Fetch all information : 319 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Historize names : 845 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build current rule values : 46 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Build target configuration: 7852 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Write node configurations : 34279 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Save expected reports : 940 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Run post generation hooks : 2 ms [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Number of nodes updated : 1597 [2018-06-04 13:56:25] DEBUG com.normation.rudder.services.policies.PromiseGenerationServiceImpl - Policy generation completed in 57238 ms
So the step "Promises written" that uses a lot of stringtemplate goes from 39s to 23s - a significant improvement !
Alas, StringTemplate 4 is not maintained any more, adapting it to make it work with our template is a pretty signigicant change (we cannot use & as a seperator in stringtemplate), it needs rewritting of techniques, due to change of syntax, plus there are some minor breaking change, especially around }, iteration, and so on.
Updated by Nicolas CHARLES almost 6 years ago
- http://software.clapper.org/scalasti/ : Scala interface to StringTemplate - no idea of the performance
- https://github.com/playframework/twirl : seems to be for html only
- scalatags : http://www.lihaoyi.com/scalatags/ : as it is also for css, may work for plain text
Bottom line: they will all require Technique rewrite, so we'll need both engine at the same time for a period of time
Updated by Alexis Mousset over 5 years ago
- Status changed from New to Discussion
- Assignee set to Nicolas CHARLES
Nicolas, IIRC you said it won't be possible?
Updated by Nicolas CHARLES over 5 years ago
- Status changed from Discussion to Rejected
I'm rejecting this issue - migrating to stringtemplate 4 is not possible (incompatible syntax, and the project is not rellay maintained), and #13987 did improve a lot performance.