Actions
Bug #15792
closedWhen writing policies, we replace the RudderUniqueId with replaceAll which is twice as slow as StringUtils.replace
Status:
Released
Priority:
N/A
Assignee:
Category:
Performance and scalability
Target version:
Pull Request:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Reviewed
Fix check:
Checked
Regression:
Description
Generation measurement shows that 110 s (probably threaded on 10 thread, so in final 10s) are used in replaceAll to replace the RudderUniqueId in template and in path
replaceAll uses internally a regex to replace all, while StringUtils.replace does not ( https://stackoverflow.com/questions/5407592/replace-all-occurrences-of-substring-in-a-string-which-is-more-efficient-in-ja )
Since we are replacing an hardcoded text, we can swith to StringUtils
following tests shows that StringUtils is twice as fast
import org.apache.commons.lang.StringUtils val TAG_OF_RUDDER_MULTI_POLICY = "RudderUniqueId" //val p = java.util.regex.Pattern.compile(TAG_OF_RUDDER_MULTI_POLICY) def randomString(length: Int) = { val r = new scala.util.Random val sb = new StringBuilder for (i <- 1 to 2*length) { sb.append(r.nextPrintableChar) } sb.toString } // create 10000 string val textToReplace= (1 to 10000).toSeq.map { case x => if (x%2 == 0) { randomString(x) } else { randomString(x).concat(TAG_OF_RUDDER_MULTI_POLICY).concat(randomString(x)) } } val start3 = System.currentTimeMillis() val replacedText3 = textToReplace.map( x => StringUtils.replace(x, TAG_OF_RUDDER_MULTI_POLICY, "coucou")) val end3 = System.currentTimeMillis() println(s"Replacing with StringUtil for ${replacedText3.length} element took ${end3-start3}") val start = System.currentTimeMillis() val replacedText = textToReplace.map( x => x.replaceAll(TAG_OF_RUDDER_MULTI_POLICY, "coucou")) val end = System.currentTimeMillis() println(s"Replacing with replaceAll for ${replacedText.length} element took ${end-start}")
outputs
Replacing with StringUtil for 10000 element took 639 Replacing with replaceAll for 10000 element took 1146
I tried also with a precompile pattern, but perfs were similar as replaceAll
Updated by Nicolas CHARLES about 5 years ago
- Status changed from New to In progress
- Assignee set to Nicolas CHARLES
Updated by Nicolas CHARLES about 5 years ago
- Status changed from In progress to Pending technical review
- Assignee changed from Nicolas CHARLES to François ARMAND
- Pull Request set to https://github.com/Normation/rudder/pull/2482
Updated by Nicolas CHARLES about 5 years ago
- Status changed from Pending technical review to Pending release
Applied in changeset rudder|40f23bfe61e77ae434591643dd653736707a0174.
Updated by François ARMAND about 5 years ago
- Fix check changed from To do to Checked
Updated by Alexis Mousset about 5 years ago
- Name check changed from To do to Reviewed
Updated by Vincent MEMBRÉ about 5 years ago
- Status changed from Pending release to Released
This bug has been fixed in Rudder 5.0.14 which was released today.
Actions