Project

General

Profile

Actions

Bug #15792

closed

When writing policies, we replace the RudderUniqueId with replaceAll which is twice as slow as StringUtils.replace

Added by Nicolas CHARLES over 4 years ago. Updated over 4 years ago.

Status:
Released
Priority:
N/A
Category:
Performance and scalability
Target version:
Severity:
UX impact:
User visibility:
Effort required:
Priority:
0
Name check:
Reviewed
Fix check:
Checked
Regression:

Description

Generation measurement shows that 110 s (probably threaded on 10 thread, so in final 10s) are used in replaceAll to replace the RudderUniqueId in template and in path

replaceAll uses internally a regex to replace all, while StringUtils.replace does not ( https://stackoverflow.com/questions/5407592/replace-all-occurrences-of-substring-in-a-string-which-is-more-efficient-in-ja )

Since we are replacing an hardcoded text, we can swith to StringUtils

following tests shows that StringUtils is twice as fast


import org.apache.commons.lang.StringUtils

  val TAG_OF_RUDDER_MULTI_POLICY = "RudderUniqueId" 
  //val p = java.util.regex.Pattern.compile(TAG_OF_RUDDER_MULTI_POLICY)

  def randomString(length: Int) = {
    val r = new scala.util.Random
    val sb = new StringBuilder
    for (i <- 1 to 2*length) {
      sb.append(r.nextPrintableChar)
    }
    sb.toString
  }

  // create 10000 string
  val textToReplace= (1 to 10000).toSeq.map { case x =>
           if (x%2 == 0) {
             randomString(x)
           } else {
             randomString(x).concat(TAG_OF_RUDDER_MULTI_POLICY).concat(randomString(x))
           }
  }

      val start3 = System.currentTimeMillis()
      val replacedText3 = textToReplace.map( x => StringUtils.replace(x, TAG_OF_RUDDER_MULTI_POLICY, "coucou"))
      val end3 = System.currentTimeMillis()
      println(s"Replacing with StringUtil for ${replacedText3.length} element took ${end3-start3}")

      val start = System.currentTimeMillis()
      val replacedText = textToReplace.map( x => x.replaceAll(TAG_OF_RUDDER_MULTI_POLICY, "coucou"))
      val end = System.currentTimeMillis()

      println(s"Replacing with replaceAll for ${replacedText.length} element took ${end-start}")

outputs

Replacing with StringUtil for 10000 element took 639
Replacing with replaceAll for 10000 element took 1146

I tried also with a precompile pattern, but perfs were similar as replaceAll

Actions

Also available in: Atom PDF