Bug #15675
closed
Leak in Cache of Node Compliance and NodeInfo and perfs improvement
Added by Nicolas CHARLES about 5 years ago.
Updated over 4 years ago.
Category:
Performance and scalability
Description
There's a leak in Node Compliance cache - leaving Rudder alone for a night caused 51 millions of object in the map of CachedReportingServiceImpl, using a wooping 2 Go of RAM (the installation is REALLY large)
There's one also in the LocalNideInfoCache
It seems that mapValues references the initial map, causing it not to be GCable
- Status changed from New to In progress
- Assignee set to Nicolas CHARLES
The problem is likely with `mapValues`:
object CountMapMemory3 {
def bigArray = new Array(10*1024*1024)
def smallArray = new Array(1)
def printMem(s: String) = {
Runtime.getRuntime.gc()
println(s"${s}: " + Runtime.getRuntime().freeMemory()/1024)
}
def main(args: Array[String]): Unit = {
def res() = {
printMem("in res 0")
val m = Map(1 -> bigArray, 2 -> bigArray, 3 -> bigArray)
val m2 = m.map { case (k,v) => (k,smallArray) }
printMem("in res 1")
m2
}
printMem("start")
val map1 = res()
printMem("end")
}
}
prints:
start: 231530
in res 0: 241548
in res 1: 120031
end: 242940
If we replace the map
line with:
val m2 = m.mapValues(_ => smallArray)
It prints:
start: 231546
in res 0: 241545
in res 1: 120003
end: 120092
- Target version changed from 5.0.14 to 5.0.15
- Target version changed from 5.0.15 to 5.0.16
- Related to Bug #16467: A lot of data is computed and stored in Policy object, that is either never used, or used only once added
- Related to Bug #16468: Computation of ComplianceLevel generates too many objects added
- Related to Bug #16513: JVM GC cannot clean objects in scope in a for { } yield {} even if they are not referenced anymore added
- Target version changed from 5.0.16 to 5.0.17
- Status changed from In progress to Discussion
all issues have been fixed on related ticket, we can close it
- Status changed from Discussion to New
- Status changed from New to Pending release
ok, marking it released as subticket are released
- Fix check changed from To do to Checked
- Name check changed from To do to Reviewed
This bug has been fixed in Rudder 5.0.17 and 6.0.5 which were released today.
- Status changed from Pending release to Released
Also available in: Atom
PDF