Bug #13014
closedStackOverflowError during policy generation in JavascriptEngine - debian 9.5 with jdk 1.8.0_181
Description
After upgrading from 4.2.7 to 4.3.3 policy generation fails with a StackOverflowError.
I determined that the exception is caused by "com.normation.rudder.services.policies.JsEngine$SandboxSecurityManager.checkPermission" calling itself recursively while trying to check for the permission to load a missing class (which appears to be "FilePermission").
Files
Updated by François ARMAND about 6 years ago
- Category set to Security
- Assignee set to François ARMAND
- Target version set to 4.3.5
- User visibility set to Operational - other Techniques | Technique editor | Rudder settings
- Priority changed from 0 to 50
Sorry for the long delay.
I'm not able to reproduce that problem, but the jvm security manager is a quite sensitive beast. From what I see, it may be that a (new ?) required permission is missing, and so the check for the permission itself trigger a permission check, which loops.
Would you mind sharing:
- your exact OS,
- your exact JVM flavor and version ?
We corrected some problems with Java 9/10 support (https://www.rudder-project.org/redmine/issues/12548) but the manifestation of the problem were not the same exactly.
Sorry for that.
Updated by Marco Kirchhoff about 6 years ago
Thanks for getting back to me.
Here's the information you asked for.
# uname -a Linux rudder 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08) x86_64 GNU/Linux # lsb_release -a Distributor ID: Debian Description: Debian GNU/Linux 9.5 (stretch) Release: 9.5 Codename: stretch # dpkg --list | grep -e rudder -e ncf ii ncf 4.3.4-stretch0 all ncf - CFEngine framework ii ncf-api-virtualenv 4.3.4-stretch0 all ncf - CFEngine framework ii rudder-agent 4.3.4-stretch0 amd64 Configuration management and audit tool - agent ii rudder-inventory-endpoint 4.3.4-stretch0 all Configuration management and audit tool - service to receive inventory data ii rudder-inventory-ldap 4.3.4-stretch0 amd64 Configuration management and audit tool - OpenLDAP ii rudder-jetty 4.3.4-stretch0 all Configuration management and audit tool - Jetty application server ii rudder-reports 4.3.4-stretch0 all Configuration management and audit tool - reports database ii rudder-server-relay 4.3.4-stretch0 amd64 Configuration management and audit tool - Server relay package ii rudder-server-root 4.3.4-stretch0 all Configuration management and audit tool - root server base package ii rudder-techniques 4.3.4-stretch0 all Configuration management and audit tool - techniques ii rudder-webapp 4.3.4-stretch0 all Configuration management and audit tool - webapp # java -version openjdk version "1.8.0_181" OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-1~deb9u1-b13) OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
Updated by François ARMAND about 6 years ago
- Related to Bug #12448: Failed generation with "Could not initialize class javax.crypto.JceSecurity" added
Updated by François ARMAND about 6 years ago
- Subject changed from StackOverflowError during policy generation in JavascriptEngine to StackOverflowError during policy generation in JavascriptEngine - debian 9.5 with jdk 1.8.0_181
- Severity changed from Major - prevents use of part of Rudder | no simple workaround to Critical - prevents main use of Rudder | no workaround | data loss | security
- Effort required set to Medium
- Priority changed from 50 to 57
1.8.0_172 is pretty recent, but I just tested with it for the same release of Rudder, and I was not able to reproduce the problem. I tested both oracle JDK and open JDK.
Finally, I tested with the exact same version of Debian / Open JDK, and I got the error. The strangest think is that even with Oracle JDK which I installed afterward, I get the error.
So there's something very fishy going on here. I fear it might be quite hard to debug :/
A workaround it to not use JS field in directive or disable these directive until we find a correction - but depending on your use case, it may be as good as not using Rudder at all.
Updated by François ARMAND about 6 years ago
Some more information on that subject:
- I reproduced it on centos 7, some JVM version.
- it seems to be here:
at com.normation.rudder.services.policies.JsEngine$SandboxSecurityManager.checkPermission(JavascriptEngine.scala:681) at sun.misc.URLClassPath.check(URLClassPath.java:642) at sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1268) at sun.misc.URLClassPath$FileLoader.findResource(URLClassPath.java:1249) at sun.misc.URLClassPath.findResource(URLClassPath.java:215) at java.net.URLClassLoader$2.run(URLClassLoader.java:569) at java.net.URLClassLoader$2.run(URLClassLoader.java:567) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findResource(URLClassLoader.java:566) at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:546) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at com.normation.rudder.services.policies.JsEngine$SandboxSecurityManager.checkPermission(JavascriptEngine.scala:681) at sun.misc.URLClassPath.check(URLClassPath.java:642)
Which correspond to code:
676 override def checkPermission(permission: Permission): Unit = { 677 if(isSandboxed.get) { 678 // there some perms which are hard to specify in that file 679 permission match { 680 // jar and classes 681 case x: FilePermission if( x.getActions == "read" && (x.getName.contains(".class") || x.getName.endsWith(".jar") || x.getName.endsWith(".so") ) ) => // ok 682 // configuration files 683 case x: FilePermission if( x.getActions == "read" && (x.getName.endsWith(".cfg") || x.getName.endsWith(".properties") ) ) => // ok 684 // cert files and config 685 ....
So it is the check itself which cause a check :/
Updated by François ARMAND about 6 years ago
So, more information: the problem is due to security check that are initiated during other security checks. Typically, trying to execute:
.... 681 case x: FilePermission if( x.getActions == "read" && (x.getName.contains(".class") || x.getName.endsWith(".jar") || x.getName.endsWith(".so") ) ) => // ok
needs to load FilePermission class file, abd that need to resolve URL (more permission), and then load a file from fs (more perm).
It seems that something changed in later JVM 1.8 JDK releases about what classes are loaded when. And that results in infinite loops due to impossibility to linearize the graphe of dependance check.
I thought about two resolutions, a correct one and a hacky one. Unfortunatly, I'm not able to implement the correct one for now, and I'm not sure I want to amass the quantity of java priviledges & security to be able to do it.
So, the correct solution is to be able to make the check permission implementation logic not subject to the permission check. This would be correct because the code for the check is statically knows and immutable, so we can assess its security. But I don't know of a way to escape the current security contexte (and it seems logical, since it's kind of the goal of the security context...)
S, the hacky one is to preload relevant classes out of the security context so that we don't have to do the resolution check later on. Of course, this is extremelly fragile.
Updated by François ARMAND about 6 years ago
- Status changed from New to In progress
Updated by François ARMAND about 6 years ago
- Status changed from In progress to Pending technical review
- Assignee changed from François ARMAND to Vincent MEMBRÉ
- Pull Request set to https://github.com/Normation/rudder/pull/2048
Updated by Rudder Quality Assistant about 6 years ago
- Assignee changed from Vincent MEMBRÉ to François ARMAND
Updated by François ARMAND about 6 years ago
- Status changed from Pending technical review to Pending release
Applied in changeset rudder|33e18852804d66a22b822a49319e24ce98c0da1a.
Updated by Vincent MEMBRÉ about 6 years ago
- Status changed from Pending release to Released