Can't profile with scala library
-
- Posts: 2
- Joined: Tue Dec 22, 2009 7:37 pm
Can't profile with scala library
On OS X, 10.6, java version "1.6.0_17"
Using a recent version of scala-library.jar (for instance 2.8.0.Beta1-RC5), Profiling fails with the following:
Exception in thread "main" java.lang.VerifyError: (class: scala/collection/immutable/Vector, method: map0 signature: (Lscala/Function1;Lscala/collection/generic/CanBuildFrom;)Ljava/lang/Object;) Unable to pop operand off an empty stack
at scala.collection.immutable.Vector$.<init>(Vector.scala:28)
at scala.collection.immutable.Vector$.<clinit>(Vector.scala)
at scala.package$.<init>(package.scala:59)
at scala.package$.<clinit>(package.scala)
at scala.Predef$.<init>(Predef.scala:32)
at scala.Predef$.<clinit>(Predef.scala)
at scala.Predef.require(Predef.scala)
at com.example.HelloWorld.main(HelloWorld.java:5)
The code runs fine normally. Here is the example:
package com.example;
public class HelloWorld {
public static void main(String[] args) {
scala.Predef.require(true);
System.out.println("Hello World");
}
}
Any ideas on how to fix this? You can find the scala-library.jar at:
http://article.gmane.org/gmane.comp.lan ... rnals/2607
Thanks!
Using a recent version of scala-library.jar (for instance 2.8.0.Beta1-RC5), Profiling fails with the following:
Exception in thread "main" java.lang.VerifyError: (class: scala/collection/immutable/Vector, method: map0 signature: (Lscala/Function1;Lscala/collection/generic/CanBuildFrom;)Ljava/lang/Object;) Unable to pop operand off an empty stack
at scala.collection.immutable.Vector$.<init>(Vector.scala:28)
at scala.collection.immutable.Vector$.<clinit>(Vector.scala)
at scala.package$.<init>(package.scala:59)
at scala.package$.<clinit>(package.scala)
at scala.Predef$.<init>(Predef.scala:32)
at scala.Predef$.<clinit>(Predef.scala)
at scala.Predef.require(Predef.scala)
at com.example.HelloWorld.main(HelloWorld.java:5)
The code runs fine normally. Here is the example:
package com.example;
public class HelloWorld {
public static void main(String[] args) {
scala.Predef.require(true);
System.out.println("Hello World");
}
}
Any ideas on how to fix this? You can find the scala-library.jar at:
http://article.gmane.org/gmane.comp.lan ... rnals/2607
Thanks!
-
- Posts: 1626
- Joined: Tue Aug 10, 2004 7:52 pm
Re: Can't profile with scala library
What is the exact version of profiler (in particular profiler agent) which you are using?
-
- Posts: 2
- Joined: Tue Dec 22, 2009 7:37 pm
Re: Can't profile with scala library
Version 8.0.20
[YourKit Java Profiler 8.0.20] JVMTI version 3001016d; 14.3-b01-101; Apple Inc.; mixed mode, sharing; Mac OS X; 64-bit JVM
[YourKit Java Profiler 8.0.20] CPU profiling is started
[YourKit Java Profiler 8.0.20] Profiler agent is listening on port 10001...
[YourKit Java Profiler 8.0.20] Launching profiler UI (/Applications/YourKit_Java_Profiler_8.0.20.app/bin/../Contents/MacOS/JavaApplicationStub)
Thanks!
[YourKit Java Profiler 8.0.20] JVMTI version 3001016d; 14.3-b01-101; Apple Inc.; mixed mode, sharing; Mac OS X; 64-bit JVM
[YourKit Java Profiler 8.0.20] CPU profiling is started
[YourKit Java Profiler 8.0.20] Profiler agent is listening on port 10001...
[YourKit Java Profiler 8.0.20] Launching profiler UI (/Applications/YourKit_Java_Profiler_8.0.20.app/bin/../Contents/MacOS/JavaApplicationStub)
Thanks!
-
- Posts: 1626
- Joined: Tue Aug 10, 2004 7:52 pm
Re: Can't profile with scala library
We reproduced the problem and now performing investigation. As soon as fix (or workaround) is available I'll notify you.
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Can't profile with scala library
Hello,
As a workaround, please use "disablealloc" startup option (see here how to specify: http://www.yourkit.com/docs/80/help/add ... ptions.jsp )
We keep on investigating what's going wrong with the bytecode of that particular method.
Best regards,
Anton
As a workaround, please use "disablealloc" startup option (see here how to specify: http://www.yourkit.com/docs/80/help/add ... ptions.jsp )
We keep on investigating what's going wrong with the bytecode of that particular method.
Best regards,
Anton
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Can't profile with scala library
Hello,
We have managed to locate the problem.
Upcoming version 8.0.21 and 9.0 EAP build 5024 will provide a fix, i.e. will work without the need to specify "disablealloc".
Technically, the problem is related with some byte code pattern generated by scala compiler. It seems that the used pattern can easily be transformed to more "regular" pattern used by javac and thus avoid such problems. If you are somehow related with scala team, please let them know we'd be happy to share technical detail with them.
Best regards,
Anton
We have managed to locate the problem.
Upcoming version 8.0.21 and 9.0 EAP build 5024 will provide a fix, i.e. will work without the need to specify "disablealloc".
Technically, the problem is related with some byte code pattern generated by scala compiler. It seems that the used pattern can easily be transformed to more "regular" pattern used by javac and thus avoid such problems. If you are somehow related with scala team, please let them know we'd be happy to share technical detail with them.
Best regards,
Anton
-
- Posts: 18
- Joined: Sun Jan 23, 2005 1:38 pm
Re: Can't profile with scala library
I'm running 5024 and am getting the below warnings. This is with scala-2.8-Beta1-RC3.
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.239]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/StringBuilder setLength
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.557]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/immutable/Vector map0
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.595]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/xml/NamespaceBinding buildString
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.595]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/xml/NamespaceBinding toString
[YourKit Java Profiler 9.0.0 EAP build 5024] [4.330]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/ArrayBuffer remove
[YourKit Java Profiler 9.0.0 EAP build 5024] [9.568]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/FlatHashTable$class newThreshold
[YourKit Java Profiler 9.0.0 EAP build 5024] [10.636]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/util/Random nextASCIIString
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.239]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/StringBuilder setLength
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.557]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/immutable/Vector map0
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.595]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/xml/NamespaceBinding buildString
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.595]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/xml/NamespaceBinding toString
[YourKit Java Profiler 9.0.0 EAP build 5024] [4.330]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/ArrayBuffer remove
[YourKit Java Profiler 9.0.0 EAP build 5024] [9.568]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/FlatHashTable$class newThreshold
[YourKit Java Profiler 9.0.0 EAP build 5024] [10.636]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/util/Random nextASCIIString
-
- Posts: 18
- Joined: Sun Jan 23, 2005 1:38 pm
Re: Can't profile with scala library
Actually my problems are much deeper. My app fully hangs during its early phases. There are verify error exceptions for nearly every worker thread.
What can I do/provide to figure out what the blocker is.
What can I do/provide to figure out what the blocker is.
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Can't profile with scala library
Hello,
The warnings are just warnings. They tell there are some strange byte code patterns detected, and allocation recording will not be registered inside mentioned method(s). There should be no other effects, so the warnings can be simply ignored, especially if you do not perform allocation recording.
As I wrote in my previous email, if we could contact scala compiler guys, we'd try to suggest them to change their byte code patterns to those allowing further byte code instrumentation. We have some ideas on how to fix their byte code, but I do not think it is a good idea since we cannot guarantee that our solution will work in all cases, because we cannot be sure that things which are correct in the cases which we've seen are also correct in other possible cases. The code generated by scala is a black box, so it seems the best solution is to try to cooperate with scala team.
As a workaround, please try options "disablealloc" and "disablecounts" or their combination.
To find out what the problem is, we need a working example which demonstrates the problem. Could you please provide it.
Best regards,
Anton
This is OK and this is expected behavior.I'm running 5024 and am getting the below warnings. This is with scala-2.8-Beta1-RC3.
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.239]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/StringBuilder setLength
The warnings are just warnings. They tell there are some strange byte code patterns detected, and allocation recording will not be registered inside mentioned method(s). There should be no other effects, so the warnings can be simply ignored, especially if you do not perform allocation recording.
As I wrote in my previous email, if we could contact scala compiler guys, we'd try to suggest them to change their byte code patterns to those allowing further byte code instrumentation. We have some ideas on how to fix their byte code, but I do not think it is a good idea since we cannot guarantee that our solution will work in all cases, because we cannot be sure that things which are correct in the cases which we've seen are also correct in other possible cases. The code generated by scala is a black box, so it seems the best solution is to try to cooperate with scala team.
Could you please provide the errors which you get.Actually my problems are much deeper. My app fully hangs during its early phases. There are verify error exceptions for nearly every worker thread.
What can I do/provide to figure out what the blocker is.
As a workaround, please try options "disablealloc" and "disablecounts" or their combination.
To find out what the problem is, we need a working example which demonstrates the problem. Could you please provide it.
Best regards,
Anton
-
- Posts: 18
- Joined: Sun Jan 23, 2005 1:38 pm
Re: Can't profile with scala library
Thanks Anton, I will try what you recommended and also try to get a small failing project.
I posted a reference to this thread on the scala development list. You should have no problems contacting the appropriate people on that forum.
http://old.nabble.com/Scala-and-YourKit ... 70054.html
I posted a reference to this thread on the scala development list. You should have no problems contacting the appropriate people on that forum.
http://old.nabble.com/Scala-and-YourKit ... 70054.html
-
- Posts: 1
- Joined: Thu Dec 31, 2009 5:32 pm
Re: Can't profile with scala library
It's one of those black boxes with full source code publicly available though. I'm guessing you mean in the sense that you have to work with it as it comes? But if you could please share whatever specifics you have in mind, I will look at it if I get time before iulian does.The code generated by scala is a black box, so it seems the best solution is to try to cooperate with scala team.
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Can't profile with scala library
Hello,
new <class>
dup
pop <-- !!! this one is the problem !!!
<push constructor parameters>
invokespecial <constructor>
which does not allow to access created object in instrumented code.
Instead, it seems that the following code pattern should be used (exactly as Javac does):
new <class>
dup
<push constructor parameters>
invokespecial <constructor>
pop
so an instrumenting agent can add instructions right after invokespecial to access the created object.
In short: the 'pop' instruction should stand after 'invokespecial', not before it.
Best regards,
Anton
Yes.I'm guessing you mean in the sense that you have to work with it as it comes?
As already mentioned in the thread http://old.nabble.com/Scala-and-YourKit ... 70054.html by Johannes Rudolph-2, the following is a sequence is generated by scala:But if you could please share whatever specifics you have in mind, I will look at it if I get time before iulian does.
new <class>
dup
pop <-- !!! this one is the problem !!!
<push constructor parameters>
invokespecial <constructor>
which does not allow to access created object in instrumented code.
Instead, it seems that the following code pattern should be used (exactly as Javac does):
new <class>
dup
<push constructor parameters>
invokespecial <constructor>
pop
so an instrumenting agent can add instructions right after invokespecial to access the created object.
In short: the 'pop' instruction should stand after 'invokespecial', not before it.
Best regards,
Anton
-
- Posts: 1
- Joined: Mon Aug 29, 2011 7:55 am
Re: Can't profile with scala library
I just want to say thank you for the information.
It is very valuable for me..newbie here and encountering the same situation.
Thanks and keep up the good work..
It is very valuable for me..newbie here and encountering the same situation.
Thanks and keep up the good work..