Can't profile with scala library

[scroyston]

On OS X, 10.6, java version "1.6.0_17"
Using a recent version of scala-library.jar (for instance 2.8.0.Beta1-RC5), Profiling fails with the following:

Exception in thread "main" java.lang.VerifyError: (class: scala/collection/immutable/Vector, method: map0 signature: (Lscala/Function1;Lscala/collection/generic/CanBuildFrom;)Ljava/lang/Object;) Unable to pop operand off an empty stack
at scala.collection.immutable.Vector$.<init>(Vector.scala:28)
at scala.collection.immutable.Vector$.<clinit>(Vector.scala)
at scala.package$.<init>(package.scala:59)
at scala.package$.<clinit>(package.scala)
at scala.Predef$.<init>(Predef.scala:32)
at scala.Predef$.<clinit>(Predef.scala)
at scala.Predef.require(Predef.scala)
at com.example.HelloWorld.main(HelloWorld.java:5)


The code runs fine normally. Here is the example:

package com.example;

public class HelloWorld {
public static void main(String[] args) {
scala.Predef.require(true);
System.out.println("Hello World");
}
}


Any ideas on how to fix this? You can find the scala-library.jar at:
http://article.gmane.org/gmane.comp.lan ... rnals/2607

Thanks!

[Vladimir Kondratyev]

What is the exact version of profiler (in particular profiler agent) which you are using?

[scroyston]

Version 8.0.20

[YourKit Java Profiler 8.0.20] JVMTI version 3001016d; 14.3-b01-101; Apple Inc.; mixed mode, sharing; Mac OS X; 64-bit JVM
[YourKit Java Profiler 8.0.20] CPU profiling is started
[YourKit Java Profiler 8.0.20] Profiler agent is listening on port 10001...
[YourKit Java Profiler 8.0.20] Launching profiler UI (/Applications/YourKit_Java_Profiler_8.0.20.app/bin/../Contents/MacOS/JavaApplicationStub)


Thanks!

[Vladimir Kondratyev]

We reproduced the problem and now performing investigation. As soon as fix (or workaround) is available I'll notify you.

[Anton Katilin]

Hello,

As a workaround, please use "disablealloc" startup option (see here how to specify: http://www.yourkit.com/docs/80/help/add ... ptions.jsp )

We keep on investigating what's going wrong with the bytecode of that particular method.

Best regards,
Anton

[Anton Katilin]

Hello,

We have managed to locate the problem.

Upcoming version 8.0.21 and 9.0 EAP build 5024 will provide a fix, i.e. will work without the need to specify "disablealloc".

Technically, the problem is related with some byte code pattern generated by scala compiler. It seems that the used pattern can easily be transformed to more "regular" pattern used by javac and thus avoid such problems. If you are somehow related with scala team, please let them know we'd be happy to share technical detail with them.

Best regards,
Anton

[memelet]

I'm running 5024 and am getting the below warnings. This is with scala-2.8-Beta1-RC3.

[YourKit Java Profiler 9.0.0 EAP build 5024] [3.239]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/StringBuilder setLength
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.557]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/immutable/Vector map0
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.595]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/xml/NamespaceBinding buildString
[YourKit Java Profiler 9.0.0 EAP build 5024] [3.595]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/xml/NamespaceBinding toString
[YourKit Java Profiler 9.0.0 EAP build 5024] [4.330]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/ArrayBuffer remove
[YourKit Java Profiler 9.0.0 EAP build 5024] [9.568]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/FlatHashTable$class newThreshold
[YourKit Java Profiler 9.0.0 EAP build 5024] [10.636]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/util/Random nextASCIIString

[memelet]

Actually my problems are much deeper. My app fully hangs during its early phases. There are verify error exceptions for nearly every worker thread.

What can I do/provide to figure out what the blocker is.

[Anton Katilin]

Hello,

I'm running 5024 and am getting the below warnings. This is with scala-2.8-Beta1-RC3.

[YourKit Java Profiler 9.0.0 EAP build 5024] [3.239]: warning: cannot inject object allocation callback due to a bad bytecode pattern (scala?): scala/collection/mutable/StringBuilder setLength

This is OK and this is expected behavior.

The warnings are just warnings. They tell there are some strange byte code patterns detected, and allocation recording will not be registered inside mentioned method(s). There should be no other effects, so the warnings can be simply ignored, especially if you do not perform allocation recording.

As I wrote in my previous email, if we could contact scala compiler guys, we'd try to suggest them to change their byte code patterns to those allowing further byte code instrumentation. We have some ideas on how to fix their byte code, but I do not think it is a good idea since we cannot guarantee that our solution will work in all cases, because we cannot be sure that things which are correct in the cases which we've seen are also correct in other possible cases. The code generated by scala is a black box, so it seems the best solution is to try to cooperate with scala team.

Actually my problems are much deeper. My app fully hangs during its early phases. There are verify error exceptions for nearly every worker thread.

What can I do/provide to figure out what the blocker is.

Could you please provide the errors which you get.

As a workaround, please try options "disablealloc" and "disablecounts" or their combination.

To find out what the problem is, we need a working example which demonstrates the problem. Could you please provide it.

Best regards,
Anton

[memelet]

Thanks Anton, I will try what you recommended and also try to get a small failing project.

I posted a reference to this thread on the scala development list. You should have no problems contacting the appropriate people on that forum.

http://old.nabble.com/Scala-and-YourKit ... 70054.html

[extempore]

The code generated by scala is a black box, so it seems the best solution is to try to cooperate with scala team.

It's one of those black boxes with full source code publicly available though. I'm guessing you mean in the sense that you have to work with it as it comes? But if you could please share whatever specifics you have in mind, I will look at it if I get time before iulian does.

[Anton Katilin]

Hello,

I'm guessing you mean in the sense that you have to work with it as it comes?

Yes.

But if you could please share whatever specifics you have in mind, I will look at it if I get time before iulian does.

As already mentioned in the thread http://old.nabble.com/Scala-and-YourKit ... 70054.html by Johannes Rudolph-2, the following is a sequence is generated by scala:

new <class>
dup
pop <-- !!! this one is the problem !!!
<push constructor parameters>
invokespecial <constructor>

which does not allow to access created object in instrumented code.

Instead, it seems that the following code pattern should be used (exactly as Javac does):

new <class>
dup
<push constructor parameters>
invokespecial <constructor>
pop

so an instrumenting agent can add instructions right after invokespecial to access the created object.

In short: the 'pop' instruction should stand after 'invokespecial', not before it.

Best regards,
Anton

[dabby]

I just want to say thank you for the information.
It is very valuable for me..newbie here and encountering the same situation.
Thanks and keep up the good work..