Hi,
I am using yourkit 11.0.3 to profile my JVM which is running with 7.0.4 version, the JVM runs the application I am profliing which includes ActiveMQ whenever I attach to the JVM, or connect through JVMTI I see that the app threads become blocked, the CPU consumption drops as result and basically, I can't profile my application since it stops working almost completely
The following is the summary of my App's JVM:
-Djava.endorsed.dirs=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../tomcat/endorsed
-Dcatalina.base=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../servers/server-0
-Dcatalina.home=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../tomcat
-Djava.io.tmpdir=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../servers/server-0/temp
-Doo.home=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/..
-Doo.node=servers/server-0
-Doo.lib=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../lib
-XX:+HeapDumpOnOutOfMemoryError
-XX:MaxPermSize=256m
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Djmx.http.port=39900
-Dcom.sun.management.jmxremote.port=39600
-Djmx.invoke.getters=true
-Dlog4j.configuration=file:C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../servers/server-0/conf/log4j.properties
-Dmgmt.uuid=server-0
-Dspring.profiles.active=nologin
-Djava.security.policy=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../tomcat/conf/catalina.policy
-XX:+HeapDumpOnOutOfMemoryError
-Xms512m
-Xmx1024m
-XX:+UseParallelGC
-Djavax.net.ssl.trustStore=
-Djavax.net.ssl.trustStorePassword=
-Djavax.net.ssl.trustStoreType=jks
-Dhttp.port=8080
-Dtomcat.https.port=8443
-Daf.central.url=
-Daf.protocol=
-Xss1024k
-Dlwsso.init.string=
-Dlwsso.trusted.domain=
-Dlwsso.protected.domains=
-Dlwsso.overwrite.xml=
-Didm.request.header=
-Dworker.uuid=307cdefa-3a4b-416d-a00e-3374611e185f
-Dbroker.uuid=2f72d74b-14ef-4b6c-9851-0113b241c2de
-Dbroker.jms.port=6565
-Dbroker.jms.http.port=6566
-Dbroker.jms.https.port=6567
-Dbroker.web.http.port=8080
-Dbroker.web.https.port=8443
-Dmgmt.url=http://localhost:8080/oo
-Djms.prefetch=500
-Dfile.encoding=Cp1252
-Duser.language=en
-Djava.library.path=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../lib
-Dwrapper.key=nPQmPD6X1r6O-9lsI0rrmKKu55jSroxt
-Dwrapper.port=32000
-Dwrapper.jvm.port.min=31000
-Dwrapper.jvm.port.max=31999
-Dwrapper.pid=7288
-Dwrapper.version=3.5.14-pro
-Dwrapper.native_library=wrapper
-Dwrapper.cpu.timeout=10
-Dwrapper.jvmid=1
-Dwrapper.lang.domain=wrapper
and the system properties:
ava.specification.vendor=Oracle Corporation
java.specification.version=1.7
java.vendor=Oracle Corporation
java.vendor.url=http://java.oracle.com/
java.vendor.url.bug=http://bugreport.sun.com/bugreport/
java.version=1.7.0_04
java.vm.info=mixed mode
java.vm.name=Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.name=Java Virtual Machine Specification
java.vm.specification.vendor=Oracle Corporation
java.vm.specification.version=1.7
java.vm.vendor=Oracle Corporation
java.vm.version=23.0-b21
javax.net.ssl.trustStore=
javax.net.ssl.trustStorePassword=
javax.net.ssl.trustStoreType=jks
jboss.i18n.generate-proxies=true
jms.prefetch=500
jmx.http.port=39900
jmx.invoke.getters=true
line.separator=\r\n
log4j.configuration=file:C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../servers/server-0/conf/log4j.properties
lwsso.init.string=
lwsso.overwrite.xml=
lwsso.protected.domains=
lwsso.trusted.domain=
mgmt.url=http://localhost:8080/oo
mgmt.uuid=server-0
oo.home=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/..
oo.lib=C:/K2_BM/oo-win64-24.6.11-SNAPSHOT/bin/../lib
oo.node=servers/server-0
os.arch=amd64
os.name=Windows Server 2008 R2
os.version=6.1
package.access=sun.,org.apache.catalina.,org.apache.coyote.,org.apache.tomcat.,org.apache.jasper.
package.definition=sun.,java.,org.apache.catalina.,org.apache.coyote.,org.apache.tomcat.,org.apache.jasper.
path.separator=;
server.loader=
shared.loader=
spring.profiles.active=nologin
sun.arch.data.model=64
sun.boot.class.path=C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\lib\resources.jar;C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\lib\rt.jar;C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\lib\sunrsasign.jar;C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\lib\jsse.jar;C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\lib\jce.jar;C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\lib\charsets.jar;C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\lib\jfr.jar;C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\classes
sun.boot.library.path=C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\java\bin
sun.cpu.endian=little
sun.cpu.isalist=amd64
sun.desktop=windows
sun.io.unicode.encoding=UnicodeLittle
sun.java.command=org.tanukisoftware.wrapper.WrapperStartStopApp org.apache.catalina.startup.Bootstrap 1 start org.apache.catalina.startup.Bootstrap TRUE 1 stop
sun.java.launcher=SUN_STANDARD
sun.jnu.encoding=Cp1252
sun.management.compiler=HotSpot 64-Bit Tiered Compilers
sun.os.patch.level=Service Pack 1
tomcat.https.port=8443
tomcat.util.buf.StringCache.byte.enabled=true
tomcat.util.scan.DefaultJarScanner.jarsToSkip=bootstrap.jar,commons-daemon.jar,tomcat-juli.jar,annotations-api.jar,el-api.jar,jsp-api.jar,servlet-api.jar,catalina.jar,catalina-ant.jar,catalina-ha.jar,catalina-tribes.jar,jasper.jar,jasper-el.jar,ecj-*.jar,tomcat-api.jar,tomcat-util.jar,tomcat-coyote.jar,tomcat-dbcp.jar,tomcat-i18n-en.jar,tomcat-i18n-es.jar,tomcat-i18n-fr.jar,tomcat-i18n-ja.jar,tomcat-juli-adapters.jar,catalina-jmx-remote.jar,catalina-ws.jar,tomcat-jdbc.jar,commons-beanutils*.jar,commons-codec*.jar,commons-collections*.jar,commons-dbcp*.jar,commons-digester*.jar,commons-fileupload*.jar,commons-httpclient*.jar,commons-io*.jar,commons-lang*.jar,commons-logging*.jar,commons-math*.jar,commons-pool*.jar,jstl.jar,geronimo-spec-jaxrpc*.jar,wsdl4j*.jar,ant.jar,ant-junit*.jar,aspectj*.jar,jmx.jar,h2*.jar,hibernate*.jar,httpclient*.jar,jmx-tools.jar,jta*.jar,log4j*.jar,mail*.jar,slf4j*.jar,xercesImpl.jar,xmlParserAPIs.jar,xml-apis.jar,dnsns.jar,ldapsec.jar,localedata.jar,sunjce_provider.jar,sunmscapi.jar,sunpkcs11.jar,jhall.jar,tools.jar,sunec.jar,zipfs.jar,apple_provider.jar,AppleScriptEngine.jar,CoreAudio.jar,dns_sd.jar,j3daudio.jar,j3dcore.jar,j3dutils.jar,jai_core.jar,jai_codec.jar,mlibwrapper_jai.jar,MRJToolkit.jar,vecmath.jar,junit.jar,junit-*.jar,ant-launcher.jar
user.country=US
user.dir=C:\K2_BM\oo-win64-24.6.11-SNAPSHOT\bin
user.home=C:\Users\cnc
user.language=en
user.name=cnc
user.script=
user.timezone=Australia/Sydney
user.variant=
worker.uuid=307cdefa-3a4b-416d-a00e-3374611e185f
wrapper.cpu.timeout=10
wrapper.java.pid=12148
wrapper.jvm.port.max=31999
wrapper.jvm.port.min=31000
wrapper.jvmid=1
wrapper.key=nPQmPD6X1r6O-9lsI0rrmKKu55jSroxt
wrapper.lang.domain=wrapper
wrapper.native_library=wrapper
wrapper.pid=7288
wrapper.port=32000
wrapper.version=3.5.14-pro
Thanks in advance for the support
Doron
Yourkit causing thread blocks
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Yourkit causing thread blocks
Hello Doron
http://www.yourkit.com/download
I cannot see which profiler agent options you have specified.
Did you use attaching the agent to a running JVM?
http://www.yourkit.com/docs/11/help/attach_agent.jsp
If yes, please try starting it with the agent instead:
http://www.yourkit.com/docs/11/help/run ... ofiler.jsp
If you already start the JVM with the profiler agent, which agent startup options have you specified?
http://www.yourkit.com/docs/11/help/startup_options.jsp
Best regards,
Anton
Please use the latest build instead (11.0.8 at the moment):I am using yourkit 11.0.3
http://www.yourkit.com/download
I cannot see which profiler agent options you have specified.
Did you use attaching the agent to a running JVM?
http://www.yourkit.com/docs/11/help/attach_agent.jsp
If yes, please try starting it with the agent instead:
http://www.yourkit.com/docs/11/help/run ... ofiler.jsp
If you already start the JVM with the profiler agent, which agent startup options have you specified?
http://www.yourkit.com/docs/11/help/startup_options.jsp
Best regards,
Anton
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Yourkit causing thread blocks
Also, which profiling modes do you use: CPU sampling/tracing, allocation recording etc.?
Is it a local profiling (the profiled application and the profiler UI run on the same machine) or a remote one (the profiled application and the profiler UI run on different machines)?
Is it a local profiling (the profiled application and the profiler UI run on the same machine) or a remote one (the profiled application and the profiler UI run on different machines)?
-
- Posts: 6
- Joined: Wed Feb 23, 2011 2:41 pm
Re: Yourkit causing thread blocks
Thanks for the quick reply, I will try with 11.0.8 as well
Regarding your questions:
I tried both with attaching and with the agent (no parameters other than port)
sampling or tracing is irrelevant since this happens as soon as I connect
When I work with disableall option the problem seems to go away, still it disables some functionality I work with (telemetry, etc')
Regarding your questions:
I tried both with attaching and with the agent (no parameters other than port)
sampling or tracing is irrelevant since this happens as soon as I connect
When I work with disableall option the problem seems to go away, still it disables some functionality I work with (telemetry, etc')
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Yourkit causing thread blocks
"disableall" is equal to the following set of individual options: disablealloc, disabletracing, disablej2ee, disableexceptiontelemetry, disablestacktelemetry, builtinprobes=none
Please see what which of them means here:
http://www.yourkit.com/docs/11/help/startup_options.jsp
You can try them one by one to see which one makes the biggest improvement.
My guess it can be disablestacktelemetry or builtinprobes=none
Please see what which of them means here:
http://www.yourkit.com/docs/11/help/startup_options.jsp
You can try them one by one to see which one makes the biggest improvement.
My guess it can be disablestacktelemetry or builtinprobes=none
-
- Posts: 3
- Joined: Tue Jan 21, 2020 10:30 am
Re: Yourkit causing thread blocks
Hello,
I'm experiencing the same behavior using 2019.8 b137 and java 11:
java version "11.0.5" 2019-10-15 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.5+10-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.5+10-LTS, mixed mode))
I'm trying to profile a public available web application developed with spring boot (https://github.com/spring-projects/spring-petclinic) during the execution of a load test suite written by me.
What happens is that when enable monitor profiling the application slows down and I can see it also from the dashboard of the load testing platform because the rate of requests per second drops dramatically.
I link a screenshot to show the performance bottleneck. Can anybody help me understanding the reason of such behavior?
https://drive.google.com/file/d/12llGhG ... sp=sharing
Thank you!
I'm experiencing the same behavior using 2019.8 b137 and java 11:
java version "11.0.5" 2019-10-15 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.5+10-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.5+10-LTS, mixed mode))
I'm trying to profile a public available web application developed with spring boot (https://github.com/spring-projects/spring-petclinic) during the execution of a load test suite written by me.
What happens is that when enable monitor profiling the application slows down and I can see it also from the dashboard of the load testing platform because the rate of requests per second drops dramatically.
I link a screenshot to show the performance bottleneck. Can anybody help me understanding the reason of such behavior?
https://drive.google.com/file/d/12llGhG ... sp=sharing
Thank you!
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Yourkit causing thread blocks
Hi,
Monitor profiling can add overhead, it's normal.
Do you use other profiling modes in parallel e.g. sampling? If you do please try to perform monitor profiling alone.
Please try to also try to stop stack telemetry during monitor profiling . See "Start/stop stack telemetry":
https://www.yourkit.com/docs/java/help/threads.jsp
Alternatively, instead of monitor profiling, you can use stack telemetry samples to analyze thread blocking/waiting states.
Best regards,
Anton
Monitor profiling can add overhead, it's normal.
Do you use other profiling modes in parallel e.g. sampling? If you do please try to perform monitor profiling alone.
Please try to also try to stop stack telemetry during monitor profiling . See "Start/stop stack telemetry":
https://www.yourkit.com/docs/java/help/threads.jsp
Alternatively, instead of monitor profiling, you can use stack telemetry samples to analyze thread blocking/waiting states.
Best regards,
Anton