Remote Java Application profiling- Profiler is not connecting Agent

[sangy]

Hi,
Using YK profiler with both agent and profiler version 2023.9-b104.
I can see agent is running on remote machine at port 10001. I can telnet from (local) profiler maching to remote machine.
Following is agent log file:
------------------------------------------------
23.9-104 0.000: Log file: /<home>/log/<application>-18770.log
23.9-104 0.000: Command line:
/<home>/java/jdk8u231/bin/java
-agentpath:<Home>/YourKit-JavaProfiler-2023.9/bin/linux-x86-64/libyjpagent.so=listen=all
-jar
<path>.jar
23.9-104 0.001: Executable: /home/java/jdk8u231/bin/java
23.9-104 0.001: uname:
sysname: Linux
release: 3.10.0-1160.105.1.el7.x86_64
version: #1 SMP Mon Nov 6 06:58:51 EST 2023
machine: x86_64
23.9-104 0.001: Linux; linux-x86-64; 64-bit process
23.9-104 0.001: hid: 360261994943Z2X3Z2489XZU30YW454394 64-bit machine
23.9-104 0.002: RAM: 15876 MB
23.9-104 0.002: Today is 20240121
23.9-104 0.002: full_build_stamp: 2023.9.104 202401111333
23.9-104 0.003: Agent library path: <Home>/YourKit-JavaProfiler-2023.9/bin/linux-x86-64/libyjpagent.so
23.9-104 0.003: getTimeNsInfo: "monotonic"
23.9-104 0.003: Agent_OnLoad: orig_options=listen=all
full_options=listen=all
23.9-104 0.003: y_setenv: _YJP_AGENT_LOADED=18770
23.9-104 0.003: jvmtiVersion: 30010203
23.9-104 0.003: phase: 1
23.9-104 0.003: vendor: Oracle Corporation
23.9-104 0.003: version: 25.231-b11
23.9-104 0.003: specVersion: 1.8
23.9-104 0.003: info: mixed mode, sharing
23.9-104 0.003: heap_sampling: capability is not available
23.9-104 0.003: virtual_threads: capability is not available (isAttached=0, version=25.231-b11)
23.9-104 0.003: asynchronous sampling: available
23.9-104 0.006: inside Kubernetes: 0
23.9-104 0.006: session GUID: HLHdmLFbKHmmy7y0J8JP
23.9-104 0.008: line_method: 1
23.9-104 0.008: can_generate_early_class_hook_events: optional capability is not available
23.9-104 0.025: Reusing existing jar file: /tmp/yjp202401111333.jar
23.9-104 0.025: JVMTI 30010203; 25.231-b11; Oracle Corporation; mixed mode, sharing; Java version: 8
23.9-104 0.050: warning: possible performance issue: probe class pattern is "*", none of @InstanceOf, @This, @RetransformIfInstanceOf specified: com.yourkit.probes.builtin.JUnitTests$Run_Probe
23.9-104 0.050: warning: possible performance issue: probe class pattern is "*", none of @InstanceOf, @This, @RetransformIfInstanceOf specified: com.yourkit.probes.builtin.JUnitTests$SetUp_Probe
23.9-104 0.050: warning: possible performance issue: probe class pattern is "*", none of @InstanceOf, @This, @RetransformIfInstanceOf specified: com.yourkit.probes.builtin.JUnitTests$TearDown_Probe
23.9-104 0.064: Successfully registered probes: 28
Registered: .AgentLoadingSupport
Registered: .AsyncChannels
Registered: .AwtEvents
Registered: .Cassandra
Registered: .CassandraLW
Registered: .Databases
Registered: .DatabasesLW
Registered: .DirectoryStreams
Registered: .Files
Registered: .FilesLW
Registered: .HBase
Registered: .HBaseLW
Registered: .JNDI
Registered: .JPA_DataNucleus
Registered: .JPA_EclipseLink
Registered: .JPA_Hibernate
Registered: .JPA_OpenJPA
Registered: .JUnitTests
Registered: .MongoDB
Registered: .MongoDBLW
Registered: .Processes
Registered: .Servlets
Registered: .ServletsLW
Registered: .Sockets
Registered: .SocketsLW
Registered: .Sync
Registered: .TestNg
Registered: .Threads
23.9-104 0.098: async_sampling: native libs: 15
23.9-104 0.683: agentStart: 1
23.9-104 0.683: agentStart: 2
23.9-104 0.685: agentStart: OK
23.9-104 0.728: tracing=off: large method: 3945 sun.nio.cs.StandardCharsets$Aliases init(java.lang.Object[])
23.9-104 0.927: vmInit: 1
23.9-104 0.929: vmInit: async-sampling initialization
23.9-104 0.930: vmInit: 2
23.9-104 0.930: agentInit: 1
23.9-104 0.930: agentInit: isAttached=0
23.9-104 0.943: agentInit: Core loaded
23.9-104 1.044: tracing=off: large method: 2252 jdk.internal.org.objectweb.asm.Frame execute(int, int, jdk.internal.org.objectweb.asm.ClassWriter, jdk.internal.org.objectweb.asm.Item)
23.9-104 1.265: setTriggers:

23.9-104 1.266: setTriggers: to unregister: 0
23.9-104 1.266: setTriggers: method listeners to register: 0
23.9-104 1.266: setTriggers: OK
23.9-104 1.266: ourUnsafe: true
23.9-104 1.287: agentInit: Core.init() called
23.9-104 1.287: [ssl-config] generating SSL certificate...
23.9-104 1.287: [ssl-config] OpenSSL 3.0.12 24 Oct 2023
23.9-104 1.287: agentInit: OK
23.9-104 1.287: vmInit: 3
23.9-104 1.287: onStartProfiling: entering...
23.9-104 1.287: onStartProfiling: done
23.9-104 1.287: vmInit: OK
23.9-104 1.542: [m: 12084 130] total classes: processed=1000 instrumented=910 279ms
23.9-104 1.603: SP: pin java.lang.ClassNotFoundException
23.9-104 1.813: SP: pin java.lang.NoSuchMethodException
23.9-104 1.822: GC counter installed to PS Scavenge
23.9-104 1.822: GC counter installed to PS MarkSweep
23.9-104 1.906: SP: pin java.io.FileNotFoundException
23.9-104 1.906: SP: pin java.security.PrivilegedActionException
23.9-104 2.796: SP: pin java.lang.IllegalArgumentException
23.9-104 2.806: SP: pin java.security.AccessControlException
23.9-104 3.055: tracing=off: large method: 4120 sun.text.resources.FormatData getContents()
23.9-104 3.071: tracing=off: large method: 8742 sun.util.resources.CurrencyNames getContents()
23.9-104 3.157: [ssl-config] generating SSL certificate done: ctx 2, key 1863, serial 0, cert 2, sign 3, use 0
23.9-104 3.157: [server] listening; port=10001; ip=::; url=https://[::1]:10001
23.9-104 3.157: [shmem] permissions: set_unrestricted
23.9-104 3.157: [shmem] creating shared_memory_object
23.9-104 3.157: [shmem] get_size
23.9-104 3.157: [shmem] get size result=1, size=106500
23.9-104 3.157: [shmem] creating mapped_region
23.9-104 3.157: [shmem] mapped address=00007F9D4AF5B000, mapped size=106500, area size=106500
23.9-104 3.166: *** HINT ***: To get profiling results, connect to the application from the profiler UI
23.9-104 3.392: SP: pin java.lang.NoSuchMethodError
23.9-104 3.953: [m: 12422 258] total classes: processed=2000 instrumented=1772 605ms
23.9-104 4.061: SP: pin java.lang..
23.9-104 4.600: SP: pin javax.naming.NoInitialContextException
23.9-104 5.490: SP: pin java.lang.IncompatibleClassChangeError
23.9-104 7.222: [m: 12426 439] total classes: processed=3000 instrumented=2710 926ms
23.9-104 7.388: SP: pin java.lang.InstantiationException
23.9-104 8.004: tracing=off: large method: 2346 com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver next()
23.9-104 8.124: SP: pin com.sun.org.apache.xerces.internal.impl.XMLEntityScanner$1
23.9-104 8.974: tracing=off: large method: 17796 sun.util.resources.TimeZoneNames getContents()
23.9-104 9.354: SP: pin java.net.MalformedURLException
23.9-104 9.379: SP: pin javax.validation.NoProviderFoundException
23.9-104 10.264: [m: 12630 626] total classes: processed=4000 instrumented=3574 1366ms
23.9-104 11.435: tracing=off: large method: 5105 org.springframework.asm.ClassReader readCode(org.springframework.asm.MethodVisitor, org.springframework.asm.Context, int)
23.9-104 13.967: [m: 12638 778] total classes: processed=5000 instrumented=4402 1760ms
23.9-104 19.052: [m: 12642 817] total classes: processed=6000 instrumented=5272 2123ms
23.9-104 19.181: tracing=off: large method: 2305 org.springframework.asm.Frame execute(int, int, org.springframework.asm.Symbol, org.springframework.asm.SymbolTable)
23.9-104 19.232: SP: pin org.springframework.cglib.core.ClassNameReader$EarlyExitException
23.9-104 19.786: SP: pin java.lang.NoSuchFieldException
23.9-104 20.584: SP: pin java.lang.UnsatisfiedLinkError
23.9-104 20.585: SP: pin org.apache.tomcat.jni.LibraryNotFoundError
23.9-104 22.797: [m: 12648 947] total classes: processed=7000 instrumented=6147 2454ms
23.9-104 24.134: tracing=off: large method: 2665 org.apache.catalina.core.StandardWrapperValve invoke(org.apache.catalina.connector.Request, org.apache.catalina.connector.Response)
23.9-104 24.166: tracing=off: large method: 2050 org.apache.tomcat.websocket.WsWebSocketContainer connectToServerRecursive(org.apache.tomcat.websocket.ClientEndpointHolder, javax.websocket.ClientEndpointConfig, java.net.URI, java.util.Set)
23.9-104 24.478: SP: pin javax.naming.NamingException
23.9-104 27.055: warning: unexpected bytecode was detected. Allocation recording for this method will be disabled. scala.util.PropertiesTrait versionOf$1(java.lang.String, int)
23.9-104 30.538: [m: 12916 1244] total classes: processed=8000 instrumented=7086 2876ms
23.9-104 31.599: tracing=off: large method: 3807 com.sun.crypto.provider.SunJCE$1 run()
23.9-104 31.651: Warning: sun.nio.fs.UnixSecureDirectoryStream.<init>(sun.nio.fs.UnixPath, long, int, java.nio.file.DirectoryStream$Filter) was not instrumented by probe .DirectoryStreams$Init: unsupported bytecode and probe combination (1)
23.9-104 31.668: SP: pin javax.crypto.BadPaddingException
23.9-104 31.669: SP: pin java.security.SignatureException
23.9-104 32.092: tracing=off: large method: 7690 com.sun.crypto.provider.DESCrypt expandKey(byte[])
23.9-104 34.831: SP: pin com.amazonaws.SdkClientException
23.9-104 40.167: [m: 13122 1362] total classes: processed=9000 instrumented=8008 3323ms
23.9-104 43.029: SP: pin org.springframework.beans.factory.NoSuchBeanDefinitionException
23.9-104 45.131: tracing=off: large method: 4947 io.swagger.v3.core.jackson.ModelResolver resolve(io.swagger.v3.core.converter.AnnotatedType, io.swagger.v3.core.converter.ModelConverterContext, java.util.Iterator)
23.9-104 46.148: [m: 13131 1480] total classes: processed=10000 instrumented=8889 3721ms
23.9-104 50.120: [m: 13529 1532] total classes: processed=11000 instrumented=9814 4162ms
23.9-104 1909.477: [http] ::ffff:127.0.0.1 - - - - handshake failed: http request (SSL routines) {}
23.9-104 1936.456: [http] ::ffff:127.0.0.1 POST "/yjp/api/v1/getStatus" 200 718 {"User-Agent":"curl/7.29.0","Host":"localhost:10001","Accept":"*/*"}
------------------------------------------------
And Following is reply for get status from the local machine to remote host where YK Agent is running.
----------------------------------------------------------------------------
curl -k https://<remothost>:10001/yjp/api/v1/getStatus

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 585 100 585 0 0 5570 0 --:--:-- --:--:-- --:--:-- 5625{
"allocationRecording" : false,
"allocationRecordingSettings" : {
"mode" : "",
"recordEach" : 0,
"sizeLimit" : 0,
"heapSamplingInterval" : 0
},
"asyncSamplingCpu" : false,
"asyncSamplingPeriodic" : false,
"callCounting" : false,
"cpuProfiling" : false,
"exceptionProfiling" : true,
"monitorProfiling" : false,
"pid" : 18770,
"sampling" : false,
"appName" : "<ApplicationId>",
"stackTelemetry" : true,
"tracing" : false,
"agentVersion" : "YourKit Java Profiler 2023.9-b104"
}

-----------------------------------------------------------------

I am using direct connection from the profiler to application. But profiler is not able to find the application.

Can you hepl me?

[Vladimir Kondratyev]

The reason for the problem might be that the machine where the UI runs is behind a proxy, or the profiler UI is incorrectly detecting the proxy settings. Please try to configure (or disable the proxy) as described in the https://www.yourkit.com/docs/java-profi ... /proxy.jsp help article.

Best regards,
Vladimir Kondratyev
YourKit GmbH
http://www.yourkit.com
"Don't get lost in data, get information!"

[sangy]

Hi Vladimir Kondratyev ,
Thanks for replying.

Proxy setting is correct, no issues with that.
Anything else that you think I can look at?

Please note I am using evaluation version.

[Vladimir Kondratyev]

Proxy setting is correct, no issues with that.

Do you have a proxy? Even if you do not have a proxy, did you try to specify "No proxy" option?

Anything else that you think I can look at?

Please do the following:

* Close profiler UI
* Remove all the files in ~/.yjp/log directoty
* Start profiler UI and try to connect to https://<remothost>:10001. Your remove connection should have single port 10001 as a range.
* Send us entire profiler UI log to support@yourkit.com

[sangy]

Hi,
The issue was , profiler UI was using proxy . Setting it to "no Proxy" did the trick.

Thanks
Sangram

[Vladimir Kondratyev]

Sangram, thank you for confirming! I'm closing this as resolved.