Hi, I'm trying to profile a Spark application running in AWS Emr using yourkit. The agent seems to start on both spark driver/spark executors (its displaying the message [YourKit Java Profiler 2022.9-b182] Log file: /tmp/ApplicationMaster-4112.log) but nothing is written in that log file (it is not even created), also none of the ports 10001-10010 will be used by the profiler.
AWS EMR version - 6.10.0
Java version:
openjdk version "17.0.6" 2023-01-17 LTS
OpenJDK Runtime Environment Corretto-17.0.6.10.1 (build 17.0.6+10-LTS)
OpenJDK 64-Bit Server VM Corretto-17.0.6.10.1 (build 17.0.6+10-LTS, mixed mode, sharing)
Spark submit command (in case it's needed):
[*]
YOURKIT_PATH="${OUTPUT_BUCKET}/yourkit/libyjpagent.so"
YOURKIT_AGENT="libyjpagent.so=sampling,logdir=/tmp"
spark-submit --master yarn --deploy-mode cluster \
--files "${YOURKIT_PATH}" \
--conf "spark.driver.extraJavaOptions=-XX:+UseG1GC -agentpath:./${YOURKIT_AGENT} -Djava.rmi.server.useCodebaseOnly=true -Dcom.sun.jndi.ldap.object.trustURLCodebase=false -Dcom.sun.jndi.cosnaming.object.trustURLCodebase=false -Dcom.sun.jndi.rmi.object.trustURLCodebase=false" \
--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -agentpath:./${YOURKIT_AGENT} -Djava.rmi.server.useCodebaseOnly=true -Dcom.sun.jndi.ldap.object.trustURLCodebase=false -Dcom.sun.jndi.cosnaming.object.trustURLCodebase=false -Dcom.sun.jndi.rmi.object.trustURLCodebase=false" \
--conf...
Any help is appreciated
Thanks,
Cornel
Problems when running Yourkit on Spark AWS Emr
-
- Posts: 4
- Joined: Tue Apr 04, 2023 3:10 pm
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Problems when running Yourkit on Spark AWS Emr
Hello Cornel,
Instead of opening agent ports on the ASW side which can be a challenge, please use YourKit connection broker approach:
https://www.yourkit.com/broker/features/
The broker is specially intended to simplify profiling setup in scenarios such as yours.
Best regards,
Anton
Instead of opening agent ports on the ASW side which can be a challenge, please use YourKit connection broker approach:
https://www.yourkit.com/broker/features/
The broker is specially intended to simplify profiling setup in scenarios such as yours.
Best regards,
Anton
-
- Posts: 4
- Joined: Tue Apr 04, 2023 3:10 pm
Re: Problems when running Yourkit on Spark AWS Emr
Hello Anton,
My clusters can't open external connections for security reasons, the broker approach does not work for us.
I fixed the issue, it was something trivial - the cluster manager decided to launch the Spark driver on a core node instead of the master node so basically I was looking on the wrong machine (the master node) and wondering why the ports 1000x are unused.
Tkx,
Cornel.
My clusters can't open external connections for security reasons, the broker approach does not work for us.
I fixed the issue, it was something trivial - the cluster manager decided to launch the Spark driver on a core node instead of the master node so basically I was looking on the wrong machine (the master node) and wondering why the ports 1000x are unused.
Tkx,
Cornel.