Docker Swarm Profiling Java

[ggr]

Hi,

In your company our Microservices runs within the clustering engine Docker-Swarm.
Each Service can be scaled up and down.

e.g.
- Host A
-- Service 1
-- Service 2
-- Service 3
- Host B
-- Service 1
-- Service 1
-- Service 5
- Host A
-- Service 1
-- Service 4
-- Service 5

So in this example above, the Service1 exists four times. It is even started twice on HostB.

Sometimes we have to monitor a single service on demand. My problem no is to connect to those service.
At first I thought that this can be used:
https://www.yourkit.com/docs/java/help/remote_profiling.jsp

But even if the agent get started with the Java-Application, there is no open Port into the container for the UI to connect to. I can't publish those port outside the container, because it can get in conflict with other containers publishing the same port. So that's not really an option for the cluster.

The only way I see here is like other APM Tools do it: The UI runs and the Agent connects to it.

Is there a way to to this? Or another solution?

[Anton Katilin]

There is absolutely no need to set a fixed port, let instead the agent choose it automatically: https://www.yourkit.com/docs/java/help/ ... s.jsp#port

Do you have ssh access to those machines? The profiler UI provides basic support for ssh tunneling, so you can easily connect to a remote JVM without exposing any additional ports:
https://www.yourkit.com/docs/java/help/ ... jsp#remote

You can also perform profiling activities (start, stop profiling) via the command line tools, then capture snapshots and manually transfer them to a developer's machine for opening them in the profiler UI.
https://www.yourkit.com/docs/java/help/ ... e_tool.jsp

[ggr]

Hi,

no, we currently have no ssh access. This is not that easy, because a Service can scale.
So one service can run just once (that would be easy) or twenty times (even several times on the same machine but in different containers).

Profiling in the command line and than transfer to the profiler UI sounds very nice.


But maybe you should think over the whole idea of the UI connection to the agent, because in environments like ours, where one service can scale to several instances (each run in its own container), there is no easy way, that the UI connects to the server. In such cases, it would be helpful if the agent connects to the server. Than there is no need to open any ports on the service-machines. Just the port of the UI must be open.

SSH has the same problem. It's not easily possible to connect to exactly the one instance I need.

[Anton Katilin]

This approach may indeed be helpful in such situations. Perhaps the agent should connect to an intermediate server/service, not directly to the UI. Anyway, the agent should explicitly be instructed (configured) where to connect by supplying target IP/host and port.

[ggr]

Connecting to an intermediate server/service seems sounds like a good solution.
But it has to be implemented...