Fully featured low overhead profiler for Java EE and Java SE platforms.
Ultimate profiling and monitoring solution for Gradle, Maven, Ant, JUnit and TestNG.
Easy to use performance and memory profiler for .NET framework.

memory inspection for excessive data structure overhead

Moderators: Vladimir Kondratyev, Anton Katilin

memory inspection for excessive data structure overhead

Postby limejuice » Tue Aug 17, 2010 5:51 am

I would like an inspection to detect cases where there is excessive overhead of the data structure compared to the actual data.

For example, in our software someone had used a ConcurrentReaderHashMap() in a class which has many instances. The map only has a few entries per object. The ConcurrentReaderHashMap has more overhead than a regular Hashmap. The datastructure has Segment[] Segment, HashEntry[], HashEntry, child classes.

This inspection would need to be able to differentiate between the data structure classes vs. the client classes. One easy heuristic would be if the retained object's class package matches the parent object's class or if it is an inner class of the parent object's class, then you should consider it as part of the data structure implementation, e.g. java.util.concurrent.ConcurrentHashMap$Segment is inner class for java.util.concurrent.ConcurrentHashMap.

One thing we did to reduce memory waste when using ConcurrentHashMap with a small set of data (< 10 elements in this case) was to tweak the constructor args to reduce the inital capacity and also reduce the concurrency level. The default capacity is 16 and the default concurrency level is 16 (that's 16 writers - readers are unlimited). In one situation, we saved a huge amount of memory waste by setting default capacity to 5 and concurreny level to 1, e.g.

new ConcurrentHashMap<Long, DataSet> (5, 0.75f, 1);

Another solution is to use a more efficient concurrent data structure, e.g. CopyOnWriteArrayList() instead of ConcurrentHashMap when expected number of elements is very small.

Thanks
limejuice
 
Posts: 120
Joined: Mon Sep 25, 2006 5:17 pm

Re: memory inspection for excessive data structure overhead

Postby Anton Katilin » Thu Nov 10, 2016 2:42 pm

Hi,

In the latest EAP build 30 we've added the inspection "Collections with biggest storage overhead":
https://www.yourkit.com/eap

Best regards,
Anton
Anton Katilin
 
Posts: 5576
Joined: Wed Aug 11, 2004 8:37 am


Return to Java Profiler

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 8 guests