memory inspection for excessive data structure overhead

Questions about YourKit Java Profiler
Post Reply
limejuice
Posts: 120
Joined: Mon Sep 25, 2006 5:17 pm

memory inspection for excessive data structure overhead

Post by limejuice »

I would like an inspection to detect cases where there is excessive overhead of the data structure compared to the actual data.

For example, in our software someone had used a ConcurrentReaderHashMap() in a class which has many instances. The map only has a few entries per object. The ConcurrentReaderHashMap has more overhead than a regular Hashmap. The datastructure has Segment[] Segment, HashEntry[], HashEntry, child classes.

This inspection would need to be able to differentiate between the data structure classes vs. the client classes. One easy heuristic would be if the retained object's class package matches the parent object's class or if it is an inner class of the parent object's class, then you should consider it as part of the data structure implementation, e.g. java.util.concurrent.ConcurrentHashMap$Segment is inner class for java.util.concurrent.ConcurrentHashMap.

One thing we did to reduce memory waste when using ConcurrentHashMap with a small set of data (< 10 elements in this case) was to tweak the constructor args to reduce the inital capacity and also reduce the concurrency level. The default capacity is 16 and the default concurrency level is 16 (that's 16 writers - readers are unlimited). In one situation, we saved a huge amount of memory waste by setting default capacity to 5 and concurreny level to 1, e.g.

new ConcurrentHashMap<Long, DataSet> (5, 0.75f, 1);

Another solution is to use a more efficient concurrent data structure, e.g. CopyOnWriteArrayList() instead of ConcurrentHashMap when expected number of elements is very small.

Thanks
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: memory inspection for excessive data structure overhead

Post by Anton Katilin »

Hi,

In the latest EAP build 30 we've added the inspection "Collections with biggest storage overhead":
https://www.yourkit.com/eap

Best regards,
Anton
Post Reply