menu

Monday, May 4, 2015

String and String Related Object's Memory Usage in Java

 Introduction:

     Strings are everywhere and almost every application heavily use them. For example, for web applications all the data coming from the view are firstly Strings, after taking them into the application they are converted to their real data types. Hence it's crucial to understand the memory consumption of String and related Objects. In this write we'll examine the history of String Object in Java, through Java versions looking into the some String methods like substring, and then we try to figure out StringBuilder Object's memory consumption and finally see how we can canonicalize String objects to reduce memory occupied by String and related Objects.

History of the Well Known String Object:

     We examine the String class in three part regarding to Java versions as before 1.7.0_06, after 1.7.0_06 before 1.8, and after 1.8.

Before 1.7.0_06;

     String class before 1.7.0_06 has a private char array, and 3 int variables; offset, count and hash respectively. That means a String "A" object will occupy 56 bytes as explained below. For details of memory calculations you can refer to this post.

12 header + 4 ref to char[] + (4*3) for 3 int = 28 rounded to 32 bytes; plus
12 header  + 4 len + 1(len of "A") * 2 = 18 rounded to 24 bytes; totally 56 bytes.

In this implementation, we had offset and count int variables, and the reason of them is to share the char array inside the String object among the substrings created with the substring method.

Let's see it on the following example which run on Java version before 1.7.0_06.

Example 1:

String s1 = "My String";  
String s2 = s1.substring(3);  
System.out.println(s1); // My String  
System.out.println(s2); // String  

Field field = String.class.getDeclaredField("value");  
field.setAccessible(true);  
char[] valueInString = (char[])field.get(s1);  
valueInString [3] = 'J';  
valueInString [4] = 'a';  
valueInString [5] = 'v';  
valueInString [6] = 'a';
 valueInString [7] = ' ';     
 valueInString [8] = ' '; 

System.out.println(s1); // My Java  
System.out.println(s2); // Java  

Here we see that, the String object s2 obtained by the substring method share the underlying char array in the object s1, so that any change to the char array of s1 will directly affect the s2. With this approach we create the substring in constant time by just changing the offset and count values of s2 and referencing to the same char array, however at the same time we possibly get a memory leak if we do not want to use the s1 object anymore, since it will cause the whole object continue to alive not the substring only. To get rid of this memory leak possibility we may use new String(string) constructor around the String obtained with substring.

An important change has been made after 1.7.0_06 to the String class, let's examine it now.

After 1.7.0_06, Before 1.8;

     String class after 1.7.0_06 before 1.8 has a private final char array, and 2 int variables, hash and hash32 respectively. The offset and count variables are removed from the implementation, and the char[] in substrings are not shared anymore. Another change to this version is, the char[] is a final variable indicating it's immutability, although we can change it's value either by reflection or array manipulation.In this version, the String "A" object will occupy 48 bytes as explained below. For details of memory calculations you can refer to this post.

12 header + 4 ref to char[] + (4*2) for 2 int = 24 bytes; plus
12 header  + 4 len + 1(len of "A") * 2 = 18 rounded to 24 bytes; totally 48 bytes.

Let's run the code we saw in Example 1 with this String version.

Example 2:

String s1 = "My String";  
String s2 = s1.substring(3);  
System.out.println(s1); // My String  
System.out.println(s2); // String  

Field field = String.class.getDeclaredField("value");  
field.setAccessible(true);  
char[] valueInString = (char[])field.get(s1);  
valueInString [3] = 'J';  
valueInString [4] = 'a';  
valueInString [5] = 'v';  
valueInString [6] = 'a';
 valueInString [7] = ' ';     
 valueInString [8] = ' '; 

System.out.println(s1); // My Java  
System.out.println(s2); // String  

We see the change to the char[] in the first String doesn't affect the second String obtained with the substring method of the first String. As we see, if you use reflection we can't anymore say that String is immutable. Immutability is valid only if you use String object without reflection.

After 1.8;

     String class after 1.8 has a private final char array, and 1 int variable, hash. The hash32 variable removed from the implementation, which was used as an alternative hash implementation if there are too many collisions with the default hashing algorithm when using with HashMap. However with Java 8 the HashMap implementation has changed and if there are too many collisions in the buckets, HashMap dynamically replaces the way of holding the buckets from linkedlist to ad-hoc implementation of TreeMap. Using this way we end up with O(logn) complexity instead of O(n) since the bucket's elements are now ordered thanks to the TreeMap implementation. In this new implementation HashMap order the bucket's elements that collison occured in. It requires the keys to be Comparable and uses the compareTo methods to order them. Thanks to this new implementation, hash32 variable removed from the String implementation.

In this version, the String "A" object will occupy 48 bytes as explained below.

12 header + 4 ref to char[] + (4*1) for 1 int = 20 bytes rounded to 24 bytes; plus
12 header  + 4 len + 1(len of "A") * 2 = 18 rounded to 24 bytes; totally 48 bytes.

StringBuilder Memory Consumption:

     We now see the StringBuilder in terms of memory usage. We use the last Java version's StringBuilder implementation. The StringBuilder class holds a char[] and an int variable 'count' in the abstract super class AbstractStringBuilder. Since the default capacity of StringBuilder is 16, the total amount of a empty StringBuilder object will occupy 72 bytes as explained below.

12 header + 4 ref to char[] + (4*1) for count = 20 bytes rounded to 24 bytes; plus
12 header  + 4 len + 16 * 2 = 48 bytes; totally 72 bytes.

The initial capacity for a non-empty StringBuilder will be 16 + length and we can get this capacity with capacity() method of it. At anytime we can shrink the capacity to the current lenght with the trimToSize() method if we pretty sure the lenght won't be increase, otherwise we lose with the effort of capacity increase. In the last implementation of the StringBuilder when we append content using the append method, StringBuilder check if the required new length (old length + new String's length) is less than the current capacity of the inner char[], and if so increase capacity to "old capacity * 2 + 2". If this value is still less than the required minimum less capacity than set the capacity to the required minimum capacity, that is old length + new String's length.

If we sure we finish appending to a StringBuilder object, we can shrink it's size with trimToSize() method at any time to reduce the amount of memory spanned with StringBuilder object.

Canonicalization of String:

Using Canonicalization we can sure there is only one unique content of an object. The JVM already maintained a canonicalization technique for all String objects, which is called pooling. If we define the Strings as hardcoded literals such as s= "a", it will be pooled automatically. The pooled String objects are kept in perm gen area until Java 7 and then removed from perm gen to the heap by Java 7 and later versions.

At this point , a small information will be helpful about perm gen space and String pooling. As we said String pool is removed from perm gen to the Heap after Java 7. By Java 8 an important change has been made and the permanent generation is completely removed. Class meta-data informations are now hold in native-memory which is called as "MetaSpace" and interned Strings and static variables are hold in Heap. By this way instead of limiting the memory of the class meta-data, interned Strings and static variables by the -XXMaxPerSize, JVM will allocate and free the machine's native memory dynamically for class meta-data and the Heap memory tuned by the -Xms and -Xmx variables will be used for static variables and interned Strings which are subject to garbage collection, as the normal java objects. By this change the difficulty of tuning perm gen space is gone.
So , keeping in mind these important changes we can say that tuning the JVM again is important when you are moving to Java 8.
You can set the metaspace size by -XX:MaxMetaSpaceSize which has no limit by default and has a 21 MB initial value for 64 bit JVM. To print meta space related statistics we have to set -XX:+UnlockDisagnosticVMOptions.

Note: There is also a canonicalization for  all Wrappers (except Double and Float) implemented as cached array inside the Wrapper classes.

If your Strings are coming from different sources such as database or file, the values will not be pooled. In this case you may use native intern method to force the JVM to pool the String objects. But be careful with the intern method with user-generated Strings as it may cause a memory leak and an OutOfMemory error if there is an attack by sending large number of different Strings to your application.
Also reconsider using intern methods if you are using Java 6 or earlier, since the pooled Strings are kept in perm gen and the perm gen has a fixed size determined by -XX:MaxPermSize JVM parameter.

If we use intern methods heavily, we have to use -XXStringTableSize JVM parameter after Java 7 to set the map size of String pool (Default value is 60013 in Java 8). We have to carefully determine the max number of distinct Strings that the application may hold and set the map size according to this.
We can also use -XXPrintStringTableStatistics JVM parameter to see the usage of String pool.

Using intern method is pretty straightforward as shown in the below example;

Example 3:

String s1 = readAsStringFromSomeExternalSource();//say it's value is "a"
s1.intern();
String s2 = "a";
if (s1 == s2)
   System.out.println("String is pooled by intern method");

When we use intern method we may use == operator instead of equals method which can also get some slight performance advantage. However this is so negligible since the first statement of the equals method also uses the == operator to check the equality, and calling the equals method may be inlined by the JVM. Also be careful that if you forget intern at some point == operator will fail.

For String pool we may use our implementation with a ConcurrentHashMap or WeakHashMap.
WeakHashMap will be the correct implementation since it will remove the String from the map when there is no other reference, which is the default behaviour of the String pool. Yes, the String pool also subject to Garbage Collection, and if there is no live reference to an Object in the native JVM String pool, it will garbage collected.
If you don't use WeakHashMap, we have to manually control the size of the map as it may grow too much after some time.
Remember also to use a synchronized version of WeakHashMap if you're in a multithread environment. You may also use Google Collections API's MapMaker to use concurrent WeakHashMap

Using Map instead of intern method causes to create one extra String object for every String used in the application as you have to pass the String object to the custom pool method and this parameter will be directly become eligible to the garbage collection. Subsequent parameter passes will be pooled by the JVM. One advantage of using map pool instead of intern method is there is no method to turn back from the intend operation.

See in the following example;

Example 4:

private ConcurrentMap<String, String> stringPool= new ConcurrentHashMap<String, String>(500);

public String getCanonicalString(String param) {
//Here the literal param String object created on heap and added to the String pool by JVM
//Thanks to the interned String pool, later calls with the same param String are not created
//on heap and just get from the interned String pool. So we can say that every String added 
//to our string pool also added to the interned String pool.
String pooledVersion= stringPool.putIfAbsent(paramparam);
return (pooledVersion == null) ? param pooledVersion ;
}

Remember to clear the stringPool if it's size grow too much.

Using byte[] instead of String:

We may use byte[] instead of String class in some situtations to reduce the memory usage, although it has limited usage because you can't use some characters with some charsets as it can hold only 256 ascii characters not full unicode. See in the following example; 

Example 5:

byte[] arr = new byte[5];
arr[0] = 'a';
arr[1] = 'b';
arr[2] = 'c';
arr[3] = 'd';
arr[4] = 'e';
System.out.println(SizeUtil.fullSizeOf(arr)); // 24 bytes
String s ="abcde";
System.out.println(SizeUtil.fullSizeOf(s));   // 56 bytes

}

For the implementation of SizeUtil look for this 
post. You see that byte array spans significantly smaller memory. Until you need the String representation you can keep the byte array, then can use
String s2 = new String(arr, CHARSET_NAME); to obtain the String object. You can especially use this technique when sending object across network or similar situation.

Conclusion:

     Today we see how String and related objects are kept in memory, and how we can reduce memory usage of them and get higher performance. We saw the history of String Object in Java through Java versions and see some methods to reduce memory usage. For example by holding byte[] instead String object we can reduce the memory usage by half or we can use StringBuilder object with trims to the current size and reduce the memory usage by this way.Also we have consider the Java version we're using, and be careful with the implementation changes. Finally we have to consider canonicalization of String objects to reduce their memory usage.

Sunday, February 1, 2015

Actual Memory Consumption of Java Objects

 Introduction:

     Today we'll be talking about the general memory consumption of java objects on the Heap, although the normal situation may sometimes change, i.e; the JVM may choose not the put a Thread local object on to the Heap instead put it to the stack. 

 General Memory Usage of Java Objects:

     Although the memory usage may changes depending on the JVM's vendor, JVM's type as 32 bit-64 bit or UseCompressedOops VM argument, the general formula for a Java object is like below.

 An object header (object class information, status flags about the synhronization locked and the reachability of the object, ID of the class) 8 bytes for 32 bit JVM and 12 bytes for 64 bit UseCompressedOops  enabled JVM, 16 bytes for 64 bit non compressed JVM + Memory for primitives according to their size (See below) + Memory for reference to the member pointer (4 bytes for 32 bit JVM and 64 bit UseCompressedOops  enabled JVM, 8 bytes for 64 bit non compressed JVM) + padding (a wasted space after the object data to make every object start at a standart address space which is a multiple of 8, so rounded to the next multiple of 8)
Be careful, all the middle calculations will be rounded to the next multiple of 8, not the total calculation. That means the outer object is rounded individually while the fields of that objects are rounded individually.

byte, boolean     1 byte
  short, char         2 bytes
  int, float            4 bytes
  long, double      8 bytes

Java Primitive Size

Here we see boolean type is holding 1 byte although it is required just one single bit. This is just for easy access to the class in memory. If we holded 1 bit for the boolean type, we would have been deal with the exact position of the boolean type each time we read or write data, but in this case we use a byte offset and the implementation is easy now.
When you need boolean array, you can use the BitSet implementation of Java language to be able to hold 1 bit for each boolean variable of the array.

 Note: In this write you can see the memory space retained for primitives depending on the place where they defined. Here the point is, if we declare the primitive as class variables they will hold the sizes above , however for the local variables they will be defined independently and they may retain larger space then their requirements. That's because the smallest unit of the memory for a JVM is changing between 32 bit and 64 bit versions.For example,some JVM's may hold a 1 byte variable on the 4 byte smallest unit for 32 bit and 8 byte unit for 64 bit.To be really able to hold 1 byte for a byte you should use array of byte , which will be holded on the heap and the array members will be reached by the reference of the array, so can hold really 1 byte.

 General Memory Usage of Java Array Objects:

     Arrays are normal java objects so their memory usage include the previous section's steps plus a memory space for the length of the array. This space is 4 bytes for 32 bit JVM's and 64 bit UseCompressedOops  enabled JVM, 8 bytes for 64 bit non compressed JVM). 

 Note: For most of the JVM implementations UseCompressedOops  is enabled by default, so you don't need to the anything to enable it. You can check whether its enabled or not using the below java code and iterating over the arguments list.

RuntimeMXBean runtimeMxBean = ManagementFactory.getRuntimeMXBean();

List<String> arguments = runtimeMxBean.getInputArguments();

Now we'll show you some examples for memory usage of java objects. But, before that we need to find out a way to measure the memory usage. In this post, we use java Instrumentation class to be able to measure the size of objects.

 How to Find the Memory Usage Programmatically Using Instrumentation:

     We'll use java.lang.instrument.Instrumentation class to get the size of the java objects. To do that follow the following instructions.

1.) Create a java project including a class that include the following static field and the following methods named premain and shallowSizeOfObject.

private static Instrumentation instrumentation = null;

public static void premain(String args, Instrumentation paramInstrumentation) {
instrumentation = paramInstrumentation;
}

public static long shallowSizeOfObject(Object obj) {
return instrumentation.getObjectSize(obj);
}

2.) Create a MANIFEST.MF file including the following definition.

Premain-Class: your_pck_name.Your_Class_Name

3.) Export the java project as a jar file using the MANIFEST.MF created in step 2.

4.) Add the jar created in step 3 to your project class path.

5.) In your project that has reference to the jar created in step 3, run the class that include a main method with the following VM argument.

-javaagent:your_jar_name.jar

That's it, when you run your program, since you tell that the jar file including the Instrumentation code is an java agent (using the javaagent VM argument) the JVM will automatically run the premain method and set the Instrumentation object to your static variable.Then you can use the getObjectSize(obj) method to get the shallow size of a java object which will include header + primitive variables total size + member pointers like we mentioned in the previous part of this post.
However if you have inner arrays or objects in your object, the getObjectSize(obj) will not give you memory usage of this arrays or objects. That's why we call our method as shallowSizeOfObject.Now ,let's see how can we get the deep memory size of a java object.

 Deep Memory Usage of Java Objects:

     To get the real retained memory space of a java object, we'll use reflection api to access the inner arrays and objects. Also we should use some techniques to skip the pooled objects, like String, Integer etc and to skip primitive and static variables.We then recursively call our deepSizeOfObject method until we finished all the fields of our object and the fields of the all super classes of our object.
Below is the last version of our SizeUtil class including the methods added to get those functionalities.

package sizeutil;

import java.lang.instrument.Instrumentation;
import java.lang.reflect.Array;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.IdentityHashMap;
import java.util.Map;

public class SizeUtil {

private static boolean SKIP_POOLED_OBJECTS = false;

private static Instrumentation instrumentation = null;

public static void premain(String args, Instrumentation paramInstrumentation) {
instrumentation = paramInstrumentation;
}

/**
* Calculates size
* @param obj
*            object to calculate size of
* @return object size
*/
public static long shallowSizeOfObject(Object obj) {
if (SKIP_POOLED_OBJECTS && isPooled(obj))
return 0;
return instrumentation.getObjectSize(obj);
}

private static boolean isPooled(Object paramObject) {
if ((paramObject instanceof Comparable)) {
if ((paramObject instanceof Enum)) {
return true;
}
if ((paramObject instanceof String)) {
return paramObject == ((String) paramObject).intern();
}
if ((paramObject instanceof Boolean)) {
return (paramObject == Boolean.TRUE) || (paramObject == Boolean.FALSE);
}
if ((paramObject instanceof Integer)) {
return paramObject == Integer.valueOf(((Integer) paramObject).intValue());
}
if ((paramObject instanceof Short)) {
return paramObject == Short.valueOf(((Short) paramObject).shortValue());
}
if ((paramObject instanceof Byte)) {
return paramObject == Byte.valueOf(((Byte) paramObject).byteValue());
}
if ((paramObject instanceof Long)) {
return paramObject == Long.valueOf(((Long) paramObject).longValue());
}
if ((paramObject instanceof Character)) {
return paramObject == Character.valueOf(((Character) paramObject).charValue());
}
}
return false;
}

/**
* Calculates deep size
* @param obj
*            object to calculate size of
* @return object deep size
*/
public static long deepSizeOfObject(Object obj) {

Map<Object, Object> previouslyVisited = new IdentityHashMap<Object, Object>();
long result = deepSizeOf(obj, previouslyVisited);
previouslyVisited.clear();
return result;
}

private static boolean skipObject(Object obj, Map<Object, Object> previouslyVisited) {
if (SKIP_POOLED_OBJECTS && isPooled(obj))
return true;
return (obj == null) || previouslyVisited == null || previouslyVisited.containsKey(obj);
}

@SuppressWarnings("rawtypes")
private static long deepSizeOf(Object obj, Map<Object, Object> previouslyVisited) {
if (skipObject(obj, previouslyVisited)) {
return 0;
}
previouslyVisited.put(obj, null);

long returnVal = 0;
// get size of object + primitive variables + member pointers
// for array header + len + if primitive total value for primitives
returnVal += SizeUtil.shallowSizeOfObject(obj);

// recursively call all array elements
Class objClass = obj.getClass();
if (objClass == null)
return 0;
if (objClass.isArray()) {
if (objClass.getName().length() != 2) {// primitive type arrays has length two, skip them (they included in the shallow size)
int lengthOfArray = Array.getLength(obj);
for (int i = 0; i < lengthOfArray; i++) {
returnVal += deepSizeOf(Array.get(obj, i), previouslyVisited);
}
}
} else {
// recursively call all fields of the object including the superclass fields
do {
Field[] objFields = objClass.getDeclaredFields();
for (int i = 0; i < objFields.length; i++) {
if (!Modifier.isStatic(objFields[i].getModifiers())) {// skip statics
if (!objFields[i].getType().isPrimitive()) { // skip primitives
objFields[i].setAccessible(true);
Object tempObject = null;
try {
tempObject = objFields[i].get(obj);
} catch (IllegalArgumentException e) {
// TODO Auto-generated catch block

} catch (IllegalAccessException e) {
// TODO Auto-generated catch block

}
if (tempObject != null) {
returnVal += deepSizeOf(tempObject, previouslyVisited);
}
}
}
}
objClass = objClass.getSuperclass();
} while (objClass != null);
}
return returnVal;
}
}

 Examples of Memory Usage:

     Here are some examples in a single TestSizeOf class about java object's memory usage. We use the SizeUtil class mentioned in the previous section in our examples. The examples are run on 64 bit Java version 1.8.0.25 where as I mentinoned UseCompressedOops enabled automatically. So our objects have 12 bytes header, 4 bytes ref to member pointers, 4 bytes ref to array elements, and extra 4 bytes for array objects to hold the length. Remember to add the -javaagent:sizeOfObjects.jar as VM argument before run your main program.

package test;

import sizeutil.SizeUtil;

public class TestSizeOf {
private String innerStr;
private int innerInt;

public TestSizeOf(String innerStr, int innerInt) {
this.innerInt = innerInt;
this.innerStr = innerStr;
}

public static void main(String[] args) {
boolean bo1 = true;
System.out.println("Retained space for boolean bo1 = true --> " + SizeUtil.deepSizeOfObject(bo1));
byte b1 = 3;
System.out.println("Retained space for byte b1 = 3 --> " + SizeUtil.deepSizeOfObject(b1));
short sh1 = 3;
System.out.println("Retained space for short sh1 = 3 --> " + SizeUtil.deepSizeOfObject(sh1));

int i1 = 3;
System.out.println("Retained space for int i1 = 3 --> " + SizeUtil.deepSizeOfObject(i1));

long l1 = 3L;
System.out.println("Retained space for long l1 = 3L --> " + SizeUtil.deepSizeOfObject(l1));

char c1 = 'A';
System.out.println("Retained space for char c1 = 'A' --> " + SizeUtil.deepSizeOfObject(c1));

float f1 = 3f;
System.out.println("Retained space for float f1 = 3f --> " + SizeUtil.deepSizeOfObject(f1));
double d1 = 3;
System.out.println("Retained space for double d1 = 3 --> " + SizeUtil.deepSizeOfObject(d1));

Object o = new Object();
System.out.println("Retained space for Object o = new Object() --> "+ 
SizeUtil.deepSizeOfObject(o));

String st1 = "A";
System.out.println("Retained space for String st1 = \"A\" --> " + SizeUtil.deepSizeOfObject(st1));

byte[] arrByte = new byte[20];
System.out.println("Retained shallow space for byte[] arrByte = new byte[20] --> " +  SizeUtil.shallowSizeOfObject(arrByte));
System.out.println("Retained deep space for byte[] arrByte = new byte[20] --> " +  SizeUtil.deepSizeOfObject(arrByte));

Byte[] arrByteObj = new Byte[] { 3, 4, 5 };
System.out.println("Retained shallow space for Byte[] arrByteObj = new Byte[] { 3, 4, 5 } --> " +    SizeUtil.shallowSizeOfObject(arrByteObj));
System.out.println("Retained deep space for Byte[] arrByteObj = new Byte[] { 3, 4, 5 } --> " +  SizeUtil.deepSizeOfObject(arrByteObj));

int[] arrInt = new int[] { 0 };
System.out.println("Retained shallow space for int[] arrInt = new int[] { 0 } --> " +  SizeUtil.shallowSizeOfObject(arrInt));
System.out.println("Retained deep space for int[] arrInt = new int[] { 0 } --> " +  SizeUtil.deepSizeOfObject(arrInt));

long[] arrLong = new long[] { 0 };
System.out.println("Retained shallow space for long[] arrLong = new long[] { 0 } --> " +  SizeUtil.shallowSizeOfObject(arrLong));
System.out.println("Retained deep space for long[] arrLong = new long[] { 0 } --> " +  SizeUtil.deepSizeOfObject(arrLong));

int[][] arrIntTwoDim = new int[5][5]; 

for (int i = 0; i < 5; i++) {
for (int j = 0; j < 5; j++) {
arrIntTwoDim[i][j] = i * j;
}
}
System.out.println("Retained shallow space for int[][] arrIntTwoDim = new int[5][5] --> " +  SizeUtil.shallowSizeOfObject(arrIntTwoDim));
System.out.println("Retained deep space for int[][] arrIntTwoDim = new int[5][5] --> " +  SizeUtil.deepSizeOfObject(arrIntTwoDim));

String[] arrStr = new String[4];
arrStr[0] = "1";
arrStr[1] = "2";
arrStr[2] = "3";
arrStr[3] = null;
System.out.println("Retained shallow space for String[] arrStr = new String[4] --> " +  SizeUtil.shallowSizeOfObject(arrStr));
System.out.println("Retained deep space for String[] arrStr = new String[4] --> " +  SizeUtil.deepSizeOfObject(arrStr));

TestSizeOf myTest = new TestSizeOf("test", 11111111);
System.out.println("Retained shallow space for TestSizeOf myTest = new TestSizeOf(\"test\", 11111111) --> " + SizeUtil.shallowSizeOfObject(myTest));
System.out.println("Retained deep space for TestSizeOf myTest = new TestSizeOf(\"test\", 11111111) --> " + SizeUtil.deepSizeOfObject(myTest));

Object obj = new String("test");
System.out.println("Retained shallow space for Object obj = new String(\"test\") --> " +  SizeUtil.shallowSizeOfObject(obj));
System.out.println("Retained deep space for Object obj = new String(\"test\") --> " +  SizeUtil.deepSizeOfObject(obj));
}

}

Here is the output of the program; 

Retained space for boolean bo1 = true --> 16
Explanation: 12 bytes header + 1 byte for boolean = 13 rounded to 16 bytes
Retained space for byte b1 = 3 --> 16
Explanation: 12 bytes header + 1 byte for byte= 13 rounded to 16 bytes
Retained space for short sh1 = 3 --> 16
Explanation: 12 bytes header + 2 bytes for short = 14 rounded to 16 bytes
Retained space for int i1 = 3 --> 16
Explanation: 12 bytes header + 4 bytes for int = 16 no need to round
Retained space for long l1 = 3L --> 24
Explanation: 12 bytes header + 8 bytes for long = 20 rounded to 24 bytes
Retained space for char c1 = 'A' --> 16
Explanation: 12 bytes header + 2 bytes for 1 char = 14 rounded to 16 bytes
Retained space for float f1 = 3f --> 16
Explanation: 12 bytes header + 4 bytes for float= 16 no need to round
Retained space for double d1 = 3 --> 24
Explanation: 12 bytes header + 8 bytes for double= 20 rounded to 24 bytes
Retained space for Object o = new Object() --> 16
Explanation: 12 bytes header  = 12 rounded to 16 bytes
Retained space for String st1 = "A" --> 48
Explanation: (for String)12 bytes header + 4 bytes ref to char[] + 4 bytes hash int value = 20 rounded to 24 bytes 

(for char[]) 12 bytes header + 4 bytes for length of char[] + 1(1 character String)*2 bytes =18 bytes rounded to 24 bytes
-> so 24 + 24 = 48 bytes
Retained shallow space for byte[] arrByte = new byte[20] --> 40
Explanation: 12 bytes header + 4 bytes for length of byte[]  + 20 (20 bytes in array) * 1 bytes = 36 rounded to 40 bytes
Retained deep space for byte[] arrByte = new byte[20] --> 40
Explanation: 12 bytes header + 4 bytes for length of byte[]  + 20 (20 bytes in array) * 1 bytes = 36 rounded to 40 bytes
Retained shallow space for Byte[] arrByteObj = new Byte[] { 3, 4, 5 } --> 32
Explanation: 12 bytes header + 4 bytes for length of byte[]  + 3 (ref to each array element) * 4 bytes = 28 rounded to 32 bytes
Retained deep space for Byte[] arrByteObj = new Byte[] { 3, 4, 5 } --> 80
Explanation: 12 bytes header + 4 bytes for length of byte[]  + 3 (ref to each array element) * 4 bytes = 28 rounded to 32 bytes
+
3 (for 3 array elements) * 16 bytes = 48 bytes
-> so 32 + 48 = 80 bytes
Retained shallow space for int[] arrInt = new int[] { 0 } --> 24
Explanation: 12 bytes header + 4 bytes for length of int [] + 1(1 int in array) * 4 bytes = 20 rounded to 24 bytes
Retained deep space for int[] arrInt = new int[] { 0 } --> 24
Explanation: 12 bytes header + 4 bytes for length of int [] + 1(1 int in array) * 4 bytes = 20 rounded to 24 bytes
Retained shallow space for long[] arrLong = new long[] { 0 } --> 24
Explanation: 12 bytes header + 4 bytes for length of int [] + 1(1 long in array) * 8 bytes = 24 rounded to 24 bytes
Retained deep space for long[] arrLong = new long[] { 0 } --> 24
Explanation: 12 bytes header + 4 bytes for length of int [] + 1(1 long in array) * 8 bytes = 24 rounded to 24 bytes
Retained shallow space for int[][] arrIntTwoDim = new int[5][5] --> 40
Explanation: (for outer array)12 bytes header + 4 bytes for  length of int [][]  + 5(ref to 5 inner array) * 4 bytes = 36 rounded to 40 bytes
Retained deep space for int[][] arrIntTwoDim = new int[5][5] --> 240
Explanation: (for outer array)12 bytes header + 4 bytes for  length of int [][]  + 5(ref to 5 inner array) * 4 bytes = 36 rounded to 40 bytes
+
(for inner arrays) 5(5 inner array) * (12 bytes  header +  4 bytes for  length of int [] + 5 * 4 bytes = 36 rounded to 40 bytes) = 200 bytes no need to round
-> so 40  + 200 = 240 bytes
Retained shallow space for String[] arrStr = new String[4] --> 32
Explanation: 12 bytes header + 4 bytes for  length of String + 4 (4 String in array) * 4 bytes = 32 bytes no need to round
Retained deep space for String[] arrStr = new String[4] --> 176
Explanation: 12 bytes header + 4 bytes for  length of String + 4 (4 String in array) * 4 bytes = 32 bytes no need to round
+
3 (4 th value is null so has no memory space) * 48 bytes (1 String with 1 character is 48 bytes as explained before) = 144 bytes
so -> 32 + 144 = 176 bytes
Retained shallow space for TestSizeOf myTest = new TestSizeOf("test", 11111111) --> 24
Explanation: 12 bytes header + 1 (ref to String field of the class) * 4 bytes + 4 bytes for int field = 20 bytes rounded to 24 bytes
Retained deep space for TestSizeOf myTest = new TestSizeOf("test", 11111111) --> 72
Explanation: 12 bytes header + 1 (ref to String field of the class) * 4 bytes + 4 bytes for int field = 20 bytes rounded to 24 bytes
+
(for String field) 12 bytes header + 4 bytes for ref of char[] + 4 bytes for hash int value = 20 rounded to 24 bytes 
+
(for char[] of String) 12 bytes header + 4 bytes length + 4 * 2 bytes = 24 bytes no need to round
so-> 24 + 24 = 48 bytes
so the final result-> 24 + 48 = 72 bytes
Retained shallow space for Object obj = new String("test") --> 24
Explanation: 12 bytes header + 4 bytes for ref of char[] + 4 bytes for hash int value = 20 rounded to 24 bytes 
Retained deep space for Object obj = new String("test") --> 48
Explanation: 12 bytes header + 4 bytes for ref of char[] + 4 bytes for hash int value = 20 rounded to 24 bytes 
+
(for char[] of String) 12 bytes header + 4 bytes length + 4 * 2 bytes = 24 bytes no need to round
so-> 24 + 24 = 48 bytes 

You can see the explanations in the below of each output.

Although we use primitive types in the first few examples, they are boxed to their wrapper classes when we call the deepSizeOfObject method. Normally we don't boxed them, we just use them like that just for explanation.

All the objects has 12 bytes header in the example. The wrapper objects has 12 bytes header + primitive size memory usage. Arrays has extra 4 bytes for their length.

The two dimensional arrays are just array of arrays.So the outer array has separate header and all the inner arrays has separate headers also.

The TestSizeOf  class has two fields as String and int. They are calculated separately in the deepSizeOfObject method.

Conclusion:

     We see the real retained memory space of java objects. Although the current state of the object like whether it is garbage collected or it's synchronization locked is contended may affect the size of the object on the heap, if we get a general understanding of  how much our objects will occupy, we can reconsider our java code to get less space and more performance. We'll be talking about memory consumption of String and StringBuilder objects and some suggestions on using String related objects in the next post.

You can download the source code from here.