How to keep Reference to a Serialized object in Java?
Question
How can i keep reference to an object that i'm sure it'll be serialized but i don't want it to be serialized through this reference ?
To make it more clear i've a Network that contains list of Nodes each Node contains a list of Connections , the Connection contains a reference to another Nodes .
The problem is when i try serializing kind of big Network this results to StackOverflowError , the situation that i'm assuming that causes this is the following:
- Serialization starts with the
Networkand tries to serialize the firstNode - Then it tries to serialize the first connection which contains a reference to another node
- It tries to serialize the next
Nodeand so on, and the serialization recursion goes through all nodes causing the overflow - If the references in
Connectionare marked astransientthere is no problem in serialization, but then the references =nullafter deserialization
Here is a sample that reproduces the problem
Network.java
import java.io.Serializable;
import java.util.ArrayList;
public class Network implements Serializable {
private static final long serialVersionUID = 1399116563470735156L;
ArrayList<Layer>layers;
public Network() {
layers= new ArrayList<Layer>();
}
//connect each layer to next layer
public void connectAllLayers(){
for (int i = 0; i < layers.size()-1; i++) {
layers.get(i).connectTo(layers.get(i+1));
}
}
}
Layer.java
import java.io.Serializable;
import java.util.ArrayList;
public class Layer implements Serializable{
private static final long serialVersionUID = -5519150448729707106L;
public ArrayList<Node>nodes;
public Layer(int nodeCount) {
nodes = new ArrayList<Node>();
for (int i = 0; i < nodeCount; i++) {
nodes.add(new Node());
}
}
//connect all nodes in a layer to all nodes in the other layer
public void connectTo(Layer layer){
for (Node myNode : nodes) {
for (Node toNode : nodes) {
myNode.connectTo(toNode);
}
}
}
}
Node.java
import java.io.Serializable;
import java.util.ArrayList;
public class Node implements Serializable{
private static final long serialVersionUID = 6323513316304801012L;
public ArrayList<Connection>connections;
public double x,y,z,a,b,c;//some variables to simulate memory usage
public Node() {
connections = new ArrayList<Connection>();
x=15;
}
public void connectTo(Node node){
Connection connection = new Connection(this, node);
this.connections.add(connection);
node.connections.add(connection);
}
}
Connection.java
import java.io.Serializable;
public class Connection implements Serializable {
private static final long serialVersionUID = 7578299749680304407L;
public Node from;
public Node to;
public double weight;
public Connection(Node from, Node to) {
this.from = from;
this.to = to;
}
}
Main.java
import java.io.*;
public class Main {
public static void saveNetwork(Network net, String filename) {
try {
// Serialize data object to a file
ObjectOutputStream out = new ObjectOutputStream(
new FileOutputStream(filename));
out.writeObject(net);
out.close();
} catch (IOException e) {
}
}
public static Network loadNetwork(String filename) {
Network net = null;
try {
FileInputStream door = new FileInputStream(filename);
ObjectInputStream reader = new ObjectInputStream(door);
net = (Network) reader.readObject();
reader.close();
} catch (Exception e) {
System.err.println(e.getMessage());
}
return net;
}
public static void main(String[] args) {
Network net;
boolean load = false;//Change to true to load the saved object
if (!load) {
net = new Network();
net.layers.add(new Layer(400));
net.layers.add(new Layer(300));
net.layers.add(new Layer(10));
net.connectAllLayers();
saveNetwork(net, "mynet");
} else {
net = loadNetwork("mynet");
}
Layer layer = net.layers.get(0);
Node node = layer.nodes.get(0);
Connection connection = node.connections.get(0);
System.out.println(connection.to.x);
}
}
To make the question more general, is there someway to serialize Graph like connected Node classes assuming there is a list that contains references to all Nodes, without causing overflow ?
2 answers
solution1
0 ACCPTED 2014-01-14 22:17:09
Java will serialize graphs fine, storing only references to objects that were already serialized. However, it does use recursion in a depth-first fashion, and your graph is very deep (548 nodes deep before I got a stack overflow).
Sort your layers by depth, and serialize them in descending order of depth. This will prevent deep recursion during serialization.
solution2
0 2014-01-19 03:22:46
I beleive this code will work for you, take care to carefully test it if used for production. For one thing, it doesnt look for custom serialization routines on classes.I think however it suites the purpose.
It consists of two classes SequentialObjectOutputStream and SequentialObjectInputStream
import java.io.*;
import java.util.*;
import java.lang.reflect.*;
import android.util.*;
public class SequentialObjectOutputStream extends DataOutputStream
implements ObjectOutput
{
interface FieldGetAction
{
void get(Object obj, Field field) throws IllegalAccessException, IOException;
}
interface ArrayGetAction
{
void get(Object array, int Index) throws ArrayIndexOutOfBoundsException, IOException;
}
public HashMap<Class, FieldGetAction> Primatives;
public HashMap<Class, ArrayGetAction> ArrayPrimatives;
public SequentialObjectOutputStream(OutputStream stream)
{
super(stream);
Primatives = new HashMap<Class, FieldGetAction>();
try
{
Primatives.put(boolean.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
boolean x = field.getBoolean(obj);
writeBoolean(x);
}
});
Primatives.put(byte.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
byte x = field.getByte(obj);
writeByte(x);
}
});
Primatives.put(short.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
short x = field.getShort(obj);
writeShort(x);
}
});
Primatives.put(int.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
int x = field.getInt(obj);
writeInt(x);
}
});
Primatives.put(long.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
long x = field.getLong(obj);
writeLong(x);
}
});
Primatives.put(char.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
char x = field.getChar(obj);
writeChar(x);
}
});
Primatives.put(float.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
float x = field.getFloat(obj);
writeFloat(x);
}
});
Primatives.put(double.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
double x = field.getDouble(obj);
writeDouble(x);
}
});
Primatives.put(String.class,
new FieldGetAction()
{
public void get(Object obj, Field field) throws IllegalAccessException, IOException
{
String x = (String) field.get(obj);
writeUTF(x);
}
});
} catch(Exception e)
{
Log.e("SOb", Log.getStackTraceString(e));
}
ArrayPrimatives = new HashMap<Class, ArrayGetAction>();
try
{
ArrayPrimatives.put(boolean.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
boolean x = Array.getBoolean(obj, index);
writeBoolean(x);
}
});
ArrayPrimatives.put(byte.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
byte x = Array.getByte(obj, index);
writeByte(x);
}
});
ArrayPrimatives.put(short.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
short x = Array.getShort(obj, index);
writeShort(x);
}
});
ArrayPrimatives.put(int.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
int x = Array.getInt(obj, index);
writeInt(x);
}
});
ArrayPrimatives.put(long.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
long x = Array.getLong(obj, index);
writeLong(x);
}
});
ArrayPrimatives.put(char.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
char x = Array.getChar(obj, index);
writeChar(x);
}
});
ArrayPrimatives.put(float.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
float x = Array.getFloat(obj, index);
writeFloat(x);
}
});
ArrayPrimatives.put(double.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
double x = Array.getDouble(obj, index);
writeDouble(x);
}
});
ArrayPrimatives.put(String.class,
new ArrayGetAction()
{
public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
String x = (String) Array.get(obj, index);
writeUTF(x);
}
});
} catch(Exception e)
{
Log.e("SOb", Log.getStackTraceString(e));
}
}
class State
{
public ArrayList<Object> OStack = new ArrayList<Object>();
public long currentId = 1;
public HashMap<Object, Long> References = new HashMap<Object, Long>();
}
public void writeObject(Object A) throws IOException, NotSerializableException
{
final State state = new State();
state.OStack.add(0, A);
LinkedList<Object> ForStack = new LinkedList<Object>();
while (!(state.OStack.size() == 0))
{
Object Current = state.OStack.get(0);
state.OStack.remove(0);
if (((Serializable) Current) == null)
{
throw new NotSerializableException();
}
//Type C = Current.getClass();
Class C = Current.getClass();
Log.i("SOb", "placing #"+Long.toString(state.currentId)+" of "+C.getCanonicalName()+" on graph");
state.References.put(Current, state.currentId);
state.currentId++;
ForStack.add(Current);
if (C.isArray())
{
//Array array = (Array) Current;
Class Ctype = C.getComponentType();
if (ArrayPrimatives.keySet().contains(Ctype) == false)
{
for (int I=0; I<Array.getLength(Current); I++)
{
Object o = Array.get(Current, I);
if ((o != null) && (state.References.keySet().contains(o) == false))
{
if (state.OStack.contains(o) == false) state.OStack.add(state.OStack.size(), o);
}
}
}
} else
{
for (Class Cur = C; Cur != null; Cur = Cur.getSuperclass())
{
Field[] fields = Cur.getDeclaredFields();
for (Field f : fields)
{
if (Modifier.isStatic(f.getModifiers()))
{
continue;
}
f.setAccessible(true);
if (f.isAccessible() == false)
{
// Log.i("SOb", " isAccessible = false");
continue;
}
Class type = f.getType();
//Log.i("SOb", " field \""+f.getName()+"\" of "+type.getCanonicalName());
if (Primatives.keySet().contains(type) == false)
{
try
{
Object o = f.get(Current);
if ((o != null) && (state.References.keySet().contains(o) == false))
{
if (state.OStack.contains(o) == false) state.OStack.add(state.OStack.size(), o);
}
} catch (IllegalAccessException e)
{
Log.e("SOb", Log.getStackTraceString(e));
}
}
}
}
}
}
writeLong(state.References.size());
for (Object O : ForStack )
{
Serializable s = (Serializable) O;
// if (s != null)
{
Class cl = O.getClass();
String name = cl.getName();
writeUTF(name);
if (cl.isArray())
{
Class components = cl.getComponentType();
ArrayGetAction action;
//Array array = (Array) O;
if (ArrayPrimatives.keySet().contains(components))
{
action = ArrayPrimatives.get(components);
} else
{
action = new ArrayGetAction()
{
public void get(Object array, int index) throws ArrayIndexOutOfBoundsException, IOException
{
Object O = Array.get(array, index);
if (O==null) writeLong(0);
else writeLong(state.References.get(O));
}
};
}
int length = Array.getLength(O);
writeInt(length);
for (int I=0; I<length; I++)
{
action.get(O, I);
}
} else
{
for (Class Cur = cl; Cur != null; Cur = Cur.getSuperclass())
{
Field[] fields = Cur.getDeclaredFields();
for (Field F : fields)
{
Class FieldType = F.getType();
F.setAccessible(true);
if (F.isAccessible() && (Modifier.isStatic(FieldType.getModifiers())))
{
FieldGetAction action;
//Array array = (Array) O;
if (Primatives.keySet().contains(FieldType))
{
action = Primatives.get(FieldType);
} else
{
action = new FieldGetAction()
{
public void get(Object obj, Field index) throws IllegalAccessException, IOException
{
Object O = index.get(obj);
if (O==null) writeLong(0);
else writeLong(state.References.get(O));
}
};
}
try
{
action.get(O, F);
} catch (IllegalAccessException e)
{
Log.e("SOb", Log.getStackTraceString(e));
}
}
}
}
}
}
}
}
}
SequentialObjectInputStream
import java.io.*;
import java.util.*;
import java.lang.reflect.*;
import android.util.*;
public class SequentialObjectInputStream extends DataInputStream implements ObjectInput
{
interface FieldPutAction
{
void put(Object obj, Field field) throws IllegalAccessException, IOException;
}
interface ArrayPutAction
{
void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException;
}
public HashMap<Class, FieldPutAction> Primatives;
public HashMap<Class, ArrayPutAction> ArrayPrimatives;
public SequentialObjectInputStream(InputStream stream)
{
super(stream);
Primatives = new HashMap<Class, FieldPutAction>();
try
{
Primatives.put(boolean.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
boolean x = readBoolean();
field.setBoolean(obj, x);
}
});
Primatives.put(byte.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
byte x = readByte();
field.setByte(obj, x);
}
});
Primatives.put(short.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
short x = readShort();
field.setShort(obj, x);
}
});
Primatives.put(int.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
int x = readInt();
field.setInt(obj, x);
}
});
Primatives.put(long.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
long x = readLong();
field.setLong(obj, x);
}
});
Primatives.put(char.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
char x = readChar();
field.setChar(obj, x);
}
});
Primatives.put(float.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
float x = readFloat();
field.setFloat(obj, x);
}
});
Primatives.put(double.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
double x = readDouble();
field.setDouble(obj, x);
}
});
Primatives.put(String.class,
new FieldPutAction()
{
public void put(Object obj, Field field) throws IllegalAccessException, IOException
{
String x = readUTF();
field.set(obj, x);
}
});
} catch(Exception e)
{
Log.e("SOb", Log.getStackTraceString(e));
}
ArrayPrimatives = new HashMap<Class, ArrayPutAction>();
try
{
ArrayPrimatives.put(boolean.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
boolean x = readBoolean();
Array.setBoolean(obj, index, x);
}
});
ArrayPrimatives.put(byte.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
byte x = readByte();
Array.setByte(obj, index, x);
}
});
ArrayPrimatives.put(short.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
short x = readShort();
Array.setShort(obj, index, x);
}
});
ArrayPrimatives.put(int.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
int x = readInt();
Array.setInt(obj, index, x);
}
});
ArrayPrimatives.put(long.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
long x = readLong();
Array.setLong(obj, index, x);
}
});
ArrayPrimatives.put(char.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
char x = readChar();
Array.setChar(obj, index, x);
}
});
ArrayPrimatives.put(float.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
float x = readFloat();
Array.setFloat(obj, index, x);
}
});
ArrayPrimatives.put(double.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
double x = readDouble();
Array.setDouble(obj, index, x);
}
});
ArrayPrimatives.put(String.class,
new ArrayPutAction()
{
public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException
{
String x = readUTF();
Array.set(obj, index, x);
}
});
} catch(Exception e)
{
Log.e("SOb", Log.getStackTraceString(e));
}
}
@Override
public Object readObject() throws ClassNotFoundException, IOException
{
long Total = readLong();
Log.i("SOb", "readObject : " + Long.toString(Total) + " objects in graph");
HashMap<Long, Object> References = new HashMap<Long, Object>();
long currentId = 1;
HashMap<Object, HashMap<Field, Long>> refCache =
new HashMap<Object, HashMap<Field, Long>>();
final HashMap<Object, HashMap<Integer, Long>> arefCache =
new HashMap<Object, HashMap<Integer,Long>>();
for (int I=0; I < Total; I++)
{
String Name = readUTF();
Class C = Class.forName(Name);
Log.i("SOb", "Object of "+C.getCanonicalName() +" on graph");
int adim = 0;
Object O = null;
if (C.isArray())
{
Class ComponentType = C.getComponentType();
int Size = readInt();
Log.i("SOb", "array of "+ComponentType.getCanonicalName() + ", " + Long.toString(Size) + " elements");
O = Array.newInstance(ComponentType, Size);
References.put(currentId, O);
currentId++;
ArrayPutAction action = null;
if (ArrayPrimatives.keySet().contains(ComponentType))
{
action = ArrayPrimatives.get(ComponentType);
} else
{
arefCache.put(O, new HashMap<Integer, Long>());
action = new ArrayPutAction()
{
public void put(Object O, int Index) throws ArrayIndexOutOfBoundsException , IOException
{
long Ref = readLong();
arefCache.get(O).put(Index, Ref);
}
};
}
for (int index=0; index< Size; index++)
{
action.put(O,index);
}
} else
{
try
{
O =
C.getConstructor(new Class[0]).newInstance(new Object[0]);
} catch(InstantiationException e)
{
Log.e("SOb", Log.getStackTraceString(e));
} catch(NoSuchMethodException e)
{
Log.e("SOb", Log.getStackTraceString(e));
} catch(IllegalAccessException e)
{
Log.e("SOb", Log.getStackTraceString(e));
} catch(InvocationTargetException e)
{
Log.e("SOb", Log.getStackTraceString(e));
}
References.put(currentId, O);
currentId++;
refCache.put(O, new HashMap<Field, Long>());
for (Field F : C.getFields())
{
if (F.isAccessible())
{
Class T = F.getType();
if (Primatives.containsKey(T))
{
try
{
Primatives.get(T).put(O, F);
} catch (IllegalAccessException e)
{
}
} else
{
refCache.get(O).put(F, readLong());
}
}
}
}
}
for (long I=0; I < Total; I++)
{
Object O = References.get(I+1);
Class C = O.getClass();
//Log.i("SOb", "get reference "+Long.toString(I)+" "+C.getCanonicalName());
if (C.isArray())
{
HashMap<Integer,Long> aref_table = arefCache.get(O);
if (ArrayPrimatives.containsKey(C.getComponentType()) == false)
{
int len = Array.getLength(O);
for (int index=0; index<len; index++)
{
long r = aref_table.get(index);
Object ref = r == 0 ? null : References.get(r);
Array.set(O, index, ref);
}
}
} else
{
HashMap<Field, Long> ref_table = refCache.get(O);
for (Field F : C.getFields())
{
if (F.isAccessible())
{
Class T = F.getType();
if (Primatives.containsKey(T) == false)
{
try
{
long r = ref_table.get(F);
Object ref = r == 0 ? null : References.get(r);
F.set(O, ref);
} catch (IllegalAccessException e)
{
Log.e("SOb", Log.getStackTraceString(e));
}
}
}
}
}
}
return References.get((Long) (long) 1);
}
}
SequentialObjectOutputStream
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
How do I peek in a java serialized object?
How to deserialize in PHP an object serialized in Java
Serialization - How to secure a serialized JAVA object?
How to pass a serialized object to appengine java task?
Update Serialized Object in Java
Transfer of a Java Serialized Object
Reference is changing after object is serialized?
How is serialVersionUID serialized in Java?
Load whole(`this`) serialized object in Java
Sort serialized object alphabetically (Java)
Related Tags