如何在Java中保留对序列化对象的引用？

Question

How can i keep reference to an object that i'm sure it'll be serialized but i don't want it to be serialized through this reference ?

To make it more clear i've a Network that contains list of Nodes each Node contains a list of Connections , the Connection contains a reference to another Nodes .

The problem is when i try serializing kind of big Network this results to StackOverflowError , the situation that i'm assuming that causes this is the following:

• Serialization starts with the Network and tries to serialize the first Node
• Then it tries to serialize the first connection which contains a reference to another node
• It tries to serialize the next Node and so on, and the serialization recursion goes through all nodes causing the overflow
• If the references in Connection are marked as transient there is no problem in serialization, but then the references = null after deserialization

Here is a sample that reproduces the problem

Network.java

import java.io.Serializable;
import java.util.ArrayList;

public class Network implements Serializable {
    private static final long serialVersionUID = 1399116563470735156L;

    ArrayList<Layer>layers;

    public Network() {
        layers= new ArrayList<Layer>();
    }

    //connect each layer to next layer
    public void connectAllLayers(){
        for (int i = 0; i < layers.size()-1; i++) {
            layers.get(i).connectTo(layers.get(i+1));
        }
    }
}

Layer.java

import java.io.Serializable;
import java.util.ArrayList;


public class Layer implements Serializable{
    private static final long serialVersionUID = -5519150448729707106L;

    public ArrayList<Node>nodes;

    public Layer(int nodeCount) {
        nodes = new ArrayList<Node>();
        for (int i = 0; i < nodeCount; i++) {
            nodes.add(new Node());
        }
    }

    //connect all nodes in a layer to all nodes in the other layer
    public void connectTo(Layer layer){
        for (Node myNode : nodes) {
            for (Node toNode : nodes) {
                myNode.connectTo(toNode);
            }
        }
    }
}

Node.java

import java.io.Serializable;
import java.util.ArrayList;


public class Node implements Serializable{
    private static final long serialVersionUID = 6323513316304801012L;

    public ArrayList<Connection>connections;
    public double x,y,z,a,b,c;//some variables to simulate memory usage

    public Node() {
        connections = new ArrayList<Connection>();
        x=15;
    }

    public void connectTo(Node node){
        Connection connection = new Connection(this, node);
        this.connections.add(connection);
        node.connections.add(connection);
    }
}

Connection.java

import java.io.Serializable;

public class Connection implements Serializable {
    private static final long serialVersionUID = 7578299749680304407L;

    public Node from;
    public Node to;
    public double weight;

    public Connection(Node from, Node to) {
        this.from = from;
        this.to = to;
    }
}

Main.java

import java.io.*;

public class Main {

    public static void saveNetwork(Network net, String filename) {
        try {
            // Serialize data object to a file
            ObjectOutputStream out = new ObjectOutputStream(
                    new FileOutputStream(filename));
            out.writeObject(net);
            out.close();
        } catch (IOException e) {
        }
    }

    public static Network loadNetwork(String filename) {
        Network net = null;
        try {
            FileInputStream door = new FileInputStream(filename);
            ObjectInputStream reader = new ObjectInputStream(door);
            net = (Network) reader.readObject();
            reader.close();
        } catch (Exception e) {
            System.err.println(e.getMessage());
        }
        return net;
    }

    public static void main(String[] args) {
        Network net;
        boolean load = false;//Change to true to load the saved object

        if (!load) {
            net = new Network();
            net.layers.add(new Layer(400));
            net.layers.add(new Layer(300));
            net.layers.add(new Layer(10));

            net.connectAllLayers();

            saveNetwork(net, "mynet");
        } else {
            net = loadNetwork("mynet");
        }
        Layer layer = net.layers.get(0);
        Node node = layer.nodes.get(0);
        Connection connection = node.connections.get(0);
        System.out.println(connection.to.x);
    }

}

To make the question more general, is there someway to serialize Graph like connected Node classes assuming there is a list that contains references to all Nodes, without causing overflow ?

Answer 1

Java will serialize graphs fine, storing only references to objects that were already serialized. However, it does use recursion in a depth-first fashion, and your graph is very deep (548 nodes deep before I got a stack overflow).

Sort your layers by depth, and serialize them in descending order of depth. This will prevent deep recursion during serialization.

Answer 2

I beleive this code will work for you, take care to carefully test it if used for production. For one thing, it doesnt look for custom serialization routines on classes.I think however it suites the purpose.

It consists of two classes SequentialObjectOutputStream and SequentialObjectInputStream

import java.io.*;
import java.util.*;
import java.lang.reflect.*;
import android.util.*;

public class SequentialObjectOutputStream extends DataOutputStream
implements ObjectOutput
{
    interface FieldGetAction
    {
        void get(Object obj, Field field) throws IllegalAccessException, IOException;
    }

    interface ArrayGetAction
    {
        void get(Object array, int Index) throws ArrayIndexOutOfBoundsException, IOException;       
    }

    public HashMap<Class, FieldGetAction> Primatives;
    public HashMap<Class, ArrayGetAction> ArrayPrimatives;

    public SequentialObjectOutputStream(OutputStream stream)
    {
        super(stream);

        Primatives = new HashMap<Class, FieldGetAction>();

        try
        {
            Primatives.put(boolean.class,
            new FieldGetAction()
            {
                public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                {
                    boolean x = field.getBoolean(obj);
                    writeBoolean(x);

                }
            });

            Primatives.put(byte.class,
                new FieldGetAction()
                {
                    public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        byte x = field.getByte(obj);
                        writeByte(x);

                    }
                });


            Primatives.put(short.class,
                new FieldGetAction()
                {
                    public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        short x = field.getShort(obj);
                        writeShort(x);

                    }
                });


            Primatives.put(int.class,
                new FieldGetAction()
                {
                    public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        int x = field.getInt(obj);
                        writeInt(x);

                    }
                });


            Primatives.put(long.class,
                new FieldGetAction()
                {
                    public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        long x = field.getLong(obj);
                        writeLong(x);

                    }
                });


            Primatives.put(char.class,
                new FieldGetAction()
                {
                    public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        char x = field.getChar(obj);
                        writeChar(x);

                    }
                });


            Primatives.put(float.class,
                new FieldGetAction()
                {
                    public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        float x = field.getFloat(obj);
                        writeFloat(x);

                    }
                });


            Primatives.put(double.class,
                new FieldGetAction()
                {
                    public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        double x = field.getDouble(obj);
                        writeDouble(x);
                    }
                });


            Primatives.put(String.class,
                new FieldGetAction()
                {
                    public void get(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        String x = (String) field.get(obj);
                        writeUTF(x);

                    }
                });
        } catch(Exception e)
        {
            Log.e("SOb", Log.getStackTraceString(e));
        }



        ArrayPrimatives = new HashMap<Class, ArrayGetAction>();

        try
        {
            ArrayPrimatives.put(boolean.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        boolean x = Array.getBoolean(obj, index);
                        writeBoolean(x);

                    }
                });

            ArrayPrimatives.put(byte.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        byte x = Array.getByte(obj, index);
                        writeByte(x);

                    }
                });


            ArrayPrimatives.put(short.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        short x = Array.getShort(obj, index);
                        writeShort(x);

                    }
                });


            ArrayPrimatives.put(int.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        int x = Array.getInt(obj, index);
                        writeInt(x);

                    }
                });


            ArrayPrimatives.put(long.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        long x = Array.getLong(obj, index);
                        writeLong(x);

                    }
                });


            ArrayPrimatives.put(char.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        char x = Array.getChar(obj, index);
                        writeChar(x);

                    }
                });


            ArrayPrimatives.put(float.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        float x = Array.getFloat(obj, index);
                        writeFloat(x);

                    }
                });


            ArrayPrimatives.put(double.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        double x = Array.getDouble(obj, index);
                        writeDouble(x);
                    }
                });


            ArrayPrimatives.put(String.class,
                new ArrayGetAction()
                {
                    public void get(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        String x = (String) Array.get(obj, index);
                        writeUTF(x);

                    }
                });
        } catch(Exception e)
        {
            Log.e("SOb", Log.getStackTraceString(e));
        }

    }

    class State
    {
        public ArrayList<Object> OStack = new ArrayList<Object>();

        public long currentId = 1;

        public HashMap<Object, Long> References = new HashMap<Object, Long>();

    }

    public void writeObject(Object A) throws IOException, NotSerializableException
    {
        final State state = new State();

        state.OStack.add(0, A);

        LinkedList<Object> ForStack = new LinkedList<Object>();

        while (!(state.OStack.size() == 0))
        {
            Object Current = state.OStack.get(0);
            state.OStack.remove(0);

            if (((Serializable) Current) == null)
            {
                throw new NotSerializableException();
            }


            //Type C = Current.getClass();

            Class C = Current.getClass();

            Log.i("SOb", "placing #"+Long.toString(state.currentId)+" of "+C.getCanonicalName()+" on graph"); 
            state.References.put(Current, state.currentId);
            state.currentId++;

            ForStack.add(Current);

            if (C.isArray())
            {
                //Array array = (Array) Current;
                Class Ctype = C.getComponentType();

                if (ArrayPrimatives.keySet().contains(Ctype) == false)
                {
                    for (int I=0; I<Array.getLength(Current); I++)
                    {
                        Object o = Array.get(Current, I);

                        if ((o != null) && (state.References.keySet().contains(o) == false))
                        {
                            if (state.OStack.contains(o) == false) state.OStack.add(state.OStack.size(), o);
                        }

                    }
                }
            } else
            {
                for (Class Cur = C; Cur != null; Cur = Cur.getSuperclass())
                {

                    Field[] fields = Cur.getDeclaredFields();

                    for (Field f : fields)
                    {
                        if (Modifier.isStatic(f.getModifiers()))
                        {
                            continue;
                        }

                        f.setAccessible(true);

                        if (f.isAccessible() == false)
                        {
                        //  Log.i("SOb", "     isAccessible = false");
                            continue;
                        }

                        Class type = f.getType();
                        //Log.i("SOb", "     field \""+f.getName()+"\" of "+type.getCanonicalName());

                        if (Primatives.keySet().contains(type) == false)
                        {       
                            try
                            {
                                Object o = f.get(Current);

                                if ((o != null) && (state.References.keySet().contains(o) == false))
                                {
                                    if (state.OStack.contains(o) == false) state.OStack.add(state.OStack.size(), o);
                                }

                            } catch (IllegalAccessException e)
                            {
                                Log.e("SOb", Log.getStackTraceString(e));
                            }
                        }
                    }
                }
            }
        }

        writeLong(state.References.size());

        for (Object O : ForStack )
        {
            Serializable s = (Serializable) O;

        //  if (s != null)
            {
                Class cl = O.getClass();

                String name = cl.getName();

                writeUTF(name);

                if (cl.isArray())
                {
                    Class components = cl.getComponentType();

                    ArrayGetAction action;

                    //Array array = (Array) O;

                    if (ArrayPrimatives.keySet().contains(components))
                    {
                        action = ArrayPrimatives.get(components);
                    } else
                    {
                        action = new ArrayGetAction()
                        {
                            public void get(Object array, int index) throws ArrayIndexOutOfBoundsException, IOException     
                            {
                                Object O = Array.get(array, index);
                                if (O==null)  writeLong(0);
                                else writeLong(state.References.get(O));
                            }
                        };
                    }

                    int length = Array.getLength(O);

                    writeInt(length);

                    for (int I=0; I<length; I++)
                    {
                        action.get(O, I);
                    }

                } else
                {
                    for (Class Cur = cl; Cur != null; Cur = Cur.getSuperclass())
                    {
                        Field[] fields = Cur.getDeclaredFields();

                        for (Field F : fields)
                        {
                            Class FieldType = F.getType();

                            F.setAccessible(true);

                            if (F.isAccessible() && (Modifier.isStatic(FieldType.getModifiers())))
                            {
                                FieldGetAction action;

                                //Array array = (Array) O;

                                if (Primatives.keySet().contains(FieldType))
                                {
                                    action = Primatives.get(FieldType);
                                } else
                                {
                                    action = new FieldGetAction()
                                    {
                                        public void get(Object obj, Field index) throws IllegalAccessException, IOException     
                                        {
                                            Object O = index.get(obj);
                                            if (O==null)  writeLong(0);
                                            else writeLong(state.References.get(O));
                                        }
                                    };
                                }

                                try
                                {
                                    action.get(O, F);
                                } catch (IllegalAccessException e)
                                {
                                    Log.e("SOb", Log.getStackTraceString(e));
                                }

                            }
                        }

                    }
                }
            }   
        }
    }
}

SequentialObjectInputStream

import java.io.*;
import java.util.*;
import java.lang.reflect.*;

import android.util.*;

public class SequentialObjectInputStream extends DataInputStream implements ObjectInput
{
    interface FieldPutAction
    {
        void put(Object obj, Field field) throws IllegalAccessException, IOException;
    }

    interface ArrayPutAction
    {
        void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException;
    }

    public HashMap<Class, FieldPutAction> Primatives;
    public HashMap<Class, ArrayPutAction> ArrayPrimatives;

    public SequentialObjectInputStream(InputStream stream)
    {
        super(stream);

        Primatives = new HashMap<Class, FieldPutAction>();

        try
        {
            Primatives.put(boolean.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        boolean x = readBoolean(); 
                        field.setBoolean(obj, x);

                    }
                });

            Primatives.put(byte.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        byte x = readByte(); 
                        field.setByte(obj, x);

                    }
                });


            Primatives.put(short.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        short x = readShort(); 
                        field.setShort(obj, x);

                    }
                });


            Primatives.put(int.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        int x = readInt(); 
                        field.setInt(obj, x);

                    }
                });


            Primatives.put(long.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        long x = readLong(); 
                        field.setLong(obj, x);

                    }
                });


            Primatives.put(char.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        char x = readChar(); 
                        field.setChar(obj, x);

                    }
                });


            Primatives.put(float.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        float x = readFloat(); 
                        field.setFloat(obj, x);

                    }
                });


            Primatives.put(double.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        double x = readDouble(); 
                        field.setDouble(obj, x);

                    }
                });


            Primatives.put(String.class,
                new FieldPutAction()
                {
                    public void put(Object obj, Field field) throws IllegalAccessException, IOException 
                    {
                        String x = readUTF(); 
                        field.set(obj, x);

                    }
                });
        } catch(Exception e)
        {
            Log.e("SOb", Log.getStackTraceString(e));
        }

        ArrayPrimatives = new HashMap<Class, ArrayPutAction>();

        try
        {
            ArrayPrimatives.put(boolean.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        boolean x = readBoolean();
                        Array.setBoolean(obj, index, x);
                    }
                });

            ArrayPrimatives.put(byte.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        byte x = readByte(); 
                        Array.setByte(obj, index, x);

                    }
                });


            ArrayPrimatives.put(short.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        short x = readShort(); 
                        Array.setShort(obj, index, x);

                    }
                });


            ArrayPrimatives.put(int.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        int x = readInt(); 
                        Array.setInt(obj, index, x);

                    }
                });


            ArrayPrimatives.put(long.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        long x = readLong(); 
                        Array.setLong(obj, index, x);

                    }
                });


            ArrayPrimatives.put(char.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        char x = readChar(); 
                        Array.setChar(obj, index, x);

                    }
                });


            ArrayPrimatives.put(float.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        float x = readFloat(); 
                        Array.setFloat(obj, index, x);

                    }
                });


            ArrayPrimatives.put(double.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        double x = readDouble(); 
                        Array.setDouble(obj, index, x);

                    }
                });


            ArrayPrimatives.put(String.class,
                new ArrayPutAction()
                {
                    public void put(Object obj, int index) throws ArrayIndexOutOfBoundsException, IOException   
                    {
                        String x = readUTF(); 
                        Array.set(obj, index, x);

                    }
                });
        } catch(Exception e)
        {
            Log.e("SOb", Log.getStackTraceString(e));
        }
    }


    @Override
    public Object readObject() throws ClassNotFoundException, IOException
    {
        long Total = readLong();

        Log.i("SOb", "readObject : " + Long.toString(Total) + " objects in graph");

        HashMap<Long, Object> References = new HashMap<Long, Object>();

        long currentId = 1;

        HashMap<Object, HashMap<Field, Long>> refCache =
            new HashMap<Object, HashMap<Field, Long>>();
        final HashMap<Object, HashMap<Integer, Long>> arefCache =
            new HashMap<Object, HashMap<Integer,Long>>();

        for (int I=0; I < Total; I++)
        {
            String Name = readUTF();
            Class C = Class.forName(Name);

            Log.i("SOb", "Object of "+C.getCanonicalName() +" on graph");

            int adim = 0;

            Object O = null;

            if (C.isArray())
            {
                Class ComponentType = C.getComponentType();

                int Size = readInt();

                Log.i("SOb", "array of "+ComponentType.getCanonicalName() + ", " + Long.toString(Size) + " elements");          
                O = Array.newInstance(ComponentType, Size);

                References.put(currentId, O);
                currentId++;

                ArrayPutAction action = null;

                if (ArrayPrimatives.keySet().contains(ComponentType))
                {
                    action = ArrayPrimatives.get(ComponentType);
                } else
                {
                    arefCache.put(O, new HashMap<Integer, Long>());

                    action = new ArrayPutAction()
                    {
                        public void put(Object O, int Index) throws ArrayIndexOutOfBoundsException , IOException
                        {
                            long Ref = readLong();

                            arefCache.get(O).put(Index, Ref);
                        }
                    };
                }

                for (int index=0; index< Size; index++)
                {
                    action.put(O,index);
                }

            } else
            {

            try
            {

                O = 
                    C.getConstructor(new Class[0]).newInstance(new Object[0]);
            } catch(InstantiationException e)
            {
                Log.e("SOb", Log.getStackTraceString(e));
            } catch(NoSuchMethodException e)
            {
                Log.e("SOb", Log.getStackTraceString(e));
            } catch(IllegalAccessException e)
            {
                Log.e("SOb", Log.getStackTraceString(e));
            } catch(InvocationTargetException e)
            {
                Log.e("SOb", Log.getStackTraceString(e));
            }

            References.put(currentId, O);
            currentId++;
            refCache.put(O, new HashMap<Field, Long>());

            for (Field F : C.getFields())
            {
                if (F.isAccessible())
                {
                    Class T = F.getType();

                    if (Primatives.containsKey(T))
                    {
                        try
                        {
                            Primatives.get(T).put(O, F);
                        } catch (IllegalAccessException e)
                        {

                        }
                    } else
                    {
                        refCache.get(O).put(F, readLong());
                    }
                }
            }

        }
        }
        for (long I=0; I < Total; I++)
        {

            Object O = References.get(I+1);

            Class C = O.getClass();

            //Log.i("SOb", "get reference "+Long.toString(I)+" "+C.getCanonicalName());


            if (C.isArray())
            {
                HashMap<Integer,Long> aref_table = arefCache.get(O);

                if (ArrayPrimatives.containsKey(C.getComponentType()) == false)
                {

                    int len = Array.getLength(O);

                    for (int index=0; index<len; index++)
                    {
                        long r = aref_table.get(index);
                        Object ref = r == 0 ? null : References.get(r);

                        Array.set(O, index, ref);   
                    }
                }

            } else
            {

            HashMap<Field, Long> ref_table = refCache.get(O);

            for (Field F : C.getFields())
            {
                if (F.isAccessible())
                {
                    Class T = F.getType();

                    if (Primatives.containsKey(T) == false)
                    {
                        try
                        {
                            long r = ref_table.get(F);
                            Object ref = r == 0 ? null : References.get(r);

                            F.set(O, ref);
                        } catch (IllegalAccessException e)
                        {
                            Log.e("SOb", Log.getStackTraceString(e));
                        }

                    }
                }
            }
            }

        }


        return References.get((Long) (long) 1);
    }

}

SequentialObjectOutputStream