从Firebase加载大型json时，Android应用返回java.lang.OutOfMemoryError

Question

I have a machine learning app that takes an array of weights from Firebase every time it updates. In the past this worked fine, but since I have begun implementing a neural network, that array size has jumped from 8,000 to 200,000. Because of this, when I pull the array as a json object, I get this error:

java.lang.OutOfMemoryError
                                                                  at java.util.HashMap.makeTable(HashMap.java:569)
                                                                  at java.util.HashMap.doubleCapacity(HashMap.java:589)
                                                                  at java.util.HashMap.put(HashMap.java:419)
                                                                  at com.google.android.gms.internal.zzahj.getValue(Unknown Source)
                                                                  at com.google.android.gms.internal.zzahj.getValue(Unknown Source)
                                                                  at com.google.android.gms.internal.zzahj.getValue(Unknown Source)
                                                                  at com.google.firebase.database.DataSnapshot.getValue(Unknown Source)
                                                                  at myapp.DataSend$5.onDataChange(DataSend.java:223)
                                                                  at com.google.android.gms.internal.zzafp.zza(Unknown Source)
                                                                  at com.google.android.gms.internal.zzagp.zzSu(Unknown Source)
                                                                  at com.google.android.gms.internal.zzags$1.run(Unknown Source)
                                                                  at android.os.Handler.handleCallback(Handler.java:733)
                                                                  at android.os.Handler.dispatchMessage(Handler.java:95)
                                                                  at android.os.Looper.loop(Looper.java:136)
                                                                  at android.app.ActivityThread.main(ActivityThread.java:5001)
                                                                  at java.lang.reflect.Method.invokeNative(Native Method)
                                                                  at java.lang.reflect.Method.invoke(Method.java:515)
                                                                  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785)
                                                                  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601)
                                                                  at dalvik.system.NativeStart.main(Native Method)

And the code in question (Datasend 223) is this:

public void checkoutListener(){
        final TextView message = (TextView) findViewById(R.id.messageDisplay);
        userValues.addValueEventListener(new ValueEventListener() {
            @Override
            public void onDataChange(DataSnapshot dataSnapshot) {
                userCheck = dataSnapshot.getValue(UserData.class);
            }

            @Override
            public void onCancelled(DatabaseError firebaseError) {

            }
        }

        );

    }

Is there any way I can prevent this, or is an array of 200,000 doubles just too much for the android heap to handle?

Answer 1

Try to make use of the flyweight design pattern here. Best would be to store your weights in external memory and read in batches while applying the same to the network nodes. You might need to change the way the weights are being set to the nodes of the hidden layers in your neural network.

Flyweight design pattern