hw can i compare two perl hashes (keys and values) extract difference and apply to a hash in javascript

Question

Caveat, I hope I'm asking the right question, feel free to suggest alternatives.

Situation; I have a site written mainly in javascript that runs on fixed and mobile devices. The site is data heavy. It maps a load of moving objects and other interactions. Each client receives an update to the data from the server every few seconds, and so, I need to minimise the amount of data sent. Currently, the server sends a rather large hash containing the current state of all the objects on each update. This is very inefficient because much of the data remains unchanged.

I believe that the approach that will make the biggest difference to the amount of data sent, with the least coding and testing, is to only send the changes to the hash each update.

So, I'm wondering if there are any tried and tested methods for comparing the data (hash) that was last sent, to the data (hash) that will be sent, extracting the differences, and then applying the difference to a hash on the javascript side? I hope that makes sense?

When I say compare, I mean keys and values. Currently the hashes are not ordered in any particular way, however it's not an issue if they need to be.

I've had a look at things like Data::Compare, however it seems to just tell me if the hashes are different, not what the differences are (unless I'm reading that wrong?).

Answer 1

I had same issue with sending diffs of objects (JSON format) from server to client in somewhat high frequency. The solution we came up with was very simple. Since there is no official or standard way to express the differences between json objects, we came up with our own protocol on how to define which objects were added, which objects were updates (ie properties of these objects where changed (added/updated/removed) and which objects were removed altogether.

One of possible delta payloads might be:

{
  added: [
    /* array of new objects */
  ],
  removed: [
    /* array of object identifiers that need to be removed */
  ],
  updated: { /* key value pairs of object identifiers with their property maps */
    obj_id_01: {
      updated: { /* key-value pairs of updated properties */ },
      removed: [ /* array of keys of removed properties in an object */ ]
    },
    obj_id_02: {
    },
    ...
  }
}

Answer 2

I developed a similar solution for a similar problem..

my $bef = {
  name => 'Fred',
  wife => 'Wilma',
  hobby => 'Breaking Rocks',
  friends => [qw! Barney Wilma Betty !],
};

my $aft = { 
  name => 'Fred',
  pet => 'Dino',
  hobby => 'Bowling',
  friends => [qw! Barney Betty Dino !],
  kids => [qw! Bam Pebbles !],
};

my $differ = Lecstor::FeedProxy::Diff->new;

my $diff = $diff->differences($bef, $aft);

$diff: {
  'pet' => 'Dino',
  'wife' => undef,
  'hobby' => 'Bowling',
  'friends' => {
    'remove' => [ 'Wilma' ],
    'add' => [ 'Dino' ]
  },
  'kids' => {
    'add' => [ 'Bam', 'Pebbles' ]
  }
};

https://github.com/lecstor/Lecstor/blob/master/lib/Lecstor/FeedProxy/Diff.pm