Saving Huge record returnd from DB in Memcache with PHP

Question

I am developing a site (using PHP + Memcache) in which users can have friends and is stuck at a point in thinking of saving a list of Friends of a user which has more than 100K friends. Suppose, if user has 100K friends, how would you save them in Memcache? Currently I have this code.

$ttl = count($result); //about 10K friends
for($i=0; $i<$ttl; $i++)
{
    $friendslist[$result[$i][0]] = $result[$i]; //$result[$i][0] = Friend's ID
}

$mem->set($usrid,$friendslist);

This creates many problems.

1) First of all memory issue and performance in saving and retrieving such huge array.

2) Cannot save more than 1mb in array

Any Einstein have good trick to save such records in memcache? What would be the code to save and retrieve such list? Any idea how Twitter or Facebook would be saving if such huge records are returnd from DB?

Answer 1

First of all I would suggest you to not save such huge record in single array instead call needed list of friends from Database just like pagination. But if still you need it then I am not sure if this is correct approach or not but you can give it a try.

$ttlsets = count($result)/1000; //100K = 100 sets

for($i=0; $i<$ttlsets; $i++)
{
    $friendslist = array();
    for($j=0; $j<100; $j++)
    {
       $friendslist[$result[$i][0]] = $result[$i]; //$result[$i][0] = Friend's ID
    }
    $mem->set($usrid.'set'.$i, $friendslist);
}

$mem->set($usrid.'counter', $ttlsets);

Now you have a list of friends with Sets. And you can also check how many sets you have. Now you can AJAX and call each Set individually having 100 friends in one set. So, now you dont have to go DB. But again this is not a full proof solution but shows everything fine till here.