So I am removing the Joins from my queries as I prepare to move to Cassandra which does not support this, but rather supports many select statements instead. I made a benchmark test on 50 rows of data in my mysql table (what I am currently using) which resulted in 101 queries (all select) and it took ~0.035 seconds to complete all of these queries. I then changed this around to some array manipulation (currently in PHP) and reduced this down to 3 queries with a bunch of O(n) for loops.
I assume whether my system is on PHP, Python, MySQL, or Cassandra (NoSQL) that it is way faster to process the data using a few O(n) for loops rather than a lot more queries, I had cut down the time from 0.035s to 0.004s using this new method as I show below.
Any alternate methods to shortening this down more? Or am I on the right track? Any cases where it's faster to run all of the queries (besides when it becomes O(n^2))? Thanks:
// Now go through and get all of the user information (This is slower in mysql, but maybe faster in cassandra)
/*foreach ($results as $key => $row)
{
// Create query
$query = DB::select('id', 'username', 'profile_picture')->from('users')->where('id', '=', $row['uid']);
// Execute it
$results2 = $query->execute(null, false);
// Join it
$data[$key] = array_merge($row, $results2[0]);
}*/
// Get all the user information (faster in mysql since less queries)
$uids = array();
$ids = array();
foreach ($results as $key => $row)
{
if (!in_array($row['uid'], $uids))
$uids[] = $row['uid'];
if (!in_array($type, array('userProfile')))
$ids[] = $row['comment_id'];
}
// Create query
$query = DB::select('id', 'username', 'profile_picture')->from('users')->where('id', '=', $uids);
// Execute it
$results2 = $query->execute(null, false);
$user_data = array();
foreach ($results2 as $key => $row)
{
$user_data[$row['id']] = array('uid' => $row['id'], 'username' => $row['username'], 'profile_picture' => $row['profile_picture']);
}
foreach ($results as $key => $row)
{
$data[$key] = array_merge($row, $user_data[$row['uid']]);
}
// End faster user info section
With Cassandra you can ask for all your keys in one query using a multi get, which is much faster than a bunch of single queries. I sometimes ask for thousands of keys in a query, and the response time is effectively instant.
There are more and more tools like playOrm(also has a raw ad-hoc tool coming) that support joins BUT only on partitions of tables(not entire tables) and do indexing with nosql patterns behind the scenes. Check out the wide-row pattern and see if that is useful to you. IT can help speed things up sometimes.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.