Wordpress display thousands of posts

Question

I'm working on a project where I need to list all the categories, sub-categories (children, grand-children etc. to about 5th level) and posts inside them. At the moment I've managed to get the site working, but the issue is that the site takes minutes to load.

To be clear on what I want the site to look like, here's a drawing:

Category Sub-category Subsubcategory Content Subsubcategory Content Sub-category You get the picture

And here's the code:

<?php
    function listing($parentcat, $list_id='', $list_name='', $path=false) {
        // parentcat is the desired category parent category, which is defined because the function is called a few times with different sets on the page.

        // list_id, list_name and path are used for purposes not related to the question

            echo "<ul><li name='$list_name'><h2><a href='#$list_name'>$list_name</a></h2>";
        }

        $args = array(
            'parent' => $parentcat,
            'include' => $cat_ids,
            'hide_empty' => 1,
            'orderby' => 'id'
        );

        $categories = get_categories( $args );
        echo "<ul>";
            foreach ($categories as $cat) {
                if ($cat->cat_name != 'Uncategorized') {

                    $flat_path = substr(get_category_parents($cat->cat_ID, false, ' &raquo;' ), 14);
                    $catnam = $cat->cat_name;

                    $listtitle = ($path ? $flat_path : $catnam);
                    echo ('<li name="' . $cat->cat_ID . '"><h2><a href="#' . $cat->cat_ID . '">' . $listtitle . '</a></h2>' );

                    if (get_posts( "category_name=$catnam" ) ) {
                        if (have_posts()) : while (have_posts()) : the_post();
                        if (in_category($cat->cat_ID)) {
                                echo '<ul><li><div>';
                                    the_content();
                                echo '</div></li></ul>';
                        } endwhile;
                        else :
                        _e('Empty list');
                        endif;
                    }

                    // Here's a recursive call for the function
                    listing($cat->cat_ID);
                    echo '</li>';
                }
            }
        echo '</ul>';
    }
    ?>

Answer 1

You might be missing 'posts_per_page' for posts

Have a look here http://codex.wordpress.org/Function_Reference/get_posts

&

'number' for categories

Have a look here http://codex.wordpress.org/Function_Reference/get_categories

Answer 2

Solution found! I'll answer to myself, so that if anybody else needs something similar, the answer is here.

Basically, the point is to first create an array which contains all the categories. After that one can simply loop through the array. This way we were able to avoid recursively using The Loop, which probably was the biggest issue in the previous solution.

Even this solution is really slow, but an incredible improvement over the last. First idea loaded few hundred posts tops before timeout after roughly 30 seconds. This one gets the whole site (about 1500 posts) in under 5-10 seconds. The speed is awful, but with the way our site will be used and the added functionality from dividing everything to separate posts outnumbers the speed issues.

$categories = get_categories('orderby=id');

    $cats_by_parent = array();

    foreach ($categories as $cat) {
        $parent_id = $cat->category_parent;
        if (!array_key_exists($parent_id, $cats_by_parent)) {
            $cats_by_parent[$parent_id] = array();
        }
        $cats_by_parent[$parent_id][] = $cat;
    }

    $posts = get_posts(['posts_per_page' => 10000000]);
    $posts_by_cats = array();

    foreach ($posts as $post) {
        $cat_id = wp_get_post_categories($post->ID)[0];
        $posts_by_cats[$cat_id] = $post->post_content;
    }

// Loop through the array and print with:
    echo $posts_by_cats[$cat->cat_ID];