how can I know that the scroll in elasticsearch if it works

Question

I'm using elasticsearch-php.

I use this code, in the tests that I do it works well.

require_once 'app/init.php';

if (isset($_GET['q'])) {
                $q = $_GET['q'];
                $query= $es->search([
                      'index' => 'rebajas',
                      //'search_type' => 'scan',
                      'scroll' => '2m',
                      //'from' => 0,
                      'size' => 1000,
                      'body'=>[
                        'query'=>[
                          'bool' => [
                            'should' => [
                              'match'=>['titulo'=>$q],

                            ]
                          ]

                        ]

                      ]


                    ]);
                  if ($query['hits']['total']>=1) {

                      $scroll_size = $query['hits']['total'];
                      $results = $query['hits']['hits'];

                      //scrol
                      $scroll_id = $query['_scroll_id'];


                        print " total results:   " . $scroll_size;
                        $count = 0;
                        // first set of scroll results
                        for ($i=0; $i<$scroll_size; $i++) {
                            $count++;
                        }
                        //scroll
                        while (isset($query['hits']['hits']) && count($query['hits']['hits']) > 0) {

                            // **
                            // Do your work here, on the $response['hits']['hits'] array
                            // **
                            $conta =0;
                            foreach ($results as $r ) {
                                $conta++;

                        ?>

                        <h1 class="mt-5"><a href=""><?php echo $conta." " .$r['_source']['titulo'];?></a> <i class="fa fa-search-plus" aria-hidden="true"></i></h1>


                        <?php
                            }

                            // When done, get the new scroll_id
                            // You must always refresh your _scroll_id!  It can change sometimes
                            $scroll_id = $query['_scroll_id'];

                            // Execute a Scroll request and repeat
                            $query = $es->scroll([
                                    "scroll_id" => $scroll_id,  //...using our previously obtained _scroll_id
                                    "scroll" => "30s"           // and the same timeout window
                                ]
                            );
                        }


                    }

                }

The questions are: How does this scroll work, when it is in production, extracting about ten thousand records, will it not block the server with so many records? How can I check what is it loading? This works the same as infinite scroll, that is to say, as I go down the page, the records are loaded.

Answer 1

Is this question how to implement an infinite scroll that is backed by elasticsearch? That's a whole class/framework worth of questions. But let me take a stab at it.

If you want an infinite scroll you'll probably want to first break out your data APIs from your webserving endpoints (ie the data shouldn't load with your page [you'll probably want your first page to get served with your webpage, but it doesn't have to and for loading next page you'll almost certainly want to have that as its own API]). Then you'll need to your AJAX to request pages as the user scrolls down your webpage (ie if user scrolls to n / PAGE_SIZE then make request for next page). You can basically google 'infinite scroll implementation' but this captures the intention pretty well .

That being said this is generally not a great use case for elasticsearch (using ES as a direct infinite scroll backing for a website). Most of the reasons why are encapsulated here . It basically comes down to the fact that having many open scrolls is expensive in elasticsearch though if you have few users and a strict timeout it may be ok? Scrolls were not intended for 'real-time' uses but more for data export use cases.