Elasticsearch 部分批量更新

Question

我有6k數據要在ElasticSearch 中更新。 我必須使用PHP 。 我在文檔中搜索，我發現了這個， 批量索引，但這並沒有保留以前的數據。

我有結構：

[
  {
    'name': 'Jonatahn',
    'age' : 21
  }
]

我要更新的代碼片段：

$params =[
    "index" => "customer",
    "type" => "doc",
    "body" => [
        [
            "index" => [
                "_index" => "customer",
                "_type" => "doc",
                "_id" => "09310451939"
            ]
        ],
        [
            "name" => "Jonathan"
        ]
    ]
];

$client->bulk($params);

當我發送['name' => 'Jonathan']我希望name會被更新並保持age ，但age會被刪除。 當然，我仍然可以逐個更新數據，但這需要很長時間，有沒有更好的方法來做到這一點？

Answer 1

我的錯誤是使用"index" ，但正確的方法來做我想要的，是"update" 。

最終的代碼是：

$params =[
"index" => "customer",
"type" => "doc",
"body" => [
    [
        "update" => [
    //   ^^^^^^ Here I change from index to update
            "_index" => "customer",
            "_type" => "doc",
            "_id" => "09310451939"
        ]
    ],
    [
        "doc" => [
            "name" => "Jonathan"
        ]
    ]
]
];

$client->bulk($params);

使用上面的代碼，我的數據保留以前的數據，只更新我傳遞的參數數據。

響應：

Array
(
    [took] => 7
    [timed_out] =>
    [_shards] => Array
        (
            [total] => 5
            [successful] => 5
            [skipped] => 0
            [failed] => 0
        )

    [hits] => Array
        (
            [total] => 1
            [max_score] => 1
            [hits] => Array
                (
                    [0] => Array
                        (
                            [_index] => customer
                            [_type] => doc
                            [_id] => 09310451939
                            [_score] => 1
                            [_source] => Array
                                (
                                    [name] => Jonathan
                                    [age] => 23
                                )

                        )

                )

        )

)

Answer 2

根據文檔 ，批量API可能的操作是索引，創建，刪除和update 。 update期望在下一行指定部分doc，upsert和script及其選項。

POST _bulk
{ "update" : {"_id" : "1", "_type" : "_doc", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }

Answer 3

這是我的最終代碼。

<?php

require_once('../elasticsearch.php');

//initialize elasticsearch
$params = array();

$params['index'] = $elastcsearch_index;
$params['type']  = $elastcsearch_type;

///////////////////////////////////////////////////
//update seeders n leechers in elasticsearch 

//get updated records
$get_updated_records = mysqli_query($conn, "SELECT content_id, seeders, leechers FROM content WHERE is_updated = '1' order by seeders DESC") ;

//create blank array
$results = array();

while($row = mysqli_fetch_assoc($get_updated_records)){
    //put all results in array
    $results[] = $row;

}   

//from https://www.elastic.co/guide/en/elasticsearch/client/php-api/current/_indexing_documents.html

$params = ['body' => []];

for($i = 0; $i < count($results); $i++) {

    $params["body"][]= [
            "update" => [
                "_index" => $elastcsearch_index,
                "_type" => $elastcsearch_type,
                "_id" => $results[$i]['content_id']
            ]
        ];

    $params["body"][]= [
            "doc" => [
                "seeders" => intval($results[$i]['seeders']) ,
                "leechers" => intval($results[$i]['leechers']) ,
            ]
        ];

    // Every 1000 documents stop and send the bulk request
     if ($i % 1000 == 0) {
        $responses = $elasticsearch->bulk($params);

        // erase the old bulk request
        $params = ['body' => []];

        // unset the bulk response when you are done to save memory
        unset($responses);
    } 
}

// Send the last batch if it exists
if (!empty($params['body'])) {
    $responses = $elasticsearch->bulk($params);
}

Answer 4

$batch_elastics 是結果數組，我只是每次都從行中取消設置這兩個值...... bcoz 在插入或更新中不需要這個值 unset($batch_row['type']); 未設置（$batch_row['diamonds_id']）；

代碼從這里開始......

    if(count($batch_elastics)){
        // echo 'hi';die;
        $params = array();                
        $params = ['body' => []]; 
        $i=1;       
        foreach($batch_elastics as $batch_row){
            $type=$batch_row['type'];
            $id=$batch_row['diamonds_id'];
            unset($batch_row['type']);
            unset($batch_row['diamonds_id']); 
            if($type=="create"){                                    
                $params["body"][]= [
                        "create" => [
                            "_index" => 'diamonds',                                                        
                            "_id" => $id,
                        ]
                    ];        
                    $params["body"][]= $batch_row;                             
                if ($i % 1000 == 0) {
                    $responses = $client->bulk($params);                                
                    $params = ['body' => []];                                
                    unset($responses);
                }
            } 
            $i=$i+1;
        }
        
        // Send the last batch if it exists
        if (!empty($params['body'])) {
            $responses = $client->bulk($params);
        }
        $params = array();                
        $params = ['body' => []]; 
        $i=1; 
        foreach($batch_elastics as $batch_row){
            $type=$batch_row['type'];
            $id=$batch_row['diamonds_id'];
            unset($batch_row['type']);
            unset($batch_row['diamonds_id']); 
            if($type=="update"){                                    
                $params["body"][]= [
                        "update" => [
                            "_index" => 'diamonds',                                                        
                            "_id" => $id,
                        ]
                    ];        
                $params["body"][]= [
                    "doc"=>$batch_row
                ];                           
                if ($i % 1000 == 0) {
                    $responses = $client->bulk($params);                                
                    $params = ['body' => []];                                
                    unset($responses);
                }
            } 
            $i=$i+1;
        }
        
        // Send the last batch if it exists
        if (!empty($params['body'])) {
            $responses = $client->bulk($params);
        }
    }