Error handling with curl and elasticsearch

Question

I'm currently developing bash scripts that use elasticsearch and I need a good error-handling. In this situation I try to add a document to elasticsearch and check if the operation succeeded.

At first I naively tried this :

response=$(curl -XPOST 'http://localhost:9200/indexation/document' -d '
{
  "content":"'"$txt"'",,
  "date_treatment":"'"$(date +%Y-%m-%d)"'"
}') && echo ok || echo fail

But curl doesn't work that way and still returns success (0 - which is actually logical) even though the json request is obviously incorrect (note the double comma on line 3) and elasticsearch displays errors.

So the answer isn't there. Now I think I should analyze the variable $response to catch errors (grep ?). I post this question to get hints or solutions on the way to do this in a reliable way and to make sure I'm not missing an obvious solution (maybe a curl option I don't know ?).

Additional useful things

Parsing JSON with Unix tools

Examples of the content of $response :

success :

{
    "_id": "AVQz7Fg0nF90YvJIX_2C",
    "_index": "indexation",
    "_shards": {
        "failed": 0,
        "successful": 1,
        "total": 1
    },
    "_type": "document",
    "_version": 1,
    "created": true
}

error :

{
    "error": {
        "caused_by": {
            "reason": "json_parse_exception: Unexpected character (',' (code 44)): was expecting either valid name character (for unquoted name) or double-quote (for quoted) to start field name\n at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@139163f; line: 3, column: 17]",
            "type": "json_parse_exception"
        },
        "reason": "failed to parse",
        "root_cause": [
            {
                "reason": "json_parse_exception: Unexpected character (',' (code 44)): was expecting either valid name character (for unquoted name) or double-quote (for quoted) to start field name\n at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@139163f; line: 3, column: 17]",
                "type": "json_parse_exception"
            }
        ],
        "type": "mapper_parsing_exception"
    },
    "status": 400
}

Answer 1

A simple workaround is to use the -f/--fail option.

As per documentation :

(HTTP) Fail silently (no output at all) on server errors. This is mostly done to better enable scripts etc to better deal with failed attempts. In normal cases when an HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.

This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407).

example:

response=$(curl -XPOST 'http://localhost:9200/indexation/document' -d '
{
  "content":"'"$txt"'",,
  "date_treatment":"'"$(date +%Y-%m-%d)"'"
}' -f ) && echo ok || echo fail