ElasticSearch：仅索引映射中指定的字段

Question

I have an ElasticSearch setup, receiving data to index via a CouchDB river. I have the problem that most of the fields in the CouchDB documents are actually not relevant for search: they are fields internally used by the application (IDs and so on), and I do not want to get false positives because of these fields. Besides, indexing not needed data seems to me a waste of resources.

To solve this problem, I have defined a mapping where I specify the fields which I want to be indexed. I am using pyes to access ElasticSearch. The process that I follow is:

1. Create the CouchDB river, associated to an index. This apparently creates also the index, and creates a "couchdb" mapping in that index which, as far as I can see, includes all fields, with dynamically assigned types.
2. Put a mapping, restring it to the fields which I really want to index.

This is the index definition as obtained by:

curl -XGET http://localhost:9200/notes_index/_mapping?pretty=true

{
  "notes_index" : {
    "default_mapping" : {
      "properties" : {
        "note_text" : {
          "type" : "string"
        }
      }
    },
    "couchdb" : {
      "properties" : {
        "_rev" : {
          "type" : "string"
        },
        "created_at_date" : {
          "format" : "dateOptionalTime",
          "type" : "date"
        },
        "note_text" : {
          "type" : "string"
        },
        "organization_id" : {
          "type" : "long"
        },
        "user_id" : {
          "type" : "long"
        },
        "created_at_time" : {
          "type" : "long"
        }
      }
    }
  }
}

The problem that I have is manyfold:

• that the default "couchdb" mapping is indexing all fields. I do not want this. Is it possible to avoid the creation of that mapping? I am confused, because that mapping seems to be the one which is somehow "connecting" to the CouchDB river.
• the mapping that I create seems not to have any effect: there are no documents indexed by that mapping

Do you have any advice on this?

EDIT

This is what I am actually doing, exactly as typed:

server="localhost"

# Create the index
curl -XPUT    "$server:9200/index1"

# Create the mapping
curl -XPUT    "$server:9200/index1/mapping1/_mapping" -d '
{
    "type1" : {
        "properties" : {
            "note_text" : {"type" : "string", "store" : "no"}
        }
    }
}
'

# Configure the river
curl -XPUT "$server:9200/_river/river1/_meta" -d '{
    "type" : "couchdb",
    "couchdb" : {
        "host" : "localhost",
        "port" : 5984,
        "user" : "admin",
        "password" : "admin",
        "db" : "notes"
    },
    "index" : {
        "index" : "index1",
        "type" : "type1"
    }
}'

The documents in index1 still contain fields other than "note_text", which is the only one that I have specifically mentioned in the mapping definition. Why is that?

Answer 1

The default behavior of CouchDB river is to use a 'dynamic' mapping, ie index all the fields that are found in the incoming CouchDB documents. You're right that it can unnecessarily increase the size of the index (your problems with search can be solved by excluding some fields from the query).

To use your own mapping instead of the 'dynamic' one, you need to configure the River plugin to use the mapping you've created (see this article ):

curl -XPUT 'elasticsearch-host:9200/_river/notes_index/_meta' -d '{
    "type" : "couchdb",

    ... your CouchDB connection configuration ...

    "index" : {
        "index" : "notes_index",
        "type" : "mapping1"
    }
}'

The name of the type that you're specifying in URL while doing mapping PUT overrides the one that you're including in the definition, so the type that you're creating is in fact mapping1 . Try executing this command to see for yourself:

> curl 'localhost:9200/index1/_mapping?pretty=true'

{
  "index1" : {
    "mapping1" : {
      "properties" : {
        "note_text" : {
          "type" : "string"
        }
      }
    }
  }
}

I think that if you will get the name of type right, it will start working fine.