elasticsearch mapping Expected map for property [fields] on field [name] but got a class java.lang.String

Question

Here is my data, It's based on a schema and I need to generate mapping to be indexed on ES. My background with ES is not much but I thought I got it until I tried it and failed and can't find the right answer online..

{
  "@context": {
    "schema": "http://schema.org/",
    "outbreak": "https://discovery.biothings.io/view/outbreak/"
  },
  "@type": "outbreak:Publication",
  "keywords": [
    "COVID-19",
    "City lockdown",
    "Epidemic",
    "Governmental action",
    "Individual reaction",
    "Mathematical modelling"
  ],
  "author": [
    {
      "@type": "outbreak:Person",
      "affiliation": [
        {
          "@type": "outbreak:Organization",
          "name": "Department of Applied Mathematics, Hong Kong Polytechnic University, Hong Kong, China. Electronic address: daihai.he@polyu.edu.hk."
        }
      ],
      "familyName": "He",
      "givenName": "Daihai",
      "name": "Daihai He"
    }
  ],
  "publicationType": [
    "Journal Article"
  ],
  "_id": "pmid32145465",
  "curatedBy": {
    "@type": "schema:WebSite",
    "name": "litcovid",
    "url": "https://www.ncbi.nlm.nih.gov/research/coronavirus/publication/32145465"
  },
  "name": "A conceptual model for the coronavirus disease 2019 (COVID-19) outbreak in Wuhan, China with individual reaction and governmental action.",
  "identifier": "32145465",
  "pmid": "32145465",
  "abstract": "The ongoing coronavirus disease 2019 (COVID-19) outbreak, emerged in Wuhan, China in the end of 2019, has claimed more than 2600 lives as of 24 February 2020 and posed a huge threat to global public health. The Chinese government has implemented control measures including setting up special hospitals and travel restriction to mitigate the spread. We propose conceptual models for the COVID-19 outbreak in Wuhan with the consideration of individual behavioural reaction and governmental actions, e.g., holiday extension, travel restriction, hospitalisation and quarantine. We employe the estimates of these two key components from the 1918 influenza pandemic in London, United Kingdom, incorporated zoonotic introductions and the emigration, and then compute future trends and the reporting ratio. The model is concise in structure, and it successfully captures the course of the COVID-19 outbreak, and thus sheds light on understanding the trends of the outbreak.",
  "license": "Copyright © 2020 The Authors. Published by Elsevier Ltd.. All rights reserved.",
  "journalName": "International journal of infectious diseases : IJID : official publication of the International Society for Infectious Diseases",
  "journalAbbreviation": "Int. J. Infect. Dis.",
  "issueNumber": "1878-3511",
  "doi": "S1201-9712(20)30117-X",
  "url": "https://www.doi.org/S1201-9712(20)30117-X",
  "datePublished": "2020-03-04",
  "dateModified": "2020-02-26"
}

and here is my mapping so far:

{
                'fields':{
                    'type': 'string'
                },
                'abstract': {
                    'type': 'text'
                },
                'pmid': {
                    'type': 'integer'
                },
                'author': {
                    'type': 'nested',
                    'properties': {
                        'name':{
                            'type': 'text'
                        },
                        'givenName':{
                            'type': 'text'
                        },
                        'familyName':{
                            'type': 'text'
                        },
                        'affiliation':{
                            'type': 'nested',
                            'properties': {
                                'name':{
                                    'type': 'text'
                                }
                            }
                        }
                    }
                },
                'isBasedOn': {
                    'type': 'text'
                },
                'funding': {
                    'type': 'nested',
                    'properties': {
                        'funder':{
                            'type': 'nested',
                            'properties':{
                                'name': 'text'
                            }
                        },
                        'identifier':{
                            'type': 'text'
                        }
                    }
                },
                'license': {
                    'type': 'text'
                },
                'keywords': {
                    'normalizer': 'keyword_lowercase_normalizer',
                    'type': 'keyword',
                    'copy_to': ['all']
                },
                'publicationType': {
                    'normalizer': 'keyword_lowercase_normalizer',
                    'type': 'keyword',
                    'copy_to': ['all']
                },
                'name': {
                    'type': 'text'
                },
                'journalName': {
                    'type': 'text'
                },
                'identifier': {
                    'type': 'text'
                },
                'doi': {
                    'type': 'text'
                },
                'datePublished': {
                    'type': 'date'
                },
                'dateModified': {
                    'type': 'date'
                },
                'issueNumber': {
                    'type': 'text'
                }
         }

I don't have a field "fields" in my data so I'm not sure what this means and "name" is a simple string

I've tried this and also including "mappings":{"properties":{...}} but that also fails. Any pointers??

Answer 1

Try to use text or keyword instead of string for field fields

string is not a valid data type in Elasticsearch. You could either use keyword or text

• Text data Type

  A field to index full-text values, such as the body of an email or the description of a product. These fields are analyzed, that is they are passed through an analyzer to convert the string into a list of individual terms before being indexed. The analysis process allows Elasticsearch to search for individual words within each full text field. Text fields are not used for sorting and seldom used for aggregations (although the significant text aggregation is a notable exception).

• Keyword data type

  A field to index structured content such as IDs, email addresses, hostnames, status codes, zip codes or tags.

Source: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html

Answer 2

There were two issues in your mapping

1. string was used, it is no longer valid data type, use text instead

2. 'properties':{'name': 'text'} should be "name":{"type":"text"}

You are using normalizer, I don't know your requirement so have a check if you need normalizer or analyzer

Corrected mapping

"fields": {
        "type": "text"
      },
      "abstract": {
        "type": "text"
      },
      "pmid": {
        "type": "integer"
      },
      "author": {
        "type": "nested",
        "properties": {
          "name": {
            "type": "text"
          },
          "givenName": {
            "type": "text"
          },
          "familyName": {
            "type": "text"
          },
          "affiliation": {
            "type": "nested",
            "properties": {
              "name": {
                "type": "text"
              }
            }
          }
        }
      },
      "isBasedOn": {
        "type": "text"
      },
      "funding": {
        "type": "nested",
        "properties": {
          "funder": {
            "type": "nested",
            "properties": {
              "name": {
                "type": "text"
              }
            }
          },
          "identifier": {
            "type": "text"
          }
        }
      },
      "license": {
        "type": "text"
      },
      "keywords": {
        "normalizer": "keyword_lowercase_normalizer",
        "type": "keyword",
        "copy_to": [
          "all"
        ]
      },
      "publicationType": {
        "normalizer": "keyword_lowercase_normalizer",
        "type": "keyword",
        "copy_to": [
          "all"
        ]
      },
      "name": {
        "type": "text"
      },
      "journalName": {
        "type": "text"
      },
      "identifier": {
        "type": "text"
      },
      "doi": {
        "type": "text"
      },
      "datePublished": {
        "type": "date"
      },
      "dateModified": {
        "type": "date"
      },
      "issueNumber": {
        "type": "text"
      }