Azure search indexer : cannot create an indexer based on mongodb data source with a collection containing a field named '$ref'

Question

As described in the title, i am facing a strange error while creating an indexer (using both portal azure and Rest api).

{
    "error": {
        "code": "",
        "message": "Error with data source: Additional content found in JSON reference object. A JSON reference object should only have a $ref property. Path '$id'.  Please adjust your data source definition in order to proceed."
    }
}

datasource was created via the azure portal without specifying delete or change strategy.

JSON Structure in comosdb (MongoDb) Post collection

{
  "_id": {
    "$oid": "....."
  },
  "author": {
    "$ref": "user",
    "$id": {
      "$oid": "...."
    }
  },
  "_class": "com.community.domain.Post"
}

bellow the indexer definition

{
"dataSourceName": "fshco-post",
"targetIndexName": "index",
"fieldMappings": [
{
"sourceFieldName": "_class",
"targetFieldName": "class"
}

    ],
    "parameters": {
        "batchSize": 1000,
        "maxFailedItems": null,
        "maxFailedItemsPerBatch": null
    }

}

To confirm that the problem is the $ref attribute.I have used one collection Post containing one document but without the child attribute $ref in the author field, and it was indexed succesfully.

I have tried the skillset **ShaperSkill **to modify the $ref naming, but also didnt work with the same error. After that, I understand that the problem is probably in the cracking data phase before the skillset execution phase. indexing phases

bellow the definition skillset that i have used:

 {
      "@odata.type": "#Microsoft.Skills.Util.ShaperSkill",
      "name": "#1",
      "description": null,
      "context": "/document",
      "inputs": [
        {
          "name": "refto",
          "source": "/document/author/$ref"
        },
        {
          "name": "id",
          "source": "/document/author/$id"
        }
      ],
      "outputs": [
        {
          "name": "output",
          "targetName": "post_author"  --> same name as the index attribute
        }
      ]
    }
  ]

In the Indexer

    "skillsetName": "skillpostshaper",
    "outputFieldMappings": [
        {
            "sourceFieldName": "/document/post_author",
            "targetFieldName": "post_author"
        }
    ],

Is there anything obvious that I've missed?

Answer 1

AFAIK, index field name should not be started with special characters as mentioned here Using field mappings in indexer, I have done one field to another field and below are steps i followed,

1. created data source, index and indexer.
2. Added new filed in index with name ref.
3. In indexer, added field mapping as shown below. Here mapping ref field to the existing field with name HotelName.
4. Once ran indexer, able to get data in ref field.

Answer 2

The error you are facing is related to the use you are giving to $ref in a JSON file and not related to the indexer per se. The $ref keyword in JSON files is a "reserved word" that can only be used to point to a reference. As suggested as part of the comments, modifying the field name in the data source before indexing it, by removing the $ref property or renaming it to a different field name, should let the indexer read the JSON file in the way you expect.

Here is the reference to JSON documentation for more information.