Azure Cognitive Search Ignore fields

Question

Is there a way to ignore any fields that don't exist on the Index or don't have a field mapping on the Indexer? Or a way to specify on the Indexer or on Import to ignore a specific field?

I am creating a Azure Search Index where the data source is CosmosDB. Because CosmosDB has a flexible schema I may periodically add new fields. I would prefer these not be indexed and it remain the original schema. Is there a way to set on either the Index or Indexer to ignore new fields (or even this specific field) when add new items to the Index?

I didn't see anything to ignore fields in the field mapping docs: https://learn.microsoft.com/en-us/azure/search/search-indexer-field-mappings

Answer 1

The solution depends on which model you use.

1. Push model: You push content via the Azure Search SDK. In this scenario you have full control over the objects you submit to search and your problem is not an issue. If you don't want a field, you don't include it in the object you submit to search.
2. Pull model: You define an indexer that pulls content from your content source. Here you can use output field mappings. As the documentation says: "Examples when you should use output field mappings: [When you] are indexing a complex type from a Cosmos DB database. You would like to get to a node on that complex type and map it into a field in your index."

See https://docs.microsoft.com/en-us/azure/search/cognitive-search-output-field-mapping

So, the solution is not to ignore anything. You rather explicitly map the bits you want.

Answer 2

FWIW: I've got a similar problem. I had already both models, the CosmosDB model and the AzureSearch model. The thing is that I need to add a flag on the latter, so in case it is not flagged perform a Delete operation on the index, or an update otherwise. (In my case, passing the flag from one model to another was the cheapest way otherwise I'd have needed a big refactor).

Since I didn't need the flag (to be persisted) on the SeachModel, using JsonIgnore attribute worked for me. If you need the value, the FieldBuilderIgnore attribute may help, see https://learn.microsoft.com/en-us/do.net/api/azure.search.documents.indexes.fieldbuilderignoreattribute?view=azure-do.net .

Example: https://github.com/Azure/azure-sdk-for.net/blob/main/sdk/search/Azure.Search.Documents/samples/Sample04_FieldBuilderIgnore.md