My Tech Memories

Quick use case : clean index mapping with reindexing data

Yassine LASRI's photo
Yassine LASRI
·

4 min read

By mistake i added some documents to my elasticsearch index without providing an explicit mapping, elasticsearch did the job and add a default mapping like this :

{
  "account_id": {
    "type": "text",
    "fields": {
      "keyword": {
        "type": "keyword",
        "ignore_above": 256
      }
    }
  }
}

I tried to update the mapping using the following API

PUT snmp-inventory-devices/_mapping
{
  "properties": {
    "account_id": {
      "type": "keyword"
    }
  }
}

Surprise! i got the following error, i can't update the mapping of an existing field, i can only add mapping for new fields 

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "mapper [account_id] cannot be changed from type [text] to [keyword]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "mapper [account_id] cannot be changed from type [text] to [keyword]"
  },
  "status" : 400
}

So the next step was to define a new index maping and reindex the data to the new index created as follow

POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "snmp-inventory-devices"
  },
  "dest": {
    "index": "snmp-inventory-devices-new"
  }
}

With this way, elasticsearch is running the reindex task on background that i can check any time and see the progress, the output is :

{
  "task" : "x7y1dRN1QqCWlrt5I3mezw:4344337"
}

With the task API, we can retrieve the status of the task and check the progress as follow

GET _tasks/x7y1dRN1QqCWlrt5I3mezw:4344337

The ouput is as follow

{
  "completed" : true,
  "task" : {
    "node" : "x7y1dRN1QqCWlrt5I3mezw",
    "id" : 4344337,
    "type" : "transport",
    "action" : "indices:data/write/reindex",
    "status" : {
      "total" : 13460,
      "updated" : 0,
      "created" : 13460,
      "deleted" : 0,
      "batches" : 14,
      "version_conflicts" : 0,
      "noops" : 0,
      "retries" : {
        "bulk" : 0,
        "search" : 0
      },
      "throttled_millis" : 0,
      "requests_per_second" : -1.0,
      "throttled_until_millis" : 0
    },
    "description" : "reindex from [snmp-inventory-devices] to [snmp-inventory-devices-new][_doc]",
    "start_time_in_millis" : 1610207973825,
    "running_time_in_nanos" : 37739653839,
    "cancellable" : true,
    "headers" : { }
  },
  "response" : {
    "took" : 37729,
    "timed_out" : false,
    "total" : 13460,
    "updated" : 0,
    "created" : 13460,
    "deleted" : 0,
    "batches" : 14,
    "version_conflicts" : 0,
    "noops" : 0,
    "retries" : {
      "bulk" : 0,
      "search" : 0
    },
    "throttled" : "0s",
    "throttled_millis" : 0,
    "requests_per_second" : -1.0,
    "throttled_until" : "0s",
    "throttled_until_millis" : 0,
    "failures" : [ ]
  }
}

With this way i can remove and re-create the old index and reindex back on the other sens ...

Happy elasticsearch

elasticsearch