Unable to update kafka cluster version in AWS MSK

Question

We have written python code to upgrade kafka version in AWS MSK and its giving error

    ..................................................
    vaibhavkanchan@vkmacbook python-code % ./update_kafka_version.py /gpn-prime/dev/msk/kafka-cluster 2.6.1
    ..................................................
    Cluster ARN: arn:aws:kafka:ap-south-1:xxxx:cluster/xxx-mks-kafka-cluster/1bdb7d3d-f9be-4d51-bc69-82e1c7a280eb-4
    Current Version :K1X6FK5RDHNB96
    cluster name: xxx-msk-mks-kafka-cluster
    Configuration ARN: arn:aws:kafka:ap-south-1:xxxx:configuration/xxx-mks-kafka-cluster/bcc660ab-ecba-4ac3-afa6-2c6013d6debf-4
    Configuration Revision: 1
    Kafka current version: 2.2.1
    ..................................................
    Traceback (most recent call last):
    File "./update_kafka_version.py", line 71, in <module>
    update_kafka_version(name, targetKafkaVersion)
    File "./update_kafka_version.py", line 40, in update_kafka_version
    update_kafka_version_response = client.update_cluster_kafka_version(
    File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
    File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
    botocore.errorfactory.BadRequestException: An error occurred (BadRequestException) when calling the UpdateClusterKafkaVersion operation: The specified parameter value is identical to the current value for the cluster. Specify a different value, then try again.
    
    vaibhavkanchan@vkmacbook python-code % aws kafka describe-cluster --cluster-arn arn:aws:kafka:ap-south-1:xxxx:cluster/xxx-mks-kafka-cluster/1bdb7d3d-f9be-4d51-bc69-82e1c7a280eb-4 --query "ClusterInfo.CurrentVersion"
    "K1X6FK5RDHNB96"
    vaibhavkanchan@vkmacbook python-code %

As per from the boto3 documentation for kafka

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kafka.html#Kafka.Client.update_cluster_kafka_version

response = client.update_cluster_kafka_version(
    ClusterArn='string',
    ConfigurationInfo={
        'Arn': 'string',
        'Revision': 123
    },
    CurrentVersion='string',
    TargetKafkaVersion='string'
)

We have stored kafka zookeeper endpoints, bootstrap nodes, cluster arn and cluster version in parameter store and fetching the cluster arn from parameter store. We are fetching current version using describe-cluster but it's still giving error that the specified current version matches with the current cluster value.

python module for boto3 in my laptop

boto3 1.17.27 botocore 1.20.44

Any advise on fix this issue will be highly appreciated.

Answer 1

@Vaibhav, same error I see with MSK 2.7.0. When I try to delete topics, it was showing still topic with one partition so I added delete.topic.enable=true in cluster configuration and I try to update 3.3.1 but showing me error The specified parameter value is identical to the current value for the cluster

So I removed and try again and it worked. Might be that is default value and AWS does not allow to pass again same, it may work only with different value.