k8s volumesnapshot is created but it returns 409 error message in python k8s client

Question

I successfully snapshot my volume using python k8s client.

However, I got the below message.

I didn't have same volumesnapshot in the cluster.

Why this happen?

Code:

def create_snapshot(namespace, pvc_name):

snapshot_class = "snapshotclass"
snapshot_name = f"snapshot-{pvc_name}"

snapshot_resource = {
"apiVersion": "snapshot.storage.k8s.io/v1beta1",
"kind": "VolumeSnapshot",
"metadata": {"name": snapshot_name},
"spec": {
    "volumeSnapshotClassName": snapshot_class,
    "source": {"persistentVolumeClaimName": pvc_name}
    }
}

res = custom_api.create_namespaced_custom_object(
    group="snapshot.storage.k8s.io",
    version="v1beta1",
    namespace= namespace,
    plural="volumesnapshots",
    body=snapshot_resource,
)

print(res)

create_snapshot("test", "test-pvc")

The volumesnapshot is created successfully, but I got a message:

  File "/home/new/my/test/rescheduler/utils/k8s_controller.py", line 72, in create_snapshot
    body=snapshot_resource,
  File "/home/new/my/test/venv/lib/python3.6/site-packages/kubernetes/client/api/custom_objects_api.py", line 225, in create_namespaced_custom_object
    return self.create_namespaced_custom_object_with_http_info(group, version, namespace, plural, body, **kwargs)  # noqa: E501
  File "/home/new/my/test/venv/lib/python3.6/site-packages/kubernetes/client/api/custom_objects_api.py", line 358, in create_namespaced_custom_object_with_http_info
    collection_formats=collection_formats)
  File "/home/new/my/test/venv/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 353, in call_api
    _preload_content, _request_timeout, _host)
  File "/home/new/my/test/venv/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 184, in __call_api
    _request_timeout=_request_timeout)
  File "/home/new/my/test/venv/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 397, in request
    body=body)
  File "/home/new/my/test/venv/lib/python3.6/site-packages/kubernetes/client/rest.py", line 280, in POST
    body=body)
  File "/home/new/my/test/venv/lib/python3.6/site-packages/kubernetes/client/rest.py", line 233, in request
    raise ApiException(http_resp=r)
kubernetes.client.exceptions.ApiException: (409)
Reason: Conflict
HTTP response headers: HTTPHeaderDict({'Audit-Id': 'dec3c73a-e5fc-4c63-8d1a-6e2e6c6600e1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'my, 25 Apr 2021 10:50:52 GMT', 'Content-Length': '346'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"volumesnapshots.snapshot.storage.k8s.io \"snapshot-test-pvc\" already exists","reason":"AlreadyExists","details":{"name":"snapshot-test-pvc,"group":"snapshot.storage.k8s.io","kind":"volumesnapshots"},"code":409}

Answer 1

Posting this answer as a community wiki to give one of the possible reasons why you can encounter error 409 when trying to create the resources with above snippet of code.

Feel free to expand it.

---

The error encountered in the question:

Reason: Conflict
HTTP response headers: HTTPHeaderDict({'Audit-Id': 'dec3c73a-e5fc-4c63-8d1a-6e2e6c6600e1', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'my, 25 Apr 2021 10:50:52 GMT', 'Content-Length': '346'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"volumesnapshots.snapshot.storage.k8s.io \"snapshot-test-pvc\" already exists","reason":"AlreadyExists","details":{"name":"snapshot-test-pvc,"group":"snapshot.storage.k8s.io","kind":"volumesnapshots"},"code":409}

"snapshot-test-pvc\" already exists","reason":"AlreadyExists"

states that the resource already exists in the cluster. To check if the resource exists in the cluster you can run following commands:

• $ kubectl get volumesnapshots -A
• $ kubectl describe volumesnapshots RESOURCE_NAME -A

I've used the code that was in the question and had no issues with it. The course of actions was following:

• first run - VolumeSnapshot created successfully
• second run - code returned the 409 error stating that resource already exists.

kubernetes.client.exceptions.ApiException: (409)
Reason: Conflict
<-- REDACTED --> 
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"volumesnapshots.snapshot.storage.k8s.io \"snapshot-example-pvc\" already exists","reason":"AlreadyExists","details":{"name":"snapshot-example-pvc","group":"snapshot.storage.k8s.io","kind":"volumesnapshots"},"code":409}

A side note!

Above error was returned with modified code from the question (mainly values).

You can also see this error when trying to run $ kubectl create -f resource.yaml -v=4 on already created resource.

---

For anyone interested here is the minimal, reproducible example of a code used in the question (it was missing import and the def was misplaced):

from kubernetes import client, config

def create_snapshot(namespace, pvc_name):

    config.load_kube_config()

    custom_api = client.CustomObjectsApi()

    snapshot_class = "snapshotclass"
    snapshot_name = f"snapshot-{pvc_name}"

    snapshot_resource = {
    "apiVersion": "snapshot.storage.k8s.io/v1beta1",
    "kind": "VolumeSnapshot",
    "metadata": {"name": snapshot_name},
    "spec": {
        "volumeSnapshotClassName": snapshot_class,
        "source": {"persistentVolumeClaimName": pvc_name}
        }
    }

    res = custom_api.create_namespaced_custom_object(
        group="snapshot.storage.k8s.io",
        version="v1beta1",
        namespace= namespace,
        plural="volumesnapshots",
        body=snapshot_resource,
    )

    print(res)

create_snapshot("default", "test-pvc")

---

Additional resources:

• Github.com: Fabric8io: Kubernetes client: Issues: 409 conflict status code when create pod
• Oreilly.com: Library: View: Managing Kubernetes: Chapter 04