Checking if the value of a key in a dictionary is empty in PySpark

Question

I am very new to Spark, but I have not managed to figure this out. I have a list of dictionaries:

list_d =  [ 

{"asset": "Anna",
 "grade": "somestring",
 "obs": "True"
},
{"asset": "Bob",
 "grade": "somestrings",
 "obs": "False"
},
{"asset": "",
 "grade": "somestrings",
 "obs": "False"
}
]

How do I write an if-else condition so that I do something (in my case, add a column, but this is unimportant) ONLY if "asset" is not empty? I tried the following but I do not get anything:

asset_class = table["asset"]
if asset_class:
    # df = df.withColumn("asset", ...) or anything

I just want to check whether that value exists. How can I do this?

Answer 1

Check this code snippet.

list_d = [
    {"asset": "Anna",
     "grade": "somestring",
     "obs": "True"
     },
    {"asset": "Bob",
     "grade": "somestrings",
     "obs": "False"
     },
    {"asset": "",
     "grade": "somestrings",
     "obs": "False"
     }
]

for table in list_d:
    asset_class = table.get("asset", None)
    if asset_class != None and asset_class != "":
        # Add your logic here
        print(table)