How to create a Spark dataframe from a dictionary with the correct orientation?

Question

I have a dictionary:

DICT = {
    "A": (0, 100),
    "B": (100, 200),
    "C": (300, 400),
    "D": (400, 500),
}

which looks like:

{'A': (0, 100), 'B': (100, 200), 'C': (300, 400), 'D': (400, 500)}

Ultimately, I would like to convert this to a Spark dataframe, looking like this:

category  lower_bound  upper_bound
   A           0           100
   B           100         200
   C           300         400
   D           400         500

This is attainable if I use:

r = (
    pd.DataFrame.from_dict(DICT, orient="index")
    .reset_index()
    .rename(columns={"index": "category", 0: "lower", 1: "upper"})
)

Is there a way I can directly create a Spark Dataframe from the dictionary that is oriented that way?

Answer 1

Found the solution using this answer

df = spark.createDataFrame(
    [[x[0], *x[1]] for x in DICT.items()], ["category", "lower_bound", "upper_bound"]
)
```