简体   繁体   English

如何从具有正确方向的字典创建 Spark dataframe?

[英]How to create a Spark dataframe from a dictionary with the correct orientation?

I have a dictionary:我有一本字典:

DICT = {
    "A": (0, 100),
    "B": (100, 200),
    "C": (300, 400),
    "D": (400, 500),
}

which looks like:看起来像:

{'A': (0, 100), 'B': (100, 200), 'C': (300, 400), 'D': (400, 500)}

Ultimately, I would like to convert this to a Spark dataframe, looking like this:最后,我想将其转换为 Spark dataframe,如下所示:

category  lower_bound  upper_bound
   A           0           100
   B           100         200
   C           300         400
   D           400         500

This is attainable if I use:这是可以实现的,如果我使用:

r = (
    pd.DataFrame.from_dict(DICT, orient="index")
    .reset_index()
    .rename(columns={"index": "category", 0: "lower", 1: "upper"})
)

Is there a way I can directly create a Spark Dataframe from the dictionary that is oriented that way?有没有一种方法可以直接从以这种方式定向的字典中创建 Spark Dataframe?

Found the solution using this answer使用此答案找到解决方案

df = spark.createDataFrame(
    [[x[0], *x[1]] for x in DICT.items()], ["category", "lower_bound", "upper_bound"]
)
```

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM