Spark将列表转换为数据框错误：不支持任何类型的架构

Question

I ma trying to make create dataframe by doing the following:

val df = Seq(
    ("tom", 25),
    ("andy", null)
).toDF("user", "rating")

but I get an error of "Schema for type Any is not supported". I think the rating column is causing the problem.

I want the rating column to be

integer (nullable=true)

but i am not sure how to achieve that

Answer 1

You can use Option :

Seq(
  ("tom", Some(25)),
  ("andy", None)
).toDF("user", "rating")

You could also be more specific about the types and use java.lang.Integer :

Seq[(String, java.lang.Integer)](
  ("tom", 25),
  ("andy", null)
).toDF("user", "rating")

but the first method should be preferred.