[英]How to get StructType object out of a StructType in spark java?
我正在处理这个 spark java 应用程序,我想访问结构类型 object 中的结构类型 object。例如-
当我们采用 spark dataframe 的模式时,它看起来像这样-
root
|-- name: struct (nullable = true)
| |-- firstname: string (nullable = true)
| |-- middlename: string (nullable = true)
| |-- lastname: string (nullable = true)
|-- language: string (nullable = true)
|-- fee: integer (nullable = true)
我想获取名称作为结构类型,以便我可以进一步分析它。 它会形成一种链条。 但问题在于,在根级别或任何级别,我们只能从 structtype 而不是其他 structtype 中提取 structfield。
StructType st = df.schema(); --> we get root level structtype
st.fields(); --> give us array of structfields but if I take name as a structfield i will lose all the fields inside it as 'name' is a StructType and I want to have it as it is.
StructType name = out of st --> this is what I want to achieve.
您可以使用官方文档中提到的参数和方法:
schema = StructType([StructField('name', StructType([StructField('firstname', StringType()), StructField('middlename', StringType()), StructField('lastname', StringType())])), StructField('language', StringType()), StructField('fee', IntegerType())])
for f in schema.fields:
if (f.name == "name"):
print(f.dataType)
for f2 in f.dataType.fields:
print(f2.name)
[Out]:
StructType([StructField('firstname', StringType(), True), StructField('middlename', StringType(), True), StructField('lastname', StringType(), True)])
firstname
middlename
lastname
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.