[英]How to execute multi line sql in spark sql
如何在 Spark SQL 中执行冗长的多行 Hive 查询? 像下面的查询:
val sqlContext = new HiveContext (sc)
val result = sqlContext.sql ("
select ...
from ...
");
例如,使用“”“代替
val results = sqlContext.sql ("""
select ....
from ....
""");
或者,如果要格式化代码,请使用:
val results = sqlContext.sql ("""
|select ....
|from ....
""".stripMargin);
您可以在SQL代码的开头/结尾使用三引号,也可以在每行末尾使用反斜杠。
val results = sqlContext.sql ("""
create table enta.scd_fullfilled_entitlement as
select *
from my_table
""");
results = sqlContext.sql (" \
create table enta.scd_fullfilled_entitlement as \
select * \
from my_table \
")
值得注意的是,长度不是问题,只是写作。 为此,您可以使用“”作为Gaweda建议或仅使用字符串变量,例如通过使用字符串构建器构建它。例如:
val selectElements = Seq("a","b","c")
val builder = StringBuilder.newBuilder
builder.append("select ")
builder.append(selectElements.mkString(","))
builder.append(" where d<10")
val results = sqlContext.sql(builder.toString())
除了上述方法之外,您还可以使用下面提到的方法:
val results = sqlContext.sql("select .... " +
" from .... " +
" where .... " +
" group by ....
");
val query = """(SELECT
a.AcctBranchName,
c.CustomerNum,
c.SourceCustomerId,
a.SourceAccountId,
a.AccountNum,
c.FullName,
c.LastName,
c.BirthDate,
a.Balance,
case when [RollOverStatus] = 'Y' then 'Yes' Else 'No' end as RollOverStatus
FROM
v_Account AS a left join v_Customer AS c
ON c.CustomerID = a.CustomerID AND c.Businessdate = a.Businessdate
WHERE
a.Category = 'Deposit' AND
c.Businessdate= '2018-11-28' AND
isnull(a.Classification,'N/A') IN ('Contractual Account','Non-Term Deposit','Term Deposit')
AND IsActive = 'Yes' ) tmp """
将您的 sql 写在三重引号内,例如 """ sql 代码 """
df = spark.sql(""" select * 来自 table1 """)
这与 Scala spark 和 Pyspark 相同。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.