带有多个 case when 语句的 Spark-sql

Question

I have created one temporary table using my dataframe in sparksql using mydf.createOrReplaceTempView("combine_table").All the fields datatype is showing as string. In this temp table I have 4 columns procuredValue,minMargin,maxMargin,Price and some other columns.In all these 4 columns i have values like 373.58...etc. Now I need to select data based on some condition and have to display data as new column ."Final Price". I am trying to do this using Case statement but getting below error. mismatched input '1st_case' expecting EOF(line 3, pos 5) can anyone suggest how should i do this.

    val d1=spark.sql(""" SELECT cast(PV as  FloatType),cast(mxM as FloatType),
    cast(mnM as FloatType ) , cast(procuredValue+ mxM as FloatType) as 1st_case, 
    cast(PV+ mnM as FloatType) as 2nd_case,
    case 
    WHEN 1st_case < price THEN 1st_case
    WHEN 2ndcse < price THEN 2ndcse 
    WHEN PV <price && saleevent = 'Sp' THEN 'price'
    WHEN price < 'PV'  && saleevent = 'Sp' && sclass = 'VH' THEN 0.9* PV
    ELSE PV 
    END AS Final_price 
    FROM combine_table""")

Answer 1

What happens to your query?

SELECT *, 
       CASE 
              WHEN Sum(i.procuredvalue + i.maxmargin) < min_val_seller.q THEN Sum(i.procuredvalue + i.maxmargin)
              WHEN Sum(i.procuredvalue + i.maxmargin) < min_val_seller.q THEN min_val_seller.q 
              WHEN Sum(i.procuredvalue < min_val_seller.q) and e.saleevent = 'Special' THEN min_val_seller.q
              WHEN min_val_seller.q < i.procuredvalue and e.saleevent = 'Special' and Min(min_val_seller.q) and s.netvalue = 'VeryHigh' THEN 0.9*i.procuredvalue
              ELSE i.procuredvalue 
       END AS final_price 
  FROM ecom_competitor_data e, 
       internal_product_data i, 
       min_val_seller, 
       seller_data s 
 WHERE e.productid = i.productid 
   AND s.sellerid = i.sellerid

• SQL Formatter
• SQL Validator

Answer 2

So many issues...

1. You cannot surround your query with double quotes and also use double quotes to qualify strings within it (without the proper escaping).
   The easier and cleaner solution would be to do something like that:

-

val myquery = """
select ...
from ...
where ...
"""   

val 1st_case=spark.sql(myquery)

PS get used to use a single quote for SQL strings. It would work for all SQL dialects, unlike double quotes.

2. 'min_val_seller.Q' is a string literal

3. The logical AND in Spark is and , not &&

4. The CASE statement starts with two identical conditions ( Sum(i.procuredvalue + i.maxmargin) < min_val_seller.q ). The 2nd condition will never be chosen.

   (please make sure you understand how CASE works)

5. ISO JOINs were introduced in the 90`s. There is no reason to use WHERE conditions instead of proper JOIN syntax.

Answer 3

    val d1=spark.sql(""" SELECT price,PV,
    case
    WHEN cast(PV + mxM as Float) < cast(price as Float) THEN PV + mxM
    WHEN cast(PV + mnM as Float) < cast(price  as Float)THEN PV + mnM
    WHEN cast(PV as Float) < cast(price  as Float) And saleevent =     'Sp' THEN price
    WHEN cast(price as Float) < cast(PV as Float)  And saleevent =   'Sp' And sclass = "VH" THEN 0.9*PV
    ELSE PV
    END AS price
    FROM combine_table""");

Thanks @David Above Query Worked for me.