火花插入覆盖测试

Question

I new to pyspark and would like to play with Insert Overwrite.

I have created a test_overwrite table with many records. Then I created an empty test_overwrite_3 table from test_overwrite and put some data in it. (I made sure the structure is the same.)

insert overwrite test_overwrite_3 select * from test_overwrite limit 3

I would expect that after executing the code above my test_overwrite_3 table with some dummy data would be deleted and it should have 3 records from my test_overwrite table.

Instead I got an error message like this:

*Error in SQL statement: ParseException: no viable alternative at input 'insert overwrite test_overwrite_3'(line 1, pos 17) == SQL == insert overwrite test_overwrite_3 -----------------^^^ select * from test_overwrite*

Can anyone help what went wrong? Thanks!

Answer 1

no viable alternative at input 'insert overwrite test_overwrite_3

a parse error, not a type error. see this

I could see table keyword is missing in your statement.

Example is :

sql(
      """
        |INSERT OVERWRITE TABLE table_with_partition
        |partition (p1='a',p2='b',p3='c',p4='c',p5='4')
        |SELECT 'blarr' FROM tmp_table
      """.stripMargin)