从 Azure Databricks 查询 SAP 表

Question

I want to query the SAP table from databricks. I have installed the JDBC library for connecting to the SAP server. I am able to connect and fetch records using spark.read.JDBC(url = jdbcUrl, table = query, properties = connectionProperties). In the table parameter, I am providing the schema and table name. I want to fetch selective records. Is there any way for providing a query?

Answer 1

You just need to use standard Spark functions like, .filter / .where - Spark will convert it into corresponding SQL query and will push it down to the database. This functionality (predicate pushdown) is enabled in Spark by default as you may see indocumentation .

df = spark.read.JDBC(url = jdbcUrl, table = query, 
       properties = connectionProperties)
filtered = df.filter("condition")