Pass a python variable into an SQL query

Question

I am working on Databricks and I am trying to pass a python variable into a SQL query:

series_name "LogTest"

query = """SELECT * FROM results_table
  WHERE name = $series_name
  """
spark.sql(query).toPandas()

I tried with $ but it does not work.

How do I do this?

Regards

Answer 1

In this case, your variable and queries are just strings. You can do:

query = f"""SELECT * FROM results_table
WHERE name = '{series_name}'
"""

... in python 3. Remember that your query string needs the single quotes around the inserted variable. However, for certain variables, you may need to pass the variable directly to the spark module so it can interpret it. I use pyodbc a lot and would do this:

query = f"""SELECT * FROM results_table
WHERE name = ?"""
cursor.execute(query, series_name)

Answer 2

Following will also work,

series_name = "LogTest"
spark.sql("SELECT * FROM results_table WHERE name = " + series_name).toPandas()

Answer 3

we can also try the following.

series_name = "LogTest"
query = "SELECT * FROM results_table WHERE name = {}".format(series_name)
spark.sql(query).toPandas() #