How to access to calling module namespace from called module in Python?

Question

I made a tiny sql renderer/wrapper for SQLite. The main idea is to write:

execute( 'select * from talbe1 where col1={param1} and col2={param2}' )

instead of

execute( 'select * from table1 where col1=? and col2=?', (param1,param2) )

Here is the code:

import re
import sqlite3

class SQLWrapper():
    def __init__(self, cursor):
        self.cursor = cursor
    def execute(self, sql):
        regexp=re.compile(r'\{(.+?)\}')
        sqlline = regexp.sub('?',sql)
        statements = regexp.findall(sql)
        varlist = tuple([ eval(_statement) for _statement in statements ])
        self.cursor.execute(sqlline, varlist)
        return self
    def fetchall(self):
        return self.cursor.fetchall()

#usage example
db = sqlite3.connect(':memory:')
cursor = db.cursor()

wrap = SQLWrapper(cursor)

wrap.execute('create table t1(a,b)')
for i in range(10):
    wrap.execute('insert into t1(a,b) values({i}, {i*2})')

limit = 50
for line in wrap.execute('select * from t1 where b < {limit}').fetchall():
    print line

It works, but when I move the class SQLWrapper to a separate module (file sqlwrap.py) and import it, the program crashes with:

Traceback (most recent call last):
  File "c:\py\module1.py", line 15, in <module>
    wrap.execute('insert into t1(a,b) values({i}, {i*2})')
  File "c:\py\sqlwrap.py", line 10, in execute
    varlist = tuple([ eval(_statement) for _statement in statements ])
  File "<string>", line 1, in <module>
NameError: name 'i' is not defined

Ie variable i is not visible from the other module. How to overcome this?

Answer 1

This goes against normal scope rules of most programming languages.

Normally you don't want that functions (or methods) that you call somehow magically do things with your own variables. That means, that only those variable values (!) are accessible to the called routine, which you explicitly added as parameter.

When you in your first code (all in one module) move your for loop with the i into an own function will run into the same problem -- than i will be local to this function and not visible to SQLwrapper.

Scope rules intentionally limit the access of variables to those which are "in scope" and don't give access to things that are "out of scope". Thus some information hiding is done and complexity is reduced.

This implies some writing overhead in some situations, but also makes the programs more save by inhibiting some dangerous practices and complexity.

I would recommend something like that, when you only intend to use SQLite (or MySQL):

execute( 'select * from table1 where col1=@param1 and col2=@param2', {'param1': param1, 'param2': param2} )

Thus you have a more readable and understandable version, but without the scope problems you encountered. The @-prefix works on SQLite and as much I know, also in MySQL -- but it is DB specific (sadly, SQL did not standardize it). In the documentation for the sqlite3-module an other prefix is used ':' this also works in SQLite, but I don't know on which other DB.

See: sqlite3.Cursor.execute documentation

BTW, if you want to reduce your writing overhead a little bit, you could write some wrapper like that:

def myExcecute(sql, **params):
   self.cursor.execute(sql, params)

Thus you can call execute with a little less overhead (and less brackets):

myExecute( 'select * from table1 where col1=@param1 and col2=@param2', param1=param1, param2=param2 )