In Python, import a function using globals()

Question

I have function "read_csv" which I use often in other Python scripts. Instead of copying it to the beginning of every Python script I use it in, I'd like to save it in a separate file and import it. The function is defined as follows:

import numpy as np

def read_csv(filename,suffix=''):
    # Read column headers (to be variable naames)
    with open(filename) as f:
        firstline = f.readline()                    # Read first line of csv
        firstline = firstline.replace("\n","")      # Remove new line characters
        firstline = firstline.replace(" ","")       # Remove spaces
        ColumnHeaders = firstline.split(",")        # Get array of column headers

    # Read in the data (omitting the first row containing column headers)
    data=np.genfromtxt(filename,skiprows=1,delimiter=",",filling_values=0)

    # Assign the data to arrays, with names of the variables generated from column headers
    Ind=0
    for Var in ColumnHeaders:
        VarName=Var+suffix                      # Define variable name appended with given suffix (if any)
        globals()[VarName]=data[:,Ind]          # Assign the columns of the data to variables names after the column headers

The function reads a csv file containing column headers and numbers beneath it, and writes the numbers as arrays into the 'workspace' with names derived from the column headers. I've saved the code above as "read_csv_nDI.py". In the same directory, I try the following script:

import numpy as np
from read_csv_nDI import read_csv
read_csv('test.csv')

where 'test.csv' is a CSV file which should work:

Track, Bin, RO_T,ZEZ2A_T,Intra,RO_T_evnt_amp,ZEZ2A_T_evnt_amp,Intra_evnt_amp,Intra_reservoir_amplitude_normalized_to_RO_T,Intra_reservoir_amplitude_normalized_to_ZEZ2A_T
              1,              1,     2149.7307,     2110.3000,     2189.5908,     1000.3883,     -766.3962,     -687.7489,       -0.6875,        0.8974
              1,              2,     2151.7307,     2112.3000,     2191.5908,     1000.3883,     -766.3962,     -687.7489,       -0.6875,        0.8974
              1,              3,     2153.7307,     2114.3000,     2193.5908,     1000.3883,     -766.3962,     -687.7489,       -0.6875,        0.8974
              1,              4,     2155.7307,     2116.3000,     2195.5908,     1000.3883,     -766.3962,     -687.7489,       -0.6875,        0.8974
              1,              5,     2157.7307,     2118.3000,     2197.5908,     1000.3883,     -766.3962,     -687.7489,       -0.6875,        0.8974
              1,              6,     2159.7307,     2120.3000,     2199.5908,     1000.3883,     -766.3962,     -687.7489,       -0.6875,        0.8974
              1,              7,     2161.7307,     2122.3000,     2201.5908,     1000.3883,     -766.3962,     -687.7489,       -0.6875,        0.8974

However, if I run the above script and type the dir() command, I don't see the variables "RO_T", "ZEZ2A_T", etc. that I would expect to see. On the other hand, if I simply add a line

read_csv('test.csv')

after the function definition in the same Python script, it works and I do see those variables after running the script. Why does it only work if the function definition is in the same script?

Answer 1

globals gives you the global variables in the file where you call it, so it is always giving you the global variables of the file where your read_csv is defined.

Cluttering your global namespace like that may not be the best idea. It is probably better to just return your entire array. Also, I recommend you look at the pandas package, which can easily read a CSV into a DataFrame object that works like a numpy array but is even more convenient for most purposes.