xlrd: want to read sheets of several xl files and store in one list/array? (better way?)

Question

I don't have much xp with xlrd/xlwt but I have managed to access one of the files I want to collect data from. I want to collect data from all files in the directory and move it to one sheet. I was thinking if there is someway I can store it all in one array/list it would be easy to output to a csv. If this is too much work and there is a simple way plz help, otherwise I am using Idle to play around with ideas and have come up with this so far:

>>> import xlrd, xlwt
>>> book = xlrd.open_workbook('c:\excelTry\Papineau.csv.xls')
>>> book.sheet_names()
[u'Charge Codes', u'Month']
>>> sh = book.sheet_by_index(1)
>>> #produces:
>>> sh.book
<xlrd.Book object at 0x01213BF0>
>>> for x in range(0, 10):
        sh.row_values(x)
[u'William Papineau', u'Pay Period 11', '', '', u' ', u' ', '', '', '', u'Weekly Total', '', '', u' ', '', '', '', '', u'Weekly Total', u'Biweekly', u'Percent of Effort']
[u'Index Number', u'Index Description', 40678.0, 40679.0, 40680.0, 40681.0, 40682.0, 40683.0, 40684.0, '', 40685.0, 40686.0, 40687.0, 40688.0, 40689.0, 40690.0, 40691.0, '', u'Total', '']
[u'E45776', u'Seat Belt Study', '', 8.0, 8.0, 8.0, 8.0, u' ', '', 32.0, '', '', '', '', '', u' ', '', 0.0, 32.0, 0.4155844155844156]
[u'E43457', u'MultiScaleWaterQuality', '', '', '', '', '', 8.0, '', 8.0, '', 5.0, 8.0, u' ', '', '', '', 13.0, 21.0, 0.2727272727272727]
[u'E45125', u'GLOSS', '', '', '', '', '', '', '', 0.0, '', '', '', 8.0, 8.0, '', '', 16.0, 16.0, 0.2077922077922078]
[u'E45131', u'GLOS AOC Trib Monitoring', '', '', '', '', '', '', '', 0.0, '', '', '', '', '', 8.0, '', 8.0, 8.0, 0.1038961038961039]

this produces what looks like a list object but every attempt I have made to manipulate or append it produces errors saying not scriptable or iterable. The file iteration will be handled with the os module using os.listdir(path) and a for loop. Any help would be greatly appreciated!

Answer 1

So far in your code you don't appear to be doing anything with the values you get from the worksheet. Maybe some of the code didn't get pasted into the question...

Would you be able to include the output of that last line of code?

You say that you want to store it all in one list.
Try something like this:

final = []
for rowx in xrange(sh.nrows):
    final.extend(sh.row_values(rowx))

Also:
Be careful with Windows paths. Single-backslashes will work only if the following letter does not, with the backslash, form an escape sequence (eg \t or tab). Other options (option 3 is probably best; unless there is a specific reason not to use it):

1. Raw strings: book = xlrd.open_workbook(r'c:\excelTry\Papineau.csv.xls')
2. Forward-slashes: book = xlrd.open_workbook('c:/excelTry/Papineau.csv.xls')
3. os.path.join :
   book = xlrd.open_workbook(os.path.join('c:','excelTry','Papineau.csv.xls'))

Answer 2

data = []
for i in xrange(sh.nrows):
    data.append(sh.row_values(i))
it will append each rows from xls file into list "data".
eg: [['a','b'],['c','d'],['e','f']] like this .