appending values to a list in python

Question

i am doing this:

def GetDistinctValues(theFile, theColumn):
  lines=theFile.split('\n')
  allValues=[]
  for line in lines:
    allValues.append(line[theColumn-1])
  return list(set(allValues))

i am getting string index out of range on this line:

allValues.append(line[theColumn-1])

does anyone know what i am doing wrong?

here's the complete code if needed:

import hashlib

def doStuff():
  createFiles('together.csv')

def readFile(fileName):
  a=open(fileName)
  fileContents=a.read()
  a.close()
  return fileContents

def GetDistinctValues(theFile, theColumn):
  lines=theFile.split('\n')
  allValues=[]
  for line in lines:
    allValues.append(line[theColumn-1])
  return list(set(allValues))

def createFiles(inputFile):
  inputFileText=readFile(inputFile)
  b = inputFileText.split('\n')
  r = readFile('header.txt')
  DISTINCTCOLUMN=12
  dValues = GetDistinctValues(inputFileText,DISTINCTCOLUMN)

  for uniqueValue in dValues:
    theHash=hashlib.sha224(uniqueValue).hexdigest()
    for x in b:
      if x[DISTINCTCOLUMN]==uniqueValue:
        x = x.replace(', ',',').decode('latin-1','ignore')
        y = x.split(',')
        if len(y) < 3:
          break
        elif len(y) > 3:
          desc = ' '.join(y[3:])
        else:
          desc = 'No description'
        # Replacing non-XML-allowed characters here (add more if needed)
        y[2] = y[2].replace('&','&amp;')

        desc = desc.replace('&','&amp;')

        r += '\n<Placemark><name>'+y[2].encode('utf-8','xmlcharrefreplace')+'</name>' \
          '\n<description>'+desc.encode('utf-8','xmlcharrefreplace')+'</description>\n' \
          '<Point><coordinates>'+y[0]+','+y[1]+'</coordinates></Point>\n</Placemark>'
    r += readFile('footer.txt')
    f = open(theHash,'w')
    f.write(r)
    f.close()

Answer 1

That is happening because line doesn't have as many elements as the code is assuming. Try the following:

for line in lines:
    if len(line) < theColumn:
        print "This line doesn't have enough elements:\n" + line
    else:
        allValues.append(line[theColumn-1])
return list(set(allValues))

That will give you a hint, that is the type of error you expect when trying to access an element out of the range of a list ie a non existent element.

Answer 2

The error isn't caused by append() , It's because the line isn't long enough. Maybe your file has a empty line at the end. You could try

def GetDistinctValues(theFile, theColumn):
  lines=theFile.split('\n')
  allValues=[]
  for line in lines:
    if line:
      allValues.append(line[theColumn-1])
  return list(set(allValues))

otherwise an exception handler can help find what's going wrong

def GetDistinctValues(theFile, theColumn):
  lines=theFile.split('\n')
  allValues=[]
  for line in lines:
     try:
        allValues.append(line[theColumn-1])
     except IndexError:
        print "line: %r"%line
  return list(set(allValues))

Answer 3

line[theColumn-1])

This will of course raise the mentioned error if the string(line) is shorted then 'theColumn'. What else would you expect?