Python list.remove items present in second list

Question

I've searched around and most of the errors I see are when people are trying to iterate over a list and modify it at the same time. In my case, I am trying to take one list, and remove items from that list that are present in a second list.

import pymysql

schemaOnly = ["table1", "table2", "table6", "table9"]

db = pymysql.connect(my connection stuff)

tables = db.cursor()
tables.execute("SHOW TABLES")
tablesTuple = tables.fetchall()
tablesList = []

# I do this because there is no way to remove items from a tuple
# which is what I get back from tables.fetchall
for item in tablesTuple:
    tablesList.append(item)

for schemaTable in schemaOnly:
    tablesList.remove(schemaTable)

When I put various print statements in the code, everything looks like proper and like it is going to work. But when it gets to the actual tablesList.remove(schemaTable) I get the dreaded ValueError: list.remove(x): x not in list .

If there is a better way to do this I am open to ideas. It just seemed logical to me to iterate through the list and remove items.

Thanks in advance!

** Edit **

Everyone in the comments and the first answer is correct. The reason this is failing is because the conversion from a Tuple to a list is creating a very badly formatted list. Hence there is nothing that matches when trying to remove items in the next loop. The solution to this issue was to take the first item from each Tuple and put those into a list like so: tablesList = [x[0] for x in tablesTuple] . Once I did this the second loop worked and the table names were correctly removed.

Thanks for pointing me in the right direction!

Answer 1

I assume that fetchall returns tuples, one for each database row matched.

Now the problem is that the elements in tablesList are tuples, whereas schemaTable contains strings. Python does not consider these to be equal.

Thus when you attempt to call remove on tablesList with a string from schemaTable , Python cannot find any such value.

You need to inspect the values in tablesList and find a way convert them to a strings. I suspect it would be by simply taking the first element out of the tuple, but I do not have a mySQL database at hand so I cannot test that.

Answer 2

Regarding your question, if there is a better way to do this: Yes.

Instead of adding items to the list, and then removing them, you can append only the items that you want. For example:

for item in tablesTuple:
    if item not in schemaOnly:
        tablesList.append(item)

Also, schemaOnly can be written as a set, to improve search complexity from O(n) to O(1):

schemaOnly = {"table1", "table2", "table6", "table9"}

This will only be meaningful with big lists, but in my experience it's useful semantically.

And finally, you can write the whole thing in one list comprehension:

tablesList = [item for item in tablesTuple if item not in schemaOnly]

And if you don't need to keep repetitions (or if there aren't any in the first place), you can also do this:

tablesSet = set(tablesTuple) - schemaOnly

Which is also has the best big-O complexity of all these variations.