Remove elements from RDD starting with certain character

Question

I am working with an RDD which has few lines which start with #. I want to remove all these lines which begin with # and keep remaining ones. I tried remove = records.filter(lambda x: x[0].startswith('#')) but this way it filters only the rows containing #. I want the opposite.

Answer 1

Try inverting your condition:

records.filter(lambda x: not x[0].startswith('#'))