python 2.6.6 “phantom” whitespaces

Question

Basically, I'm opening some files and removing all whitespaces from every line in that file. code snippet:

for filepath in filelist:
        if filepath.endswith(".shader"):
            shaderfile = open(filepath,"r").readlines()
            for line in shaderfile:
                line = Left(line, line.find("\n"))+"\n"
                line = line.replace(" ","")
                if line.find("common/")>-1:
                    print(line.replace("\n","\\n"))

As per request, I removed the less important code.

There are two weird things going on:

1) Some lines end with "\n\n"

2) I'm getting this output:

textures/common/lightgrid\n
textures/common/mirror1
 \n
textures/common/mirror2
 \n
maptextures/common/invisible.tga
 \n
textures/common/watercaulk
 \n
textures/common/clipnokick
 \n
textures/common/invisible\n

3) When I pasted the output here, it looked like:

textures/common/lightgrid\n
textures/common/mirror1\n
textures/common/mirror2\n
maptextures/common/invisible.tga\n
textures/common/watercaulk\n
textures/common/clipnokick\n
textures/common/invisible\n

I seriously have no idea what's going on. Is it a bug with print()? Sorry for the bad formatting, but it's not my fault, it's stackoverflow's.

Answer 1

from StringIO import StringIO

a = StringIO("""textures/common/lightgrid
textures/common/mirror1

textures/common/mirror2

maptextures/common/invisible.tga

textures/common/watercaulk

textures/common/clipnokick

textures/common/invisible""")


def clean_lines(fileobj):
    for line in fileobj:
        if line:
            line = line.strip()
            if line:
                yield "%s\n" % line


print [line for line in clean_lines(a)]

I used stringIO to emulate a file just replace a with whatever your fileobj is.

Answer 2

It seems that you want output like

textures/common/lightgrid\ntextures/common/mirror1\n...

and instead you're getting

textures/common/lightgrid\n
textures/common/mirror1\n
...

This is because print statement adds an implicit newline.

You can use normal file output:

from sys import stdout
# ...
stdout.write("foo") # adds no implicit newline

You can use function print() ported back from Python 3:

from __future__ import print_function
#
print("foo", end="") # use empty string instead of default newline

Also, if you need to remove whitespace from inside of your strings, you can make it far simpler and probably more efficient.

import re # regular expressions
whitespace_rx = re.compile(r"\s+") # matches any number of whitespace
# ...
splinters = whitespace_rx.split(raw_line) # "a b\nc" -> ['a','b','c']
compacted_line = "".join(splinters) # ['a','b','c'] -> 'abc'

Of course you can just replace splinters with the .split() call.