通过模式正则表达式删除空格

Question

I tried to remove the white spaces of some strings, as below

import re
test_str = '''PERSON_ID<|>DEPT_ID<|>DATE_JOINED
    AAAAA<|>S1<|>2021/01/03
    BBBBBB<|>S2<|>
    2021/02/03
    CCCCC<|>S1<|>2021/03/05'''

regex = r's*<|>s*\<|>s*\n+'
subst = "<|>"
# print(re.sub(regex, subst, test_str, 0, re.MULTILINE).split('\n'))
for line in re.sub(regex, subst, test_str, 0, re.MULTILINE).split('\n'):
    # print(line)
    if line != '':
        print (line.strip())

I've got the output,

PERSON_ID<|>|>DEPT_ID<|>|>DATE_JOINED
AAAAA<|>|>S1<|>|>2021/01/03
BBBBBB<|>|>S2<|>|<|>    2021/02/03
CCCCC<|>|>S1<|>|>2021/03/05

I am looking for a solution to remove the white spaces in the second last line before the date in the output.

Answer 1

The problem is that you have \\n+ after >\\s* . So it stops removing whitespace at the \\n , and leaves the spaces at the beginning of the next line. Change that to >\\s+ , since newline is included in whitespace.

import re
test_str = '''PERSON_ID<|>DEPT_ID<|>DATE_JOINED
    AAAAA<|>S1<|>2021/01/03
    BBBBBB<|>S2<|>
    2021/02/03
    CCCCC<|>S1<|>2021/03/05'''

regex = r'\s+<|>\s+<|>\s+'
subst = "<|>"
# print(re.sub(regex, subst, test_str, 0, re.MULTILINE).split('\n'))
for line in re.sub(regex, subst, test_str, 0, re.MULTILINE).split('\n'):
    # print(line)
    if line != '':
        print (line.strip())

Answer 2

Figured I'd try a solution that tries to guarantee each row as three columns separated by a common separator, that doesn't depend on regex

Specifically, if you're getting this dataset from an unknown source, then it isn't "regular" and there's no expression that can guarantee its always the date column that needs fixed

test_str = '''PERSON_ID<|>DEPT_ID<|>DATE_JOINED
    AAAAA<|>S1<|>2021/01/03
    BBBBBB<|>S2<|>
    2021/02/03
    CCCCC<|>S1<|>2021/03/05'''

lines = [l.strip() for l in test_str.splitlines(keepends=False)]
sep = '<|>'

def get_columns(line, separator):
  return [x.strip() for x in line.split(separator) if x.strip()]

header = get_columns(lines[0], sep)
columns = len(header)

# prepare case if second line needs to join first line
i = 1
lines[i] = sep.join(get_columns(lines[i], sep))

# check remaining lines
while True:
  i += 1
  if i >= len(lines):
    break
  # print(i, lines[i])
  parts = get_columns(lines[i], sep)
  previous_parts = get_columns(lines[i-1], sep)
  num_parts = len(parts)
  # skip lines that shouldn't be relocated
  if num_parts == columns or len(previous_parts) == columns:
    continue
  else:
    if len(parts) + len(previous_parts) > columns:
      # TODO: handle case where this line has all columns and part of the previous line
      pass  
    previous_parts.extend(parts)
    lines[i-1] = sep.join(previous_parts)
    del lines[i]  # remove the current line that was moved to the previous

for l in lines:
  print(l)

Sample output

PERSON_ID<|>DEPT_ID<|>DATE_JOINED
AAAAA<|>S1<|>2021/01/03
BBBBBB<|>S2<|>2021/02/03
CCCCC<|>S1<|>2021/03/05

Same output for

test_str = '''PERSON_ID<|>DEPT_ID<|>DATE_JOINED
    AAAAA<|>S1<|>
    2021/01/03
    BBBBBB<|>S2<|>
    2021/02/03
    CCCCC<|>S1<|>2021/03/05'''

and

test_str = '''PERSON_ID<|>DEPT_ID<|>DATE_JOINED
    AAAAA<|>
    S1<|>2021/01/03
    BBBBBB<|>S2<|>
    2021/02/03
    CCCCC<|>S1<|>2021/03/05'''