Python - Regex to Get directory name from HDFS

Question

Im trying to extract a folder name from the result of a subprocess command. The result is Found 1 items

drwxr-xr-x   - user user          0 2017-05-04 17:19 /user/oozie/share/lib/lib_20170406204755

I want to extract lib_20170406204755 . I was able to do it using

process = subprocess.check_output(['hdfs','dfs','-ls','/user/oozie/share/lib'])
print process.split(' ')[-1].rstrip().split('/')[-1]

The folder is always lib_timestamp

How can do this using regex?

Answer 1

No regex required here, you may as well use split() :

string = "drwxr-xr-x   - user user          0 2017-05-04 17:19 /user/oozie/share/lib/lib_20170406204755"

folder = string.split('/')[-1]
print(folder)
# lib_20170406204755

But if you insist:

[^/]+$

---

In Python :

 import re string = "drwxr-xr-x - user user 0 2017-05-04 17:19 /user/oozie/share/lib/lib_20170406204755" rx = re.compile(r'[^/]+$') folder = rx.search(string).group(0) print(folder) # lib_20170406204755 

See a demo on regex101.com .

Answer 2

This should do the trick:

(?!/)(lib_\\d*)

This regex is searching for for something that starts with lib_ followed by a bunch of numbers, should be enough if no similar folders are found on the result.

(?!/) is just to make sure that the folder is preceded by a /

Example

Answer 3

A clean approach would be to use the os.path module to pick apart paths.

import os
import subprocess

output = subprocess.check_output(['hdfs','dfs','-ls','/user/oozie/share/lib'])

# there are 8 columns in the output, i.e. we need a maximum of 7 splits per line
output_table = [line.split(maxsplit=7) for line in output.splitlines()]

# we are interested in the basename of that path
filenames = [os.path.basename(row[7]) for row in output_table]

with this test input:

drwxr-xr-x   - user user          0 2017-05-04 17:19 /user/oozie/share/lib/lib_20170406204755
drwxr-xr-x   - user user          0 2017-05-04 17:19 /user/oozie/share/lib/lib_20110523212454

filenames will be ['lib_20170406204755', 'lib_20110523212454']