with python how to find a string from all files in a directory and subfloders

Question

i am trying to find which files contains "RunInstances" from aws cloudtrail logs, with grep i could easily run this command to find out: grep -r "RunInstances" *

but i want to try to use python, i tried os.walk, and something is wrong:

john@john-HP-ProBook-4411s:~/Downloads$ python
Python 2.7.12 (default, Nov 20 2017, 18:23:56)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> for path,dir,file in os.walk("."):
...     for fileNames in file:
...             if fileNames.endswith("json"):
...                     fileName = str(os.path.join(path,dir,file))
...                     print(fileName)
...
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File "/usr/lib/python2.7/posixpath.py", line 68, in join
    if b.startswith('/'):
AttributeError: 'list' object has no attribute 'startswith'
>>>

could you please offer me with some suggestions please?

Answer 1

The problem is here:

fileName = str(os.path.join(path,dir,file))

You're trying to join a path, a list of names, and a list of names into a path. That doesn't make any sense. If you look at the sample code you copied, I'm pretty sure it's joining the path and a single dir or file from the list, not a path plus both lists.

In particular, you probably want to os.path.join(path, fileNames) .

That may seem confusing, but that's because your variable names are confusing. Passing file to join fails because file is, despite the name, a whole list of file names, while passing fileNames would work because, again despite the name, it's just a single file name.

Answer 2

for语句中的file是os.walk（）所在目录中所有文件的列表。如果只有一个文件，则是一个元素的列表。

Answer 3

import os
for path,dir,file in os.walk("."):
    for fileNames in file:
        if fileNames.endswith("json"):
            fileName = str(os.path.join(path,fileNames))
            print(fileName)

You were close, just dir is a list and so ist file . fileNames on the other hand is just a string. And you cant join a path with a list as one argument.