How to synchronize a windows folder in python flask app with a sqlite database table?

Question

I am looking for a solution to compare filenames inside sqlite database with filenames in a windows directory (app.config['UPLOAD_FOLDER']).

I have a sqlite database which has a table called "file", where i have all data from my file.

file is defined as: files = [file.to_displayable_dict() for file in File.query if is_file_visible(file)] and this is the function:

def to_displayable_dict(self):
        return {
            "Dateiname" : "<div class=hasTooltip> <a href=/download/" + self.name + ">" + self.name + "</a> <span>" + self.beschreibung + "</span></div>",
            "Kategorie" : self.kategorie,
            "Bezeichnung" : self.bezeichnung,
            "TV" : self.tv,
            "Stand" : self.stand,
            "Upload-Benutzer" : self.uploadbenutzer,
            "Vertraulichkeitsstufen" : self.Vertraulichkeitsstufen
            }

Inside the sqlite table file I have this kind of data if I print("Files:", files) LOOK additional comments added.

If I go into my windows instance folder (app.config['UPLOAD_FOLDER']) and delete the file (Dateiname: EPEX_SPOTPRICE_STROM), it should automatically delete the file in my file table of my database.db.

For the solution there must be a comparison of filenames in WIndows directory and filename in the table file.

I dont know because of href how to make the comparison. My thoughts are, but doesnt work. Post actions over a route are not a good solution.

something like get_files = os.listdir(app.config['UPLOAD_FOLDER'])
for file in Files.query():
if file.filename not in files_in_folder:
db.session.delete(file.filename)
db.session.commit()
flash (data was deleted )

How would you do it under python flask? Any suggestions with own code example? Thanks!

I tried to word around this:

import os
path = r"C:/Users/add706/Documents/NRL_webseite/instance/uploads"

def get_files(path):
    for file in os.listdir(path):
        if os.path.isfile(os.path.join(path, file)):
            yield file  
for file in get_files(path):
    print("file:", file)

output of that file: EPEX_SPOTPRICE_STROM.docx

Answer 1

If you want to have an endpoint with which all file references that are no longer physically present in the file system are deleted from the database, the code should be as follows.

@app.route('/sync')
def sync():
    files = os.listdir(app.config['UPLOAD_FOLDER'])
    if File.query.filter(File.name.notin_(files)).delete():
        db.session.commit()
    return redirect(url_for('index'))

Another option is to use filesystem events, which I'm not very familiar with. Below is an example code using watchdog .

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

# ...

class MyHandler(FileSystemEventHandler):
    def on_deleted(self, event):
        if not event.is_directory:
            target = event.src_path[len(app.config['UPLOAD_FOLDER'])+1:]
            if File.query.filter_by(name=target).delete():
                db.session.commit()

def run_watcher(path):
    fs_handler = MyHandler()
    observer = Observer()
    observer.schedule(fs_handler, path, recursive=False)
    observer.start()

run_watcher(app.config['UPLOAD_FOLDER'])

It would also be possible to have a script which is executed at regular intervals and removes all invalid entries.