Processing images with Watchdog in Python

Question

I'm using the watchdog library to detect when new images are created in a specific folder. When watchdog detects a newly created image, I kick off some image processing functions with SimpleCV / OpenCV.

However, the pictures are being taken from a Raspberry Pi camera, and from the below error I don't believe the entire image is saved when it first appears in the directory. (Essentially the file is being saved in "pieces", or multiform).

Please note that when I copy & paste an image into the respective folder, the script runs successfully.

Ask: Is there any way to kick off the image processing only once the entirety of the file has been saved?

import time
from SimpleCV import Image
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class ExampleHandler(FileSystemEventHandler):
    def on_created(self, event):
        image = Image(event.src_path)
        do image processing stuff
        print "Got event for file %s" % event.src_path 

observer = Observer()
event_handler = ExampleHandler() # create event handler
observer.schedule(event_handler, path='/path/to/images') # set observer to use created handler in directory
observer.start()

# sleep until keyboard interrupt, then stop + rejoin the observer
try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    observer.stop()

observer.join()

Error Dump:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/watchdog/observers/api.py", line 241, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "/usr/local/lib/python2.7/dist-packages/watchdog/observers/api.py", line 408, in dispatch_events
    self._dispatch_event(event, watch)
  File "/usr/local/lib/python2.7/dist-packages/watchdog/observers/api.py", line 403, in _dispatch_event
    handler.dispatch(event)
  File "/usr/local/lib/python2.7/dist-packages/watchdog/events.py", line 361, in dispatch
    _method_map[event_type](event)
  File "picture_classifier_cube.py", line 11, in on_created
    image = Image(event.src_path)
  File "/usr/local/lib/python2.7/dist-packages/SimpleCV/ImageClass.py", line 1073, in __init__
    self._pil = pil.open(self.filename).convert("RGB")
  File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 1980, in open
    raise IOError("cannot identify image file")
IOError: cannot identify image file

Edit After updating the handler to sleep a few seconds after recognizing a new file, I receive a different error.

class ExampleHandler(FileSystemEventHandler):
    def on_created(self, event):
        time.sleep(3)
        cans = Image(event.src_path)

Error

IOError: [Errno 2] No such file or directory: '/path/to/images/test.jpg~'

Note that I am capturing an image with the following Pi command: raspistill -o 'test.jpg'

Answer 1

While there may not be a way to know for certain if a file is finished being written to without modifying the program that does the writing, you could monitor the file size:

from os import stat
from time import sleep

def wait_for_write_finish( filename ):
    last_size, size= -1, 0
    while size!=last_size:
        sleep(1)
        last_size, size= size, stat(filename).st_size

You may want to do this with appropriate threading