how to speed up display of large images from local file system in html / javascript

Question

I have an html app that I am working on to process large number of large images. We're talking about possibly 5,000 photos that are around 3-5MB each.

So far I am testing with around 1000 images and things are already getting pretty slow.

I am using drag and drop and a FileReader to load the images then setting the FileReader result as the source of the image:

    private loadImageFromDisk(image: IImage): Rx.Observable<IImage> {

        return Rx.Observable.defer( () => {
            console.log( `loading ${image.file.name} from disc` );
            console.time( `file ${image.file.name} loaded from file system` );

            const reader = new FileReader();
            setTimeout( () => reader.readAsDataURL(image.file), 0 ) ;

            const subject = new Rx.Subject();

            reader.onload = event => {
                subject.onNext( reader.result );
                subject.onCompleted();
            }

            return subject
                .safeApply(
                    this.$rootScope,
                    result => {
                        console.timeEnd( `file ${image.file.name} loaded from file system` );
                        image.content = reader.result;
                    }
                )
                .flatMap( result => Rx.Observable.return( image ) );
        } );
    }

html:

        <div
            ng-repeat="photo in controller.pendingPhotos"
            class="mdl-card photo-frame mdl-card--border mdl-shadow--4dp">
            <div class="mdl-card__title">
                {{photo.file.name}}
            </div>

            <div class="img-placeholder mdl-card__media">
                <div
                    ng-if="!photo.content"
                    class="mdl-spinner mdl-js-spinner is-active"
                    mdl-upgrade
                    ></div>

                <img class="img-preview" ng-if="photo.content" ng-src="{{photo.content}}"/>
            </div>

            <div class="mdl-card__supporting-text" ng-if="photo.response">
                {{controller.formatResponse(photo.response)}}
            </div>

        </div>

I know that ng-repeat can be a performance issue and I will sort that but at the moment even displaying one image can take a few seconds. If I load the image from disc but don't actually display it it only takes around 50-100 ms per image to load from disc. If I display it things get MUCH slower.

I suspect that the slowdown is the browser (chrome) having to resize the image.

In a test I did with 70 images I loaded all of them into the browser and after everything was loaded and rendered scrolling performance was slow the first few times I scrolled up and down the page, after that it was smooth.

These images are around 3,000 pixels by 2,000. I am resizing them to be 200 pixels long to display them.

What are the best approaches to speed this up?

Answer 1

I was facing the same problem some time ago (when doing service for photographers, using angular).

Problem is not about RxJS or angular, it is rather about browser itself - it is not optimised for displaying lots of big images this way.

At first if you need to display lots of images(does not matter is it local or remote files):

1. Resize them before displaying (faster loading, no need for resize, lower memory consumption).
2. If you can - display only visible images (otherwise page would be really slow until all images would be loaded). Check this answer: How do I get the x and y positions of an element in an AngularJS directive originally trackVisibility was written to display images only when they become visible.

---

About displaying images from local files, things are even more complicated:

In your case you are loading files as data urls, and there is a problem: 70 images you mentioned for 3 mb each will consume at least 2.1 Gb of RAM(actually more, and obliviously will affect performance)

First recommendation is - if you can: do not use data urls, better use URL.createObjectURL and use URL.revokeObjectURL when you do not need it anymore.

Second: if you need just thumbnails - resize images locally(using canvas) before displaying them. There would be an issue with antialiasing, if it is important for you case - take a look at step-down technic described here: Html5 canvas drawImage: how to apply antialiasing And if you are supporting iOS - there can be a problem with canvas size limitation, so you will need to detect it somehow. (both issues were addressed in example below)

And the last one: if you need to create thumbnails for lots of images - do not do this at once, instead - schedule work pieces over event loop (otherwise browser would be not responsive while resizing images). And for better preformance: do this sequentially(not in parallel for all images), it may sound strange - but, it would be faster(due too lower memory consumption, and less disk reads at the same time).

In summary:

1. Use trackVisibility directive mentioned above to display only visible images
2. Do not use, data urls especially for big images.
3. Create resized thumbnails, before displaying them

Libraries you may find useful for implementing this:

• https://github.com/blueimp/JavaScript-Load-Image
• https://github.com/blueimp/JavaScript-Canvas-to-Blob

Rough code example about doing image thumbnails (most of code was copied from working project - so, it is expected to work. canvasToJpegBlob and makeThumbnail was written just now and was not tested, so there can be small mistakes):

 function loadImage(imagePath) {
   return Rx.Observable.create(function(observer) {
     var img = new Image();
     img.src = imagePath;
     image.onload = function() {
       observer.onNext(image);
       observer.onCompleted();
     }
     image.onError = function(err) {
       observer.onError(err);
     }
   });
 }

 // canvas edge cases detection
 var maxDimm = 32000;
 var ios5 = false, ios3 = false;
 (function() {
   if (navigator.userAgent.indexOf('MSIE') !== -1 || navigator.appVersion.indexOf('Trident/') > 0) {
     maxDimm = 8000;
   } else {
     var canvas = document.createElement('canvas');
     canvas.width = 1024 * 3;
     canvas.height = 1025;
     if (canvas.toDataURL('image/jpeg') === 'data:,') {
       ios3 = true;
     } else {
       canvas = document.createElement('canvas');
       canvas.width = 1024 * 5;
       canvas.height = 1025;
       if (canvas.toDataURL('image/jpeg') === 'data:,') {
         ios5 = true;
       }
     }
   }
 }());

 function stepDown(src, width, height) {
   var
     steps,
     resultCanvas = document.createElement('canvas'),
     srcWidth = src.width,
     srcHeight = src.height,
     context;

   resultCanvas.width = width;
   resultCanvas.height = height;

   if ((srcWidth / width) > (srcHeight / height)) {
     steps = Math.ceil(Math.log(srcWidth / width) / Math.log(2));
   } else {
     steps = Math.ceil(Math.log(srcHeight / height) / Math.log(2));
   }

   if (steps <= 1) {
     context = resultCanvas.getContext('2d');
     context.drawImage(src, 0, 0, width, height);
   } else {
     var tmpCanvas = document.createElement('canvas');

     var
       currentWidth = width * Math.pow(2, steps - 1),
       currentHeight = height * Math.pow(2, steps - 1),
       newWidth = currentWidth,
       newHeight = currentHeight;

     if (ios3 && currentWidth * currentHeight > 3 * 1024 * 1024) {
       newHeight = 1024 * Math.sqrt(3 * srcHeight / srcWidth);
       newWidth = newHeight * srcWidth / srcHeight;
     } else {
       if (ios5 && currentWidth * currentHeight > 5 * 1024 * 1024) {
         newHeight = 1024 * Math.sqrt(5 * srcHeight / srcWidth);
         newWidth = newHeight * srcWidth / srcHeight;
       } else {
         if (currentWidth > maxDimm || currentHeight > maxDimm) {
           if (currentHeight > currentWidth) {
             newHeight = maxDimm;
             newWidth = maxDimm * currentWidth / currentHeight;
           } else {
             newWidth = maxDimm;
             newHeight = maxDimm * currentWidth / currentHeight;
           }
         }
       }
     }

     currentWidth = newWidth;
     currentHeight = newHeight;

     if ((currentWidth / width) > (currentHeight / height)) {
       steps = Math.ceil(Math.log(currentWidth / width) / Math.log(2));
     } else {
       steps = Math.ceil(Math.log(currentHeight / height) / Math.log(2));
     }


     context = tmpCanvas.getContext('2d');
     tmpCanvas.width = Math.ceil(currentWidth);
     tmpCanvas.height = Math.ceil(currentHeight);

     context.drawImage(src, 0, 0, srcWidth, srcHeight, 0, 0, currentWidth, currentHeight);

     while (steps > 1) {
       newWidth = currentWidth * 0.5;
       newHeight = currentHeight * 0.5;

       context.drawImage(tmpCanvas, 0, 0, currentWidth, currentHeight, 0, 0, newWidth, newHeight);
       steps -= 1;
       currentWidth = newWidth;
       currentHeight = newHeight;
     }

     context = resultCanvas.getContext('2d');
     context.drawImage(tmpCanvas, 0, 0, currentWidth, currentHeight, 0, 0, width, height);
   }
   return resultCanvas;
 }

 function canvasToJpegBlob(canvas) {
   return Rx.Observable.create(function(observer) {
     try {
       canvas.toBlob(function(blob) {
         observer.onNext(blob);
         observer.onCompleted();
       }, 'image/jpeg');
     } catch (err) {
       observer.onError(err);
     }
   });
 }

 function makeThumbnail(file) {
   return Observable.defer(()=> {
     const fileUrl = URL.createObjectURL(file);
     return loadImage(fileUrl)
       .map(image => {
         const width = 200;
         const height = image.height * width / image.width;
         const thumbnailCanvas = stepDown(image, width, height);
         URL.revokeObjectURL(fileUrl);
         return thubnailCanvas;
       })
       .flatMap(canvasToJpegBlob)
       .map(canvasBlob=>URL.createObjectURL(canvasBlob))
       .map(thumbnailUrl => {
         return {
           file,
           thumbnailUrl
         }
       })
   });
  }