Angular 7 upload to blob storage with progress indicator

Question

I am trying to create a service that will allow me to:

1. Upload files to Azure blob storage
2. Return progress
3. When failed retry
4. When successful call my service to save the file path

To achieve this I started by following this tutorial on Medium . I have been able to save files in the storage and return the current progress.

My problems come when I want to call my service with the file path saved.

I have looked at the following to try and work out how to achieve this but without any success making multiple http requests , the rxjs docs making subsequent http requests .

I'm struggling to understand how I change the examples into something I can use.

Please note that I am trying to make the azureBlobStorageService reusable, therefore I am not making the second HTTP call within the azure service, that will be the responsibility of the caller.

Below is my code with comments where I have tried to add the mergeMap or flatMap etc, but without success. I have removed those references as I have tried so many variations I feel comments give a clearer picture of what i'm trying to achieve

Upload component

this.uploadProgress$ = from(inputNode.files as FileList).pipe(
  map(file => this._uploadService.updateCertificate(file)),
  combineAll()
);

Upload service

// this is where I would like to call my REST api when the file has uploaded to azure
updateCertificate(file: File): Observable<IUploadProgress> {
      return this._azureBlobStorage
        .uploadCertificateToBlobStorage(file, this.group)
        .pipe(
          map(
            progress => this.mapProgress(file, progress)
          ),
          //  flatMap(x => this._httpClient.post('xcv', JSON.Stringify(sasToken.filename))) <--fail 1
        )
        .pipe(flatMap(x => this._httpClient.post('', JSON.stringify('')))); <-- fail 2
  } // also tried merge map and a couple of others

  private mapProgress(file: File, progress: number): IUploadProgress {
    return {
      filename: file.name,
      progress: progress
    };
  }

Azure BlobStorage Service

uploadCertificateToBlobStorage(file: File, group: string): Observable<number> 
{
  this.populateSasToken('/cert/' + group + '/' + file.name);
  return this.uploadToBlobStorage(this.sasToken, file);
}

private populateSasToken(filename: string): void {
    //create sasToken stuff
  }

private uploadToBlobStorage(sasToken: ISasToken, file: File): Observable<number> {
  const customBlockSize = this.getBlockSize(file);
  const options = { blockSize: customBlockSize };
  const blobService = this.createBlobService(sasToken.storageAccessToken, sasToken.storageUri);

  blobService.singleBlobPutThresholdInBytes = customBlockSize;

  return this.uploadFile(blobService, sasToken, file, options);
}

  private createBlobService(sasToken: string, blobUri: string): IBlobService {
    return this._blobStorage
      .createBlobServiceWithSas(blobUri, sasToken)
      .withFilter(new this._blobStorage.ExponentialRetryPolicyFilter());
  }

// Need to change this to return a custom object with number and the sasToken.filename
// but when I change this return type and the return of the associated methods I errors, I can't see what i'm missing
private uploadFile(
    blobService: IBlobService,
    sasToken: ISasToken,
    file: File,
    options: { blockSize: number }
  ): Observable<number> {
    return new Observable<number>(observer => {
      const speedSummary = blobService.createBlockBlobFromBrowserFile(
        sasToken.container,
        sasToken.filename,
        file,
        options,
        error => this.callback(error, observer)
      );
      speedSummary.on('progress', () => this.getProgress(speedSummary, observer, sasToken.filename));
    }).pipe(
      startWith(0),
      distinctUntilChanged()
      // retry(4) I think this will allow me to retry failed called to azure. 
    );
  }

  private getProgress(speedSummary: ISpeedSummary, observer: Subscriber<number>, fileName: string): void {
    const progress = parseInt(speedSummary.getCompletePercent(2), 10);
    observer.next(progress === 100 ? 99 : progress);
  }

  private callback(error: any, observer: Subscriber<number>): void {
    if (error) {
      console.log(error);
      observer.error(error);
    } else {
      observer.next(100);
      observer.complete();
    }
  }

================================

Changes to upload file

the below causes

Type Observable is not assignable to type Observable

================================

export class Xxx {
  y: number;
  x: string;
}




private uploadFile(
    blobService: IBlobService,
    sasToken: ISasToken,
    file: File,
    options: { blockSize: number }
  ): Observable<Xxx> {
    return new Observable<Xxx>(observer => {
      const speedSummary = blobService.createBlockBlobFromBrowserFile(
        sasToken.container,
        sasToken.filename,
        file,
        options,
        error => this.callback(error, observer)
      );
      speedSummary.on('progress', () => this.getProgress(speedSummary, observer, sasToken.filename));
    }).pipe(
      startWith(0),
      distinctUntilChanged(),
      retry(4)
    );
  }

  private getProgress(speedSummary: ISpeedSummary, observer: Subscriber<Xxx>, fileName: string): void {
    const progress = parseInt(speedSummary.getCompletePercent(2), 10);
    // observer.next(progress === 100 ? 99 : progress);
    observer.next(new Xxx());
  }

  private callback(error: any, observer: Subscriber<Xxx>): void {
    if (error) {
      console.log(error);
      observer.error(error);
    } else {
      // observer.next(100);
      observer.next(new Xxx());
      observer.complete();
    }
  }

Answer 1

I used https://npmjs.com/package/angular-progress-http

It has been a while since I looked at this code but here is some code snippets that might help

file.service.ts

import * as FileSaver from 'file-saver';
import { Injectable } from '@angular/core';
import { ProgressHttp, Progress } from "angular-progress-http";
import { RequestOptions, Headers, Response, ResponseContentType } from '@angular/http';
import { AuthHttp } from 'angular2-jwt';

import { Observable } from 'rxjs/Observable';

import { environment } from '../environments/environment';

@Injectable()
export class FileService {

  constructor(private http: ProgressHttp, private authHttp: AuthHttp) { }

  upload(url: string, files: File[], listener: (progress: Progress) => void): Observable<Response> {
    let formData: FormData = new FormData();
    files.forEach(file => {
      if (file) {
        formData.append('uploadFile', file, file.name);
      }
    });
    let headers = new Headers();
    headers.append('Authorization', `Bearer ${localStorage.getItem('token')}`);
    let options = new RequestOptions({ headers: headers });
    return this.http.withUploadProgressListener(listener).post(url, formData, options);
  }

  download(url: string, filename: string) {
    let options = new RequestOptions(new Headers({ 'Content-Type': 'application/json' }));
    options.responseType = ResponseContentType.Blob;

    this.authHttp.get(url, options).subscribe(r => {
        this.saveFileContent(r, filename);
    });
  }

  private saveFileContent(res: Response, filename: string) {
    let fileBlob = res.blob();
    let blob = new Blob([fileBlob]);
    FileSaver.saveAs(blob, filename);
  }
}

and the api end point action.

    [Authorize(Roles = "Administrator"), HttpPost("AddFile/{id}")]
    public async Task<IActionResult> AddFile(int id)
    {
        var files = Request.Form.Files;
        if (files.Count > 0)
        {
            var sectionId = dbContext.Articles.Where(a => a.Id == id).Select(a => a.SectionId).Single();
            using (var fileStream = files[0].OpenReadStream())
            {
                await fileService.SaveAsync($"sections/{sectionId}/articles/{id}/{files[0].FileName}", fileStream);
            }
        }
        return Content("Ok");
    }

and the file service

using ContactManager.API.Models;
using Microsoft.Extensions.Options;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;

namespace ContactManager.API.Storage
{
    public class AzureFileService : IFileService
    {
        AppSettings appSettings;

        CloudStorageAccount storageAccount = null;

        CloudStorageAccount StorageAccount
        {
            get
            {
                if (storageAccount == null)
                {
                    storageAccount = new CloudStorageAccount(new Microsoft.WindowsAzure.Storage.Auth.StorageCredentials(this.appSettings.AzureStorage.Account, this.appSettings.AzureStorage.Key), true);
                }
                return storageAccount;
            }
        }

        CloudBlobClient blobClient = null;

        CloudBlobClient BlobClient
        {
            get
            {
                if (blobClient == null)
                {
                    blobClient = StorageAccount.CreateCloudBlobClient();
                }
                return blobClient;
            }
        }

        private CloudBlobContainer GetContainerReference(Permission permission)
        {
            return BlobClient.GetContainerReference(permission == Permission.Public ?  appSettings.AzureStorage.PublicFolder : appSettings.AzureStorage.PrivateFolder);
        }

        public AzureFileService(IOptions<AppSettings> appSettings)
        {
            this.appSettings = appSettings.Value;
        }

        public async Task SaveAsync(string path, Stream stream, Permission permission = Permission.Public)
        {
            var container = GetContainerReference(permission);
            var blockBlob = container.GetBlockBlobReference(path);
            await blockBlob.UploadFromStreamAsync(stream);
        }

        public void Delete(string path, Permission permission = Permission.Public)
        {
            var container = GetContainerReference(permission);
            var blockBlob = container.GetBlockBlobReference(path);
            blockBlob.DeleteAsync();
        }

        public async Task<Stream> GetAsync(string path, Permission permission = Permission.Public)
        {
            var container = GetContainerReference(permission);
            var blockBlob = container.GetBlockBlobReference(path);
            var stream = new MemoryStream();
            await blockBlob.DownloadToStreamAsync(stream);
            stream.Position = 0;
            return stream;
        }
    }
}

Hope this gives you a help.

Answer 2

Sorry, a bit late, but it looks like your this._httpClient.post('xcv', JSON.Stringify(sasToken.filename)) in the upload method is not returning an IUploadResponse. You'll need to map the response from the HttpClient call to match the IUploadResponse interface

I've also updated the article mentioned in the description to use the newer libraries - https://medium.com/@stuarttottle/upload-to-azure-blob-storage-with-angular-8-2ed80dfc6672