使用从 nodeJs 到 angular 应用程序的流下载大文件

Question

i am struggling to get this stream downloading file works if i try the same request using curl the stream works fine and streaming the data. in my angular app the file is completely download before the client see the download file tab it seems like the subscribe only happened after all stream body fully downloaded

what i am trying to achive is after first chunk of data send to angular app i want the file to start downloading.

instead after observing the network only after all file downloaded from the backend the downLoadFile methood is called and the ui expirence is stuck

this is a minimal example of what i am trying todo at the backend i have a genrator that genreate a huge file and pipe the request

Node JS

const FILE = './HUGE_FILE.csv'

const lines = await fs.readFileSync(FILE, 'utf8').split('\n')

function* generator() {
    for (const i of files) {
        console.log(i)
        yield i
    }
}
app.get('/', async function (req, res) {
    res.setHeader('Content-Type', 'text/csv');
    res.setHeader('Content-Disposition', 'attachment; filename=\"' + 'download-' + Date.now() + '.csv\"');
    res.setHeader('Cache-Control', 'no-cache');
    res.setHeader('Pragma', 'no-cache');
    const readable = Readable.from(generator());
    readable.pipe(res);
});

at the client side calling the endpoint and waiting for the resonse

Angular code

@Component({
    selector: 'is-optimizer-csv',
    templateUrl: './optimizer-csv.component.html',
    styleUrls: ['./optimizer-csv.component.scss']
})
export class OptimizerCsvComponent implements OnInit {

    private onDownloadButtonClicked() {
        const data = {...this.form.get('download').value, ...(this.advertiserId ? {advertiserId: this.advertiserId} : null)};
        this.loading$.next(true);
        this.optimizerService
            .downloadOptimizerCsvData(data)
            .pipe(
                take(1),
                tap(_ => this.loading$.next(false))
            )
            .subscribe();
    }

  
}





@Injectable({
    providedIn: 'root'
})
export class OptimizerService {
    constructor(private readonly http: Ht, private datePipe: DatePipe) {}

    downloadOptimizerCsvData(data: any) {
        this.http.get(`${environment.apiUrl}`,{
            responseType: 'arraybuffer',headers:headers} 
           ).subscribe(response => this.downLoadFile(response, "text/csv"));
  
      }
      downLoadFile(data: any, type: string) {
        let blob = new Blob([data], { type: type});
        let url = window.URL.createObjectURL(blob);
        let pwa = window.open(url);
        if (!pwa || pwa.closed || typeof pwa.closed == 'undefined') {
            alert( 'Please disable your Pop-up blocker and try again.');
        }
    }
}

Answer 1

it look like angular common http client does not support streaming out of the box.

see this github issue by @prabh-62

https://github.com/angular/angular/issues/44143

from this thread the why to solve your issue is by implementing the fetch stream logic using native fetch

Answer 2

The reason why your file gets immediately downloaded is because you are subscribing upon the construtor in which your service is injected gets initialized.Meaning your component is just telling the service to fire the download method with the HttpClient call in it, while it seems you are having a (click)="onDownloadButtonClicked()" event in your template.

The solution is to delegate the firing of the action until the onClick event from the UI is fired. To do so

1- Remove the subscription from your service method.

 downloadOptimizerCsvData(data: any) {
        this.http.get(`${environment.apiUrl}`,{
            responseType: 'arraybuffer',headers:headers} 
           )
  
      }

2- Return an Observable to the consumer component

 downloadOptimizerCsvData(data: any) {
       return this.http.get(`${environment.apiUrl}`,{
            responseType: 'arraybuffer',headers:headers} 
           )
  
      }

3- Subscribe inside the onDownloadButtonClicked() by refactoring the method to somthing like

 private onDownloadButtonClicked() {
        const data = {...this.form.get('download').value, ...(this.advertiserId ? {advertiserId: this.advertiserId} : null)};
        this.loading$.next(true);
        this.optimizerService
            .downloadOptimizerCsvData(data)
            .pipe(
                take(1),
                tap(_ => this.loading$.next(false))
            )
            .subscribe(response => this.optimizerService.downLoadFile(response, "text/csv"));
    }

If you want to download the file as soon as the view becomes available then you need to call the method programmatically inside the ngOnInit(){} lifecyclehook without a user based triggered event.

Read here on this lifecyclehook and how you can use it