Scala Source.fromInputStream not generating full results

Question

We are working on writing an existing python utility into Scala. Utility downloads compressed data from a REST API as ".gzip" file.

Python code :

response=requests.get(url,stream=True,headers=self.header,proxies=config.PROXIES,timeout=config.TIMEOUT)
        with open(file_path, 'wb+') as f:
            shutil.copyfileobj(response.raw, f) 

While in scala I am writing it this way :

var out: GZIPOutputStream = new GZIPOutputStream(new FileOutputStream(outputFile))
    var writer= new PrintWriter(out)
    try {
      val inputStrem: InputStream = response.getEntity.getContent
      val gzipInputStream = new GZIPInputStream(inputStrem)
      for(line <- Source.fromInputStream(gzipInputStream).getLines){
        writer.write(line+"\n")
      }          
    } catch {
      case e: Exception => throw e
     }finally {
      if (writer != null) writer.close
    }

But the output file is getting produced of different sizes every time with scala code. Python code is running fine. REST POST and GET requests are same for both utilities. I am not getting any errors in scala , its getting executed successfully but creating different sizes files. Any help would be really appreciated .

Thank You !!

Answer 1

If this is Apache HTTP then HttpEntity has a writeTo method that writes the content directly to an OutputStream .

response.getEntity().writeTo(new FileOutputStream(outputFile))

This does not re-encode the data like the original Scala does, but the Python version does not do that either.