How to send file over TCP in one time in Nodejs

Question

I want to send file over tcp/ip using Nodejs which I do suceess with the file which has size less than 65536 bytes.

Do you set buffersize? Yes, I do.

I try to send picture which has 233565 bytes, but it looks like nodejs seperate my file to send it multiple times.

Here is my Example Codes

Server.js

const net = require('net');
net.bytesWritten = 300000;
net.bufferSize = 300000;
const fs = require('fs');
const server = net.createServer((c) => {
  // 'connection' listener
  console.log('client connected');
  c.on('end', () => {
    console.log('client disconnected');
  });
  // c.write('hello\r\n');
  fs.readFile('daemon0.png' , (err, data) =>{
    if(!err){
      console.log(data.length);
      c.write(data);
    }
    else {
      console.log('readfile daemon0 err');
    }
  });
  c.pipe(c);
});
server.on('error', (err) => {
  throw err;
});
server.listen(8124, 'localhost', () => {
  console.log('server bound');
});

Server.js output

server bound

client connected

233565

client disconnected

Client.js

const net = require('net');
net.bufferSize = 300000;
net.bytesRead = 300000;
const client = net.connect({port: 8124, address: 'localhost' }, () => {
  // 'connect' listener
  console.log('connected to server!');
  client.write('world!\r\n');
});
client.on('data', (data) => {
  // console.log(data.toString());
  console.log(net.bufferSize,data.length);
  client.end();
});
client.on('end', () => {
  console.log('disconnected from server');
});

Client.js output

connected to server!

300000 5840

300000 65536

300000 65536

300000 65536

300000 31125

disconnected from server

as both output of client it looks like client has a limit to receive the data which maximum is 65536 bytes. How to extend the buffersize limit in TCP nodejs?

I ask this question in nodejs gitter chat and mpotra answer this make sense explanation for me.

That's not a problem. That's how streaming works: sending something large into smaller packets. If your only desire is to increase the buffer size so that you would receive the file into one single data event, then you're thinking of it wrong. If your file would have 500MB, based on your expectation, both the server and the client would have to allocated 500MB! into memory, just to satisfy the need of one single data event; which is obviously the wrong approach to data transfer.

Answer 1

Node is sending the whole Buffer just fine, but your client will emit data events in chunks. If you want to receive the whole file, you need to buffer the chunks:

const chunks = []
client.on('data', chunk => chunks.push(chunk))
client.on('end', () => {
  const file = Buffer.concat(chunks)
  // do what you want with it
})

I am not aware of a method to control when data events are emitted in Node.

Answer 2

I think you need to tune up your kernel buffer:

# echo 'net.core.wmem_max=12582912' >> /etc/sysctl.conf
# echo 'net.core.rmem_max=12582912' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_rmem= 10240 87380 12582912' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_wmem= 10240 87380 12582912' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_window_scaling = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_timestamps = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_sack = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_no_metrics_save = 1' >> /etc/sysctl.conf
# echo 'net.core.netdev_max_backlog = 5000' >> /etc/sysctl.conf
//reload
sysctl -p
//view changes
tcpdump -ni eth0

For windows take a look here