How to improve the performance of HTTP server based on POCO C++ library

Question

I have a hardware based on ARM processor with QNX RTOS, on which I need to stream http data between two applications http_client and http_server, both based on POCO C++ library. I have create simple client and server programs that stream contents of a file. On the hardware I get http throughput of 1.8 Gbps, whereas iperf give tcp throughput of 27 Gbps. (note that file read time has been deducted to arrive at the http throughput numbers)

I tried the same programs on Linux on Laptop, there too I see a difference in the throughput between http and tcp.

Can someone suggest ways through either Poco configuration, compile-time flags, API usage or setup which can help me achieve higher throughput nearer to the iperf tcp throughput.

Other details of my setup :

I am using the POCO shared release version 1.10.1, compiled with O2 optimization and making use of the template of HTTPFormServer which is given under poco-> Net -> Samples.

I have set the TCPNODELAY to false, which helped me in improving the performance on Linux on laptop, but not on QNX. I wonder why it did not help on hardware. Any pointers on this ?

Answer 1

iperf is a pure, bulk-transfer operation. Just like netperf TCP_STREAM. While it may appear to be "streaming" to you at the application layer, likely as not this POCO software is doing successive request/response pairs when "streaming" the data.

In theory you can see that in a packet capture. When you look at the packet capture of the iperf test, you will see large TCP data segments going from the sender to the receiver and only TCP ACKnowledgements going from the receiver to the sender. If there is some request/response going on with the HTTP "streaming" you will see at least some data segments (HTTP requests) going from the receiver back to the sender. And depending on how many requests this software will have outstanding at one time, you may see very different performance compared to iperf.