boost.asio : multi ports application-layer protocol

Question

I am designing a client-server app and I need to use three different channels between them. Below on the schema, the client opens a connection to the server (1), and then the server opens two connections to the client on other ports (2).

|-----------|                  |-----------|
|           |  (1) port: 8000  |           |
|           |<-----------------|           |
|           |                  |           |
|           |                  |           |
|  Server   |  (2) port: 8001  |  Client   |
|           |----------------->|           |
|           |                  |           |
|           |  (2) port: 8002  |           |
|           |----------------->|           |
|-----------|                  |-----------|

A simple channel server example : http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/example/cpp11/echo/async_tcp_echo_server.cpp

My questions is how to implement this, based on boost.asio framework ? Or, how to open the new connections (2) from the server ?

PS Maybe a better protocol could be to open the two other connections from client when the first is established ? But what could be the structure used in asio in this case ?

Thanks a lot in advance !

Answer 1

As has been pointed out, any system that's required to be robust should not assume that routing/firewalling is such that connections can be made back to the client from the server.

How would we go around this?

• client connects to server:8000
• server responds with a session UID
• client is responsible for connecting two more "channels", using the session UID as a correlation ID
• server decides that a session is completely ready if all 3 "channels" (ie sockets) participating in with a given session UID have been connected. If there's a timeout for this to happen, the server jots the whole session and closes any sockets that had been opened.

This is fairly easy, but requires a wire protocol to coordinate the sessions and channel "roles" for connections. I did initially elect not to implement this as a demo. Instead I thought it would be a nice exercise to learn Boost Asio, and implemented the back-channels (initiated from the server side) as in your original drawing.

The full code is on github: https://gist.github.com/sehe/9946161

Notes:

• there is a 'general' listener implementation that is used for both the server (port 8000) and "back-channels" (ports 8001,8002). See listener.hpp
• I opted for the stackless coroutine approach. This requires Boost Asio 1.54
• This approach has led to relative abuse of shared_ptr<> , at least in my view. The reason is that it's very beneficial if the coroutine (which is also the completion functor) is copyable without any issues. I could probably clean this up by making the class itself enable_shared_from_this and bind to shared_from_this instead. The benefit now is that there are (almost) no bind -expressions.

• Creating the 'back channels' is done in the server class which overrides on_accept :

   virtual bool on_accept(tcp::socket& socket) override { auto host = socket.remote_endpoint().address().to_string(); // for now setting up the back-connections is all synchronous - // that might not work well in practice (scaling, latency) but... try { tcp::resolver resolver(socket.get_io_service()); auto ep1 = resolver.resolve(tcp::resolver::query(host, "8001")); auto ep2 = resolver.resolve(tcp::resolver::query(host, "8002")); backsock1 = make_shared<tcp::socket>(socket.get_io_service()); backsock2 = make_shared<tcp::socket>(socket.get_io_service()); backsock1->connect(*ep1); backsock2->connect(*ep2); std::cerr << "on_accept: back channels connected for " << host << "\\n"; } catch(std::exception const& e) { std::cerr << "on_accept: '" << e.what() << "' for " << host << "\\n"; return false; } return base_type::on_accept(socket); } 

• If on_accept fails (eg for our server cannot connect the back-channels) an error is returned on the "main" socket (initial connection) and the session is aborted

There are three programs:

• run_server (which listens on port 8000)

• run_client (which connects to port 8000 and listens on 8001,8002), and sends 1 message. You can observe how the the server responds by connecting on the back-channels and sending different messages on all three sockets.

• test (which combines the two):

   #include <boost/asio.hpp> #include <boost/thread.hpp> #include "server.hpp" #include "client.hpp" int main() { boost::asio::io_service svc; // start service on a separate thread boost::thread th([&svc] { svc.post(demo::server(svc)); svc.run(); }); boost::this_thread::sleep_for(boost::chrono::milliseconds(500)); // allow server to start accepting // post client traffic to the service std::cerr << "Starting a test client that sends a message...\\n"; demo::client client(svc, "localhost", "8000"); // await interrupt (or new connections) th.join(); } 

The output of the last program looks like:

Starting a test client that sends a message...
on_accept: back channels connected for 127.0.0.1
listener 127.0.0.1:8000: accepting connection from 127.0.0.1:40999
listener 127.0.0.1:8000: 'hello world from demo client' received from 127.0.0.1:40999
listener 127.0.0.1:8001: accepting connection from 127.0.0.1:40132
listener 127.0.0.1:8002: accepting connection from 127.0.0.1:37970
listener 127.0.0.1:8001: 'We've received a request of length 29' received from 127.0.0.1:40132
listener 127.0.0.1:8002: 'We're handling it in void demo::server::do_back_chatter(const string&)' received from 127.0.0.1:37970
listener 127.0.0.1:40999: 'ECHO hello world from demo client' received from 127.0.0.1:8000