NOTE: The C++ portion of this blog entry is incorrect. I've made some pretty stupid mistakes. These are discussed in a newer blog entry, which can be found here.
In this entry I will be discussing using Google's message format,
Protocol Buffers, to communicate between a Java client and a C++ server using TCP sockets.
I came across protobuf in my search for a good means of communicating over the network between apps that may not be written in the same language. I wanted an Android phone, an iPhone, or even just a laptop to be able to communicate with the same server program. I feel like this is an increasingly desirable feature for mobile apps, so I'm going to regurgitate the information that I compiled on my search.
The way protocol buffers works is this: You define your message (data types and fields) in
protobuf's own language. Then, once you have a properly formed message with all of the fields you need, you find a compiler for your language. Google provides official compilers for C++, Java, and Python.
There is also a huge list of 3rd party compilers for many other languages (pretty much any one you can think of, including Objective C for you iPhone guys!).
Then you run the protobuf message through the compiler, and the compiler generates a library (source code/headers) for you to use that includes the message object, parsers, getters and setters, serializers and deserializers, and many other nifty little things that you would normally have to painstakingly code out for your own message. You "include" this in your program, and you're good to go!
Another advantage of protobuf is that Google encodes the messages in to their own binary format rather than ASCII, so the messages are a lot smaller than a message sent via JSON or XML.
I found an unsatisfactory amount of stuff on the internet about using protobuf across platforms. In my case, I wanted to know how to send a message from Java -> C++, and from C++ -> Java. So, I needed to know how to send and receive a message on both sides via TCP.
Here is a code example of a Java client sending a protobuf message to a C++ server, and the C++ server replying with another protobuf message.
Note: I am using Java NIO because it's just way easier to deal with than Streams, and it's more efficient.
Java Send
//set up socket
SocketChannel serverSocket;
serverSocket=SocketChannel.open();
serverSocket.socket().setReuseAddress(true);
serverSocket.connect(new InetSocketAddress(servIP,servPort));
serverSocket.configureBlocking(true);
//create BAOS for protobuf
ByteArrayOutputStream baos=new ByteArrayOutputStream();
//mClientDetails is a protobuf message object, dump it to the BAOS
mClientDetails.writeDelimitedTo(baos);
//copy the message to a bytebuffer
ByteBuffer socketBuffer=ByteBuffer.wrap(baos.toByteArray());
//keep sending until the buffer is empty
while(socketBuffer.hasRemaining())
serverSocket.write(socketBuffer);
C++ Receive
//set up server socket
sockaddr_in *sa=(sockaddr_in *)malloc(sizeof(struct sockaddr_in));
memset(sa, 0, sizeof(struct sockaddr_in));
sa->sin_family = AF_INET;
sa->sin_addr.s_addr = htonl(INADDR_ANY);
sa->sin_port = htons(port);
serverSock=socket(PF_INET,SOCK_STREAM,IPPROTO_TCP);
int reuse=1;
setsockopt(serverSock, IPPROTO_TCP, SO_REUSEADDR, &reuse, sizeof(reuse));
//bind socket to port, listen
bind(serverSock, (sockaddr *)sa, sizeof(*sa));
listen(serverSock, 1);
//create protobuf object
protobuf::ClientDetails client;
//accept a client connection
sockaddr_in *clientAddr;
long unsigned int caSize=sizeof(clientAddr);
int clientSock=accept(serverSock, (struct sockaddr *)&clientAddr, (socklen_t *)&caSize);
//receive message from the client, where BUFFER_SIZE is large enough to contain your message
int received=recv(clientSock, buffer, BUFFER_SIZE, 0);
//read varint delimited protobuf object in to buffer
//there's no method to do this in the C++ library so here's the workaround
google::protobuf::io::ArrayInputStream arrayIn(buffer, received);
google::protobuf::io::CodedInputStream codedIn(&arrayIn);
google::protobuf::uint32 size;
codedIn.ReadVarint32(&size);
google::protobuf::io::CodedInputStream::Limit msgLimit = codedIn.PushLimit(size);
client.ParseFromCodedStream(&codedIn);
codedIn.PopLimit(msgLimit);
C++ Send
//already set up a message object called serverAck
//make a buffer that can hold message + room for a 32bit delimiter
int ackSize=serverAck.ByteSize()+4;
char* ackBuf=new char[ackSize];
//write varint delimiter to buffer
google::protobuf::io::ArrayOutputStream arrayOut(ackBuf, ackSize);
google::protobuf::io::CodedOutputStream codedOut(&arrayOut);
codedOut.WriteVarint32(serverAck.ByteSize());
//write protobuf ack to buffer
serverAck.SerializeToCodedStream(&codedOut);
//send buffer to client
send(clientSock, ackBuf, ackSize, 0);
delete(ackBuf);
Java Receive
//receive message from the client, where BUFFER_SIZE is large enough to contain your message
socketBuffer=ByteBuffer.allocate(BUFFER_SIZE);
int bytesRead=serverSocket.read(socketBuffer);
//copy message byte array from socket buffer
socketBuffer.flip();
byte[] ackBuf = new byte[socketBuffer.remaining()];
socketBuffer.get(ackBuf);
//create ByteArrayInputStream from byte[] (this is what protobuf wants)
ByteArrayInputStream ackStream=new ByteArrayInputStream(ackBuf);
//parse message
ServerAck serverAck=ServerAck.parseDelimitedFrom(ackStream);
//done!
socketBuffer.clear();
For further reading on implementing protobuf, I recommend taking a look at
Google's tutorials.