1. Turbo Exchange
  2. Developers Area
  3. Socket Transport Streaming

Socket Transport Streaming

    Table of contents
    1. 1. Steps to Encode:
    2. 2. Steps to decode:

    Protocol Buffers Serialises & Deserialises structured message types.  It does not define how we handle it beyond that.

    Problem 1: It is not possible to know in advance how long (in bytes) a serialised proto message is.  To solve this we attach a header describing the length if the serialised data.  In protocol buffer fashion, we use base 128 varints for this. See http://code.google.com/apis/protocolbuffers/docs/encoding.html#varints

    Problem 2: If more than one type of structure is to be sent, you need to know in advance what type of message it is so that is can be deserialised.  We solve this, by wrapping our proto message, in another proto message.  We call it the MessageWrapper, which defines a message type, and the payload (amongst other things).

    For integraty, we add a sync byte (STX).  Whilst it's assumed the data transport (e.g TCP/IP socket) is error corrected, in theory the receiver should never loose it's place.  However, software bugs can never be ruled out, so a single constant byte is added to the start of the message. 

     MessageWrapper.png

     

    Steps to Encode:
    1. Serialise the MessageWrapper.
    2. Encode the length of serialised MessageWrapper to a varint 128 (1 or more bytes).
    3. Send an STX (hex aa) followed by the varint, followed by the serialised MessageWrapper
    Steps to decode:
    1. Look for the STX (it should be the first byte you encounter).
    2. Decode the Length (varint 128).
    3. Is there enough data in the buffer to satisfy the length of data we are looking for? If not, wait.
    4. Verify that the last byte of the given length is the end of the stream, or followed by another STX.
    5. Deserialise the MessageWrapper