We’ve started working on making Collage endian safe to be able to communicate between an IBM BlueGene and X86 workstations. This requires that all data exchanges can be byte-swapped, which necessitates heavy refactoring. Long story short, this stalled the 1.0 API definition since the API for Node/LocalNode is not yet final. Therefore I’ll throw in a preview on how the new Collage peer-to-peer communications will look like.
Every communication will be stream-based (see last week’s post). The receiver knows the endianness of the sender and will swap, if necessary, the byte order in the corresponding DataIStream. This approach has the benefit that byte swapping only occurs in mixed-endian environment, whereas the traditional network big endian order nowadays requires two swaps in an x86-only cluster. The packets exchanged between nodes today will disappear completely, since Collage can’t examine their structure.
You can observe this work in the corresponding issue ticket. Sending a packet in the old way looks like this:
NodeAttachObjectPacket packet;
packet.requestID = _localNode->registerRequest( object );
packet.objectID = id;
packet.objectInstanceID = instanceID;
_localNode->send( packet );
This is replaced by a DataOStream, which will send the data once it goes out of scope, making the code much nicer:
const uint32_t requestID = _localNode->registerRequest( object );
_localNode->send( CMD_NODE_ATTACH_OBJECT ) << id << instanceID << requestID;
The receiving side doesn’t improve as much, since we still need to extract all data into local variables. The old code accesses the raw dat by casting it to the appropriate packet:
const NodeAttachObjectPacket* packet = command.get();
[...]
_attachObject( object, packet->objectID, packet->objectInstanceID );
The new code extracts the data into local variables, which will be endian-converted if necessary:
const UUID& objectID = stream< UUID >.get();
const uint32_t instanceID = stream< uint32_t >.get();
[...]
_attachObject( object, objectID, instanceID );
The good news is that Equalizer application code is not affected at all by this. Besides the byte swapping, this will enable other features, since all data passes through the DataI/OStream, for example automatically compressing any packet over a certain size.
This work will continue the following weeks, and once it’s merged into the master branch I’ll continue with my introduction to the then new-and-shiny Collage.
Parallel Rendering 