Archive for the ‘Windows’ Category

Equalizer 1.4 released

7. September 2012


The last two weeks have been quiet, since I was on biking through Switzerland. Meanwhile, poor Daniel back at work churned through most of the Collage changes outlined in the last post. You can see the changes in the Collage endian branch on github, which will be merged back into master in the next couple of weeks.

Now back to the news: After finally figuring out how to build Equalizer and dependencies using MacPorts portfiles on Mac OS X, I released the long-standing 1.4 version of Equalizer, GPU-SD, Lunchbox and vmmlib. Below is the release announcement – enjoy!

Neuchatel, Switzerland – September 7, 2012 – Eyescale is pleased to announce the release of Equalizer 1.4.

Equalizer is the standard framework to create and deploy parallel, scalable 3D applications. This modular release includes Collage 0.6, a cross-platform C++ library for building heterogenous, distributed applications, GPU-SD 1.4, a C++ library and daemon for the discovery and announcement of graphics processing units using zeroconf networking and Lunchbox 1.4, a C++ library for multi-threaded programming. All software packages are available for free for commercial and non-commercial use under the LGPL open source license.

Equalizer 1.4 is a feature release extending the 1.0 API, introducing major new features, most notably asynchronous readbacks, region of interest and thread affinity for increased performance during scalable rendering. It culminates over seven years of development and decades of experience into a feature-rich, high-performance and mature parallel rendering framework and related high-performance C++ libraries.

Equalizer enables software developers to easily build interactive and scalable visualization applications, which optimally combine multiple graphics cards, processors and computers to scale the rendering performance, visual quality and display size.

Equalizer Applications

Eyescale provides software consulting and development services for parallel 3D visualization software and GPU computing applications, based on the Eyescale software products or other open and closed source solutions.

Please check the release notes on the Equalizer website for a comprehensive list of new features, enhancements, optimizations and bug fixes. A paperback book of the Programming and User Guide is available.

We would like to thank all individuals and parties who have contributed to the development of Equalizer 1.4.

Left image courtesy of Cajal Blue Brain/ / Blue Brain Project. Second from left copyright Realtime Technology AG, 2008. Right image courtesy University of Siegen, 2008.

Sneak Peek: The new Collage

17. August 2012

We’ve started working on making Collage endian safe to be able to communicate between an IBM BlueGene and X86 workstations. This requires that all data exchanges can be byte-swapped, which necessitates heavy refactoring. Long story short, this stalled the 1.0 API definition since the API for Node/LocalNode is not yet final. Therefore I’ll throw in a preview on how the new Collage peer-to-peer communications will look like.

Every communication will be stream-based (see last week’s post). The receiver knows the endianness of the sender and will swap, if necessary, the byte order in the corresponding DataIStream. This approach has the benefit that byte swapping only occurs in mixed-endian environment, whereas the traditional network big endian order nowadays requires two swaps in an x86-only cluster. The packets exchanged between nodes today will disappear completely, since Collage can’t examine their structure.

You can observe this work in the corresponding issue ticket. Sending a packet in the old way looks like this:

NodeAttachObjectPacket packet;
packet.requestID = _localNode->registerRequest( object );
packet.objectID = id;
packet.objectInstanceID = instanceID;
_localNode->send( packet );

This is replaced by a DataOStream, which will send the data once it goes out of scope, making the code much nicer:

const uint32_t requestID = _localNode->registerRequest( object );
_localNode->send( CMD_NODE_ATTACH_OBJECT ) << id << instanceID << requestID;

The receiving side doesn’t improve as much, since we still need to extract all data into local variables. The old code accesses the raw dat by casting it to the appropriate packet:

const NodeAttachObjectPacket* packet = command.get();
[...]
_attachObject( object, packet->objectID, packet->objectInstanceID );

The new code extracts the data into local variables, which will be endian-converted if necessary:

const UUID& objectID = stream< UUID >.get();
const uint32_t instanceID = stream< uint32_t >.get();
[...]
_attachObject( object, objectID, instanceID );

The good news is that Equalizer application code is not affected at all by this. Besides the byte swapping, this will enable other features, since all data passes through the DataI/OStream, for example automatically compressing any packet over a certain size.

This work will continue the following weeks, and once it’s merged into the master branch I’ll continue with my introduction to the then new-and-shiny Collage.

Introducing Collage: Barrier

15. July 2012

While I personally think that barriers are an anti-pattern, they have exactly one valid use case in my line of work — as swap barriers synchronizing the display of a new frame across multiple segments of a display wall or immersive installation.

In an ideal work, swap synchronization would be done using hardware support. Equalizer supports this for nVidia G-Sync, but I haven’t seen many installations using hardware swap synchronization. First, it’s expensive since you need a professional grade card with a special synchronization board. So lower cost installations such as display walls typically don’t even have the hardware. Installations which need the frame synchronization, such as active stereo setups, oftentimes only use the frame (retrace) synchronization and use a software barrier for swap synchronization. The reason is that getting hardware swap synchronization running is such a mess due to driver issues that most people don’t bother. For these reasons, Equalizer both supports hardware and software swap synchronization. The software synchronization used a co::Barrier.

Back to the Collage barrier: Once it is set up, the only call needed is enter, which will block until the height has been reached.

Barriers are versioned, distributed objects. Any process can set up a barrier, register it and communicate the barrier identifier and version to all users of the barrier. The users map, sync and enter the barrier. Since it’s versioned, the master instance can be committed any time, and enter requests are versioned, that is, a barrier operation for an old version will be finished before the enter requests for the new version are processed.

The protocol right now is very simple, a master instance simply tracks enter requests until the height is reached and then unlocks all users. While there are other algorithms which use optimized tree structures or broadcast to reduce the latency, we haven’t seen the need to implement a more complex algorithm.

Introducing Collage

6. July 2012

Collage evolved from the Equalizer network library. Now it’s used by a few other projects as well, and I’ve been mentioning it quite a few times on this blog already.

Looking back over the last seven years of Equalizer, Collage has received by far the largest investment in manpower compared to all the other components. It seems innocent enough, but trust me, getting a distributed network library right and bug free is no small task. On the one hand, the ‘fun’ implementation of the Windows IP stack (WSASYSCALLFAILURE and stack corruptions) took a lot of debugging to get going, and on the other hand advanced features such as InfiniBand support (thanks Dardo!) and fast, reliable multicast are no small task to implement.

What’s wrong with boost::asio?

Nothing. It provides about the same functionality as co::Connection and co::ConnectionSet. We use it as the UDP backend for the RSP multicast implementation. It wasn’t around when I started eq::net, which became Collage. We’ld love to replace co::Connection and co::ConnectionSet with it, but the effort of porting the RDMA and RSP connections to asio has prevented this so far. Ultimately Collage tries to provide higher level abstractions.

What’s wrong with 0MQ?

Again: Nothing. It provides higher level abstractions then asio. It’s less likely that we’ll use it as the backend for Collage, since some of the design decisions are somewhat different from what Collage is doing.

So, what’s next?

Collage, similar to Lunchbox, aims to provide high-level abstractions. Similarly to the introducing Lunchbox series, I’ll present them over the next few weeks. In the same timeframe, we are planning to separate the Equalizer and Collage projects as well as to define and document the ‘1.0’ Collage API.

For now, you can find a technical overview presentation on the Collage website. Enjoy!

Introducing lunchbox::Servus

22. June 2012

Yes, we’re back to Lunchbox. During the 1.4 beta release (see previous post) we decided to merge a separate library in Lunchbox, since keeping it separate would have been overkill. In the end, it’s just a single class: Servus.

For the 999‰ of you who can’t get the reference: It’s a play on words to Bonjour. This class implements a simple C++ interface to announce and discover key-value pairs over zeroconf networking (aka. Bonjour).

The usage is really simple: You instantiate the class with the service name you’re interested in, e.g., “_collage._tcp”. Then you can register key-value pairs and start announcing them, and you can discover existing key-value pairs announced by other processes, typically within the subnet. It’s also legal to update key-value pairs on an announced service. Sounds simple enough, but if you’ve ever used the callback-driven dns_sd C API, you’ll know it’s much easier to use.

We use this class for two things right now: Collage node discovery and in GPU-SD.

In Collage, each co::LocalNode announces his node identifier and connection descriptions. This information is used as a fallback path when performing a LocalNode::connect( NodeID ) to identify and to connect to a previously unknown node. The Servus handle is exposed to applications through co::Zeroconf, which is a wrapper to ensure thread-safety. Applications can then use additional key-value pairs specific to their implementation.

In GPU-SD, Servus is used to announce and discover graphics cards in a visualization cluster, which in turn is used by Equalizer for auto-configuration.

Equalizer 0.9 Released!

11. August 2009

Cross-Segment Load-Balancing
We are pleased to announce the release of Equalizer 0.9, the standard framework to create and deploy parallel, scalable OpenGL applications. The most notable new features in this release are:

Please check the release notes on the Equalizer website for a comprehensive list of new features, enhancements, optimizations and bug fixes. A paperback book of the Programming and User Guide is available from Lulu.com.

We would like to thank all individuals and parties who have contributed to the development of Equalizer 0.9.

RTT DeltaGen 8.5 (RTT Scale) video

24. January 2009

RTT just published a DeltaGen 8.5 video on youtube, including a shot of RTT Scale, which is based on Equalizer. Enjoy some realtime raytraced Audi Q5 goodness with global illumination:

Youtube link

List of available affinity GPU’s

24. January 2009

If you just want to check if your Windows system can address GPU’s individually in OpenGL, there is a new tool on the Equalizer website which lists the GPU’s visible through the WGL_NV_gpu_affinity API. This is often useful if you want to cross-check your own code, or just verify a system setup. Feel free to post questions below!

Equalizer 0.6 released!

3. December 2008

Here is the release announcement:

Scalable Display System

We are pleased to announce the release of Equalizer 0.6, a major advance in parallel OpenGL rendering, supporting:

  • Automatic 2D and DB load-balancing
  • DPlex (time-multiplex) compounds
  • Paracomp compositing backend

Please check the release notes on the Equalizer website for a comprehensive list of new features, enhancements, optimizations and bug fixes. The Programming Guide can be ordered at lulu.com.

We would like to thank all individuals and parties who have contributed to the development of Equalizer 0.6.

About RTT Scale

RTT Scale is the new scalability module for RTT’s leading high-end visualisation software suite RTT DeltaGen. The module is to be integrated into the forthcoming Version 8.5. RTT Scale uses Equalizer 0.6 to boost OpenGL and ray-tracing performance on highly realistic, complex scenes. Please visit http://www.rtt.ag for more information.

About Equalizer

Equalizer is the standard middleware to create parallel OpenGL-based applications. It enables applications to benefit from multiple graphics cards, processors and computers to scale rendering performance, visual quality and display size. Please visit http://www.equalizergraphics.com for more information.

Commercial support, custom software development and porting services are available from Eyescale Software GmbH. Please contact info@eyescale.ch for further information.

Image copyright Realtime Technology AG, 2008.

New Poll, Equalizer 0.6 release coming

14. November 2008

I’ve just tagged the release candidate for the upcoming 0.6 release. You can get the release notes here.

Since the blogflux polls are broken for some time and to celebrate the new release, I’ve also created a new poll. Here it is: