Archive for the ‘OS X’ Category

Equalizer 1.4 released

7. September 2012


The last two weeks have been quiet, since I was on biking through Switzerland. Meanwhile, poor Daniel back at work churned through most of the Collage changes outlined in the last post. You can see the changes in the Collage endian branch on github, which will be merged back into master in the next couple of weeks.

Now back to the news: After finally figuring out how to build Equalizer and dependencies using MacPorts portfiles on Mac OS X, I released the long-standing 1.4 version of Equalizer, GPU-SD, Lunchbox and vmmlib. Below is the release announcement – enjoy!

Neuchatel, Switzerland – September 7, 2012 – Eyescale is pleased to announce the release of Equalizer 1.4.

Equalizer is the standard framework to create and deploy parallel, scalable 3D applications. This modular release includes Collage 0.6, a cross-platform C++ library for building heterogenous, distributed applications, GPU-SD 1.4, a C++ library and daemon for the discovery and announcement of graphics processing units using zeroconf networking and Lunchbox 1.4, a C++ library for multi-threaded programming. All software packages are available for free for commercial and non-commercial use under the LGPL open source license.

Equalizer 1.4 is a feature release extending the 1.0 API, introducing major new features, most notably asynchronous readbacks, region of interest and thread affinity for increased performance during scalable rendering. It culminates over seven years of development and decades of experience into a feature-rich, high-performance and mature parallel rendering framework and related high-performance C++ libraries.

Equalizer enables software developers to easily build interactive and scalable visualization applications, which optimally combine multiple graphics cards, processors and computers to scale the rendering performance, visual quality and display size.

Equalizer Applications

Eyescale provides software consulting and development services for parallel 3D visualization software and GPU computing applications, based on the Eyescale software products or other open and closed source solutions.

Please check the release notes on the Equalizer website for a comprehensive list of new features, enhancements, optimizations and bug fixes. A paperback book of the Programming and User Guide is available.

We would like to thank all individuals and parties who have contributed to the development of Equalizer 1.4.

Left image courtesy of Cajal Blue Brain/ / Blue Brain Project. Second from left copyright Realtime Technology AG, 2008. Right image courtesy University of Siegen, 2008.

Advertisements

Introducing Collage

6. July 2012

Collage evolved from the Equalizer network library. Now it’s used by a few other projects as well, and I’ve been mentioning it quite a few times on this blog already.

Looking back over the last seven years of Equalizer, Collage has received by far the largest investment in manpower compared to all the other components. It seems innocent enough, but trust me, getting a distributed network library right and bug free is no small task. On the one hand, the ‘fun’ implementation of the Windows IP stack (WSASYSCALLFAILURE and stack corruptions) took a lot of debugging to get going, and on the other hand advanced features such as InfiniBand support (thanks Dardo!) and fast, reliable multicast are no small task to implement.

What’s wrong with boost::asio?

Nothing. It provides about the same functionality as co::Connection and co::ConnectionSet. We use it as the UDP backend for the RSP multicast implementation. It wasn’t around when I started eq::net, which became Collage. We’ld love to replace co::Connection and co::ConnectionSet with it, but the effort of porting the RDMA and RSP connections to asio has prevented this so far. Ultimately Collage tries to provide higher level abstractions.

What’s wrong with 0MQ?

Again: Nothing. It provides higher level abstractions then asio. It’s less likely that we’ll use it as the backend for Collage, since some of the design decisions are somewhat different from what Collage is doing.

So, what’s next?

Collage, similar to Lunchbox, aims to provide high-level abstractions. Similarly to the introducing Lunchbox series, I’ll present them over the next few weeks. In the same timeframe, we are planning to separate the Equalizer and Collage projects as well as to define and document the ‘1.0’ Collage API.

For now, you can find a technical overview presentation on the Collage website. Enjoy!

Programming and User Guide for Equalizer 1.4

29. June 2012

Equalizer Programming and User Guide

Equalizer Programming and User Guide


I’ve just uploaded the review version of the Programming and User Guide for the upcoming 1.4 release of Equalizer, and to a certain extent, Collage.

This one packs 111 pages of content (118 total) and 63 figures. In a couple of weeks I’ll create the final hardcopy version on Amazon/CreateSpace.

What’s New?

This edition has, among the customary full review pass, a lot of new content. Starting with a full new chapter on Sequel and the associated polygonal rendering example, continuing with new section for application-specific scaling factors in immersive environments, region of interest for compositing and load-balancing and zeroconf discovery in Collage, finally finishing with a substantial rewrite of the section on distributed objects in Collage.

What?

The book is structured in two parts: The User Guide, laying the foundation on parallel rendering and scalability algorithms, and then explaining the configuration of visualization systems for Equalizer applications. The appendix contains a full reference on the file format.

The second part, the Programming Guide, gradually introduces programming parallel rendering applications. Starting with the basics in eqHello, the complexity is gradually increased with a chapter on Sequel and Equalizer using the respective example application. After this, an advanced features section focuses on introducing and demonstrating on specific features in isolation. It finishes of with a chapter on the Collage network library.

Why?

The Programming and User Guide is the ‘OpenGL red book’ of Equalizer. It consolidates all the documentation available in various places (Equalizer website, mailing list, github feature issues, my head) into a single document. Apart from gathering this information, through the format emerges a bigger picture, putting design decisions in context.

Lunchbox Folly

8. June 2012

Facebook recently released folly, which caught my eye due to its similarity to Lunchbox. It’s definitely a library to watch.

I don’t think we’ll be using it right now, since it’s missing some of the stuff we’re using, e.g, the LFVector, and since the implementation so far seems to be mainly tested on Linux + gcc 4.6. On the other hand, it has some interesting components which we might need in the future, e.g, atomic hash containers. It also contains optimized version of some standard components such as vector and string, which haven’t really shown up as hotspot during profiling our code.

Interesting enough, folly also has some components which are almost identical to the Lunchbox counterparts, such as r/w spin locks and lock-free queues. It’s always good to see when ideas converge to a common design.

Lock Performance

5. July 2011

I’m currently working on a low-level library where locked data access has to be optimized. Therefore I benchmarked the performance of the three lock types in Collage on Linux and Mac OS X. The test just runs a number of threads which just set and unset the lock without any other operation. Click on the image below to get a full-resolution image. Be aware the chart uses double-log scale.

The two benchmarks can not be directly compared since they did not run on the same hardware. There are nevertheless a few interesting observations:

(1) Spinlocks are faster than ‘real’ locks. I’ve blogged about this before. Since they consume CPU time while spinning they should only be hold for a very short time, i.e., to read a value. The Collage implement immediately backs off when encountering a set lock by yielding the thread. This avoids priority inversion, which can be observed by some pthread spin lock implementations.

(2) pthread locks are dead slow on Mac OS X. Be aware that the graph uses log scale – a spin lock is up to three orders of magnitude faster than a pthread lock!

(3) Timed locks are slower than un-timed. This meets my intuitive expectation, since the timed implementation is more complex. The timed lock in Collage is implemented using pthread_cond_timedwait.

(4) The Spinlock is faster on OS X on slower hardware than on Linux. Not sure why that is the case. The Collage spin lock uses an atomic variable and compare_and_set. Either these operations are faster on the Core i5, or the thread yield behaves ‘better’ on OS X.

(5) Single-threaded lock access in pthread libraries seems to be optimized.

(6) pthread conditions on Linux observe a steep performance drop once you have more threads than cores. Could be a scheduling issue again.

Next I’ll work on benchmarking and optimizing read/write locking in the Collage Spinlock. Stay tuned for updates!

EDIT: I discovered a bug in my micro-benchmark which wrongly multiplied the results with the number of threads – doh! The figure is fixed now with a new test run.

Equalizer 0.9 Released!

11. August 2009

Cross-Segment Load-Balancing
We are pleased to announce the release of Equalizer 0.9, the standard framework to create and deploy parallel, scalable OpenGL applications. The most notable new features in this release are:

Please check the release notes on the Equalizer website for a comprehensive list of new features, enhancements, optimizations and bug fixes. A paperback book of the Programming and User Guide is available from Lulu.com.

We would like to thank all individuals and parties who have contributed to the development of Equalizer 0.9.

Equalizer 0.6 released!

3. December 2008

Here is the release announcement:

Scalable Display System

We are pleased to announce the release of Equalizer 0.6, a major advance in parallel OpenGL rendering, supporting:

  • Automatic 2D and DB load-balancing
  • DPlex (time-multiplex) compounds
  • Paracomp compositing backend

Please check the release notes on the Equalizer website for a comprehensive list of new features, enhancements, optimizations and bug fixes. The Programming Guide can be ordered at lulu.com.

We would like to thank all individuals and parties who have contributed to the development of Equalizer 0.6.

About RTT Scale

RTT Scale is the new scalability module for RTT’s leading high-end visualisation software suite RTT DeltaGen. The module is to be integrated into the forthcoming Version 8.5. RTT Scale uses Equalizer 0.6 to boost OpenGL and ray-tracing performance on highly realistic, complex scenes. Please visit http://www.rtt.ag for more information.

About Equalizer

Equalizer is the standard middleware to create parallel OpenGL-based applications. It enables applications to benefit from multiple graphics cards, processors and computers to scale rendering performance, visual quality and display size. Please visit http://www.equalizergraphics.com for more information.

Commercial support, custom software development and porting services are available from Eyescale Software GmbH. Please contact info@eyescale.ch for further information.

Image copyright Realtime Technology AG, 2008.

Streaming compositing for database decomposition

14. November 2008

I’ve published a new parallel rendering video showing streaming compositing for DB decomposition. The algorithm is pretty simple, but keeps the compositing workload and network load evenly distributed by streaming the results through all rendering nodes:

Enjoy!

New Poll, Equalizer 0.6 release coming

14. November 2008

I’ve just tagged the release candidate for the upcoming 0.6 release. You can get the release notes here.

Since the blogflux polls are broken for some time and to celebrate the new release, I’ve also created a new poll. Here it is:

DPlex support, including parallel rendering video

27. October 2008

The Equalizer source tree now contains support for DPlex decompositions. This was the last scalable rendering feature missing, which means that the client library is now 1.0-ready. Yeah!

But fear not, we’ve got plenty in store for the following Equalizer releases leading to the 1.0. And even after version 1.0 we have plenty of features and optimizations in store to push the boundaries of scalable rendering performance.

I’ll now test and fix some corner cases of DPlex, like multi-level decompositions, before the 0.6 release. For normal DPlex mode, the feature is already working well, see for yourself: