A couple of MIT electrical engineering and computer science alums have recently published details on how they spent a few dollars of cloud CPU time to re-render 16,544 circuit diagrams in under an hour using VMware’s vFabric RabbitMQ (see VMware’s summary also).
If you’ve done just about any website or online app development in the past decade, you’ve probably created graphics of various sizes to use on various website templates or possibly made graphics available in various formats. Sometimes, changes to the website or app can create a need for every graphic file to be re-rendered, and this is the exact problem CircuitLab.com ran into.
When CircuitLab.com users save and share their circuit designs, the system creates various renderings, and each rendering process takes about 10+ seconds of core CPU time. With over 16,000 circuit images on their site, the team estimated about 44 hours of time to re-render everything. This speed was too slow. So, what did the team do to speed things up? They leveraged the RabbitMQ architecture inside their application to connect with 64 high-speed CPU cores in the cloud.
By adding new queues, they could let the existing code-base run in their production environment while re-rendering production images in the cloud. With a built-in message passing architecture in place, a new function was added with “just a bit of modification” to the existing codebase. With these changes, 16,544 re-rendering messages were processed in under 1 hour (61 total core hours) and for a few dollars of CPU time.