Simulating MapReduce in Mathematica

MapReduce is one of the most important algorithm types in big data. Perhaps its biggest draw is the fact that it can be massively distributed across commodity hardware to crunch massive amounts of data. This puts super computing in the hands of those without specialized parallel hardware. In a very basic example for a Hadoop class I did, I implemented a basic MapReduce simulator in Mathematica in order to illustrate how MapReduce works. I also got some extra credit for it, as well.

A Practical Guide to pthread And Pretty Fractals

Threads. Many programs need them, and they're important on multiprocessor systems (which, recently, is basically every system). Sometimes, they can be confusing, so I've taken the most most important bits and I've distilled them down into the basics of how to do threads in a basic C program using POSIX Threads (pthreads, for short). Let's get started!

Render Farm - 3D Edition!

A few months ago I wrote a simple render farm in PHP using imagemagick. But it could only deal with very simple animations in 2D space. What if we could deal with complex scenes in 3D space? And, what if instead of using hacky HTTP transactions to move data back and forth, we used Socket.io to form a true protocol? Well, that's exactly what we're going to do here.

Easy Parallel Computing with Old Android Phones

I have some relatively old Android phones lying around (including the Nexus 4 with a broken screen I rooted) and I wanted to cluster them together in some way to see what they were capable of. But the problem was that none of them had the same platform. They all had different Android flavors and different permissions. So I made a few crucial design decisions up front to make this method of clustering as compatible with everything as possible. Here's the outline of how this is going to work.

Building a Basic Render Farm

Parallel computing is really neat. Obviously, it's important in science where you can break computational tasks into smaller parts and distribute the work among many discrete systems. But for rendering, it's incredibly important. Take Pixar, for example. They have a massive rendering farm and yet it sometimes takes them more than a day to render a single frame because of the complexity of the scene (hundreds of lights, shadows, etc.) This project clearly won't reach those levels, it's just to show that it can be done fairly easily to tween images across another using ImageMagick on multiple computers.