Erlang and Cloud Computing: A Fine Pair Indeed
[update]: Some clarification (in it’s favor!) on Mnesia’s limitations thanks to commenter Gleb Peregud
“The Cloud”. Infrastructure as a resource. Whether or not you’ve bought into the hype (or HiPE as you prefer :D ) some of the largest software/IT companies in the world are throwing piles of cash at this idea of a hardware-service. But, for the purposes of new web applications that are looking to take advantage of hardware scaling to meet demand, there’s a lot of work to be done. For a quick roundup on some of the issues facing web applications with the EC2 platform you can check out Tony Arcieri’s Post on Rails with EC2. Just as Tony Points out in his article, Erlang has a lot of tools ready made for these demands, and, with the added side benefit of proven stability/scalability in intense environments, it’s certainly worthy of some consideration for your next cloud ready app.
A Language (and VM) Built to Scale
Erlang has been around the block a few times ( short history ), and it’s been used in some situations with incredible requirements and results. From it’s home at Ericsson to other telecom applications with the likes of Nortel, T-Mobile, and Motorola, where it has achieved 5 nines of availability in some instances, Erlang has been proven as a reliable platform on which to build applications.
Platform is the key word there, because it’s not just the language and its syntax that make it great for it’s appointed task but also the VM that it runs on. Erlang’s VM comes with some really great features that also make concurrent programming a lot easier.
- “Green Threads” – There’s some disagreement over whether the name is correctly applied in Erlang’s case, but the benefit is clear: Cheap process creation. In comparison system threads are “heavy”, and if a programmer is working with them on a intensive concurrent application they are forced to worry about going thread crazy because of performance degredation. While Erlang doesn’t completely fix this issue, it does the system thread managment for you, allowing for more attention to be paid to building the application as it makes sense for the problem set. If you need a thousand processes, then use a thousand.
- Message Passing Primitive’s – Erlang has been designed to pass messages between processes. It has built in syntax constructs specifically for this purpose, and while there are many languages that have something similar, it’s a great indicator of what the language was meant for. Here’s a very simple example: receive_msg () ->receive{ message , Text } ->io : format ( "Someone told me this: ~s~n " , Text ),forward ( Pid )end .
As long as you know the Erlang process id of a given bit of running code you can dial it up and tell it what to do.
- Hot Code Swapping – This is a big one here. This is where the 5 – 9 nines up-time comes from because, as long as your build your applications with a little forethought, you don’t have to bring them down for the purposes of providing bug fixes or features. Before jumping at the “with a little forethought”, take a read here and see how easy it really is. And it is easy, but even if it was hard when was the last time you worked on an application that didn’t need to be restarted when new code was added?
Libraries Ready Made for Distributing Load
Erlang comes with a whole host of libraries and modules built in support of its primary goal as a massively concurrent programming language. These libraries are important enough that Erlang is often referred to in conjunction with them as “Erlang/OTP”.
- Mnesia – “is a distributed DataBase Management System (DBMS), appropriate for telecommunications applications and other Erlang applications which require continuous operation and exhibit soft real-time properties.” (ref) I’m sure many of you are saying to yourselves, “I can distribute load with MySQL so what’s the advantage?”. The advantage comes from being able to add a table copy to a new node as easy as: mnesia : create_schema ([ somenode ]),mnesia : add_table_copy ( Tab , somenode , Type )
That’s pretty amazing when you consider what it means for easy data distribution.
Mnesia is not without its drawbacks though. Most importantly it wasn’t designed to store enormous tables like SQL databases. In fact it has a 2gb table size limit for disc_only_copy configuration of tables, but the in memory and memory and disc table copies are only limited by ram size. Also, with more than 8-10 nodes the amount of network communication may begin to inflict performance degredation. (Big thanks to comment poster Gleb Peregud, for both of those tidbits).
- gen_server – gen_server is a module that provides an interface that allows your code to take advantage of much of the OTP goodness like process supervision and logging without lifting a finger. Supervision itself is enough to get excited about, as it allows safe and reliable process topologies to be created with ease. Processes monitoring and restarting other processes sounds like a recipe for reliability when implemented correctly.
That’s only two notable examples from an enormouse set of libraries all targeted at reliable distributed infrastructure. Most of which have been proven in the harsh environments mentioned earlier.
Alternatives to Erlang
This is a big enough market that there are bound to be a lot of solutions out there for the problem of easy node addition to applications, and it only takes one look at the comments in one of Yariv’s posts ( Erlang vs Scala ) to see there are a lot of opinions.
At this point I have to confess I can’t really comment with full knowledge on how something like Scala stacks up to Erlang in this specific scenario. What I do know is that Scala runs on the JVM, which is great from a stability perspective but may not be as stellar for massive concurrency because of the way JVM handles threading with real system threads. Much the same, the CLR from Microsoft uses real system threads, but has the ability to use something akin to green threads (reference?). Clearly using either of these has some great advantages, like access to Java libraries (though many argue this might make concurrency more difficult), so it’s not as cut and dry as Erlang > All.
Clouds on the Brain
However you choose to tackle it, the cloud is growing in popularity and importance when considering web applications. Looking to the future, even the most basic hosting plans may consist of on demand hardware for our disposal, and then it will really be up to you how you want to utilize the resources you have to better serve out your apps.
Let me know what you think in the comments!