Erlinda (Parallel computing framework for Erlang)

Project Hompage

http://code.google.com/p/erlinda/

Vision Statement

Provide a high level framework for developing high performance applications that can scale linearly with higher resources.

Abstract

We live in the world where Moore's law and Von Neumann's model of programming will soon be obsolete. As Dr. Dobb's article (chinese version) on "Free Lunch is over" indicated we will have to adjust in this world. Fortunately, parallel programming field has been dealing with this for several decades and there are plenty of solutions available. Unfortunately, most of them are still very research oriented and difficult to use in the real world. I wrote similar system called JavaNOW several years ago as a research project in Java that I will port to Erlang. I plan to leverage concurrency capability of Erlang language and its supported libraries such as OTP and provide a framework for writing high performance applications. Though, Erlang is concurrent language and OTP provides sophisticated support for writing fault tolerant services and provides high availability (with hot code swaps), but it does not inherently support writing parallel applications. Erlinda fill in that gap and provide abstraction to write high performance applications. It is built on top of Erlang and OTP and uses RabbitMQ for messaging middleware.

Introduction

Erlinda provides a framework for writing parallel applications. It provides a number of abstractions for creating parallel applications such as:

  • Tuple Space model based on Linda memory model (JavaSpaces)
  • MPI like APIs for communication (Scatter/Gather)
  • Map/Reduce
  • Master/Slave (Computing Farm)
  • Service oriented (OTP)
  • Clustering
  • Agents and Mobililty
  • Messaging oriented (Queues/Topics)
  • Service/Resource discovery mechanism (similar to JINI)
  • Integration with other languages and middlewares
  • Code Server (similar to JINI/RMI)
  • Security

Reference: JavaNow: A Linda based parallel framework

JavaNow is a framework to develop parallel and high performance applications using Linda based tuple space model, which runs on network of workstations. It is a rewrite of original system I wrote back in '97-99. Though, for client APIs the system behave same as original JavaNow and Linda, i.e., it provides simple APIs like

  • get - removes a matching tuple and blocks if it does not exist.
  • getIfExists - removes a matching tuple in non-blocking fashion.
  • read - returns a copy of matching tuple and blocks if it does not exist.
  • readIfExists - returns a copy of a matching tuple in non-blocking fashion.
  • put - stores a tuple to the tuplespace.
  • eval - executes a thread that turns into a tuple, which is added to the tuple space.
  • subscribe - registers a listener that is notified when a tuple is added or removed.
  • size - returns size of tuplespace

In addition to basic Linda operations it provides following bulk and data flow operations:

  • getAllIfExists - removes all tuples in non-blocking fashion.
  • readAllIfExists - returns copy of all tuples in non-blocking fashion.
  • clear - removes all tuples.
  • putDelayed - It takes two tuple spaces and performs blocking get operation on first tuple and stores it to the second tuple space.
  • evalDelayed - It takes two tuple spaces as arguments and performs blocking get operation on first tuple space and then evaluates a thread passing that tuple and stores it in second tuple space.

Master/Slave

JavaNow provides Master/Slave framework that can be used to implement parallel applications similar to map/reduce, where master distributes the work to workers (map phase) and then collects the results (reduce phase).

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License