Improving Scalability of Distributed Computing Environments

Richard Croucher, Barclays Bank

Scaling distributed computing environments to extend from 100's to 1000's of nodes is a challenge for programming environments such as Erlang/OTP.   Traditional large scale compute environments use MPI or offer simple Grid computing using Distributed Resource Managers.   Functional languages such as Erlang provide  greater flexibility and programmer productivity than MPI or Grid based programs but don't scale as well.  This session will look at the scaling constraints of ErlangOTP and discuss opportunities for utilising some of the techniques used by MPI, such as RDMA and Multicast to improve its scalability.


Bio: Richard is a platform architect who specializes in high performance systems, including those used by financial institutions for high frequency trading and huge compute clusters with thousands of nodes used in the Cloud. Richard discovered Erlang and OTP three years ago and has adopted this as his platform of choice.   He has designed and helped build a large and complex Cloud based application using Erlang/OTP and has been exploring the design challenges of getting this to scale to millions of users for the last 12 months.

Richard is a multi-disciplinarian with experience across hardware, storage, network, operating systems, DevOps and systems programming.  He has to use all of these when designing a new platform. Over the years, he's programmed in Assembler, Basic, Fortran, Pascal,  C, C++, Java, C# and Erlang.  He has designed and built his own computers working at the chip level and designing his own circuit boards. Richard is currently VP of High Frequency Engineering at Barclays and has had previous roles as  Chief Architect at Sun Microsystems, Principle Architect at Microsoft (Azure).

Richard is a Fellow of STAC Research, a Fellow of the British Computer Society and a Chartered IT Practitioner and holds degrees from Brunel University, the University of Berkshire and the University of East London.