Scottish Programming Languages Seminar

Location

Acknowledgement

Programme

Functional programming and stream-processing have shared history -- from early work on dataflow architectures, VAL, and SISAL, to Haskell's use of stream-based IO (before monads) or the modern-day resurgence of Haskell stream libraries (iteratees, pipes, conduit). These days, "streaming" can mean a lot of things; StreamIt, based on synchronous-dataflow, has totally ordered streams and will not duplicate stateful stream processors, whereas Apache Storm makes the opposite decisions. The first part of this talk will overview this broad landscape.
We argue that the degree of dynamism (e.g. in data-rates and stream topologies) is the major axis along which various stream technologies are differentiated. In the second part of this talk, we describe our past and ongoing work on navigating this spectrum, by developing technologies that leverage regularities where they occur, but tolerate dynamism. We have studied profile-driven program partitioning, and other compilation topics, and our current thrust for developing stream DSLs overlaps heavily with work on data-parallel DSLs (e.g. Accelerate).

In the current era of "Big Data", distributed complex event processing is the prevailing approach to analyzing streams of raw data in real time. The sheer volume of raw data events demands that aggregation operations take place as close as possible to the sources of raw data. Many such sources are constrained, wireless sensor nodes (such as constitute the Internet of Things); increasingly, smart phones are also providing socially-related streams of raw data. Traditional complex event processing technologies, based upon stream database management systems or publish/subscribe systems, are adept at handling many demands for complex event processing. A growing number of hybrid complex event detection scenarios require features of both technologies. Previous work in my group [1] has focused on a unification of publish/subscribe and stream database concepts to tackle all complex event processing scenarios, with particular emphasis upon hybrid scenarios. Fundamental to this unification is an automaton programming language for specifying aggregation operations and an asymmetric publish/subscribe broker architecture for aggregating at different levels of abstraction. This architecture enables runtime composition of ensembles of automata, an essential feature for an open, distributed complex event processing platform. It also enables the placement of aggregation operations into the infrastructure.
In this talk, I will describe the automaton programming language, GAPL, its runtime, give some examples, and provide information about the performance and scalability of the system.

[1] J Sventek, A Koliousis, "Unification of publish/subscribe systems and stream databases: the impact on complex event processing", Proceedings of the 13th International Middleware Conference, 292-311.

We have just finished the first provisional release of a standard we have been working on with a group of semiconductor companies and software developers. It's called SYCL and it enables C++ metaprogramming on a wide range of heterogeneous processors supported by OpenCL. This talk will cover some of the problems standardizing a programming model on a range of heterogeneous devices and operating systems and how as a group we overcame those problems with SYCL. We came up with some interesting new solutions to the challenges posed by heterogeneous systems, such as:

• How do you provide a standard way for different compilers for very different devices to interoperate and compile the same integrated source file?
• How do you expose the multiple levels of parallelism that exist in OpenCL's execution model in an easy-to-use way?
• How do you handle the differences in data access between the "host" CPU and different devices in the system?
• How does the type system handle the difference in pointer types between host and device?
• We also had a lot of discussions about asynchronous error-handling, which is a hard problem in C++ right now.

I'll show how (a type theoretic generalisation of) multi-sorted signatures and their terms algebras can be used to write client and server programs that interact according to some pre-specified protocol. The programs written in this style will be correct-by-construction with regard to the protocol, similar to programs written in a language with session types. It's not clear to me how far this approach can be pushed, in particular it's not obvious how to handle concurrency nor recursive protocols -- if there's time, I shall try to touch upon those topics.

There has been significant interest in recent months in finding new ways to implement composable and modular effectful programs using handlers of algebraic effects. In previous work, I have shown how an algebraic effect system can be embedded directly in a dependently typed host language, Idris. In this talk I will show how, additionally, parameterising algebraic effects over a resource state allows us to reason about run-time state transitions, where a transition may depend on run-time information about resource usage.
Using this, I will show how we can describe communication protocols for concurrent and distributed systems, and build implementations of those protocols which are verified by the Idris type checker.

DSLs are supposed to capture abstractions appropriate for well characterised classes of problems. But if a DSL is Turing complete (TC), or embedded in a TC language, or its domain maps to other domains, then is it really domain specific? Perhaps domain specific-ness is a pragmatic rather than a semantic property?
This is an extended version of the presentation given at the Real World DSL Workshop on 1/5/14.