Schedule

Nearly thirty years have passed since the release of the infamous Dragon Book by Aho, Lam, Sethi, and Ullman. Since then the available memory and CPU power has grown by a factor of at least 1000 and 10000 respectively. The use of multiprocessors and fast networks has become the rule rather than the exception. Even though compiler and programming language have advanced greatly in this period of time, their basic design has not been adjusted to reflect the enormous hardware improvements.

In this presentation, which reflects the content of my master thesis, I present a D compiler written in D that exploits these tremendous hardware advances. To utilize multiprocessors the lexer and parser are run in different communicating threads. On top of that different semantic analysis are spread across multiple threads. To speedup compilation, by use of a cache and work distribution among a network, datastructures are created that store results, like the abstract-syntax- tree, in a consecutive chunk of memory. To remove manual labor a lexer- and parser-generator were created that use a custom library that, not only, consists of stl inspired containers.

Besides the conceptual architecture of this new compiler, I will report on the implementation, the use of datastructures and the lessons learned during this project with respect to compiler design as well as the D Programming Language.

Modern large-scale software design often requires ever-growing team sizes. As team size scales up, so does the amount of churn (and number of bugs introduced) to a code base in a release. Unfortunately, the amount of manual developer or QA testing necessary to find those bugs scales quadratically (or worse) as N features are tested by hand in M configurations. The solution is to keep coders, not QA, responsible for maintaining the quality of their individual components, and to make it trivial and fast to run just the right tests that cover the code the coder changes. D comes pre-baked with lots of tools to make this easier, including a standard unit testing harness, but there'smuch more we can build on top of that to help write not only tests for code, but code that can be tested in an automated fashion.

We'll discuss techniques and tools for designing large-scale testable code in D, including unit testing, mocks, fakes, inversion of control, and dependency injection.

The Concurrent D Garbage Collector (CDGC) was developed mainly to address this issue, by making the mark phase run in parallel with the actual application, reducing the pause times by 2 orders of magnitude. Since our systems at sociomantic labs GmbH require to have very low latency at all times, we've been using this collector for almost 2 years now with good results.

This presentation gives a brief introduction to garbage collection, describes original D's GC implementation details and explains how it has been extended to do perform the mark phase in parallel. After showing some benchmark results, shortcomings and future work and improvements are discussed.