Google MillWheel

Google MillWheel
Development status Active
Operating system Cross-platform
Website http://research.google.com/pubs/pub41378.html

MillWheel is a framework for building low-latency data-processing applications that is widely used at Google. Users specify a directed computation graph and application code for individual nodes, and the system manages persistent state and the continuous flow of records, all within the envelope of the framework's fault-tolerance guarantees. This paper describes MillWheel's programming model as well as its implementation. The case study of a continuous anomaly detector in use at Google serves to motivate how many of MillWheel's features are used. MillWheel's programming model provides a notion of logical time, making it simple to write time-based aggregations. MillWheel was designed from the outset with fault tolerance and scalability in mind. In practice, we find that MillWheel's unique combination of scalability, fault tolerance, and a versatile programming model lends itself to a wide variety of problems at Google.[1]

References

This article is issued from Wikipedia - version of the 8/14/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.