In this talk, Tom Wilkie looks at Castle, an open-source project, is a ground-up overhauling of RAID, file systems, and the POSIX interface
The standard Linux storage stack wasn’t designed for write-heavy big data workloads, nor is it well-suited to modern hardware: large, slow SATA disks, SSDs or many cores. Castle, an open-source project, is a ground-up overhauling of RAID, file systems, and the POSIX interface. It is released under the GPL and runs as part of the Linux kernel. Our target is 1 million random inserts per second to disk on a $1,000 commodity box, and we’re nearly there. Castle is also the core of the Acunu Data Platform, which delivers up to 100x higher performance for applications written for Cassandra and other tools.
YOU MAY ALSO LIKE:
- Monitoring Kubernetes With Prometheus (SkillsCast recorded in September 2018)
- Fast Track to Machine Learning with Louis Dorard (in London on 15th - 17th July 2019)
- Real-time Systems with Spark Streaming and Kafka (in London on 23rd - 24th September 2019)
- Infiniteconf 2019 - A one-day community celebration of Big Data, Machine Learning and AI (in London on 4th July 2019)
- CloudNative London 2019 (in London on 25th - 27th September 2019)
- Causal inference and the data-fusion problem (in London on 18th June 2019)
- The wonders of IBM: Watson, artificial intelligence, Swift coding and databases! (in London on 19th June 2019)
- AI Auditing Framework (SkillsCast recorded in June 2019)
- Applying Monte Carlo Tree Search (MCTS) to the Protein Folding (SkillsCast recorded in June 2019)
Castle: Re-inventing storage for Big Data
Tom is one of the Prometheus Developers and VP Product at Grafana Labs. Previously he founded Kausal, a new company working on Prometheus & Cortex. and has worked at companies such as Weaveworks, Google, Acunu and XenSource. In his spare time, Tom likes to make craft beer and build 3D printers.