Wdgymyenyiu7kspvelmy
SkillsCast

Predicting congestion on London’s roads with Beam and Tensorflow - Intermediate

6th July 2017 in London at CodeNode

There are 42 other SkillsCasts available from Infiniteconf 2017 - the conference on Big Data and Fast Data

Please log in to watch this conference skillscast.

Https s3.amazonaws.com prod.tracker2 resource 41088130 skillsmatter conference skillscast o9nohu

The talk will be about a project Oliver and the Datatonic team did with TfL, where they used Apache Beam and Tensorflow to predict congestion. The talk will focus in detail on how and why these technologies were used in the use case at hand. n-episode-ii-predicting.html

redicting congestion is an important part of any traffic management system. With accurate forecasting traffic can be effectively regulated, ensuring safe and fast journeys on the roads. Join Oliver as he shares a deep learning model which accurately forecasts congestion based on road sensor data from Transport for London (TfL).

The IoT (internet of things) nature and scale of the raw sensor data (5 TB, 120 billion rows) demands extensive preprocessing as a first step towards a predictive traffic model. Oliver and his team used Apache Beam for this task as it let them create efficient data pipelines which can be executed in distributed frameworks such as Apache Spark or Apache Flink. Beam natively handles streaming workloads which makes it an ideal candidate for large scale preprocessing of real-time data such as streams from road sensors.

The preprocessed data was then used to train a neural network to predict the congestion ahead of time. A recurrent neural network (RNN) was chosen to model the traffic time-series for each of the sensors. This deep learning architecture was implemented in Tensorflow and it allowed us to accurately model the time-series including the correlations between the sensors on the road network. With this end-to-end example Oliver will demonstrate how Beam and Tensorflow can be used to build predictive models for time series data.

More information on the project (including some results and images) can be found here and here.

YOU MAY ALSO LIKE:

Thanks to our sponsors

Predicting congestion on London’s roads with Beam and Tensorflow - Intermediate

Oliver Gindele

Oliver Gindele is the head of Machine Learning at Datatonic.

SkillsCast

Please log in to watch this conference skillscast.

Https s3.amazonaws.com prod.tracker2 resource 41088130 skillsmatter conference skillscast o9nohu

The talk will be about a project Oliver and the Datatonic team did with TfL, where they used Apache Beam and Tensorflow to predict congestion. The talk will focus in detail on how and why these technologies were used in the use case at hand. n-episode-ii-predicting.html

redicting congestion is an important part of any traffic management system. With accurate forecasting traffic can be effectively regulated, ensuring safe and fast journeys on the roads. Join Oliver as he shares a deep learning model which accurately forecasts congestion based on road sensor data from Transport for London (TfL).

The IoT (internet of things) nature and scale of the raw sensor data (5 TB, 120 billion rows) demands extensive preprocessing as a first step towards a predictive traffic model. Oliver and his team used Apache Beam for this task as it let them create efficient data pipelines which can be executed in distributed frameworks such as Apache Spark or Apache Flink. Beam natively handles streaming workloads which makes it an ideal candidate for large scale preprocessing of real-time data such as streams from road sensors.

The preprocessed data was then used to train a neural network to predict the congestion ahead of time. A recurrent neural network (RNN) was chosen to model the traffic time-series for each of the sensors. This deep learning architecture was implemented in Tensorflow and it allowed us to accurately model the time-series including the correlations between the sensors on the road network. With this end-to-end example Oliver will demonstrate how Beam and Tensorflow can be used to build predictive models for time series data.

More information on the project (including some results and images) can be found here and here.

YOU MAY ALSO LIKE:

Thanks to our sponsors

About the Speaker

Predicting congestion on London’s roads with Beam and Tensorflow - Intermediate

Oliver Gindele

Oliver Gindele is the head of Machine Learning at Datatonic.