Google I/O 2013 - Behind the Data Sensing Lab: Gathering, Processing, and Analyzing Data at Scale

Amy Unruh, Kim Cameron Highly scalable and rapid data collection and analysis is a key need for many mobile and gaming apps, as well as for sensor networks and the ""Internet of Things."" We'll show how the Data Sensing Lab incorporates a key Google Cloud Platform pattern: a high-throughput pipeline for data collection, processing, and analysis. We use the Cloud Endpoints API to collect constantly streaming data; process large amounts of data with high throughput using App Engine, Cloud Storage, and data transformation on Compute Engine; and query many GBs of collected data in just a few seconds using BigQuery. For all I/O 2013 sessions, go to https://developers.google.com/live