From Farm to Cloud:Building a massive Realtime Streaming Ingest Wieneke Keller&Sebastian Lenartowicz

Summary Aurea Imaging is an AgTech scaleup focusing on precision farming in orchards. By retrofitting our TreeScout sensor package to their tractors, farmers are able to collect data about their orchard down to the tree level as they perform other tasks. Using onboard stereo cameras, the tractor’s high-precision GPS, a machine-vision pipeline running in real time on the device, and near-real-time cloud-based analytics, this data is turned into maps used by other agricultural machines that enable the grower to utilize less labour and chemical products to produce more food. As the device fleet scales, one of our greatest challenges has been real-time data ingest - a TreeScout operating normally generates thousands of data points per second, and the highly seasonal nature of agriculture means that large chunks of the fleet are online and sending at the same time. Traditional data-science and GIS-analytics approaches break down in this kind of environment, so join us as we walk you through the architecture and challenges of engineering a streaming data system capable of tackling these challenges