A Deep Dive into Query Execution Engine of Spark SQL - Maryann Xue

Spark SQL enables Spark to perform efficient and fault-tolerant relational query processing with analytics database technologies. The relational queries are compiled to the executable physical plans consisting of transformations and actions on RDDs with the generated Java code. The code is compiled to Java bytecode, executed at runtime by JVM and optimized by JIT to native machine code at runtime. This talk will take a deep dive into Spark SQL execution engine. The talk includes pipelined execution, whole-stage code generation, UDF execution, memory management, vectorized readers, lineage based RDD transformation and action. About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business. Read more here: https://databricks.com/product/unifie... Connect with us: Website: https://databricks.com Facebook:   / databricksinc   Twitter:   / databricks   LinkedIn:   / databricks   Instagram:   / databricksinc   Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-nam...

Join Today
A Deep Dive into Spark SQL's Catalyst Optimizer with Yin Huai
▶︎

A Deep Dive into Spark SQL's Catalyst Optimizer with Yin Huai

Bucketing in Spark SQL 2 3 with Jacek Laskowski
▶︎

Bucketing in Spark SQL 2 3 with Jacek Laskowski

SparkSQL: A Compiler from Queries to RDDs: Spark Summit East talk by Sameer Agarwal
▶︎

SparkSQL: A Compiler from Queries to RDDs: Spark Summit East talk by Sameer Agarwal

A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets - Jules Damji
▶︎

A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets - Jules Damji

Spark Declarative Pipelines (SDP) Explained in Under 20 Minutes
▶︎

Spark Declarative Pipelines (SDP) Explained in Under 20 Minutes

Spark + Parquet In Depth: Spark Summit East talk by: Emily Curtin and Robbie Strickland
▶︎

Spark + Parquet In Depth: Spark Summit East talk by: Emily Curtin and Robbie Strickland

Physical Plans in Spark SQL—continues - David Vrba (Socialbakers)
▶︎

Physical Plans in Spark SQL—continues - David Vrba (Socialbakers)

A Deeper Understanding of Spark Internals - Aaron Davidson (Databricks)
▶︎

A Deeper Understanding of Spark Internals - Aaron Davidson (Databricks)

Spark SQL: A compiler from Queries to RDDS with Sameer Agarwal
▶︎

Spark SQL: A compiler from Queries to RDDS with Sameer Agarwal

Deep Dive: Apache Spark Memory Management
▶︎

Deep Dive: Apache Spark Memory Management

But what is a neural network? | Deep learning chapter 1
▶︎

But what is a neural network? | Deep learning chapter 1

Tuning and Debugging Apache Spark
▶︎

Tuning and Debugging Apache Spark

Intro to Apache Spark for Java and Scala Developers - Ted Malaska (Cloudera)
▶︎

Intro to Apache Spark for Java and Scala Developers - Ted Malaska (Cloudera)

Apache Spark Core – Practical Optimization Daniel Tomes (Databricks)
▶︎

Apache Spark Core – Practical Optimization Daniel Tomes (Databricks)

Deep Dive into Project Tungsten Bringing Spark Closer to Bare Metal -Josh Rosen (Databricks)
▶︎

Deep Dive into Project Tungsten Bringing Spark Closer to Bare Metal -Josh Rosen (Databricks)

Making Apache Spark™ Better with Delta Lake
▶︎

Making Apache Spark™ Better with Delta Lake

From Query Plan to Performance: Supercharging your Apache Spark Queries using the Spark UI SQL Tab
▶︎

From Query Plan to Performance: Supercharging your Apache Spark Queries using the Spark UI SQL Tab

Something is jamming GPS over Europe. Here's what we found
▶︎

Something is jamming GPS over Europe. Here's what we found

SQL Course for Beginners [Full Course]
▶︎

SQL Course for Beginners [Full Course]

A Deep Dive into the Catalyst Optimizer (Herman van Hovell)
▶︎

A Deep Dive into the Catalyst Optimizer (Herman van Hovell)