In this video ‪@mehdio‬ dives into the new experimental feature of DuckDB : running PySpark code but with DuckDB engine ⚡

Note : This is not yet supported on MotherDuck

Resources
* Github Repo of the tutorial : https://github.com/mehdio/duckdbpys...
* Niels Claes's benchmark on SQL engines :   / headtoheadcomparisonofdbtsqlengines  

➡ Follow Us
LinkedIn:   / motherduck  
X (formerly known as Twitter) :   / motherduck  
Blog: https://motherduck.com/blog/

0:00 Intro
0:53 Challenges of Apache Spark development
3:24 The Java boat load
6:01 Pyspark with DuckDB demo
8:10 A word about benchmarks
8:50 Limitations
9:26 Conclusions

#duckdb #pyspark #apachespark #dataengineering