Secret sauce that brings YouTube followers, views, likes
Get Free YouTube Subscribers, Views and Likes

Bulk Inserts with PostgreSQL: Four+ Methods for Efficient Data Loading

Follow
Ryan Booz

This talk was recorded at the FOSDEM 2023 PGDay in Brussels. Please see the original recording along with other great talks from the day at https://fosdem.org/2023/schedule/trac...

Talk Description:
In a world where data continues to grow at an overwhelming rate, knowing how to load and insert data into PostgreSQL as efficiently as possible is a key skill for any software or data engineer. Whether loading data from files, manually generating multivalued INSERT or UPSERT statements, or using some of the more popular language SDKs, knowing which option is right for each situation can sometimes be difficult to figure out.

This talk will review how to efficiently insert large amounts of data into PostgreSQL using freely available tools and wellcrafted SQL. We'll demonstrate the usage of tools like COPY, how to batch data using multivalued INSERT statements, functions to look for in language SDKs, and even demonstrate how to use array parameters to speed up multivalued inserts.

By the end of this session, you will know at least four ways to insert bulk data, the impact it will have on your PostgreSQL database, and options for improving your current application code to perform at its best!

posted by shistatszj