This week I am revisiting a web scraper that I wrote almost a year ago, and improving and cleaning it up. Using requests, beautifulsoup and some user defined functions we deal with pagination and export products to a csv file using pandas. Itertools helps to flatten out a list of lists and is a handy solution to know.


Learning Python with Web Scraping is a good fun and practical way to improve your coding abilities. It was interesting for me to see how much I have improved in the last year going over and rewriting a scraping script.


Support Me:

Amazon US: https://amzn.to/2OzqL1M
Amazon UK: https://amzn.to/2OYuMwo
Hosting: Digital Ocean: https://m.do.co/c/c7c90f161ff6
Gear Used: https://jhnwr.com/gear/ (NEW)
Patreon:   / johnwatsonrooney   (NEW)


Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases