How To Build a Web Scraping API for Large-Scale Data Collection Using FastAPI

Channel:

Akamai Developer

Subscribers:

1,040,000

Published on October 16, 2024 9:30:52 PM ● Video Link: https://www.youtube.com/watch?v=_rLm9O2nVJU

Duration: 0:00

4,997 views

191

New to Cloud Computing? Get started here with a $100 credit → https://www.linode.com/lp/youtube-viewers/?utm_source=youtube

‪@TechWithTim‬ shows viewers how to build a Scalable Web scraping API for large scale data collection using FastAPI and Python. He walks through all the deployment steps and how to make it take advantage of multiple CPU cores.

Chapters:
0:00 - Introduction
1:06 - Setup Server
2:20 - Create a new Linode
4:40 - Configure Linode for deployment
10:15 - Write API code
19:30 - Use FastAPI
24:44 - Create roots & end points
32:00 - Build Web scraping code
38:15 - Write Web scraping logic
39:40 - Test the API locally
41:42 - Test with Uvicorn
43:00 - Move the code to the Linode
46:24 - Create a virtual environment using Python
47:45 - Make API publicly available
54:51 - Configure FastAPI to run automatically
57:43 - View the general & error logs
58:19 - Test the API using Postman
1:00:05 - Conclusion

Check out Tim's code on GitHub → https://github.com/techwithtim/Akami-Scaleable-Web-Scraping-API
Learn more about Fast API → https://fastapi.tiangolo.com/
Learn more about Python → https://www.python.org/about/
Subscribe to get notified of new episodes as they come out →    / @akamaideveloper  


#AkamaiDeveloper #Webscraping #datacollection #FastAPI #Python
Product: Akamai, FastAPI, Python; ‪@TechWithTim‬