Tutorial #8: Create a Web Scraper using ChatGPT and Selenium!

Channel:
Singapore John Tan Chong Min
Subscribers:
5,330
Published on ● Video Link: https://www.youtube.com/watch?v=4xsBJQs2Hao


Category:
Tutorial
Duration: 25:01
218 views
8

I created a web scraper using ChatGPT and Selenium!
To run this notebook for yourself, get an OpenAI API, as well as a Custom JSON API (from Google) and Programmable Search Engine (from Google).
Get your OpenAI API key at https://platform.openai.com/account/api-keys
Get your Custom JSON API key and cx at https://developers.google.com/custom-search/v1/introduction


Jupyter Notebook Link: https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Tutorial/Web_Scraper.ipynb
Excel sheet for scraped Large Language Model courses (as of 18 Aug): https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Tutorial/Large%20Language%20Model%20courses.xlsx

Strict JSON Framework Video: https://www.youtube.com/watch?v=A6sIh-lmApk
Strict JSON Framework Code: https://github.com/tanchongmin/strictjson

~~~~~~~~~~~~~~~~~~~~~~~~

0:00 Introduction
10:06 Main Program #1: Get Websites from Google Search + Secondary URLs
15:31 Man Program #2: Extract Information from Website
19:46 Main Program #3: Sort Websites by Relevance and output it into an Excel file

~~~~~~~~~~~~~~~~~~~~~~~~

AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.

Discord: https://discord.gg/bzp87AHJy5
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin



Other Videos By John Tan Chong Min


2023-10-24ChatDev: Can LLM Agents really replace a software company?
2023-10-17LLMs and Robotics: An Overview by Daniel Tan!
2023-10-17LLM Q&A #1: Prompting vs Fine-Tuning, More vs Fewer Sources for RAG, Prompting vs LLMs as a System
2023-10-10LLMs as a System of Multiple Expert Agents to solve the ARC Challenge (Detailed Walkthrough)
2023-09-26Everything about LLM Agents - Chain of Thought, Reflection, Tool Use, Memory, Multi-Agent Framework
2023-09-19Moving Beyond Probabilities: Memory as World Modelling
2023-09-05Symbolic Regression: Doing What LLMs cannot - Deriving Arbitrary Mathematical Relations!
2023-08-29LLM Agents as a System (Prelim Findings Sharing): An Attempt to solve a 2-player 2D Escape Room!
2023-08-23LLM as Pattern Machines(Part 2) - Goal Directed Decision Transformers, 10-Year Plan for Intelligence
2023-08-18Tutorial #9: Evolution Game v2: ChatGPT (Text) and Dall-E (Image) API Integration!
2023-08-17Tutorial #8: Create a Web Scraper using ChatGPT and Selenium!
2023-08-17Tutorial #7: Create a Chatbot with Gradio and ChatGPT!
2023-08-15LLMs as General Pattern Machines: Use Arbitrary Tokens to Pattern Match?
2023-08-08Tutorial #6: LangChain & StrictJSON Implementation of Knowledge Graph Question Answer with LLMs
2023-08-08Large Language Models and Knowledge Graphs: Merging Flexibility and Structure
2023-07-31Tutorial #5: SymbolicAI - Automatic Retrieval Augmented Generation, Multimodal Inputs, User Packages
2023-07-27How Llama 2 works: Ghost Attention, Quality Supervised Fine-tuning, RLHF for Safety and Helpfulness
2023-07-27Llama 2 vs ChatGPT
2023-07-11I-JEPA: Importance of Predicting in Latent Space
2023-07-09Gen AI Study Group Introductory Tutorial - Transformers, ChatGPT, Prompt Engineering, Projects
2023-07-03Tutorial #5: Strict JSON LLM Framework - Get LLM to output JSON exactly the way you want it!