Design Web Crawler (Part-VI) : Google, Amazon, Meta : STEP-3 : Design Deep Dive Complete & WRAP-UP

Channel:

codestorywithMIK

Subscribers:

92,300

Published on December 3, 2022 4:17:24 AM ● Video Link: https://www.youtube.com/watch?v=hoD3uw5RX-U

Duration: 32:04

193 views

20

This is the 6th video on designing our "WEB CRAWLER'' (PART-VI).
We will complete our Step-3 (Design Deep Dive) and Step-4 (WRAP-UP)

It is a very popular and classic System Design Interview Questions.
It has been asked in Google, Amazon and Meta with some variations.

We were doing STEP-3 : Design Deep Dive 🏗⚙️👷
It will be divided in various important components :
DFS vs BFS
URL Frontier
HTML Downloader
Robustness
Extensibility
Detect and Avoid Problematic Contents
We were doing STEP-4 : WRAP UP 🏗⚙️👷

Please use the pointers below for different time stamps. Hope that helps :

Introduction + Recap Design Deep Dive : (00:00)
HTML DOWNLOADER : (01:08)
Robots.txt (Robots Exclusion Protocol) : (02:19)
Distributed Crawling : (06:30)
Cache DNS Resolver : (08:07)
Locality : (10:23)
Set Timeout : (11:17)
Robustness : (12:35)
Extensibility-Block Diagram Recap: (17:24)
Detect & Avoid Problematic Content : (23:40)
WRAP-UP : (27:38)



My GitHub Repo for interview preparation : https://github.com/MAZHARMIK/Interview_DS_Algo
Subscribe to my channel : https://www.youtube.com/channel/UCaw58edcO3ZqMw76Bvs0kGQ

╔═╦╗╔╦╗╔═╦═╦╦╦╦╗╔═╗
║╚╣║║║╚╣╚╣╔╣╔╣║╚╣═╣
╠╗║╚╝║║╠╗║╚╣║║║║║═╣
╚═╩══╩═╩═╩═╩╝╚╩═╩═╝

#coding #helpajobseeker #easyrecipes #leetcode #leetcodequestionandanswers #leetcodesolution #leetcodedailychallenge #leetcodequestions #leetcodechallenge #hindi #india #hindiexplanation #hindiexplained #easyexplaination #interview #interviewtips
#interviewpreparation #interview_ds_algo #hinglish