The past, present, and future of robots.txt

Subscribers:
751,000
Published on ● Video Link: https://www.youtube.com/watch?v=nSjUI2lUUJs



Duration: 0:00
13,892 views
176


In this episode of Search Off the Record, Gary, Lizzi, and Martin from the Google Search team are joined by special guest David Price, a Product Counsel for Google. They discuss human initiated crawl requests vs. bot initiated crawl requests, the robots.txt file, and more!\n\n Episode transcript → https://goo.gle/sotr028-transcript\n\n Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.

Subscribe to Google Search Central Channel → https://goo.gle/SearchCentral

#SOTRpodcast #GoogleSearch