Special files in robots.txt

Channel:
United States Google Search Central
Subscribers:
751,000
Published on ● Video Link: https://www.youtube.com/watch?v=Kruk8MhSzPk


Duration: 1:56
9,102 views
299

In this episode of Ask Google Webmasters, John Mueller goes over special files in robots.txt such as *.css, php.ini, and .htaccess, and explains why one does not have to and should not disallow these files from being crawled. The question was submitted by @svperflvid. Thank you!

Send us your questions on Twitter with the hashtag AskGooglebot and your question might be answered!
Google Search Central on Twitter → https://goo.gle/3f4Z0a8

Watch more AskGooglebot episodes → https://goo.gle/2OjWcvS
Subscribe to the Google Search Central Channel → https://goo.gle/SearchCentral

#AskGooglebot



Other Videos By Google Search Central


2020-08-11English Google Webmaster Central office-hours from August 7, 2020
2020-08-10Announcing the Virtual Webmaster Unconference
2020-08-06Mobile-First Indexing
2020-08-05Japanese Webmaster Office Hours（ウェブマスター オフィスアワー 2020 年 08 月 06 日）
2020-08-04English Google Webmaster Central office-hours from August 4, 2020
2020-08-03German / Google Webmaster Central Sprechstunden auf Deutsch vom 30. Juli 2020
2020-07-31Google Search News (July ‘20) - Web Stories, Page Experience Benchmark, and more
2020-07-27Page Speed: SEO Mythbusting
2020-07-24English Google Webmaster Central office-hours from July 24, 2020
2020-07-24English Google Webmaster Central office-hours from July 21, 2020
2020-07-20Special files in robots.txt
2020-07-14Crawl Budget: SEO Mythbusting
2020-07-08Rich Results & Google Search Console
2020-07-01Japanese Webmaster Office Hours（ウェブマスター オフィスアワー 2020 年 07 月 02 日）
2020-07-01JavaScript SEO office hours, June 24th 2020
2020-06-30SEO Mythbusting with Google & Bing
2020-06-26English Google Webmaster Central office-hours from June 26, 2020
2020-06-26German / Google Webmaster Central Sprechstunden auf Deutsch vom 25. Juni 2020
2020-06-24SEO Mythbusting Is Back - Official Trailer (Season 2)
2020-06-23English Google Webmaster Central office-hours from June 23, 2020
2020-06-23JavaScript SEO office hours June 17th, 2020


Tags:
GDS: Yes
Special files in robots.txt
robots.txt
what happens when you disallow css files
rendering
rendering problems
What SEOs need to know
how to increase seo
seo tips
search engines
index
crawl
googlebot
intro to seo
seo tutorial
getting started with seo
search engine optimization
#AskGoogleWebmasters
ask Google Webmasters
webmasters
Google Search
Google Webmasters
John Mueller
Martin Splitt
wta
Google Search Central