0:00
Ever wondered how websites tell search
0:01
engines what to crawl and what to skip?
0:04
That's where robots text comes in. A
0:07
robots.txt file is a simple text file
0:10
that guides search engines like Google
0:12
on which pages to index or ignore. If
0:15
you found this helpful, hit like and
0:17
subscribe for more easy web tips.
