some of the basics about how webmasters can use a file called robots.txt to control how search engine crawlers (aka bots) crawl their websites. But there is so much more to talk about with bots. So let's take a bit of a deeper dive into the subject. Topic 1: Using the proper text file encoding The robots.txt file is used by webmasters to either specifically define which files and directories that