# prevents crawling of forum pages and logs, stats pages. User-agent: * #Disallow: /forums/robo Disallow: /stats Disallow: /logs Disallow: /bahamas # This bot looks for student plagarism so fuck him User-agent: TurnitinBot Disallow: /