Learn exactly how to configure robots.txt for GPTBot, PerplexityBot, ClaudeBot, and Gemini in 2026. Step-by-step tactics to control AI crawlers without killing