Dynamic Sitecore MVC robots.txt

Here’s a simple way to create a dynamic Sitecore MVC robots.txt.

1) RouteConfig

2. Create a RobotsController

3. Add /robots.txt to IgnoreUrlPrefixes setting in Sitecore.config

4. Ensure   <modules runAllManagedModulesForAllRequests=”true”> is set to true within web.config.

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">