Add X-Robots-Tag: noindex for .mintlify.app subdomains when custom domain is configured

[Pietrus-914]

Problem
When a custom domain is configured (e.g., docs.example.com), the default .mintlify.app subdomain remains publicly accessible and crawlable by search engines. This creates a duplicate content issue that can negatively impact SEO.
Both URLs serve identical content and both get indexed by Google.

Why canonical tags are not enough
Mintlify correctly sets the canonical URL to the custom domain, but search engines treat canonical tags as hints, not directives. Google may still:

Index the .mintlify.app subdomain
Split link equity between both URLs
Show the wrong URL in search results
Waste crawl budget on duplicate pages

Proposed solutions
Option 1: X-Robots-Tag HTTP header (recommended)
Automatically add the following HTTP header for .mintlify.app subdomains when a custom domain is configured:
X-Robots-Tag: noindex, nofollow
This is the industry standard approach. Vercel and Netlify already do this automatically for preview deployments and non-production URLs.

Option 2: 301 redirect
Implement automatic 301 redirects from *.mintlify.app to the configured custom domain. This would:

Pass full link equity to the custom domain
Ensure users always land on the correct URL
Completely eliminate duplicate content

Option 3: Configuration option
Add a setting in docs.json to control subdomain behavior:

"seo": {
"disableMintlifySubdomain": true
}
Or in the dashboard under Domain Settings.

Impact
This affects all Mintlify users who configure custom domains. Without this fix, users must either:

Accept potential SEO penalties from duplicate content
Manually request URL removal through Google Search Console (temporary fix)

Thank you for considering this feature. Happy to provide additional details if needed.

[laurenschristian]

+1 on this. We have this exact problem — Google is indexing our .mintlify.app subdomain despite having a custom domain properly configured with canonical tags set in docs.json.

Searching the .mintlify.app subdomain on Google returns multiple indexed pages pointing to the mintlify.app version instead of our custom domain.

The canonical tag approach is clearly not sufficient — Google is treating it as a hint and continuing to index the mintlify.app version. A 301 redirect or X-Robots-Tag: noindex header on .mintlify.app subdomains when a custom domain is active would solve this immediately.

Netlify has been doing this automatically for years. Would love to see Mintlify implement this.

[00al]

+1

[vokiel]

+1

Also there is a little problem with the SEO -> meta tags -> robots setting. We wanted to disable indexing by adding this tag, so that the .mintlify.app an .mintlify.dev/docs pages are excluded from search engines. And then, on CloudFront level, dedicated Lambda@Edge would rewrite those <meta name="robots" content="noindex,nofollow"> into <meta name="robots" content="index,follow">. But the JavaScript on front side is adding <meta name="robots" content="noindex,nofollow"> again, based on the configuration.

So, IMHO it would be better to have some switch that would exclude .app and .dev domains from search engines only by adding X-Robots-Tag header.
Or would add meta tags based on the domain/host. If request sets the Host to defined one: e.g. example.com then send index,follow, but if the host is .mintlify.app an .mintlify.dev then add noindex,nofollow.

[Blargian]

There is a no index option from within the dashboard. Does this address the issue?

[obalais]

We hit this too, and we now have an answer from Mintlify support that is worth putting on the record here.

Our setup: custom domain developers.semji.com, mirror semji.mintlify.app.

What is already correct on Mintlify's side: the mirror serves a proper per-page, cross-domain canonical, and the sitemap served by the mirror lists only custom-domain URLs (64 out of 64 in our case). So the "your canonical is misconfigured" explanation does not apply here.

What is missing: the mirror answers 200 on every page, including with a Googlebot user agent, with no X-Robots-Tag, no <meta name="robots">, no 301, and a permissive robots.txt identical on both hosts. The canonical is a signal rather than a directive, so pages get indexed on the mirror anyway.

We opened a support ticket about this. The reply, verbatim:

Mintlify doesn't currently support X-Robots-Tag, noindex, or a 301 redirect from the .mintlify.app subdomain to a custom domain. Since .mintlify.app isn't under customer control, this needs to be implemented on Mintlify's side, and it's a known limitation that's on the roadmap with no timeline to share yet.

Support also pointed us to the feature-requests board to get the use case tracked, which is exactly what this thread already is, open since January. So the issue is acknowledged internally and owned by Mintlify. It would help to have that status reflected here, and ideally a rough timeline.

@Blargian on the dashboard no-index toggle: it does not address this. It is project-wide, so it would noindex the custom domain too, which is the domain we want ranking. It also applies a meta tag rather than a response header, which is what @vokiel ran into when trying to rewrite it at the edge. The fix has to be host-aware and served as an HTTP header.

One implementation note: the information needed is already available at request time. The mirror already renders the custom domain in both the canonical and the sitemap, so the canonical host for the project is known while the response is being served. Emitting X-Robots-Tag: noindex, nofollow when the request Host ends in .mintlify.app or .mintlify.dev is the same decision, one layer earlier.