If you built an indie product and organic traffic is still near zero, do not start by publishing ten generic SEO articles. First find the earliest failure: Google cannot reach the page, cannot read the important facts, cannot tell which URL is the real one, has not discovered the page yet, or has no reason to trust the product.
This order matters. A sitemap cannot fix a page blocked by robots rules. More content cannot fix a page marked noindex. A new backlink cannot fix a canonical URL that points to the wrong site.
Start with the exact page#
Search Google for site:yourdomain.com/page-path. Replace the domain and path with the exact product or landing page you want indexed. Also search the product name in quotes if it is unique.
These searches are clues, not proof. A missing result does not tell you which technical setting is wrong, and a result does not prove that Google will keep the page indexed or rank it for a useful query.
Write down the exact URL you tested. Check the public URL, not a staging URL or a version with tracking parameters.
Check whether crawlers can enter#
Open yourdomain.com/robots.txt in a browser. Look for a broad rule such as User-agent: * followed by Disallow: /. Check whether a more specific rule blocks the path you care about.
Then run the AI Crawler Access Checker. It compares the served robots rules with a live request. This catches an important mismatch: robots.txt may allow a page while a CDN or WAF returns 403 or 429 to the request.
Do not block every AI crawler by default. Search crawlers, user-requested fetchers, and training crawlers serve different purposes. Choose a training-data policy separately from the access needed for search and citations.
Check the HTML Google receives#
Open the page source, not only the browser inspector. Look for the page title, description, main heading, product or service explanation, and important links in the initial HTML.
The JavaScript SEO Render Checker compares raw HTML with the rendered page. If the important facts appear only after JavaScript runs, some crawlers may not see the same page a person sees.
This does not mean all JavaScript is bad. It means the facts that define the page should be available in a server response that a crawler can read without depending on a browser session.
Check for noindex and the wrong URL#
Search the page source and response headers for noindex. A page with a noindex directive is asking search engines not to include it. Remove it only when the page is meant to be public.
Next look for a canonical link. The canonical should point to the public URL you want to represent the page. A product page that canonicals to a staging host, a different product, or the homepage sends a confusing identity signal.
Run the Canonical URL Identity Checker against the exact page. It checks the page URL, canonical tag, redirects, and related signals. Fix the earliest mismatch before requesting another crawl.
Check whether the sitemap works#
Open yourdomain.com/sitemap.xml. It should return XML, not an HTML app shell, login page, or error screen. Each URL should be the canonical public version, and pages you do not want indexed should not be included.
The Sitemap Health Checker checks the response, locations, dates, duplicates, and basic alignment with robots and canonical signals. A clean sitemap helps discovery, but it does not guarantee indexing.
Submit the sitemap in Google Search Console after the public files are correct. Google may take days or weeks to crawl and process a new site. A crawl request is a request, not a promise of inclusion.
If the technical checks pass#
When access, HTML, canonical identity, and sitemap discovery all look right, the remaining problem may be relevance, trust, or demand rather than a broken setting.
Make the page answer one clear question. Say who the product is for, what it does, and what outcome it supports. Link to useful evidence such as documentation, examples, pricing, or a public product profile.
For a new product, build a small set of relevant references instead of submitting to every site that offers a link. The startup directory collection helps compare destinations by audience, submission access, pricing, link type, and Directory Score. Read how to choose startup directories before preparing a batch of listings.
What this diagnosis cannot prove#
- A passing technical check does not guarantee indexing.
- Indexing does not guarantee a first-page ranking.
- A sitemap does not create authority or demand.
- A backlink does not guarantee traffic or ranking improvement.
- An AI crawler check does not prove that an assistant will cite your product.
Fix the earliest failure, request a crawl when the page is ready, and measure impressions and clicks in Search Console over time. If the page is technically sound but still invisible, improve its usefulness and earn relevant references before publishing more keyword variations.
See How We Checked It
We name the source, record the review date, and separate reported claims from our own checks. Read the methodology and editorial policy before citing a finding or sending a correction.
Put this guide into practice
Open the founder distribution tools hub, then choose the check that matches your next decision.
Found an outdated fact? Report a correction.
Back to articles