Technical SEO guide and checklist
Technical SEO sounds specialist and mostly is not. It is a small number of questions about whether Google can reach, render and index your pages, asked in the right order.
Last updated
Quick answer
Technical SEO is the work that lets search engines crawl, render and index your pages: the site's structure, status codes, robots rules, canonical tags, sitemaps, JavaScript rendering, speed and mobile layout. It doesn't make a page relevant to a search, but if it fails, nothing else on the page gets a chance to rank.
Key takeaways
- Google works in three stages: crawling, indexing and serving. A page has to pass each one.
- robots.txt controls crawling, not indexing. Use noindex to keep a page out of Google.
- A canonical tag is a hint, not a rule.
- Core Web Vitals are used in ranking, but relevance comes first.
What technical SEO covers
On-page SEO is about what a page says. Technical SEO is about whether Google can get to it, see it and keep it. Google describes Search as three stages, crawling, indexing and serving, and notes that not every page makes it through each one.
| Stage | The question | What usually stops it |
|---|---|---|
| Crawling | Can Google reach the page? | No internal links, robots.txt rules, server errors |
| Rendering | Can Google see the content? | Content that only appears after JavaScript fails |
| Indexing | Did Google decide to keep it? | noindex, duplicates, canonical tags, low value |
| Serving | Does it show for relevant searches? | Relevance and quality; mostly not technical |
How to tell you have a technical problem
Compare the number of pages you published with the number Google has indexed. If they differ materially and you can't explain the gap, something structural is wrong.
Then use URL Inspection in Search Console on a few important pages. It shows whether the page is indexed, which canonical Google chose, and what Google saw when it rendered the page.
Page indexing statuses explained
Search Console's page indexing report groups unindexed pages by reason. Google's own advice is not to expect every URL to be indexed; the goal is the canonical version of every important page.
| Status | What it means | Usual cause and fix |
|---|---|---|
| URL blocked by robots.txt | Google wasn't allowed to crawl it | Check the rule was intended; robots.txt doesn't remove pages from the index |
| URL marked 'noindex' | The page asked not to be indexed | Remove noindex if the page should rank |
| Crawled - currently not indexed | Crawled, not kept | Often thin or duplicate content; improve or consolidate the page |
| Discovered - currently not indexed | Found, not crawled yet | Weak internal links, or a large site Google is crawling slowly |
| Duplicate without user-selected canonical | A duplicate with no canonical set | Add a canonical tag pointing to the preferred URL |
| Duplicate, Google chose different canonical than user | Google disagreed with your canonical | Make internal links and the sitemap use the URL you want |
| Soft 404 | Looks like an error page but returns 200 | Return a real 404, or give the page real content |
| Page with redirect | A URL that redirects elsewhere | Usually fine; remove it from the sitemap and internal links |
Crawling: links, status codes and sitemaps
- Internal links: every page you want found should be linked from another page with a normal <a href> link.
- Status codes: important pages return 200; moved pages return a single permanent (301) redirect; removed pages return 404 or 410.
- Sitemaps: list only canonical, indexable URLs. Google ignores the priority and changefreq values.
- Server errors: repeated 5xx errors slow crawling down.
Indexing controls: robots.txt, noindex and canonicals
These three are confused more than anything else in technical SEO.
| Control | What it does | What it doesn't do |
|---|---|---|
| robots.txt | Stops Google crawling a URL | Keep it out of Google: a blocked URL can still appear in results |
| noindex | Removes the page from Google's results | Work if robots.txt blocks the page, because Google never sees the tag |
| Canonical tag | Tells Google which duplicate you prefer | Guarantee it: Google treats it as a hint, not a rule |
Sources: Google's robots.txt introduction, blocking indexing with noindex and canonicalization.
Rendering and JavaScript
If your content only appears after JavaScript runs, Google has to render the page before it can see anything. Google's JavaScript SEO basics say a page may wait in the render queue for a few seconds, or longer, and recommend server-side or pre-rendering because it makes the site faster for users and crawlers.
The test is simple: view the page source rather than the inspector. If your headline and body copy aren't in the raw HTML, you're relying on rendering, and that is a risk you're taking rather than a decision you made.
Page experience and Core Web Vitals
Google says Core Web Vitals are used by its ranking systems, and also that it always seeks to show the most relevant content, even if the page experience is sub-par. Fix a slow site because it loses visitors; don't expect speed to outrank a more relevant page. Core Web Vitals explained covers the three metrics, and the mobile SEO guide covers mobile-first indexing.
Technical SEO checklist
| Check | Free way to check | Priority |
|---|---|---|
| Important pages are indexed | Search Console page indexing report and URL Inspection | High |
| No important page blocked by robots.txt or noindex | URL Inspection; view source for noindex | High |
| Content is in the raw HTML | View page source | High |
| One URL per page (https, www, trailing slash) | Type the variants and check they redirect | High |
| Canonical tags point to the preferred URL | View source; URL Inspection shows Google's choice | High |
| Every important page is internally linked | Click through from the homepage | High |
| Redirects are single hops | A browser redirect checker on old URLs | Medium |
| Sitemap lists only canonical, indexable URLs | Open the sitemap; compare with URL Inspection | Medium |
| No soft 404s or 5xx errors | Page indexing report | Medium |
| Core Web Vitals pass on key templates | Search Console Core Web Vitals report; PageSpeed Insights | Medium |
| Mobile content matches desktop | URL Inspection rendered HTML | Medium |
| Structured data validates | Rich Results Test | Low to medium |
What matters less than you've been told
A perfect PageSpeed score. Sitemap priority values, which Google ignores. Chasing every warning a crawling tool reports, most of which affect nothing. Spending a quarter on those while important pages sit unindexed is the most common misallocation in technical SEO.
Common questions
What is technical SEO in simple terms?
Making sure search engines can find, read and keep your pages. It covers crawling, rendering and indexing, rather than what the pages say.
How is technical SEO different from on-page SEO?
On-page SEO is about the content and wording of a page. Technical SEO is about whether Google can reach, render and index that page in the first place.
Can robots.txt stop a page appearing in Google?
No. Google says robots.txt is not a mechanism for keeping a page out of Google; a blocked URL can still appear in results. Use a noindex tag, and don't block the page in robots.txt, or Google won't see the tag.
Why does Search Console say "Crawled - currently not indexed"?
Google crawled the page and decided not to index it, often because it is thin, duplicated or low value. Google says there is no need to resubmit it; improve or consolidate the page instead.
Does JavaScript hurt SEO?
Not necessarily, but content that only appears after JavaScript runs depends on rendering. Google recommends server-side or pre-rendering because it is faster for users and crawlers.
How much do Core Web Vitals affect rankings?
They are used by Google's ranking systems, but Google says it shows the most relevant content even if page experience is sub-par. Relevance comes first.
Do I need a developer for technical SEO?
For diagnosis, no. For most fixes, yes, or a CMS that exposes the settings. The diagnosis is the specialist part; implementation is usually straightforward once somebody has said precisely what to change.
Where to next
If you would rather have someone do this than read about it, the commercial page for this work is Technical SEO services.
All guides Core Web Vitals explainedMobile SEO guideEcommerce SEO checklist