Module 9 · 9.2

English

Nederlands English Deutsch

Search visibility: the basics

Module 9: Email, search visibility, GDPR, maintenance and migrating · Reading time: 9 minutes

Learning objective: After this lesson, you'll be able to get the technical basis of a WordPress website's search visibility (SEO) right and check it.

Key concepts: search engine optimisation (SEO), indexing, title, meta description, headings, permalinks, sitemap (wp-sitemap.xml), robots.txt, noindex, Google Search Console, 301 for changed addresses, duplicate content, canonical, SEO plugin

What a search engine needs

A search engine does three things with your website:

Finding (crawling): a program follows links and the sitemap to discover your pages.
Including (indexing): the page is read and added to the index, unless you say it may not be.
Ranking: for a search query, the search engine decides which pages fit best.

The technical basics cover steps 1 and 2: make sure search engines can find your pages and are allowed to include them, and that they understand what each page is about. Step 3 mostly depends on the content: is your page the best answer to the question? No plugin can sort that out for you.

The basics WordPress already does

A title per page (<title> in the HTML), usually "Page title - Site name".
Clean addresses if your permalinks are set to Post name (lesson 2.4).
A sitemap: since WordPress 5.5, WordPress generates its own sitemap at https://example.com/wp-sitemap.xml, with all your public pages, posts, categories and authors. An SEO plugin often replaces it with its own (for example sitemap_index.xml).
A robots.txt: WordPress creates a virtual file https://example.com/robots.txt that refers to the sitemap. If you place a real robots.txt file in your web directory yourself, that one takes precedence.
Noindex on all pages if the "Discourage search engines" checkbox is ticked (lesson 2.4).

What you handle yourself

Title and meta description

The title is the clickable line in the search results. The meta description is the bit of text underneath it (the search engine doesn't always use it, but often does). WordPress doesn't generate a meta description itself; for that you need an SEO plugin or a theme that offers it.

<title>Remmen afstellen van je e-bike - Fietsenwinkel De Spaak</title>
<meta name="description" content="Zo stel je de remmen van je e-bike zelf af, en wanneer je beter langs de werkplaats komt.">

Write for people: does the title say what the page is about, and does the description give someone a reason to click? Give every page its own title and description, no copies.

Headings and content

Use headings for structure (lesson 4.1): one H1 (the page title), H2 for the sections. Write in the words your visitors use. Link from your text to other pages on your website (internal links); this helps search engines find your pages and understand how they relate. Describe images with alt text (lesson 4.4).

robots.txt and noindex: the difference

These two are often confused:

robots.txt says which addresses a search engine may not visit. Disallow: / means: don't go anywhere. But a page that may not be visited can still turn up in the search results (without a description) if other sites link to it.
noindex (a <meta name="robots" content="noindex"> in the page, or the X-Robots-Tag: noindex header) says: don't include this page. For that, the search engine does need to be able to visit the page.

If you want to keep a page out of Google, use noindex, not robots.txt. And after launch, check that your robots.txt doesn't contain Disallow: / and that noindex isn't set anywhere on your normal pages. A forgotten checkbox or a robots.txt left over from the build phase can keep a website out of Google for months.

A standard WordPress robots.txt looks roughly like this:

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Sitemap: https://example.com/wp-sitemap.xml

Google Search Console

Google Search Console is a free service from Google that shows you how Google sees your website: which pages are included and which aren't (and why), which search terms you're found for, errors, and security issues (lesson 7.5). You prove the website is yours with a TXT record in your DNS (DNS course lesson 4.2) or a file. Submit your sitemap there.

Bing has a similar service (Bing Webmaster Tools).

When addresses change

If you change a page's slug, migrate to a different domain, or rebuild the website with different addresses, links from outside and the search results point to addresses that no longer exist. Always set up a 301 redirect from the old address to the new one (lesson 3.2). You can do this with a redirect plugin, in .htaccess, or in your hosting panel. You can get a list of old addresses from your old sitemap or from Search Console.

Duplicate content

The same content at multiple addresses confuses search engines. Common causes with WordPress:

www and without www, http and https that don't redirect (lesson 3.2);
a staging site that search engines can find (lesson 8.3);
archives and tags with the same content (lesson 4.2).

WordPress adds a <link rel="canonical" href="..."> to posts and pages: "this is the real address of this content". That helps, but it's not a substitute for proper redirects.

SEO plugins, realistically

There are many SEO plugins (for example Yoast SEO, Rank Math, SEOPress, The SEO Framework). They help with: a title and meta description per page, a better sitemap, structured data (schema), breadcrumbs, redirects, and a check of your text. Useful, with two caveats:

A green dot in the plugin means your text meets a checklist, not that you'll rank highly. Good content for your visitors matters more than a readability score.
Choose one; two SEO plugins will produce duplicate titles, sitemaps and canonical rules.

Speed (module 6) and a website that works well on mobile also count.

Summary

Search engines find, include and rank; the technical side handles finding and including, the content handles the rest.
WordPress generates a title, clean addresses, a sitemap (wp-sitemap.xml) and a robots.txt by itself.
A meta description per page requires an SEO plugin or a theme that can do it.
robots.txt controls visiting, noindex controls including; after launch, check that neither is blocking your website.
Use Search Console, set up 301 redirects for changed addresses, and choose at most one SEO plugin.

Quiz

Exercise

Exercise A9.2: The technical basis of your search visibility. Enter your website's address. The tool checks: robots.txt is reachable and doesn't block the entire website (Disallow: / for all search engines), there is a sitemap (named in robots.txt, or /wp-sitemap.xml) that responds as XML, and the homepage has a non-empty title, no noindex (in the HTML or the X-Robots-Tag header) and exactly one H1. A missing meta description shows as orange.

Cookies at WebOké

Cookie settings

Necessary

Marketing