Index
An index is another name for the database used by a search engine. Indexes contain the information on all the websites that Google (or any other search engine) was able to find. If a website is not in a search engine’s index, users will not be able to find it.
The index as the main reference point
It is the source data in Google’s index that ultimately determines the value of different search terms and keywords. Search engines apply their algorithms to the available data, and measure the frequency of different factors under different conditions, which factors are related to one another and so on. The index includes not just the URLs, but all content, including texts, images, videos and, in principle, everything within the HTML code of the URL.
The information garnered from this analysis flows back into Google’s algorithm to provide a new assessment of the index data, which attempts to understand which content best meets which user intent. The Google search results, or rankings, are then calculated on the basis of this content assessment.
Country indexes
Globally operating search engines, such as Google, usually have a separate index for each market. This means that, for example, there is a Google Index for US (google.com), a Google Index for Japan (google.co.jp) etc. Having national indexes helps the search engine tailor results to the search behavior (including but not limited to language) of each market. This provides a more reliable information resource that is more closely related to what users in the country are looking for. An inferior alternative approach would be to base results on what would be a universal index, including data from all markets, but this would make it impossible to meet the specific needs of users in each country.
Whilst several global players have high-ranking websites in multiple country indexes, as demonstrated in our SEO World Rankings , it pays to be aware of the different ranking factors applicable in each country. In particular, search data for every keyword is unique to each national index. Without this data, it is impossible to base SEO and content marketing decisions on the actual behavior of users.
Local indexes
A local search engine index follows the same principle as a country index, just at a regional or city level. Local indexes are mainly important for searches for local services or places, as they make it possible to return information specific to the location of the user. The most obvious example is for search queries containing “near me” or something like “phone number taxi”, where users in Miami would clearly expect very different answers to users in Portland.
The local indexes for Miami and Portland contain quite different results for “pizza near me”.
Search market in constant change
As with any other economic market, the search market is also in a constant state of flux, meaning that search engines must conduct continuous indexing and re-indexing of the available data, if they wish to provide users with information that is most relevant to their search requests. As new websites and/or new content constantly appear on the internet, it is essential for search engines to constantly update their indexes. Sometimes, webmasters may be surprised that a new page – even if it has been built paying close attention to SEO and content marketing requirements – is not ranking in the Google search results. Normally, the reason will be that it has not yet been included in Google’s index.
Technical specifications for index inclusion
By definition, if a website is not in the index, it can’t and won’t be displayed in any search results and it will lose all value. Understanding how Google etc. crawl the web and making a domain technically accessible is essential, meaning that the crawler can view all pages – and all content on those pages – without difficulty and all URLs on a domain can be included in the relevant index. If search engines crawling your page.
Mobile (first) index
Until now, search engines have used desktop crawlers as their primary means of gathering information for their index. Whether or not there is a separate mobile index, or whether the mobile results are tweaked from the desktop index, has been subject to speculation. In 2016, however, a Google webmaster blog post announced the plan to switch to mobile-first indexing. This will mean that the primary information for Google’s index will come from crawling the mobile versions of websites, with the desktop index set to be an adaptation.
Whilst it is not yet clear what this new way of indexing will mean for websites with long-form content on their desktop pages and shorter mobile content, it does indicate that any webmaster looking for sustained success should be aware of the latest developments in Google’s indexing practices, as the way the search engine views a page is ultimately what will determine how well it performs in the rankings.
