Google's Search Engine Optimization Starter Guide

Search Engine Optimization Starter Guide This document first began as an effort to help teams within Google, but we thought it'd be just as useful to webmasters that are new to the topic of search engine optimization and wish to improve their sites' interaction with both users and search engines. Although this guide won't tell you any secrets that'll automatically rank your site first for queries in Google (sorry!), following the best practices outlined below will make it easier for search engines to crawl, index and understand your content. Search engine optimization is often about making small modifications to parts of your website. When viewed individually, these changes might seem like incremental improvements, but when combined with other optimizations, they could have a noticeable impact on your site's user experience and performance in organic search results. You're likely already familiar with many of the topics in this guide, because they're essential ingredients for any web page, but you may not be making the most out of them. Even though this guide's title contains the words "search engine", we'd like to say that you should base your optimization decisions first and foremost on what's best for the visitors of your site. They're the main consumers of your content and are using search engines to find your work. Focusing too hard on specific tweaks to gain ranking in the organic results of search engines may not deliver the desired results. Search engine optimization is about putting your site's best foot forward when it comes to visibility in search engines, but your ultimate consumers are your users, not search engines. Your site may be smaller or larger than our example site and offer vastly different content, but the optimization topics we discuss below should apply to sites of all sizes and types. We hope our guide gives you some fresh ideas on how to improve your website, and we'd love to hear your questions, feedback, and success stories in the Google Webmaster Help Forum. Welcome to Google's Search Engine Optimization Starter Guide Table of Contents 4 6 8 10 14 16 18 20 21 22 24 26 28 30 Googlebot Crawling content on the Internet for Google's index every day, every night, non stop. From here on, I'll be explaining various points on search engine optimization (SEO)! SEO Basics Create unique, accurate page titles Make use of the "description" meta tag Improving Site Structure Improve the structure of your URLs Make your site easier to navigate Optimizing Content Offer quality content and services Write better anchor text Optimize your use of images Use heading tags appropriately Dealing with Crawlers Make effective use of robots.txt Be aware of rel="nofollow" for links SEO for Mobile Phones Notify Google of mobile sites Guide mobile users accurately Promotions and Analysis Promote your website in the right ways Make use of free webmaster tools An example may help our explanations, so we've created a fictitious website to follow throughout the guide. For each topic, we've fleshed out enough information about the site to illustrate the point being covered. Here's some background information about the site we'll use: Website/business name: "Brandon's Baseball Cards" Domain name: brandonsbaseballcards.com Focus: Online-only baseball card sales, price guides, articles, and news content Size: Small, ~250 pages “Paid” Search, AdWords Organic Search Search engine optimization affects only organic search results, not paid or "sponsored" results such as Google AdWords. Create unique, accurate page titles SEO Basics Glossary Indicate page titles by using title tags Page title contents are displayed in search results (1) The title of the homepage for our baseball card site, which lists the business name and three main focus areas. (2) A user performs the query [baseball cards]. Our homepage shows up as a result, with the title listed on the first line (notice that the query terms the user searched for appear in bold). If the user clicks the result and visits the page, the page's title will appear at the top of the browser. (3) A user performs the query [rarest baseball cards]. A relevant, deeper page (its title is unique to the content of the page) on our site appears as a result. A title tag tells both users and search engines what the topic of a particular page is. The (2) A user performs the query [baseball cards]. Our homepage appears as a result, with part of its description meta tag used as the snippet. (3) A user performs the query [rarest baseball cards]. One of our deeper pages, with its unique description meta tag used as the snippet, appears as a result. A page's description meta tag gives Google and other search engines a summary of what the page is about (1). Whereas a page's title may be a few words or a phrase, a page's description meta tag might be a sentence or two or a short paragraph. Google Webmaster Tools provides a handy content analysis section that'll tell you about any description meta tags that are either too short, long, or duplicated too many times (the same information is also shown for (4) This nofollows all of the links on a page. Make sure you have solid measures in place to deal with comment spam! 23 Notify Google of mobile sites SEO for Mobile Phones Configure mobile sites so that they can be indexed accurately Verify that your mobile site is indexed by Google Glossary (1) Example of a search for [baseball cards] on Google’s desktop search (above) and mobile search (left). Mobile search results are built for mobile devices and are different from "standard" desktop results. Make sure your mobile site is properly recognized by Google so that searchers can find it. It seems the world is going mobile, with many people using mobile phones on a daily basis, and a large user base searching on Google’s mobile search page. However, as a webmaster, running a mobile site and tapping into the mobile search audience isn't easy. Mobile sites not only use a different format from normal desktop sites, but the management methods and expertise required are also quite different. This results in a variety of new challenges. While many mobile sites were designed with mobile viewing in mind, they weren’t designed to be search friendly. Here are troubleshooting tips to help ensure that your site is properly crawled and indexed: If your web site doesn't show up in the results of a Google mobile search even using the site: operator, it may be that your site has one or both of the following issues: 1. Googlebot may not be able to find your site Googlebot must crawl your site before it can be included in our search index. If you just created the site, we may not yet be aware of it. If that's the case, create a Mobile Sitemap and submit it to Google to inform us of the site’s existence. A Mobile Sitemap can be submitted using Google Webmaster Tools, just like a standard Sitemap. Mobile Sitemap An XML Sitemap that contains URLs of web pages designed for mobile phones. Submitting the URLs of mobile phone web content to Google notifies us of the existence of those pages and allows us to crawl them. User-agent Software and hardware utilized by the user when said user is accessing a website. XHTML Mobile XHTML, a markup language redefined via adaptation of HTML to XML, and then expanded for use with mobile phones. Compact HTML Markup language resembling HTML; it is used when creating web pages that can be displayed on mobile phones and with PHS and PDA. 24 Improving Site Structure Optimizing Content Dealing with Crawlers Promotions and Analysis SEO Basics SEO for Mobile Phones Google’s mobile search page http://www.google.com/m/ site: operator http://www.google.com/support/webmasters/bin/answer.py?answer=35256 Mobile Sitemap http://www.google.com/support/webmasters/bin/topic.py?topic=8493 Submitted using Google Webmaster Tools http://www.google.com/support/webmasters/bin/answer.py?answer=156184 Use DNS Lookups to verify Googlebot http://googlewebmastercentral.blogspot.com/2006/09/how-to-verify-googlebot.html Mobile Webmaster Guidelines http://www.google.com/support/webmasters/bin/answer.py?answer=72462 Links Verify that Google can recognize your mobile URLs SetEnvIf User-Agent "Googlebot-Mobile" allow_ua SetEnvIf User-Agent "Android" allow_ua SetEnvIf User-Agent "BlackBerry" allow_ua SetEnvIf User-Agent "iPhone" allow_ua SetEnvIf User-Agent "NetFront" allow_ua SetEnvIf User-Agent "Symbian OS" allow_ua SetEnvIf User-Agent "Windows Phone" allow_ua Order deny,allow deny from all allow from env=allow_ua (2) An example of a mobile site restricting any access from non-mobile devices. Please remember to allow access from user agents including “Googlebot-Mobile”. (3) An example of DTD for mobile devices. 2. Googlebot may not be able to access your site Some mobile sites refuse access to anything but mobile phones, making it impossible for Googlebot to access the site, and therefore making the site unsearchable. Our crawler for mobile sites is "Googlebot-Mobile". If you'd like your site crawled, please allow any User-agent including "Googlebot-Mobile" to access your site (2). You should also be aware that Google may change its Useragent information at any time without notice, so we don't recommend checking whether the User-agent exactly matches "GooglebotMobile" (the current User-agent). Instead, check whether the Useragent header contains the string "Googlebot-Mobile". You can also use DNS Lookups to verify Googlebot. Once Googlebot-Mobile crawls your URLs, we then check for whether each URL is viewable on a mobile device. Pages we determine aren't viewable on a mobile phone won't be included in our mobile site index (although they may be included in the regular web index). This determination is based on a variety of factors, one of which is the "DTD (Doc Type Definition)" declaration. Check that your mobile-friendly URLs' DTD declaration is in an appropriate mobile format such as XHTML Mobile or Compact HTML (3). If it's in a compatible format, the page is eligible for the mobile search index. For more information, see the Mobile Webmaster Guidelines. 25 Guide mobile users accurately SEO for Mobile Phones Running desktop and mobile versions of your site Redirect mobile users to the correct version Homepage Product page Mobile version Homepage Redirect Redirect Mobile user (1) An example of redirecting a user to the mobile version of the URL when it's accessed from a mobile device. In this case, the content on both URLs needs to be as similar as possible. Desktop version Product page Glossary One of the most common problems for webmasters who run both mobile and desktop versions of a site is that the mobile version of the site appears for users on a desktop computer, or that the desktop version of the site appears when someone accesses it on a mobile device. In dealing with this scenario, here are two viable options: When a mobile user or crawler (like Googlebot-Mobile) accesses the desktop version of a URL, you can redirect them to the corresponding mobile version of the same page. Google notices the relationship between the two versions of the URL and displays the standard version for searches from desktops and the mobile version for mobile searches. If you redirect users, please make sure that the content on the corresponding mobile/desktop URL matches as closely as possible (1). For example, if you run a shopping site and there's an access from a mobile phone to a desktop-version URL, make sure that the user is redirected to the mobile version of the page for the same product, and not to the homepage of the mobile version of the site. We occasionally find sites using this kind of redirect in an attempt to boost their search rankings, but this practice only results in a negative user experience, and so should be avoided at all costs. On the other hand, when there's an access to a mobile-version URL from a desktop browser or by our web crawler, Googlebot, it's not necessary to redirect them to the desktop-version. For instance, Google doesn't automatically redirect desktop users from their mobile site to their desktop site; instead they include a link on the mobileversion page to the desktop version. These links are especially helpful when a mobile site doesn't provide the full functionality of the desktop version—users can easily navigate to the desktop-version if they prefer. Redirect Being automatically transported from one specified web page to another specified web page when browsing a website. 26 SEO Basics Improving Site Structure Optimizing Content Dealing with Crawlers SEO for Mobile Phones Promotions and Analysis Be sure to guide the user to the right site for their device! (2) Example of changing the format of a page based on the User-agent. In this case, the desktop user is supposed to see what Googlebot sees and the mobile user is supposed to see what Googlebot-mobile sees. Website Desktop user Googlebot Googlebot-Mobile Mobile user Must be same Must be same Desktop contents Mobile contents Can be different Some sites have the same URL for both desktop and mobile content, but change their format according to User-agent. In other words, both mobile users and desktop users access the same URL (i.e. no redirects), but the content/format changes slightly according to the User-agent. In this case, the same URL will appear for both mobile search and desktop search, and desktop users can see a desktop version of the content while mobile users can see a mobile version of the content (2). However, note that if you fail to configure your site correctly, your site could be considered to be cloaking, which can lead to your site disappearing from our search results. Cloaking refers to an attempt to boost search result rankings by serving different content to Googlebot than to regular users. This causes problems such as less relevant results (pages appear in search results even though their content is actually unrelated to what users see/want), so we take cloaking very seriously. So what does "the page that the user sees" mean if you provide both versions with a URL? As I mentioned in the previous post, Google uses "Googlebot" for web search and "Googlebot-Mobile" for mobile search. To remain within our guidelines, you should serve the same content to Googlebot as a typical desktop user would see, and the same content to Googlebot-Mobile as you would to the browser on a typical mobile device. It's fine if the contents for Googlebot are different from those for Googlebot-Mobile. One example of how you could be unintentionally detected as cloaking is if your site returns a message like "Please access from mobile phones" to desktop browsers, but then returns a full mobile version to both crawlers (so Googlebot receives the mobile version). In this case, the page which web search users see (e.g. "Please access from mobile phones") is different from the page which Googlebot crawls (e.g. "Welcome to my site"). Again, we detect cloaking because we want to serve users the same relevant content that Googlebot or Googlebot-Mobile crawled. Switch content based on User-agent Google mobile http://www.google.com/m/ Cloaking http://www.google.com/support/webmasters/bin/answer.py?answer=66355 Links Can be different 27 Promote your website in the right ways Promotions and Analysis About increasing backlinks with an intention to increase the value of the site Master making announcements via blogs and being recognized online My blog Website Product page Glossary (2) By having your business registered for Google Places, you can promote your site through Google Maps and Web searches. News: “I have a new card!” User’s blogs Social media service Newsletter, DM, Posters, etc. Online Offline (1) Promoting your site and having quality links could lead to increasing your site’s reputation. While most of the links to your site will be gained gradually, as people discover your content through search or other ways and link to it, Google understands that you'd like to let others know about the hard work you've put into your content. Effectively promoting your new content will lead to faster discovery by those who are interested in the same subject (1). As with most points covered in this document, taking these recommendations to an extreme could actually harm the reputation of your site. A blog post on your own site letting your visitor base know that you added something new is a great way to get the word out about new content or services. Other webmasters who follow your site or RSS feed could pick the story up as well. Putting effort into the offline promotion of your company or site can also be rewarding. For example, if you have a business site, make sure its URL is listed on your business cards, letterhead, posters, etc. You could also send out recurring newsletters to clients through the mail letting them know about new content on the company's website. If you run a local business, adding its information to Google Places will help you reach customers on Google Maps and web search. The Webmaster Help Center has more tips on promoting your local business. RSS feed Data including full or summarized text describing an update to a site/blog. RSS is an abbreviation for RDF Site Summary; a service using a similar data format is Atom. 28 Improving Site Structure Optimizing Content Dealing with Crawlers Promotions and Analysis SEO for Mobile Phones SEO Basics Google Places http://www.google.com/local/add/ Promoting your local business http://www.google.com/support/webmasters/bin/answer.py?answer=92319 Links Best Practices Know about social media sites Sites built around user interaction and sharing have made it easier to match interested groups of people up with relevant content. Reach out to those in your site's related community Chances are, there are a number of sites that cover topic areas similar to yours. Opening up communication with these sites is usually beneficial. Hot topics in your niche or community could spark additional ideas for content or building a good community resource. spamming link requests out to all sites related to your topic area purchasing links from another site with the aim of getting PageRank instead of traffic Avoid: attempting to promote each new, small piece of content you create; go for big, interesting items involving your site in schemes where your content is artificially promoted to the top of these services Avoid: Is your site doing OK? 29 Make use of free webmaster tools Promotions and Analysis Major search engines, including Google, provide free tools for webmasters. Google's Webmaster Tools help webmasters better control how Google interacts with their websites and get useful information from Google about their site. Using Webmaster Tools won't help your site get preferential treatment; however, it can help you identify issues that, if addressed, can help your site perform better in search results. With the service, webmasters can: Make Googlebot crawling smoother by using Webmaster Tools see which parts of a site Googlebot had problems crawling notify us of an XML Sitemap file analyze and generate robots.txt files remove URLs already crawled by Googlebot specify your preferred domain identify issues with title and description meta tags understand the top searches used to reach a site get a glimpse at how Googlebot sees pages remove unwanted sitelinks that Google may use in results receive notification of quality guideline violations and request a site reconsideration If you've improved the crawling and indexing of your site using Google Webmasters Tools or other services, you're probably curious about the traffic coming to your site. Web analytics programs like Google Analytics are a valuable source of insight for this. You can use these to: High-level analysis is possible via Google Analytics and Website Optimizer get insight into how users reach and behave on your site discover the most popular content on your site measure the impact of optimizations you make to your site - e.g. did changing those title and description meta tags improve traffic from search engines? Yahoo! (Yahoo! Site Explorer) and Microsoft (Bing Webmaster Tools) also offer free tools for webmasters. For advanced users, the information an analytics package provides, combined with data from your server log files, can provide even more comprehensive information about how visitors are interacting with your documents (such as additional keywords that searchers might use to find your site). Lastly, Google offers another tool called Google Website Optimizer that allows you to run experiments to find what on-page changes will produce the best conversion rates with visitors. This, in combination with Google Analytics and Google Webmaster Tools (see our video on using the "Google Trifecta"), is a powerful way to begin improving your site. 30 Improving Site Structure Optimizing Content Dealing with Crawlers SEO for Mobile Phones SEO Basics Promotions and Analysis Google Trifecta http://www.youtube.com/watch?v=9yKjrdcC8wA Links http://www.google.com/support/forum/p/webmasters/ Have questions or feedback on our guide? Let us know. Google Webmaster Help Forum http://googlewebmastercentral.blogspot.com/ Frequent posts by Googlers on how to improve your website. Google Webmaster Central Blog http://www.google.com/support/webmasters/ Filled with in-depth documentation on webmaster-related issues. Google Webmaster Help Center https://www.google.com/webmasters/tools/ Optimize how Google interacts with your website. Google Webmaster Tools http://www.google.com/webmasters/guidelines.html Design, content, technical, and quality guidelines from Google. Google Webmaster Guidelines http://www.google.com/analytics/ Find the source of your visitors, what they're viewing, and benchmark changes. Google Analytics http://www.google.com/websiteoptimizer/ Run experiments on your pages to see what will work and what won't. Google Website Optimizer http://www.google.com/support/webmasters/bin/answer. py?answer=35291 If you don't want to go at it alone, these tips should help you choose an SEO company. Tips on Hiring an SEO Make the most of useful tools and information! Except as otherwise noted, the content of this document is licensed under the Creative Commons Attribution 3.0 License. This booklet is also available in PDF format. You can download the PDF version at ... http://www.google.co.jp/intl/en/webmasters/docs/search-engine-optimization-starter-guide.pdf 31 ©Copyright 2010 Google is a trademark of Google Inc. All other company and product names may be trademarks of the respective companies with which they are associated. http://www.google.com/webmasters/ Check out Google's SEO resources and tools. Google Webmaster Central Search 32