To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.
If you create memorable content, people will want to come back for more. So instead of churning out lackluster content that can be found anywhere on the web, write higher quality, unique content that caters directly to your audience. Speak your opinion on a subject matter, instead of just objectively providing facts. Create useful, thought-provoking content. Posting three so-so blog posts a week will not be nearly effective as posting one superb blog post per week.
Great article, Brian. Like that you’re finally talking about Domain Authority (DA). It’s essential to make skyscraper technique work as well. Also, a great pointer on comments as I have personally seen articles perform well because of comments. Do you recommend closing the comments as well a few days after the article is published? Kinda like Copyblogger does now.
For example, we regularly create content on the topic of "SEO," but it's still very difficult to rank well on Google for such a popular topic on this acronym alone. We also risk competing with our own content by creating multiple pages that are all targeting the exact same keyword -- and potentially the same search engine results page (SERP). Therefore, we also create content on conducting keyword research, optimizing images for search engines, creating an SEO strategy (which you're reading right now), and other subtopics within SEO.
WOW. I consider myself a total newbie to SEO, but I’ve been working on my Squarespace site for my small business for about 3 years and have read dozens of articles on how to improve SEO. So far, this has been the MOST USEFUL and information-packed resource I’ve found so far. I’m honestly shocked that this is free to access. I haven’t even completely consumed this content yet (I’ve bookmarked it to come back to!) but I’ve already made some significant changes to my SEO strategy, including adding a couple of infographics to blog posts, changing my internal and external linking habits, editing meta descriptions, and a bunch more. Thanks for all the time and passion you’ve out into this.