1)  Was not doing anything that could be considered black hat.

2)  Was providing any SEO benefit for their clients.

The problems you face with projects like this is that Google doesn’t provide enough information, and you cannot post corner-case questions like this in public Webmaster forums. To do so would violate your NDA, and potentially reveal your client’s intellectual property. So, what option do you have left? Well, you set up a honeypot!

A honeypot is a term that comes from the information security industry. Honeypots are a set of files that, to an automated program, appear like regular files, but they allow for the monitoring and “capturing” of specific viruses, e-mail harvesters, etc. In our case, we set up a honeypot with the purpose of detecting and tracking search engine bot behavior in specific circumstances. We also wanted to track the outcome (positive, neutral or negative) in the search engine results pages (SERPs).

Let me walk you trough a few ways you can learn advanced SEO by using a honeypot.

Goals of the honeypot

First, let’s define the goals in terms of questions for which we don’t have public answers. Here are some interesting questions you and I might have:

1. Which search bots support the if-modified-since and/or the if-unmodified-since headers?

2. Is Googlebot really a headless browser?

3. Which search bots crawl AJAX URLs? Which ones support Google’s crawlable scheme?

4. Does Google follow links inside PDFs? Do they count for indexation and rankings?

5. Does the in-page canonical tag carries more weight that the canonical link header?

Add your own questions to this list. For the purpose of this post, I’m going to explain how you go about answering first question. The recent work I did for a client was related to AJAX style fragment URLs. Unfortunately, I can’t share any details.

Setting up the Honeypot

The first thing you need to do is understand the problem really well. In our case, if-modified-since is a header that browsers and bots can send to a webserver, and the webserver will avoid resending a resource (image, document, page, video, etc.) if it hasn’t changed since the last time it was requested. The primary goal is to save bandwidth.

If-unmodified-since does the opposite. It returns the resource if it hasn’t changed.

There is technical protocol that HTTP clients and servers must obey, and a typical conversation looks like this:

CLIENT/BOT Request:

GET / HTTP/1.1

Host: hamletbatista.com

If-Modified-Since:  Thu, 26 Jan 2012 17:32:59 GMT

SERVER Response:

HTTP/1.1 304 Not modified

Date: Thu, 26 Jan 2012 17:32:59 GMT

CLIENT/BOT Request:

GET / HTTP/1.1

Host: hamletbatista.com

If-Unmodified-Since:  Thu, 26 Jan 2012 17:32:59 GMT

SERVER Response:

HTTP/1.1 412 Precondition failed

You can learn more about this here.

The most common way to follow these conversations between servers and bots is to setup and analyze traffic logs. However, the typical format of a traffic log does not store ‘if-modified-since’ header information. Sometimes it is practical to set up a custom log to track this information, but other times it isn’t.

Here is how a typical log entry looks like for valid Googlebot request .

66.249.67.9 - - [26/Jan/2011:02:29:32 -0500] "GET / HTTP/1.1" 200 157 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Getting the answers

One simple alternative is to look for the response code. In the case of a request that includes the ‘if-modified-since’ header, the web server will return status code 200 if the page changed, and status code 304 if it hasn’t changed. On the other hand, it will return 412 if the resource changed, and the client sent an ‘if-unmodified-since’.

Because 200 is a code that can be returned when the ‘if-modified-since’/’if-unmodified-since’ headers are not sent, the most reliable way to tell if a request included the header we want to check, is to track responses that returned 304 (the response that say nothing changed) or 412 (something changed).

You also want to make sure your webserver support the corresponding headers. You can use Firebug for this.

As you should have guess by now, it is easy to check if Googlebot supports this header by checking the traffic log for entries coming from Googlebot and seeing if the responses include the 304 or 412 status codes.

I wrote a simple log parsing script in Python to look for response codes 304 or 412 and see if any entry came for Googlebot. In order to make it work, you will need the excellent Python log parser, apachelog.

import apachelog, sys, glob

format = r'%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"'

files = glob.glob("access_log*")

p = apachelog.parser(format)

for log in files:
	for line in open(log):
	      try:
		data = p.parse(line)

		status = data['%>s']
		ua = data['%{User-agent}i']
		rq = data['%r']
		referrer = data['%{Referer}i']

		if rq.indexof('/feed/') < 0  and ( status == '304' or status == '412'):
			#print referrer
			print rq
			print ua
			print status

	      except:
		#sys.stderr.write("Unable to parse %s" % line)
		pass

This is the partial output.

GET /feed/ HTTP/1.1

Netvibes (http://www.netvibes.com/; 58 subscribers; feedID: 1503582)

304

GET /feed/ HTTP/1.1

Alltop/1.1

304

GET /feed/ HTTP/1.1

Xianguo.com 1 Subscribers

304

GET /feed/ HTTP/1.1

Alltop/1.1

304

GET /feed/ HTTP/1.1

Alltop/1.1

304

GET /feed/ HTTP/1.1

Alltop/1.1

304

GET /feed/ HTTP/1.1

NetNewsWire/3.2.15 (Mac OS X; http://netnewswireapp.com/mac/; gzip-happy)

304

All entries came from newsreaders and related bots. There wasn’t a single entry from Googlebot or any other search bot.

Conclusion: No evidence of support.

I know I said I would only cover one example, but I feel like I need to give you a little bit more to get you really excited about this stuff.

Let’s say you didn’t think about looking at the response codes to track ‘if-modified-since’ or that you need to track which search bots support the canonical header element or that you want to know if Googlebot requests compression when making requests. In order to track this easily, you need to log extra header information that is not part of the typical log setup.

This is how you do it:

1. You create a separate log file so you don’t mess up the ability to use log analysis tools that rely on standard log formats.
2. You filter this separate log so it only records the traffic you want to track. In our case, we want to track search bot traffic.
3. You change the log format so it records the additional fields

Here is the partial configuration I used for to perform tests for this post:

SetEnvIf User-Agent ".*Googlebot/2.1.*" gbot

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" \"%{Accept-encoding}i\"" proxy2

# I use CloudFlare to speed up this blog, so I need to record the X-Forwarded-For instead of the reverse proxy IP address

CustomLog "|/usr/sbin/rotatelogs -l /var/www/hamletbatista/logs/googlebot_log.%Y-%m-%d 86400" proxy2 env=gbot

You don’t need to wait for Googlebot to come to the site to test your honeypot. You can use Google Webmaster Tool’s ‘Fetch as Googlebot,’ and Googlebot will come right away. The main difference I’ve seen using this method is that if you provide a URL with a redirect, Googlebot won’t follow it. The regular Googlebot crawler, however, will.

This post is just scratching the surface of all the possible insights you can gain by setting up honeypots to answer your more complex technical SEO questions. If you use this approach and get some really useful results, please make sure to share them in the comments.

I did not get started as an affiliate marketer by promoting information products on Clickbank, but I’m pretty sure the vast majority of affiliates got started this way. Clickbank accepts almost anybody, and their system is the simplest and most straight forward there is. When you notice that many of the “guru” affiliates who are teaching everyone how to get rich online often promote Clickbank products, you can see how so many people become Clickbank affiliates.

The more experience and success you get, the more sophisticated your marketing becomes. Your need for more powerful tools, more intelligence data, and more control grows as your business grows.

While sophisticated affiliate networks like MediaTrust, NeverBlue, and Clickbooth all provide advanced tools that can help you run very complex campaigns, their barrier to entry is very high.  CPA networks won’t accept you unless you are a proven and ethical marketer. They don’t have time for newbies. I believe CPA networks are where most experienced affiliates go once they learn the ropes on sites like ClickBank and Commission Junction.

But there is a new alternative for experienced affiliates. I was lucky enough to be invited to see it in person, and I can confidently say it is the next step in the evolution of affiliate marketing. It is the new and highly innovative network known as TrendRevenue.

The People Behind Trend Revenue Are The Reason It Is So Unique And Exciting!

I worked with many TrendRevenue members back in my glorious pharmacy days. We made a LOT of money, money we didn’t even think it was possible to make (I talked about it in more detail in this interview). In fact, back in the day, they flew me and my beautiful wife to Maui for a week with all expenses paid just to convince me to drop my current affiliate relationship and work with them. They did a great job persuading me, and I am really glad I made the switch.

I managed to get top rankings in Google not only for hot pharmaceutical products, but also for other industries like mortgages, and life insurance. They all paid decently, but only the pharmacy marketing made a huge amount of money. Now that I think about it, the real reason for this is blatantly obvious.

I made most of my money from return customers. When you look at it, people who purchase pharmaceuticals are very likely to purchase them again and again. It doesn’t take a genius to realize that once you hook them with a single sale, you have them as a valuable customer for a very long time.

It also helps that you don’t have to create an aggressive sales pitch for pharmaceutical products that people already know they need. As soon as my pages made it to the top of Google, I just sat back and enjoyed the success. Customers wanted the products, and we happily provided them.

The vast majority of affiliate programs only pay for leads (or first sale), they don’t pay commissions on repeat sales. The backend, as it is commonly known, is where most merchants make their profits. Why would they share their lifetime customer value? The people at TrendRevenue did it in the past with huge success and are preparing to do it again.

They are aiming at replicating their past successes with this model. They have designed an affiliate program that takes all of the hassle out of being an affiliate, giving you more time to do your job as a marketer.  You don’t have to setup a website, and you don’t have to install a thing. You simply pick a theme and decide which products you want to sell. You only need to focus on the marketing aspect.

The truly great thing about them is the fact that you can only sell products that have an excellent reputation for generating repeat business. You can select between Housewares, Ink & Toner, and Pet Products. Nobody who buys these products only buys them once. As soon as you make a single sale, you’ve got a customer for life.

TrendRevenue also offers a level of transparency that is yet to be seen in the online affiliate world. With TrendRevenue, you get to see the frontend and backend revenue for any of your affiliate projects. When most people see how much they are making compared to how much TrendRevenue gets, they usually can’t believe it. It’s actually a problem for TrendRevenue because so many potential affiliates believe it is too good to be true.

I want to take this opportunity to tell you that TrendRevenue is the real deal, and they are pretty serious and determined to make their platform the next big thing in the affiliate space. If you have read my blog for some time, you know I don’t give my endorsement easily.  I need to really believe in something before I recommend it.

Last weekend, the TrendRevenue team provided me with a first class flight to Santa Monica, California. I got a chance to tour their offices and see their other successful businesses. Here are a few pictures I took on my trip.

Bill Appleyard, their CTO and someone who I have worked with in the past, went to pick me up at the airport

You can see the CEO of TrendRevenue in the middle of some pretty famous people!

A real state-of-the-art office. I have more pictures in my Facebook account.

They booked me on a suite in the Beverly Hilton. It’s five times the size of the studio I have in Manhattan right now!

We met on my hotel suite to brainstorm how they can get the word out about their system. I am really impressed with the TrendRevenue team.

If you are reading this blog, you probably have enough experience and intelligence to qualify for their system. There are no affiliate links on this post. If you decide to join them, apply to their program and when you talk to them, tell them you read about them on this blog. It will give you a better chance at getting accepted. It might even help you to negotiate a really nice deal

Disclaimer: TrendRevenue paid for my trip to their offices, but they did not pay me to write this. There are no affiliate links on this post.

But here is the reality. SEO is thriving! I might even go so far as to say it is in its prime. A number of key factors have come together very recently to cause a dramatic uptick in demand for SEO services. Among them are the following:

• Paid search is becoming more and more popular and very expensive as an advertising method
• Google is no longer shunning most SEO as black hat. The company is embracing it with guides, videos, and search engineers who now act as SEO evangelists
  
• The complexity of paid search is almost on par with SEO
• Serious investors are getting involved in SEO

Paid search is becoming more and more popular and very expensive as an advertising method

I still love paid search because of the instant gratification and all the nice tools and information Google provides to create winning campaigns, but being forced to raise your bids or get extremely creative to “stay in the game” is very challenging even for an experience search marketer like me, I can feel the pain and frustration of all those business owners that give up right out of the gate.

The great thing about this for SEO is that once people experience some success in paid search and see how valuable it is, organic search or not having to pay for the clients becomes a rather enticing proposition.

Google is no longer shunning most SEO as black hat. The company is embracing it with guides, videos, and search engineers who now act as SEO evangelists

I never thought I’d see the day when Google’s engineers will be proclaiming themselves as “the real SEO experts”. These are not their exact words, I’m paraphrasing from “SEO Site Advice from the Experts”. Here is the link to the video. As an SEO I’d definitely not want this kind of competition, but I am really happy Google is giving the SEO industry a lot of the credibility it deserves by having Matt Cutts and other Googlers evangelize best practice SEO principles and strategies.

One of my favorite pieces of SEO advice provided by Google is their own SEO score card, where they scored how their own product pages do in terms of SEO. I use the format and some of the ideas presented in their document in some semi-automated and extensive SEO audits I’ve been performing for large e-commerce sites.

Again, nice validation to the usefulness of SEO directly from Google!

The complexity of paid search is almost on par with SEO

Sometimes I feel like Google launches a new feature for Google Adwords every week. Not to mention all the features they have in beta that are only accessible to a small group of advertisers. In short, Google Adwords give the really sophisticated marketer far more tools than he needs to outsmart his competitors, but for the regular business user, the learning curve is simply getting steeper by the day. If you have to put so much effort to learn paid search, why not invest the time to learn SEO and get the clicks for free?

Serious investors are getting involved in SEO

Perhaps the best sign of the maturity of an industry is when outsiders start betting with their money on the success of companies creating products and services to serve the market. Companies such as SEOmoz, DIYSEO, Altruik, BrightEdge, Covario and Conductor have received millions of dollars in funding to create powerful technology that can help companies of different sizes improve their SEO.

As a side note, I accepted a really good offer to join Altruik as their Chief Search Strategist to help with the evolution of their SEO software platform. I’m really excited about the work I will be doing there. I’m relocating with my family to New York next month. I plan to write a more detailed follow up post, but didn’t want to miss the opportunity to thank my SEO celebrity friends Will Critchlow, Dr. Peter Myers, Curtis R. Curtis, Adam Audette, Ralph Tegtmeier, Eric Enge, Sean Golligher and Stephan Spencer. They all wrote fantastic endorsement letters for my O-1A visa petition, which got approved and now I am able to live and work in the US for the next three years. I’m handing my baby, RankSense, to a team to help me operate it while I’m gone. The bottom line is that launching a technology startup with target customers in the US from the Dominican Republic is a terrible idea. Don’t try this at home.

Now, going back on topic …

As always, the prospect of more traffic for little to no cost entices many companies to get involved with SEO. This dream brings them right up to your door, paycheck in hand, ready to hire out your services. Because so many companies see the successful SEO strategies that others are implementing, more and more of them want to get in on the action and compete.

After all these years, SEO is still mostly misunderstood.

While the demand for SEO services is on the rise, the actual services offered aren’t getting any better. Many SEOs are still playing a cat and mouse game with search engines. They are trying to find little rules of thumb and quick fixes, thinking it will amount to more than short term success. The main problem with that is that search engines can and do change a lot their parameters in an effort to find and push more quality content to the top of the results and to kill any spam. I talked about this in one of the first posts I wrote on this blog and I think it is still very valid today.

SEO is not a game. It is a public relations activity. If you understand this very simple concept, you automatically understand 90% of what SEO actually is. I will elaborate more about this in a future post.

A lot of people get this wrong. They want to take the easy way out. If you can show your clients why your competitor has the wrong idea and how you can create legitimate testable results, you will never have a problem making money in the SEO field.

Only the SEOs who can truly deliver will prevail.

Despite the fact that demand for SEO services is at an all time high, much of the SEO world resembles the medieval era. There are too many fragmented ideas, too many witch doctors with magic SEO spells and potions that offer no real guarantee of success. People are all too willing to blindly accept what they believe to be the best practices without taking the time to validate what they hold so dearly to be true.

Don’t think of this as a bad thing. It’s an opportunity in disguise. Companies are tired of the overly polished salesmen who are only good at creating pretty charts and studies that merely look sophisticated on the surface. You can be their knight in shining armor, someone who takes the time to perform experiments and test SEO strategies before declaring which ones work and which ones do not.

In the end, SEO isn’t about following a strict set of rules. It’s about understanding Google’s priorities and aligning your client’s content with them. Simply put, Google makes its money by providing highly relevant top-notch content to its searchers. The best SEO strategy is to do the same.

SEO might be dead for gamers and cheats, but it is very much alive for people who are willing to invest the time into performing useful experiments and using hard data to guide them in all of their decisions. You have an opportunity to step in and help out companies who are in desperate need of real expertise. All things considered, there is no better time to be in the SEO business!

On Friday, we organized the very first RankSense SEO webinar entitled Searching for Results. If you missed the webinar you can access a recording of the live event here…

During this inaugural webinar, Hamlet covered several key SEO topics including:

• Solutions to Common SEO Problems
• Creating successful content
• How to attract successful traffic
• How to use the “Brand Hijacker” technique to attract visitors
• A look at the concept of link opportunity
• Viral Link Building
• How to scale your SEO efforts
• Hamlet’s SEO mind map

The SEO Mind Map
My personal favorite was a look at Hamlet’s SEO mind map. I think this mind map is a good visual representation of what SEO is all about and how a good SEO campaign is developed and implemented. The chart summarizes all of the key steps involved in the often complex SEO process.

More SEO Webinars coming soon…
We will be organizing future webinars as well. Be sure to add your name to our mailing list to receive exclusive invitations to future live webinars. If you have any suggestions as to what type of topics you’d like to see covered, do let us know! You can contact me directly at ednar@ranksense.com. Your suggestions are always appreciated and valued. =)

About Edna Rodriguez – Connect with me on LinkedIn.
Edna is Director of Business Development at NEMedia, the creators of the RankSense SEO suite. A native of the Dominican Republic, Edna was born in Mayaguez, Puerto Rico and grew up in Santo Domingo, capital of the Dominican Republic. Prior to joining NEMedia, Edna resided in Atlanta, GA where she was Senior Program Officer at the Atlanta Women’s Foundation. On her spare time Edna enjoys spending time with her kids, reading, learning about SEO and working on her small business projects.

Apparently Adam Audette from AudetteMedia thinks so. Thanks for the recognition, Adam!

Going Back to the Source

Certainly you can do all sorts of things to your site to “improve” the likelihood that the visitor will stay around longer and perhaps take action eventually. But I think that too much effort and thought is put into driving traffic from as many sources and keywords as possible and in trying to optimize everything that can be optimized on a website. Yet little thought is put into something as obvious as trying to understand clearly what each visitor is expecting when they see your landing page.

Think about it. Visitors land on your website all the time, but the path of clicks they followed to get there, and their expectations once they arrive, are often completely different from visitor to visitor. One visitor may arrive after reading a favorable review of your services from a reputable blog site or online magazine. Another may arrive directly from your Google AdWords ad that promised a discount on purchases “today only.” This is what I call the visitor’s frame of mind. It is a very important concept for conversion: the source of your traffic preconditions your visitor to take (or not to take) action on your content.

Once you really understand how much the visitor frame of mind changes from one traffic source to another, you will place more emphasis and effort in the sources that provide the most valuable visitors. You will conclude, like I did, that the conversion from online advertising is abysmal compared with the quality traffic you get from editorial pieces. Incidentally, the editorial frame of mind is also why I’m a big fan of affiliate marketing: you have third-party sellers endorsing your product and services.

I consider search engine marketing the most powerful way to get clicks that are likely to take action. Remember, high-ranking organic search listings are also perceived as a form of editorial content and endorsement. If Google ranked your site as number 1, people believe it’s worth a try. Ads, on the other hand, are not seen as endorsements, but rather “try at your own risk” advice. Think about your own experience searching. Do you trust the PPC ads on the side of the screen, or do you jump to Google’s “recommendations” first?

Opening day madness

These days, the keywords that convert best are usually the most expensive or the hardest to rank for organically. Savvy advertisers have identified these keywords and start bidding wars that increase the overall cost of acquiring clicks. But you do not necessarily need to get the most expensive ones — you just need to get the cheap ones and lure people back to you a few more times. It’s similar to the way we shop for movies. Sometimes we’re willing to pay more money to see a movie on its opening day at the theater, while in many cases we prefer to pay less and wait for the film to come out on DVD so we can see it a few more times for less money.  You too can get cheap clicks if you are willing to wait for people who are not ready to buy right away.

How? In order to get the action you expect — a purchase, a signup, a download, etc. — the visitor needs to feel like your content presents the ultimate solution to his/her problem. If you solve specific problems affecting many users and gain the credibility and visibility to be a recommended resource, the traffic and the conversions will pour in. This is why I say that conversion rates are affected by the source of the traffic more than you might expect.

The stream of clicks that got the visitor to your site in the first place can tell you what that problem might be. If you select the right keywords to map the problems/needs your potential visitors are expressing, you can better serve them with the content that is going to address those particular needs when they land on the website. That will boost your conversion rate — and your profits.

Another important tactic is to build a relationship over time. Offer a free e-zine subscription, for example, and you can share useful tips with these visitors until you get the person into the right frame of mind. People are primarily looking for content/information, and you can provide that info while slowing building up their desire to purchase from you at a later date. The idea is to trade your time in creating valuable content, tips and advice in exchange for traffic and sales down the line. The visitors will come back to your site with the right frame of mind. This is the approach I’ve taken on this blog and it is working out very well.

Now, I just need to find more time to write/share more useful content! Please let me know what you think of this concept in the comments.

A couple of months ago I found a really clever and easy way to build thousands of links to any site (with your choice of anchor text). In the spirit of sharing I try to promote in this blog, here it is…

PAD your link-building regimen

There are thousands of software developers that provide their solution as a software download (RankSense is one of those), but getting visibility for your solution is an uphill battle due to the large number of competing products in any software niche. One of the easiest ways to get a lot of visibility is to offer a free version of your product and syndicate it through free software download directories.

Now, there are thousands of such directories and submitting your software to each one manually is definitely a pain. Fortunately, there is already a solution for that. All of the software distribution sites support a standard syndication format for content similar to RSS. It’s called PAD (Portable Application Description). Essentially, it is an XML format with all the details about your software, your company, and so on. And most important of all for us marketers, it includes a space to provide our company/product URL and description. In most cases, both result in a link with the anchor text you specify.

You are probably going to say that you don’t have a software download to distribute. Here are two very simple options for you:

1.      Hire a programmer on Elance or Guru and have him develop a very simple tool (like a calendar or to-do list) or enhance an existing open source one. If you search diligently, you can get lots of proposals for anywhere between $50 to $200 bucks. Most of the software directories—at least the ones that are most useful for link juice—review each submission, so if you want to increase your chances try to create something simple but original.

2.      Turn your blog’s content into an e-book in executable format using one of the popular e-book creators. This strategy is even easier and I guess most of you will go this route, but I need to warn you that you will have less success with the directories that carefully review submissions.

After you have your “free new software” you need to create the syndication file—the PAD file. There is a free tool that you can use for this called <a
href=”http://www.asp-shareware.org/pad/padgen.php” target=”_blank”>PadGen. Simply fill out the forms, which are self explanatory, and let the software generate the PAD file for you. Pay special attention to the fields for your link and company name. Those are the ones that control the link back to your site and the anchor text. Upload the file to your server, and submit the URL to the directories.

Anchors away

As a regular reader of this blog, you probably understand the importance of mixing your anchor text. Getting thousands of links back with the same anchor text is not a good idea. So here is a clever strategy you can use to control the anchor text:

Write a simple PHP script and map it to your PAD file URL using Apache’s ScriptAlias, or map the .xml extension (if you don’t have other files with the same extension) or do something similar. The idea is that the script must intercept the request and randomly or serially alter the anchor text in the XML with the values of a predetermined list of anchor text that you want to target. Performing basic keyword research will give you a suitable list. The goal is to have every submission use a different (but relevant) description for your company/product. To be on the safe side, I suggest you always leave the product name with the keyword-rich description. I also recommend that in some of the variations you use only the product name and not a keyword-rich description. This will make everything look much more natural.

This sounds like a lot of work, but really it isn’t. Syndicating software is not very different from syndicating content.

All that’s left is to find a big list of software directories to start submitting your PAD file to. A search in Google for “software downloads” returns more than 70 million results and all I see are software download sites, so the sky is the limit! I recommend you don’t include your personal e-mail in the PAD but create one that you can actively monitor because many of the sites will e-mail you additional steps. I’m sure spammers set up fake download sites to harvest e-mails so be careful with that, too.

There are software tools that can automate the submissions too; do a quick search in Google and you will find some really useful ones.

Happy link hunting!

Update: I’m glad to learn that SEO expert, Nick Wilsdon, has been using this technique for years! Check  out his really useful post for more useful tips

The problem is scammers who claim to be SEOs and who disrepute all of us with what they do. Wherever we draw the line between who is and who isn’t a “real SEO” is not important. There is no central authority or watchdog group that validates which SEO claims are true and which are false. But—and this is a big but—there is one way we can turn the tables. You guessed it: unlike scammers, we can guarantee our results.

A Knock at the Door

It’s well known that many people at the moment associate the term SEO consultant with snake oil salesmen. This is in large part thanks to people that don’t have a clue about SEO, but do know a lot about telemarketing and how much money can be made by proffering SEO services. Picture somebody coming to you and offering an irresistible service that will send mountains of customers right to your door; he’ll charge several thousand dollars for the service and you need to wait six months to see the results, which he doesn’t guarantee. Would you buy it?

Actually, a lot of people do buy into it…and that’s a problem for us. By offering our services with no guarantee at all, we are encouraging others to do the same. It has opened the door for scammers to prey upon innocent people. When a site owner comes into contact with an SEO, how can he tell the good ones from the bad ones? Most small business owners don’t attend search marketing conferences and don’t know who Danny Sullivan or who Aaron Wall is. They do know that their inbox is flooded and they get called maybe five times a week from different companies pitching SEO services.

You don’t need to know SEO to confirm whether a practitioner is providing a useful service or not. Just measure words against tangible results. How many extra visitors and/or sales/leads have resulted from the work of this SEO? Many will say that SEO has nothing to do with sales/leads/actions/conversions but, as I have laid out before, I strongly disagree. And I’m not talking about conversion optimization, which I consider a separate concern from SEO. If the consultant can show only rankings as proof, run away. Why? Because rankings alone are not useful unless they result in an increase in valuable traffic.

If you read my previous post, I think you’ll agree that rankings are not the best way to measure the success of an SEO campaign. There has to be a bottom-line benefit for the site owner, and that will depend on the specific goals of the campaign. Is it to increase awareness and trust of the brand, or to generate leads and sales? At the end of the day, the campaign is successful by reaching those predetermined set goals. This makes the SEO campaign valuable, both to the customer and the SEO providing the services.

A Line in the Sand

What I am trying to get across is that we can (and should) guarantee results—the right kind of results—because a well-executed SEO campaign will certainly drive more search traffic and affect conversions. This is true for at least a couple of reasons:

1.       With proper SEO you can pretty much define the right audience for a product or service. That is, if the consultant does a good job and selects the right keywords, those visitors should already express an immediate interest in whatever it is they are expecting on the landing page. Choose the wrong keywords and the results and conversions are going to be drastically the opposite.

2.       People tend to trust editorial pieces more than ads. That being said, when someone does a search in Google they see “editorial content” on the left (which is the result of SEO) and they see paid ads on the right (the result of PPC). Which content do you think they trust more? Which will result in higher conversions more often? Remember, most people trust Google, and that trust is conferred upon the search results. This is why Google is so protective of their users; every search result is perceived as an endorsement.

The larger point I’m trying to make is that we shouldn’t be fighting amongst ourselves about different hat colors. It doesn’t matter if you’re a white hat, black hat or gray hat if at the end of the day you’re providing results and the customer understands the risks of your methods. No matter what hat color we wear, we should be fighting together against the scammers that are infiltrating and destroying our industry. Our real concern are those people scamming our potential customers and not providing any value at all. We need to set a clear line in the sand that separates “us” from “them.” That is why I believe the smartest thing to do is guarantee our results—not top rankings—but measurable results.

If you get a customer and you don’t think you can provide measurable results, then don’t take the job. There will be plenty more knocking at your door because, when you deliver on what you say and exceed your customers’ expectations, they will come back to you for more and refer you to others.

Okay, I’ve said my piece. I’d appreciate your thoughts on this in the comments.

A better way to check search rankings

I definitely feel the pain of those that need to rely on tools that are constantly getting blocked by Google. But there is no need to get Google mad at you and block your IP. I learned early in life that there are always many different ways to achieve the same goals. You can get far more information about your rankings, for instance, directly from your traffic log files. You can determine all the keywords that you are ranking for, their relative positions, the number of visits each keyword is sending, and, with the IP address, you can also determine the physical locations where searchers are coming from. We’ve had this functionality built in to RankSense since we started developing it four years ago! Plus, you don’t need to query Google to get this information. (BTW, the new 2.0 version of RankSense is coming out, and the Discover Rankings tool can now detect conversions as well. That way, you’ll be ranking first for the keywords that actually bring in the money!)

The problem with fixation on ranking reports

But even if you can still check your rankings, this belies a notion that SEO is all about moving a site to the #1 spot in the search results. There is a camp that thinks ranking reports like this are important because customers are asking for them; the other camp argues that it is a waste of time because different people get different search results, and that the success of an SEO campaign must be measured in another way, like traffic and conversions.

There certainly are some fundamental problems with letting your customers get fixated on moving up in the rankings. Let me explain a few of them:

1.       A #1 ranking for a keyword means nothing if the traffic that derives from it does not give you the desired results: leads, sales, downloads, sign-ups, etc. Is the customer clear on his or her specific goals for the SEO campaign? How does he or she know a ranking for “blue widgets” will help achieve those goals? Has the keyword phrase been tested with PPC? Those are more important questions.

2.       Rankings can move up and down for a multitude of reasons, and fixation on those fluctuations can make any normal human being crazy. For example, Google delivers different results based on the physical location of the searcher, the language setting, the search history, and so on. That makes perfect sense because people living in different places and having different interests would consider relevant things that other people in other places do not.

3.       There are more important metrics of success for an SEO campaign. We should not let the client forget the main purpose of SEO, and ranking for “blue widgets” should not be the end goal in and of itself. As consultants, we need to be the ones telling clients which keywords are going to produce the best results, not rankings.

Does this means that trying to rank #1 is impossible or a bad idea?

I personally don’t think so. Fighting for #1 is like trying to be the best at what you do, and it is well known that #1 rankings bring with them a huge amount of benefits. They receive more than twice as many clicks as the second result (provided it doesn’t have a crappy snippet) and being #1 is a sign of trust to searchers. Users trust that if Google recommends the site as #1, it has to be good.

Also, while it is true that for many keywords the location in the search results might shift for different users, for a great many searches this is not the case. Consider navigational or brand searches. People doing brand searches are expecting the same thing and it doesn’t matter where in the world they are, their language or their search habits. A search for RankSense anywhere in the world should hopefully bring you to my RankSense software! For many searches, trying to be #1 (or on the first page) for queries that bring in money and don’t necessarily vary much from searcher to searcher is a very good idea, IMO.

My SEO Key Performance Indicators

As I said, rankings are not a measure of success unless they result in tangible goals such as traffic and conversions. That is why my favorite KPIs for measuring/tracking the success of my search campaigns are:

1.       Search traffic growth and corresponding conversions. I want to know week-to-week and month-to-month how my organic traffic is growing and, more importantly, how many conversions I’m getting from the overall search traffic.

2.       How many rankings I have on the first page vs the rest of the SERPs. As I said, I can derive this information from the traffic logs. Getting more visitors from the first page of results than from page two downwards means that I will be getting more traffic, so this indicator supports the first. In addition, I want to make sure my most profitable keywords are on the first page. That needs to be a priority because that will drastically improve my results.

3.       How many new incoming links I am getting every week/month etc. If I were the obsessive type, instead of fixating on rankings I’d fixate on how many new links I was getting. Getting links and increasing the visibility of the site will result in deeper index penetration (because the site is considered more important) and the ability to rank for more competitive terms that can potentially result in a lot more traffic. It could also help me adapt my content development strategy to focus on more competitive terms.

4.       How many pages I have in the search engine index vs. the total crawlable pages of the site. Last but not least, the easiest way to start getting search traffic from long-tail queries is to create lots of quality content and promote that content. Every page we manage to get into the search engine index is another source of search referrals, so making sure all or most of the pages are getting indexed is a winning strategy.

There are several other metrics I use to measure success, but for me the most critical/important ones are these four. Do you use any KPIs to measure the success of your SEO efforts? If you do, please share them in the comments.

This is a guest post by Paul Burani from Clicksharp Marketing, a very sharp search marketer I met in NY

In the search engine world, it’s easy to think in terms of black and white.  Some traffic you pay for, some you don’t.  There’s page one, and there’s everything else.  And of course we use the terms “black hat” and “white hat” SEO to differentiate the practices which push (or exceed) the limits of what is deemed acceptable by each search engine’s terms of service.

This view often creates a temptation to pursue Search Engine Optimization at all costs, when in fact it may be an ill-advised strategy.  What would lead a CEO, marketing manager, publisher or webmaster to make a dubious investment in SEO?  In many cases, this is based on the simplistic notion that clicks from Pay-Per-Click (PPC) advertising come at a significant price, whereas in organic search, they cost nothing.

The idea that SEO is free is patently false.

What you’re not paying in Cost Per Click (CPC), you’re paying in time: the time required to generate and/or syndicate content, format your markup properly, research competitors, and stay current on trends in search engine algorithms.  We all know that time is money.  And if you’re outsourcing any of the above, you’re paying money on top of that money.

So whereas in PPC you build your campaigns around a certain monthly ad budget, with SEO your budget is effectively measured in content and not dollars.  Writing more content and creating more pages increases the reach of your site.  It broadens the spectrum of keyword queries for which you can be reasonably competitive — from the high traffic core keywords to the “long tail” terms with comparatively low volumes but high upside (in the form of conversion potential).  This creates a very powerful virtuous cycle:

• Googlebot and other search engine crawlers have more content to crawl, and begin to view your domain more favorably.

• A growing number of unique users find your site through their own natural search behavior.

• A certain percentage of these users link to you in the process of generating their own content.

• Your steadily growing network of inbound links improves your authority in search engines, indirectly improving your crawl rate, your search engine visibility, and the overall stature of your website on the internet.

If you haven’t yet gained any key insights from your PPC campaigns, SEO may not be as valuable.

Another myth about SEO is that it’s a dark science, driven by a small handful of hyper-intelligent individuals who stand around a bubbling cauldron sipping nerve tonic, talking about how much smarter they are than Google.

That’s far from the case.  While it is true that an SEO campaign is driven by basic competencies in content generation, keyword and competitor research, server dynamics, usability and so on — ultimately SEO is about running a business.  The same way an offline business might invest in a billboard placement to pull in new customers, search engines are simply a newer medium for practicing all the same rules of business.  Thanks to all its campaign administration tools, and support from the major search engines, PPC tends to be more generally embraced as a business development tool.  For instance, measuring and enhancing Return On Investment (ROI) in PPC has evolved much more quickly than in SEO.  Maybe that’s why the industry doesn’t consider Search Engine Optimization (SEO) to be a part of the broad strategies of Search Engine Marketing (SEM)?

Making the transition from PPC to SEO, however, is easier than you think.

If a PPC campaign is well-designed to attract a wide variety of clientele, and has been running long enough to have statistically-significant volumes of impressions, clicks and conversions, the next step is to skim the cream off the top of the campaign and create the foundation for an SEO campaign.

Imagine you’re running your favorite analytics program, looking at a list of all your PPC keywords, sorted by conversion rate.  Once you’ve filtered out a few outliers with inadequate sample sizes (go back to your Statistics 101 textbook if need be), you find maybe a dozen keywords that are significantly outperforming the overall campaign.  (Note: in PPC/SEO-speak, both single words and multiple-word phrases are referred to as keywords.)

These keywords now become the bedrock of your campaign to migrate business potential from PPC to SEO — we’ll call them the Dynamic Dozen.  Your copywriters are instructed to create content which repeatedly brings the Dynamic Dozen to prominence.  Your developer or webmaster organizes supporting elements of the markup (e.g. title tags, alt tags, meta descriptions, etc) accordingly.  Your PR firm (or guerilla link builders) generate inbound links using the Dynamic Dozen as rotated anchor text.

You’re acting on a very simple premise: once upon a time, the people who searched for those core half-dozen terms found you and became customers — and other people would do the same, regardless of whether they found you in the sponsored results or the organic results.

Checking your rankings in the search engines for the Dynamic Dozen might seem like a worthy benchmark for success, but it’s important not to give too much credibility to this metric.  A better strategy is to become very friendly with your analytics program, and monitor your search engine traffic on a regular basis.  Trends in absolute numbers of unique visits will give you a very general idea of the SEO campaign’s progress, but for a more objective view, ask yourself questions such as:

• How much traffic is being generated specifically by the Dynamic Dozen (including phrases that contain any of the these terms)?

• The people who reach my site through these keywords, do they develop a better-than-average relationship with the site content?  Do I see a lower bounce rate, higher pageviews per visit, growth in repeat visits?

• Are the Dynamic Dozen converting at the same rate as they were in PPC?

• Here’s the million dollar question — for the amount of time and money I’ve invested in SEO, is my organic search traffic reaching the same rate overall ROI as it was in PPC?

How would I begin to estimate the cost of an SEO campaign?

Understanding the return on your SEO investment can be tricky, but as always, you have a lot of objective data to use to your advantage.

1. Costs.  Come up with a reasonable estimate of the value of one hour of time for each of the key people involved: copy writers, webmasters, link builders, etc.  There are a variety of very cheap (or free) time tracking tools which facilitate this kind of record-keeping.  Add to this any costs for submission and/or syndication.

2.Assign value to conversions.  These numbers can usually be input directly into your analytics program.  If you’re selling product directly, the value is measured in profits.  If a conversion is represented by something less concrete, such as a white paper download, use your own business data to calculate the rate how many white paper downloads it takes to generate one new customer — and how much that new customer is worth.

3. Competition.  You can set a reasonable benchmark for ROI by isolating a few key competitors and studying their SEO campaigns.  Count up all the inbound links pointing to an important page on their site (you might also apply a filter such as PageRank, to account for the varying levels of inbound link quality).  Using your own human resources as a model, estimate how much it would cost to create and promote that content, and divide by the number of inbound links.

Once you’ve accomplished this, an objective view of ROI is your reward, and you’re essentially left with one of two results.

1. The ROI is lower than what you’ve observed for your PPC (or lower than what management has required).  In this event, you’ll need to return to your analytics software, extract the laggard keywords, and adapt both existing and new content toward better performance for these terms.

2. The ROI is higher — and which case you’ve earned a pat on the back.  You now have a mandate to return to the C-level executives asking for more budget.  Or if they don’t want to invest in marketing, maybe they’d like to invest in a big bonus for their new marketing guru.

There are many compelling reasons to invest in SEO.  Though it is not as “free” as many people believe, much of the intelligence needed to manage your campaign already exists in the form of PPC data.  Transferring these assets to the organic search channel, and measuring their impact on the bottom line, are a viable marketing strategy for many businesses competing in the digital economy.

So when is this a good idea, and when is it not?  Consider these criteria:

• Are you prepared to spend human resources, as well as your marketing budget, on SEO?

• Can you come up with an ongoing list of themes to use for developing new content?

• Have you gained a clear understanding of which keywords are convert your website’s visitors into customers?

• Do you regularly use your analytics software to measure your marketing investments?

• Do you have a good sense of which competitors are doing the same thing?

If the answer to any of these questions is no, you may not yet be ready for the spoils of Search Engine Optimization. But if you believe your marketing agenda is prepared to meet these challenges head-on, the upside is yours for the taking.

[Editor's note: BTW, don't miss my interview at Meetinnovators ]