How to do a Domain Traffic Test – What are you measuring?

“What are you measuring” seems like an obvious question when you are conducting a traffic test with a new monetisation company. From experience, most people get the answer to this question completely wrong.

When you are running a traffic test you are NOT seeing if you can earn more money. This may sound strange but it’s the simple truth. What you should be actually measuring is the capabilities of the new company to optimise your traffic for a better result. More money (although important) is merely a by-product of the optimisation activity.

For example, we recently had a client place their domains with ParkLogic for optimisation and as time went by we just couldn’t beat their numbers. In fact, it seemed that the more effort that we put into the optimisation the worse thing became.

It was at this time that I had an epiphany (ie. a brainwave!). It just so happened that the client provided their baseline data in Euros from back in August and we were being benchmarked against these numbers. Since the ParkLogic system uses $USD we dutifully converted the baseline data from Euros to $USD when we received it.

The below graph tells you the story of the relationship between the Euro and USD during this period of time. The Euro has fallen from $USD1.36, when we did the initial conversion for the baseline data, to $USD1.09 (bank rate). That was a fall of 25%!

So the fact that we were providing the client around a 5% uplift wasn’t the real result at all. The real result was 5% + 25% = 30%. To clearly show this we applied today’s exchange rate to the baseline data so that we could clearly measure our optimisation efforts and nullify the impact of the exchange rate.

Another example is we were optimising a domain portfolio and the results were absolutely stellar! In fact, we were beating the baseline data by about 250%. Naturally, the client was really happy with the results but something was annoying me about the numbers….

A quick comparison analysis showed that one domain was getting a number of clicks paying $80+ each. We removed the impact of this domain as we did not believe that it was sustainable and redid our analysis. The uplift was still around 135% but I was more comfortable with this number being sustainable.

In both these examples, only paying attention to the money earned column would cause you to completely miss the actually impact of the optimisation effort. It’s not just the data but interpreting the data that is critical to getting the most out of your domain portfolio.

Another simple example is when you have domains that are seasonal in nature that affect the result. These should be removed from any analysis to determine whether the new company is actually adding value. They have the potential to either inflate or deflate the numbers and cause you to come away with a completely incorrect picture of the new monetisation company.

I live a breath numbers all day every day and I would like to say that I’ve seen everything…..but I know that I haven’t. When measuring the impact of any change you must first of all have a clear baseline and then have a metric that you can actually use to measure the impact of that change. It’s the art combined with the science that produces results….and both are constantly evolving.

In a future article I will begin to unpack what metric is critical for determining whether a new monetisation company is performing or not.

---------------------------------------------------

Michael Gilmour has been in business for over 32 years and has both a BSC in Electronics and Computer Science and an MBA. He was the former vice-chairman of the Internet Industry Association in Australia and is in demand as a speaker at Internet conferences the world over. Michael is passionate about working with online entrepreneurs to help them navigate their new ventures around the many pitfalls that all businesses face.