This is the second in the series on running a traffic test. The first may be viewed by clicking on the link below:
Part 1 – Traffic Test – Baseline Data
One of the challenges of running a traffic test is to determine which metrics should be used for comparison purposes. I’ve had so many people tell me that all that matters is revenue…..I beg to disagree.
Although revenue is a very important metric it is not the only metric that should be paid attention to. For example, let’s imagine the baseline data for an education related domain is from May and the traffic test started mid-June.
Many education advertisers wind down during June and therefore the pay-out rates per click will typically be less for this type of traffic. In addition, depending upon the type of education domain the traffic is likely to be less as well.
What we need to do is to interpret what the data is telling us and not just look at it and say, “The revenue is down, I’m pulling the test.”
One of the challenges in running a traffic test is using views as a comparison metric. Because every provider counts views differently then using this metric can end up becoming quite subjective....that being said, sometimes it's all that is provided in the baseline data so it's better than nothing.
Many years ago, I was sitting in a meeting with a person that controlled a very large domain portfolio. They were running a traffic test with us and we were taking them through our typical analysis and discussing what was happening at the individual domain level. This surprised the prospective client as they said they’d never done that before.
This brings up a VERY important point. Domains are unique and should be treated that way. A portfolio test may not be performing overall due to a few domains pulling the results down. Our advice would be to re-baseline those domains and then re-compare the results.
As an aside, we provide this advice because we are focused on what is best for the partner and not what is best for our own pocket. It’s a long-term view where if we do what’s best for the partner then they will trust us to always provide them the best advice into the future.
Let’s take a look at a real live example of a portfolio going through an optimisation process (all data is anonymized). Since the partner did not provide baseline data, we had to let the portfolio sit for the first week and send all the traffic back to their existing parking solution. This then provided a daily baseline of $52.19. On the 11th June we then began optimising the portfolio.
Even after so few days the high-level results are quite interesting with an overall improvement of 18%. If all of the domains that were performing at ParkLogic were left with ParkLogic and all the non-performing domains were moved back to where they were then the improvement would be 42.6%. Even after such a short amount of time this is an outstanding result….the optimisation has barely kicked in as yet.
What’s more, if we removed the ten worst performing domains from the analysis, then the improvement on the ParkLogic platform is 35%. If we sent all the non-performing domains back to where they came from then the revenue improvement is 49.3%.
What this shows is that getting involved at the domain level makes a considerable difference to the performance of the entire portfolio. For obvious reason, we focus our efforts on the worst performing high-value domains to achieve the maximum benefit or our partner.
As time goes by, we anticipate that we will see a consistent upward trend for the partner as the algorithms migrate the traffic to where it will be paid the most. Since every provider has different filters on their traffic, we also see the traffic recognition rates increase. This is because traffic is automatically migrated to those providers that are recognising and are willing to pay for it.
In the next article I will discuss some of the metrics that we pay close attention to and take a look at some of the other measurments in this live example.