How accurate is our data?

Observed Data

Estimated Data

† Accuracy of Click-Through-Rate and Cost-Per-Click Estimates (CTR & CPC)

These vary from client to client, as well as between our clients and their competitors. We measure the accuracy of the estimated client CTR and CPC on a daily basis. At present the mean absolute percentage error of estimates for CTR is roughly 12%, while the mean absolute percentage error of the estimates for cost-per-click is approximately 17%. Measuring the accuracy of click-through-rate and cost-per-click estimates is a more difficult problem and is currently under work.

†† Accuracy of Impressions and Clicks

To estimate the number of impressions that an advertiser obtains for a given search term, we take the search volume from Google for that term* and then multiply it by the frequency with which the advertiser’s advert is displayed. We use the number of times we observe a specific advert to estimate the frequency with which an advertiser’s advert is displayed on that search term. To estimate the number of clicks we take the estimate for the number of impressions that an advertiser will receive on a search term, then multiply it by our estimate for the click-through-rate of that advertiser on that search term.

*This figure is provided directly by Google

Observing Competitors in the market

For each search term, how likely are we to see a given advertiser in a day’s worth of scrapes? This depends on various things, such as the number of advertisers in the market, the likelihood that a given advertiser will appear on the search term and the number of times we scrape the search term in a given day.

If all advertisers on the market are equally likely to appear on the given search term, then the likelihood that we will see a particular advertiser on that search term is given in the following table.

Alternatively, if an advertiser appears on a search term for a certain percentage of the time, then the likelihood that we will see a particular advertiser on that term is as follows.

Given that Google restricts advertisers’ appearances each hour, depending on their budget, it is crucial to scrape as many times per day as possible to ensure that smaller advertisers are picked up accurately.

Example: if an advertiser appears 50%+ of the time per day on a specific search term then there is a 99% chance that Adthena will see them. However, if they appear only 10% of the time then the likelihood of seeing them drops to less than 60%. If it is a competitive term with more than 10 Advertisers appearing on page 1, then we can increase the scrape frequency to improve the likelihood of seeing the smaller advertiser from 60% to 80%.

Competitors’ Advert Duration

When a competitor runs advertising on a search term, how accurately can we estimate the length of time that they ran the campaign? Again, this depends on factors like the number of competitors in the market, the likelihood that the competitor will successfully appear on the search term and the number of times a day that we scrape that  term.

Assuming we scrape the search term 16 times a day, and that the campaign is running for two weeks, then on average we obtain the following accuracy.

If we know the likelihood with which an advertiser will appear on the given search term, then we will typically obtain the following accuracy.

Search Term Overlap

If you and a competitor are bidding on a group of search terms, then on what percentage of these overlapping terms are we likely to see the competitor? Assume that there are 1000 overlapping terms and we scrape each term 12 times a day. In the following table we give the 95% confidence interval of the percentage of the overlap for which we will see the competitor on a given day. The table gives difference confidence intervals dependent on the likelihood that we will see the competitor on a given scrape.