Ask HN: statistical significance on group of Adwords keywords

1 points by tucson 13 years ago | 1 comment

I am trying to optimize my adwords campaign, and running into the 'statistical significance' question.

I read this great post from Paras Chopra here: http://visualwebsiteoptimizer.com/split-testing-blog/ab-testing-significance-calculator-spreadsheet-in-excel/

Though in my case I started my campaign with many keywords (about 400 keywords), and I now have the following:

Keyword group A (10 keywords - the "best"): 47 clicks, 16 conversions

Keyword group B (390 keywords - the rest): 244 clicks, 18 conversions

If I apply these data into the A/B Testing Significance calculator, I obtain a 99% confidence that A is better than B. But does it really work with a group of keywords, or is this calculation meant to be used only for strict "singular" test cases?

VorticonCmdr 13 years ago |

You can use it this way. But what does the data tell? Group A is definitely performing better than group B or better it is not a random coincidence that group A is better performing.