A/B Testing Trango.co
This article was originally published on Trango.co by the author.
When we launched Trango, we wanted to keep a very minimalist theme to try and give the greatest weight to the content and this proved to be a very successful strategy, by the end of our first month we were getting 2,000 visits per day. Its at this point we started thinking about how we want to take the website forward. We realized we had a lot of GREAT content that wasnt getting a lot of attention becuase it wasnt on the home page anymore, and the theme we had been using upto that point was excellent at attracting content, but mainly to the newest articles we were publishing. This meant great articles such as 10 Signs to Spot an Idiot on LinkedIn and The Making of Artsly were getting overlooked.
At this point we added features such as “Related Posts” that appear underneath each article which helped in improving our conversion rate (more on this later), but we felt it wasn’t enough. We decided that we should try and test whether we could use a different theme that would increase our conversion rate.
Tools
Finding a theme was easy, but then running a test seemed like a very tough call as in my mind I though we would have to make a subdomain where we would then direct traffic, or use a split testing tool such as VWO or Optimizely. All of these actually had their drawbacks, but then i came across some great wordpress plugins that would do the trick. With some quick testing on dummy sites to see how the plugins would work we decided on using two plugins.
The first was to set up the new theme properly called Theme Test Drive, which allowed us to configure the new theme properly with its menus, widgets and fonts.
The second plugin, SES Theme Split Test, was the one that actually split the traffic from one theme to the other and segmented the data within GA. This meant that we could start tracking within GA metrics like our bounce rate, conversion rate, Ad CTR.
Testing Themes
Before I go further into the test I wanted to introduce the 2 themes we had been testing. The original one is called On Topic, and the new test theme is called Memorable. Both are developed from the great hands of Woothemes (which i have used for the last 3 years now and can vouch for their awesome support team)
On Topic — Original Theme
Once we had the themes ready and the test running it was all a waiting game to see which one would have the best results. The primary measure for us is our conversion rate. Conversion rates are a tough thing to measure for blog/news sites. Unlike ecommerce ventures where conversions are easily measured by number of orders generated, or SaaS applications by the number of signups. At Trango we care most about writing compelling content that will create a loyal audience of readers for us. We want our customer to not only read the articles we write but also read a lot of articles, based on this we created two goals in GA.
3+ Page views in a visit
5+ min on the website in a visit
If either of these criteria is meet we could that as a conversion. This measure is somewhat arbitrary but we like it because it helps us see how much our readers are interacting with our content. No one (unless they leave the browser on) would stay on a website for more than 5 minutes unless they found it compelling, similarly people may end up visiting the home page and one article, but unless they found something they like they would not move on to a 3rd page.
Results
So with our conversion goals in mind we can now look at the results (GA results are posted here).
ThemeVisitorsConversion RateAd CTRmemorable114210.95%0.19%ontopic11238.51%0.15%
Both theme got very a similar number of views over the course of the test, but their Conversion Rate and Ad CTR (click thru rate) are significantly different (at the 98% level to be exact). An extra 2.44% of visitors were more likely to convert (either visit 3+ pages or spend 5 min on the website) and the ad CTR went up by 0.04% which over the course of a year could be a significant amount of revenue.
Just the numbers alone shows that the new theme is performing better than the last. Further to that we were also, from our readers and writers, getting good feedback about the theme which is always good to hear. So with all this in mind we at Trango would like to present to you our new look and hope you enjoy it
