The Web has fundamentally changed the business of advertising in just a few years. So it stands to reason that the process of creating ads is bound to change too.
The persuasive power of a message is a crucial ingredient in any ad. But settling on the best combination of words is at best a black art and at worst, little more than guesswork.
So advertisers often try to test their ads before letting them out into the wild.
The traditional ways to test the effectiveness of an advertising campaign are with a survey or a focus group. Surveys are shown to a carefully selected group of people who are asked to give their opinion about various different forms of words. A focus group is similar but uses a small group of people in more intimate setting, often recorded and watched from behind a one way mirror.
There are clear disadvantages with both techniques. Subjects are difficult to recruit, hard to motivate (often requiring some kind of financial reward ) and the entire process is expensive and time consuming.
What’s more, the results are hard to analyse since any number of extraneous effects can influence them. Focus groups, for example, are notoriously susceptible to group dynamics in which the view of one individual can come to dominate. And there is a general question over whether recruited subjects can ever really measure the persuasiveness of anything.
Then there is the obvious conflict created by the fact that a subject is not evaluating the messages under the conditions in which they were designed to work ie to get the attention of an otherwise disinterested reader.
So there’s obvious interest in finding a better way to test the value of persuasive messages. One approach is to use crowdsourcing services such as Mechanical Turk to generate an immediate readership willing to take part.
But Turks are paid to take part. So the results are no better than those that conventional methods produce, although they are cheaper and quicker to collect.
Today, Marco Guerini at the Italian research organisation Trento-Rise and a couple of buddies say they’ve found an interesting way round this: to test messages on Google’s AdWords service.
The idea here is to use Google Adwords to place many variations of a single message to see which generates the highest click through rate.
That’s a significant improvement over previous methods. The subjects are not paid and make their choice in the very conditions in which the message is designed to work. And the data is quick and relatively cheap to collect.
Google already has a rudimentary tool that can help with this task. The so-called AdWords Campaign Experiments (ACE) tool allows users to test two variations of an ad side-by-side.
But to really get to the heart of persuasiveness requires a much more rigorous approach. Guerini and co make some small steps in this direction by testing various adverts for medieval art at their local castle in Trento.
These guys used Google’s ACE tool to test various pairs of adverts and achieved remarkable success with some of their ads. One ad, for example, achieved a click through rate of over 6 per cent from just a few hundred impressions–that’s an impressive statistic in an industry more used to measuring responses in fractions of a percent.
However, this click through rate was not different in a statistically significant way from its variant so there’s no way of knowing what it was about the message that generated the interest.
So while Guerini and co’s experiments are interesting pilots they are not extensive enough to provide any insight into the nature of persuasive messaging. That will need testing on a much larger scale.
These will not be easy experiments to perform and present numerous challenges. For example, the process of changing the wording of an advert is fraught with difficulty. Then there is the question of whether this method is able to test anything other adverts designed for AdWords. It might have limited utility for testing the messages in magazine adverts or billboard posters, for instance.
But the important point is that these kinds of experiments are possible at all. And it’s not hard to imagine interesting scenarios for future research. For example, AdWords could be used as part of an evolutionary algorithm. This process might start with a ‘population’ of messages that are tested on Adwords. The best performers are then selected to ‘reproduce’ with various random changes to form a new generation of messages that are again tested. And so on.
Who knows what kind of insight these kinds of approaches might produce into the nature of persuasiveness and the human mind. But we appear to have a way to carry out these experiments for the first time.
Ref: arxiv.org/abs/1204.5369: Ecological Evaluation of Persuasive Messages Using Google AdWords