Qualified Understanding

18 May 2023

I was always told that the most accomplished acrobats, the best ice skaters, were the clowns. You had to be really good at the basics to hurl yourself around without hurting yourself. You had to be smart to appear to be an idiot, in other words.

Knowing the basics should be fundamental. In our world, where does the data we all take for granted come from? How is it collected? What are the assumptions behind it?

How can anyone advise anyone to do anything without at least a basic understanding of the tools we all use to make recommendations, or spend someone else’s money?

We are all asked to do surveys. All the time. Next time you do one just stop and ask yourself how much thought you’re giving to it; how accurate you’re being; or how easy or clear the questions are to answer.

Remember – someone is taking action, or spending money on the basis of what you say.

Professionally, it’s never a bad idea to ask if it would be possible to complete the questionnaire or task as a respondent. Obviously sometimes it isn’t practical but whenever I have been allowed to do this it is inevitably revealing.

A few years ago, a service appeared that claimed to measure, via an online after-the-event questionnaire, all exposures to every piece of branded communication. Taking a beer brand as an example, the claim was that one study allowed planners to compare the efficacy of beer mats with pump decals, TV commercial, sponsorships, social media, OOH. All on a like-for-like basis. All based around what the respondent claimed to remember.

We asked if everyone at the client, plus the agencies could take the questionnaire. The research supplier wasn’t over-joyed with the notion, but we insisted. Our experience allowed us to put the results into context. In short, and to be polite not to take them too literally.

On the flipside early in my career I sat as an agency representative on the BARB Technical Sub-Committee. For a technical reason to do with boundary changes some panel homes were going to be dropped by BARB. Their data was already being discarded.

We were offered the chance to visit a few of these homes, to meet some ex-BARB panel members. At this stage they were unaware that their responses were no longer included in the study.

What struck me was how incredibly seriously these people took their task. They believed they had a responsibility for shaping TV, for deciding what was recommissioned, what dropped. They were of course to an extent right.

When faced with a bunch of cynical London-based ad people they were shocked to think that we thought they made up the data or didn’t answer the questions they were asked accurately and in a timely fashion.

It may be considered nerdy by some to worry about audience measurement data, or brand metrics but I don’t see how agencies can do their job properly without understanding where the basics come from, and what they mean.

I also think advertisers should, prior to spending any money insist that the people they’re spending it with have at least some degree of understanding of the data they’re using to guide their decisions.

After all, financial advisors have to take regular exams to update and keep their qualifications. Why aren’t agency media people responsible for investing large sums of money subject to the same scrutiny?

We could create a system where different activities, like taking an exam, completing a course, attending a conference all generate points towards a qualification. The points would lapse after so many years, meaning you need to keep up-to-date.

An agency would only qualify to handle a client’s business if they could show they have an aggregate number of points amongst their teams. If they so wish, agencies can be approved to run regular internal courses thus making the whole thing easier to participate in.

Right now what we’re doing in spending our clients’ money is in essence educated guessing, what we think likely to work best. In fact, it’s not even that educated as a lot of the time we don’t really pay much attention to the source of the data we’re using.

The fact that clients’ trust in agencies has declined is well-known. The growth of in-house teams, and the direct sales efforts of the major vendors threaten agencies’ very existence.

A scheme to ensure a level of understanding in the tools we use would help give clients’ back their confidence in the professional capabilities of media agencies.

Raising professional standards in understanding would impact all aspects of the media industry positively.