Google Blogoscoped

Friday, February 15, 2008

The Presentation Paradox

There’s a problem with presentations by web agencies done for clients, and it leads to the problem of overly complicated interfaces on the web. We can define the Presentation Paradox as this:

The effectiveness of a presentation is inversely proportional to the effectiveness of the tool presented.

Now this is more of a general observation so it doesn’t apply to all cases. Not all websites are tools, for starters – a tool as in: a hammer, a search engine, a shopping form, and so on –, though a lot more are tool-like than we may think. Also, sometimes clients or the team are so clued-in to what is presented that they can sense true effectiveness. But we need to keep in mind that clients are often not that technically adept, or they might have done the implementation themselves.

To explain the paradox, let’s use an example. The goal of the given hypothetical site is to sell more products, and the client envisioned a “related products” tool for visitors. We’ll peek into two alternate realities below. The left is a solution from an agency which envisions to print a simple link to a related product on the product page. The right-hand solution from another agency is to have a full-featured and visual Related Products Explorer application. On button press, the Related Products Explorer opens a plug-in based app in a new window and uses colorful nodes mapped onto an auto-rotating 3D sphere; it also contains priority sliders for attributes like “price” and “sales rank” so that the user can adjust what will be shown. There’s also mellow background music and superb sound effects.

Now, the left-hand side screen may lack some features, and the right-hand screen may offer some features which may be cool in this or that context, but let’s look beyond the specifics of this example – it’s just metaphorical. (Given that Amazon already has related product links, the right-hand tool may in fact be superior if you’re e.g. an Amazon associate who wants to settle for some, albeit minor, available niche, and also create some buzz.) For the sake of argument, imagine something simple and usable to the left side, say a Google search homepage, and a cluttered and complicated tool to the right, say Microsoft’s Silverlight-based search tool Tafiti.

During presentation

Let’s list the pros and cons of these tools in relation to their effectiveness when presented:

The plain solutionThe Full-Featured App
Is boring to look atIs exciting to look at
Does not require any explanation, the presentation is over in a secondRequires you to actually go into an interesting story about what all the things on the screen mean, as they’re not self-explanatory
Is just a feature; it’s a small addition to the siteIs a full-fledged application, with its own good-sounding name; it’s a product in itself
The presentation slide has a loading time of 1 second (if it’s a prototype, then all plug-ins are installed, speakers are on, things are cached, the presenter knows exactly how to use the tool etc.)The presentation slide has a loading time of 1 second (if it’s a prototype, then all plug-ins are installed, speakers are on, things are cached, the presenter knows exactly how to use the tool etc.)
The presentation only contains a single slideThe presentation contains multiple slides which taken together create real story-telling, as there’s surprise, plot, pace, etc.
The technical implementation is complicated but it’s all in the background and heavily advanced-algorithm-based The complexity of the tool is expressed in the interface; it can be explained, to awe-inspiring effect, by pointing at certain arrows, nodes, buttons and so on
Is not the type of thing to win an award (you can’t review it easily, there’s no interesting screenshots to be made from it etc.)Is the type of the thing to win awards (you can write verbose reviews about it, you can make good-looking screenshots or movie clips out of it)
and so on ...and so on ...

During use

Let’s list the pros and cons of these tools in relation to their effectiveness when used:

The plain solutionThe Full-Featured App
Loads in 1 secondLoads in 1 minute
Is understood without manualNeeds elaborate help text to be understood
Just worksRequires the plug-in SuperCoolAnimation4 which is deployed on only 80% of all computers
Supports back button, can be linked to, isn’t rejected by pop-up blockers etc.Is inaccessible in many contexts (e.g. breaks back button)
The computer speakers are perhaps turned off, but it doesn’t matterAudio information is lost when the computer speakers are turned off
Solves 99.9% of all use cases very effectivelySolves 100% of all use cases, but all very ineffectively
Does not get in the way of the user task at handGets in the way of the user task at hand
and so on ...and so on ...

Recipient groups

And let’s compare the two recipient groups and the situation they’re in – the ones judging the tool during presentation, versus the ones using the tool in real-life. Let’s call it the Presentation Recipient Paradox:

The motivations of the people judging the tool are inverse to the motivations of the people using the tool.

As there are many exceptions to this paradox too, it’s only a generalization. But side-by-side, very often we get:

Those judging the toolThose using the tool
Have time (the presentation meeting was scheduled for around one hour, and they’re paid for their time)Are in a hurry because they want to do something else later on
Tool complexity will allow the recipient to make statements showing their good understanding of the tool Tool complexity will cause user misunderstandings and frustration
Are receiving a personalized guided tour Don’t have any personalized tour
Don’t do anything but look at the tool Do a lot of stuff other than looking at the tool (listening to music, TV is in the background, a chat program is open, another browser window started a download a minute ago, they’re leaving to get another coffee, and so on)
Are limited in their criticism because of political reasons (“if I say X then Y may be offended so I better say Z”).Are in a non-political context
Don’t want to be sold something which they think they could’ve thought of themselves, as that’s no added value (and it would open them for attack by internal competitors in the style of “why didn’t you think of this super-easy solution during all those years?”) Want to see something simple they immediately grasp
Don’t want to see something which looks like a copy of something existing, but want novelty (unless they specifically ordered a product clone)Don’t want unexpected behavior, don’t want interface surprises, but rather tools that remind them of stuff they’ve used elsewhere before
Don’t always understand the userAre the user
and so on ...and so on ...

 

Now, sometimes a tool’s effectiveness can be easily measured in retrospect. If you give one “user” a boring looking hammer and the other “user” the latest Goald-Coat-Flex-O-2000-Hammer-Solution, then you can count who will drive more nails into defined spots in the wall in a given time. However, strictly speaking this would consist of developing a near-infinite amount of solutions and then performing a near-infinite amount of user testing... which would require near-infinite money.

More realistically, many companies will chose a solution based on a couple of presentations, and later on, based on specific prototypes of the tool originally presented. But even at that point, changes will cost. If at all, the company will mostly only do comparison tests within the solution space defined based on instinct, not testing. To go back to the hammer example, they may be testing the Goald-Coat-Flex-O-2000-Hammer-Solution side-by-side with the Silver-Coat-Flex-O-3000-Hammer-Solution With Plastic Grip. Doing so, they’ll never find out if a mere working hammer would have been both cheaper, and more effective for the task.

Solutions

Perhaps you know of some solutions for this problem, or you know which kind of contexts, or teams or clients, make it easier to choose real effectiveness when judging solution proposals. I’m interested to hear those.

One mindset leading to better solutions is to really understand the user by being the user, i.e. using the tools you’re developing yourself for real tasks. But that takes for granted that you’ve already settled on a solution space, and it also still won’t route around the reality of some agencies or development houses having to sell their tools to people who aren’t typical users.
A second mindset is to be a one-person “development house” – you don’t need to do a presentation as you don’t need to convince anyone of the merits of your solution! – but that doesn’t seem to scale for all purposes.
A third mindset is to introduce red herrings for the client, and hope to get the actual tools that matter to the end user without client or manager interruption, and then retroactively present statistics showing that the tools-that-matter were used by 50,000 site visitors and the tools-that-merely-look-cool were used by just 500 visitors during a given period... but once you’re thinking about compromising by investing energy to create red herrings, you might ask yourself if you’re really working in the most effective environment for your skills.
A fourth mindset is to focus on the background math and algos during a presentation. “The tool may look simple, but look what great things are under the hood.” It will however (for better or worse) be hard to judge on these things for the client; added to that, sometimes the best back-ends are elegantly & ridiculously simple... think utilizing some open API with 3 lines of code versus code that runs for 5,000 lines, including super-cool algos, but doing the same.
(A fifth mindset perhaps is “whatever, the client paid and was happy, and I’m working for the client and not the end user.” But the point here isn’t really about how to make money selling snake oil, which, admittedly, worked in some situations in the past and will probably continue to work in the future. The point is more that in the long term, it’s end users who pay the client, which means unhappy end users will cause an unhappy client.)

I could imagine one other solution for presentations would be to focus not on the product (the tool itself), but on user stories associated with the product (the effect the tool has). “Mary was looking for a book by Douglas Adams. On seeing the product, she also saw a related link leading to a book by Terry Pratchett. She ended up buying both and was happy ever since.” Or: “Michelangelo used hammer and chisel to create this amazing statue of David.”

But how do you present these user stories, without resorting to hypothetical stories? One way would be to film actual users using the tool, then interview them about what they did. (It’s important to note that these users would not be given an actual presentation to the tool; just like online you don’t start with a manual, but dive right into using the service in question.) Users would tell what they wanted to achieve, and why, and how long it took them to achieve it. They would be asked whether or not they enjoyed performing the task at hand with the given tools.
Perhaps even more than that – as sometimes when explaining our actions we unconsciously lie about them (to avoid dissonance because we don’t actually know ourselves what we were thinking when using the tool!) – the presentation would simply consist of the facial expressions and gestures of end users using the tool. Imagine the forehead-wrinkled puzzled looks, and mouse-click rattling, when users tackle the Related Products Explorer app above; imagine, on the other hand, the smiles when their task is performed with ease and great results.

Advertisement

 
Blog  |  Forum     more >> Archive | Feed | Google's blogs | About
Advertisement

 

This site unofficially covers Google™ and more with some rights reserved. Join our forum!