Profile

Cover photo
Craig Sullivan
Works at Optimal Visit Ltd
Attended Boclair Academy
Lives in London
644 followers|39,920 views
AboutPostsPhotosVideosReviews

Stream

Craig Sullivan

commented on a video on YouTube.
Shared publicly  - 
 
Excellent video - short, to the point and really helped.  My laptop for £42 now has the screen back - saving me lots of money.  Thank you very much.  Only one thing missing - the little data cable at the top of the screen for the webcam, but that's me being picky!
1
Add a comment...

Craig Sullivan

Shared publicly  - 
 
*Truth, Lies and Damned Analytics *

I've now spent a week looking at four businesses - a B2B lead gen company, a travel operator, an e-commerce site and a B2C furniture supplier.    

As a hands on kinda guy, I get to see a lot of analytics setups and it takes you out of your comfort zone every time.  Why? 

 Because most of them are FUBARed* in a way that makes each one uniquely untruthful about that business and the way it REALLY works and flows online.  It's like being shot from a large cannon into a thorned hedge every week - I never quite get over the variety, purity and depth of pain that I experience - the same but slightly different every time.

And so I have been exposed to these setups and forced to dig and model and construct the real flows and numbers.  To go back to pretty much the raw data - making manual funnels, performing my own segmentation and constructing flow and leak models.  Spending lots of time looking at the page flows and trying to figure out how it all works.

But isn't that what the analytics package is for?  ARF!  I have not found one setup this year (and I count that at nearly 20 now) that isn't broken in some horrendous way.  And by examining the flows, page constructions and technical workings - you can eventually deduce the truth and make a proper model.

Some examples this week:

• A key step count in a funnel inflated by a second flow joining.

• New buyers and existing logging in customers going to the identical page, mixing the two flows.

• A microsite where there were more people using it to log in than engaging with the content.

• A charity site where recruitment content so dominated traffic that it obscured the truth.

• A set of pages fragmented across many URL constructions, making them hard to clump together in analytics, grossly undercounting their contribution.

This is the sometimes miserable and sometimes fun part of my work - decoding the truth from all the inputs I'm getting.  Let me give an example of how the inputs can skew your thinking:

• I've watched quite a few people doing expert reviews of sites and sometimes I've seen the analytics data before they did this.  I can see that they are heavily biased by the way the site looks on the page.  If you were to ask an expert reviewer to describe the flow, key areas and content on a site - they'd give you a visually led answer, based entirely around their exploration, opinion, biases or just the interesting stuff they checked out.  On most sites, the reviewer will start with the homepage, even if 90% of the traffic lands elsewhere.  It's not the truth.

• And if you just watch the usability test, depending on how the scenario is scripted, you could arrive at a second and different view of where the key flows and areas of a site were.

• And then there is the business layer.  To some it's invisible but I'm always thinking about the business outcomes of human behaviour on sites - it's the most important one of the lot.  So what happened then?  Did they leave the funnel and call us or what?  Did they come to our store?  How much did they spend?  Working the money or business outcomes you want is another dimension to this data - what do you WANT them to do and how much MORE do you want them to do it.  

• And then the data - if you accept the reports and data being pumped at you, therein lies the biggest flaw.  Not checking how it works.  We get seduced by what we think the analytics data are telling us - even when the collection method is flawed or broken.

And I've found all of these ways to be useless on their own for optimising sites - it's about how you integrate them holistically.  About the micro AND the macro.  The tiny interactions at field level in a form all the way to the lifetime value or improving your Net Promoter Score.

But it's all built on nothing if you get the wrong data into the mix.  And that's where we come back to the analytics and the truth.  

So unfortunately, most of the time I find myself having to construct models manually.  Sometimes because I can't segment Google Analytics funnels but mainly because the analytics configuration is recording site traffic - but not in a way that lets you readily interpret it
.
The fundamental problem is that there is (a) The way the developers, CMS and platform have worked to record pages and stuff that happens and (b) The way the analytics setup marries with this.  The two groups of people with a stake here - the developers and the entire business reporting layer of the company - often don't have nice fireside chats together about this stuff enough.  Most marketing execs would be surprised to know how much developers care about instrumentation if only you gave them time to fix it instead of working on something less useful.

When there is a mismatch between what the data should be recording and the site construction is impeding this - it has to be fixed.  All the reports, filters, goal outcomes, funnels and dashboards you can use will never ever fix the fact that the underlying data is wrong.  

And when your data isn't matching the business flows or where customers flow - the behaviours and outcomes - then you begin to lose faith in the analytics.  I've noticed that people just start trusting something less when it diverges hugely from other data points.  They should fix it or someone else should - but this isn't always what happens.  If your car speedo looked busted, would you just sort of ignore it?

So how do you fix this?  By auditing and fixing the instrumentation.  A couple of days is usually enough to find 90% of the dumb mismatch between site and analytics setup.  And yes, sometimes developers will need to fix stuff but generally, most of what you find is easy and quick to fix.  Investing time here will spread layers of greater truth and clarity through all your reporting and insight generation.  

My plea to everyone is to spend money on instrumenting, fixing and improving their analytics continually.  Towards measuring more and better stuff, with greater fidelity and also usability for everyone in the reporting chain.  

It all starts with the instrumentation budget though.  You probably spend less than 1% of your time or money on this area and you'll be delighted if you follow my advice and spend LOTS more.

It will also make me less miserable when for the 900th time, I will conclude that I have to manually construct a layered model for a site, simply because I can't see the real flow.  And when that's happening, it means everyone in your company isn't seeing the real flow either.  I don't want to teach everyone how to do manual funnels or layered models - it's very time consuming - I'd just like to help people fix this distortion of reality in your basic reporting!

I'm still writing up the methodology I've been using to review sites - using a whole range of GA reports, tools and review methods.  Every site, business and customer audience - they're all wonderfully and uniquely different.  However, there are common aspects to how people choose to interact - to enter a site, progress to different conceptual levels or reach one or more goals.  I'd like to explain this conceptual model more - as it offers a nice simple way of seeing where the pinch points are - where the money is.

Armed with the flow model, you can then construct a split testing or improvement plan that maximises conversion holistically - across the entire model.  Not starting on the flashy page you think it's on - but where the money is actually being won or lost.  Knowing where the battlefield actually bloody is can be critical to winning the war, please note!

So a perfect example.  

5 weeks had been spent arguing over the new landing page.  It had been kicked around the brand team, marketing, UX and engineering like some unwanted guest.  Immense amounts of time, large meetings and endless redesigns sucked at productivity like a massive black hole.  It still wasn't ready so oh boy - it sure better be good when it arrives eh?

So here's a hypothesis - what if this nice high traffic landing page doubled the conversion rate?  What if these guys really did get, amazingly a HUGE 100% lift?

Well, the answer is less than 20 checkouts a month.  Yes, 20.  The plain fact was that the problem was with the targeting and traffic - and the elasticity you get playing with the landing page is NOT ENOUGH to shift the business layer.  

All that effort pushing the wrong lever.  As Duke Nukem says about truth: "Come get some".

That now concludes my series of "101 Whines about broken analytics and other miscellany".

If you need your site going over, get in touch.  I know a very small bunch of people who are really good at this and will give you the exact changes to make - the fixes to apply, that will get you started on the road to a better truth.

*FUBAR  http://en.wikipedia.org/wiki/Military_slang#FUBAR

  
10
1
Tim Leighton-Boyce's profile photoCharles Meaden's profile photoStephane Hamel's profile photoAl Wightman's profile photo
12 comments
 
Cutting through the BS as always Craig, great stuff
Add a comment...

Craig Sullivan

Shared publicly  - 
 
Hacking the Company - Lean UX in Action

I've had quite a few experiences recently worth sharing - and all of these are centered around the concept of using the website as a way to hack your business model, pivot in a new direction or otherwise 'hypothesise' about how things could be.

When I worked at Belron (who own Carglass, Autoglass and Safelite around the world) - we often did usability testing with hypotheses.

In France, for example, we took our core service proposition on our web channels and thought hard about the usability test we planned.  Since we were going to be changing the entire process for handling customer bookings, how about we model that - instead of the status quo?

So - before the business had even begun putting this in place, we mocked the website process up to reflect the future reality.  We then got the feedback.  The important thing is that customers were feeding back on a server we hadn't yet invested in designing.

This allowed us to adjust the new process to more closely match customer emotional needs, before we'd actually kicked the project off.  This is where the web scores - it's like having a retail store where you employ staff who just test stuff out.  The good stuff gets exported to the rest of your retail empire.

You can create interfaces, products, pages, promotions - that don't exist, that cost little to produce and yet give you warning of the f***ups you're about to make.  Hypothesise the future, with customer feedback NOW.

Another good example is a couple of freemium SaaS vendors I've worked with.  Both were looking closely at how much stuff they gave away for free.

I explained the situation like this - it's a multi step conveyor belt - inbound marketing to landing to engagement, conversion and then the next step.  Either Lifetime Value and other metrics but especially, the conversion of FREE members to PAID members.  And movement up the hierarchy of product or feature mix that drives revenue.  A conveyor belt of conversion steps, outcomes and insights.

Aside from asking them to work the whole chain, I helped design two feature comparison tables.

You know - the ones where you see the ticks and crosses against the feature list.  A visual reminder of what you get for nothing, and what the potential implied value of paid means to you.

A very interesting page to test of course.  But how about we totally mess with their entire business model?  WHAT?  How are we going to do that - without loads of work?

Well let's assume you have 7 core features, 5 of which are available in the free product.  What happens if you design an A/B split test - where you vary the mixture of free and paid for features?  Mix up what people see on the feature table and gain insight from the results?

What I see from tests like these are interesting insights - the first thing they tell you is that your mix may not be optimal.  Either you may be giving away too much for free or you are under converting as you don't make free compelling enough.

It's possible from this kind of testing to arrive at a smaller 'free'  featureset that converts MORE people to both paid and free, whilst holding back more stuff to convert later.

And this is the best bit - if you've found your sweet spot, you can hold back more stuff to use as an upsell to paid or premium features.

If you test this kind of stuff, it might be confusing for customers as the experiment won't match the actual service (yet).  One way round this is to get to the end of the conversion process and say "Congratulations - we've given you the full deal.  Just for today, we're giving away the full feature set to everyone".

So - do be careful about Business Hypothesis Testing - especially if you're trying it on a live website.  However, remember that useful point - you can model the future and dekink the product design, before it's even built, deployed or arrived.

There is nothing new about this stuff - in the UX world we prototype all the time.  However, this isn't about prototyping the interface - it's about getting feedback on your business model, service design or experience - when imagined differently.  And that can be a game changer.

I love these kinds of test because they gain huge leaps in product but in a preventative way - rather than a reactive, looking in the wing mirror kind of way.  Welcome to the new Lean.

C.
4
Add a comment...

Craig Sullivan

Shared publicly  - 
4
Tim Leighton-Boyce's profile photo
 
Something I've noticed is that if you have a survey on the confirmation page (one of my favourite things) the response rate tends to be lower if the confirmation page itself has little else of interest.

If the thank you page includes details of the order itself (some don't) and other material, the response rate goes up.

My theory is that if people see a minimalist page they leave. If there's a bit more on the page, more of them stick around and re-engage. There's an opportunity there.
Add a comment...
 
Conversion Camp

Firstly - sorry for spamming you.  I mail my infrequent articles around directly but wanted your help with something.

 I'm really keen to run a London event, with the best #CRO and analytics crossover experts - and do it similarly to #Brightonseo in these ways:

* Reasonable prices  - limited sellout but priced for wide access
* Great speakers - conversion optimisers, growth hackers and vertical specialists - the best in their area
* Collaborative - workshop, collaboration and interaction - not PPT to death.  Post event discussions and community.
* Great publicity - the network of people to support this is pretty huge so it should be well attended

I'm interested in speakers (I'll approach some of you directly - you know who you are <g>)

If you'd like to help shape, steer, run or just do something to blag a free ticket - then let me know.

Most of all this should be FUN - I want people to argue, learn, swap notes and come away inspired to go forth and optimise.

If there's a conference organiser wants to partner with us, I'm open to talking this through too.

If you have ideas for topic areas, let me know.  If you want to help run this, do get in touch.
3
Tim Leighton-Boyce's profile photoColin Tate's profile photoGerry White's profile photoNiels Tybjerg's profile photo
13 comments
 
Hi, sounds great, measurecamp is the best thing I've been to and something similar would be awesome. If I can help at all, shout. 
Add a comment...

Craig Sullivan

Shared publicly  - 
 
Pollination

When I was a young boy, I learned a lot about beekeeping from my grandfather.  Many a summer afternoon was spent asking questions, making wax templates, building hives or helping move the bees to 'new pastures'.

 I spent a lot of time observing the bees - mainly because I was s*** scared of being stung and yes, bees CAN smell fear.  

This was my first taste of watching 20,000 visitors, with attitude, do something that totally hooked my attention.  I read voraciously and absorbed everything I could find at the library.

I was amazed at how they could engineer perfect hexagons out of wax secreted from their own bodies.  I read of Their Queen and her subjects, their complex dances, their calculations of distance - always taking account of wind and the movement of the sun overhead.  Like an organic machine, I watched over 50 kilos of honey come from one beehive in the hot summer of 1976 from ceaseless industry and coordination.

But one thing really stood out.  That call to action, that small orange dot worn by thousands of bees - one on each leg.  Pollen.

This amazing stuff is collected in the wild and used to feed the young.  But like any interface or process you observe, there is another layer going on here.  The bees are having a two way relationship.  In return for collecting this multi-coloured, protein rich substance, the bee performs a vital service for the plant.  It acts as the dating website.  

Every time that leg is flown around the foraging area, it collects (and drops off) some of that pollen at most flowers the bee visits.  And in doing this, the plants are pollinated.  Yup, that's where food comes from and if you own a fruit tree for long enough, you may remember what it's like when pollination doesn't happen.  No food.

I could write a lot of preachy sentimental stuff here about how you ought to support bees or the beekeeper.  Nope.  Just go and find a local beekeeping chapter or Apiarist and take a look.  Once you see the big picture, you'll support not hammering their tiny brains with chemicals.  If they disappear - a lot of the stuff you love will become too expensive to grow in the western world or vanish completely.  Fair warning.

But back to this article and my central point!  Pollination - it's the vehicle for message (using DNA) that creates the variety of natural selection for plants.  With reduced pollination, species can't compete or evolve against others and become less resistant to disease.  Over a long period, an entire and unique species might vanish.  They simply go out of business.

The Pollination of ideas within companies is something I've observed too - working on digital products from startups to lean corporates, small companies to publicly quoted ones.  Here are the key attributes I've seen, where the teams really 'get' the problem space and effectively 'explore' or test the solution space:

• One flat team approach - No silos, No 'pass the parcel' design
• Team members are polymaths - multi skilled, multiple disciplines
• Excellent analytics and customer research tools, properly invested in
• Strong Senior Management and Budget support
• Agile, Lean and evidence based continuous product improvement
• UX owned by everyone, not by team, agency or role
• Rich interaction with other teams and viewpoints
• Collaborative, lightweight, software tools
• Sharing of knowledge through conversation, not process
• Less big boring meetings, more interaction at desk, online, iterative
• Not using all the corporate toolkit,  rolling your own data universe
• Remote and completely flexible working - distributed global teams possible
• Leadership around belief in product, mission and way of being
• Enjoyment - teams that have fun build great products.  FUN!

So where do the best ideas come from in these teams?  Well mainly from customers and the people that support the product at every level.  For example, here is my shortlist for an employee working on an online clothing brand.  As part of induction and every month after that, time should be spent:

• Listening to customer online calls (call centre)
• Reading emails or feedback (website, call centre)
• Working in-store with back-office or customer facing IT systems (retail)
• In the warehouse, understanding the flow of online orders (warehouse)
• Understanding Social feedback (what are they saying)
• Reading Reviews (online and especially 3rd party sites)
• Working with business owners, finance people, legal teams
• Attending meetings of other teams - marketing, IT, agency, SEO, Content etc.
• Ordering complete service or product(s) - comparing with competitors

Why?  Surely these people should be coding, building product or running tests?  Yup.  But they'll run far better tests and improvements by seeing the bigger picture.  Because they'll be designing better solutions to business problems.

In most of these trips outside your comfortable IT, Management or Product bunker you'll be wandering into a sea of interesting psychology.  Let's take the call centre or warehouse - you're 'someone from head office' now visiting for a bit - and that doesn't encourage brutally open and honest communication.  They're not going to tell you what sucks without a bit of work on your part.

If you practice your interviewing (Steve Krug has good advice for the beginner) then you can get much better answers than direct interrogation.  You have to make people comfortable about sharing stuff they know is wrong but feel defensive or worried to talk about.  

If people are so terrified of being honest that you'd have to use torture, I find beer has a calming effect.  Take everyone out for beers after your visit - carefully timed to finish at end of shift.  Then be honest and open - ask 'What's really grinds your gears about the online stuff?" "What do customers get really upset or delighted about?”  Well worth the beer costs.  And this is just one of the sources of vital information you might need.

Spending time immersed in different viewpoints or connections that your digital product has across an entire organisation and customer cycle is important.  This helps you see the real customer problems, the major sources of cost or loss and the places where you need to fix things first.  You're running a digital business, not a website - it's not a self-contained bubble.

Gone are the days when we'd run a usability test on a site, pronounce the problems being A, B and C and then fix a couple of issues or just ignore the report in favour of some new shiny distraction.  Now we're trying to solve problems of overarching customer experience, cost, complexity, process or flow across an entire business setup.  And those are a game changer - as you can focus effort where it really hits the bottom line or causes pain to customers.  

A good example of this is maybe a Freemium SaaS product.  A company tries very hard to increase their conversion through site testing on 100,000 visitors a week, converting at 2%.   A usability test is run, without knowing the numbers behind the story. The whole team focuses on pushing that metric higher, fixing stuff and burning time.  Meanwhile, they're sitting on 8M subscriber emails they don't try to upsell to the paid subscription models.  D'oh.

And after 13 years working in UX, I still bemoan the fact that many disciplines are the same.  Where is the ethnographer with social analytics skills?  Where is the usability moderator with google analytics smarts?  The multi-variate testing and forms usability hacker? 

These people are all out there but they're sold, packaged and hired as disciplines - not the business solutions providers I know that some of them are.  Although I complain about the roles not moving with the times, I know it's not the fault of the UX staff - more what clients want to buy, what agencies expect to sell and how the time is then broken down to people.

And this is one reason why single disciplines, without pollination or inputs from varied sources, just don't have a place in the teams I build these days.  I need people who can spot problems with the entire flow, understand the bigger picture, see opportunities and then test hypotheses.  Hacking the growth of a business requires multiple skills and toolset use.   Whether it's a startup or a lean UX corporate team, you all have to roll up your sleeves and absorb problems and data - in order to design the best responses.

And if this doesn't sound much like your job right now, you might actually be a Drone.  These are the males, kept around the hive to mate with the Queen.  They don't get out of the hive much, they're pretty expendable and they're only a vehicle for DNA, a box to be ticked.  

If your company has a lot of these around and you feel capable of more than a single disciplined experience, you need to find a new job.  If your agency is selling units of blinkered thinking - assumptions, opinion or guessing - disguised as real insight - then you need to find a newer style of place.

Fast moving digital products need a team who are comfortable breaking way from assumptions and cherished notions, getting out of the office, escaping the hive to forage and find delicious opportunity.  

Silos or single discipline approaches are anathema to building wonderful customer experiences - they just get in the way of the best ideas, communication and sharing. You also simply can't layer Agile or Lean approaches onto silos or single discipline process or approaches - it doesn't work because the foundations still suck (Agile Silos anyone?).

The overlap between people who all care about the product (and influence each other’s work) is the intersection and spark of ideas, disciplines, insights and change.  If you never overlap within and outside teams, you're missing that pollination.  

Given flight, the stuff that's brought back to us and the exchange made by sharing ideas drives survival, evolution and growth of a product.   Without it, the barrier to competing with and beating your company is much lower - you're just easier to weed out of the corporate gene pool.  

New buzzwords or methodologies (lean, agile, kaizen, kazinga*) aren't the answer, when they're just overlaid on the old BS.    You have to fundamentally change the way you build, measure, test and improve digital products and that starts with breaking down silos, removing barriers and running multi-disciplinary teams.

I can still recall the smell of the honey and the sound of 20,000 bees inside my head, as we opened up a hive for the first time.  Trying to stay calm inside a cloud of stinging insects is pretty hard work but I relaxed by trying to watch them.  

They're incredibly efficient and tiny biological robots.  In realtime, they quest for and spot new opportunities (flowers) and react in minutes, weighing up the best options and handing out directions.  Can you say the same about your digital marketing teams?

I certainly won't forget my Grandfather, the bees or the useful lessons.

*OK, Kazinga - I made that one up.  Just checking you’re awake here.  Keepin you lean.
13
Brian Clifton's profile photoCharles Meaden's profile photo
3 comments
 
Funny enough I had thought about bee keeping a few years ago... the idea of building and growing a community appeals to me ;)
Add a comment...

Craig Sullivan

Shared publicly  - 
 
UX & FORMS OPTIMISATION

I was reading this article and found it very good.

http://designmodo.com/ux-form-validation/

I was spurred into sharing an example for you of intense field level validation - where you concentrate on one field and optimise the heck out of it.  Really focus your effort on squeezing all the bugs out of just one field or interaction.  

First - and this is the important bit of advice.  Read up on how to make forms for people in the real world - like application forms.  That's very useful - you should also follow and read @cjforms work here.

Secondly - it did me no harm at all to read and listen to everything by  Luke Wroblewski voraciously and I still do – his qual/quant work here inspired me to try and squeeze out every drop of misery and improve clarity, simplicity, UX and revenue, all at the same time.

Psychology, user context, device, error messages, validation, traffic source, behaviour - all of these come into play in complex ways with what we sometimes call funnels.  They are just a set of steps, like filling in a form but BOING! - remember one very important thing - please don't think of them as pages but a set of steps, chunks, interactions - all the way down in smaller pieces until you get to the field level.  Try to optimise all of these parts - not just pages.

When you do field level optimisation, here is some prep work:

(1) Get session capture.  Use a session replay tool to capture visitor interactions on the page where the field lives.

(2) Get stats.  Use event tracking or your session replay tool to tell you things like how long visitors spent on a field, how many times they submitted the form, what errors or error groupings they received and what their exit rates were as a result.  Get stats.  If you don't have any, instrument some.

(3) Find all the data.  If you have some sort of submission, try to find out what all the data contained in the failures actually was.  Yes - dig out all the recordings in session replay or have your developers store the failures (or add in an analytics tag).  Capture every failure, every single little warning flag.  If you have contextual error messages, fire an analytics event tag when these display to the user.  You need to know this data.

(4) Get users.  Try real people using your forms and breaking them.  Consider making a prototype of one field or that area - and mocking up a page to try stuff out.  This will give you rich qual feedback to go with the numbers.

(5) Spot patterns. Are there any patterns in the failures you have reviewed?  Try looking by eye at the raw data and see if you can group them.  If you can't spot a pattern -  have a good developer or  analytics person try to spot patterns algorithmically.

(6) Make it vanish.  Either way - look at the failures and think hard.   "What can I do to rid the earth of this foul problem?"  How can I auto correct or solve the input errors which drive abandonment.

I once spent a couple of days looking at all the form failures on a movie website.  Most of that time, I was looking at a postcode lookup system which was just not working.  UX testing told me some useful things but I suspected there was further data lurking in the field level submission data themselves.

Firstly, the usual things - people used mixed case and sometimes used no spaces, one space or two spaces between the parts of the postcode.  So far so good.  Some people put in complete bull*** postcodes.  Then I found more obscure ones - where the postcode had a full stop or some extra, random character appended.

The full stop and extra character could be fixed by trimming or checking the code using a sliding algorithm to find the bit that looked 'right'.  The spaces could be fixed by removing them all from the input.  The lower or upper case letters we simply converted..

Then I looked much further as, obsessed now, I chased down the last bits:

The postcode example is here : http://www.slideshare.net/sullivac/postcode-auto-correct

This last part is about reading and  understanding the algorithm that the British Royal Mail uses, you can auto correct the most common errors people make.

http://en.wikipedia.org/wiki/Postcodes_in_the_United_Kingdom#Operation_and_application

For example, the postcode CR0 1LH can be entered a number of ways. The visitor might change the zero (0) to a letter O and the 1′s and L’s frequently get swapped.

People also shift the characters – it’s common to write the postcode in capitals (for those who remember writing letters lol). UX folks know from observation that many people are less expert typists than us tech types and look at the keyboard and not the screen, when typing.  This means that what's in the field isn't always what's going on 'down there'.

In the example above - that turned a valid postcode into this one : CR)!LH

Most websites would reject that but hang on - what's happening?  I began to see £$%&*()!" symbols in there, in place of numbers.  What the heck?

What was happening here was that people used the SHIFT key to capitalise the letters in the postcode, like you'd write them.  They just forgot to lift their finger off the SHIFT key when they were typing the numbers.  Because they weren;t looking at the screen, they made mistakes.  And still submitted what they had typed!

There are quite a few patterns like this with phone numbers, fields, dropdowns – where you can optimise at an atomic level by systematically analysing failures.

Auto cleaning the postcodes for the example above (and not generating any error message) was accomplished by working out what they actually meant.  Like when Google corrects your search spelling - it knows what you really want.  

Using the rules of the royal mail. CR) !LH – really only can be the postcode we are looking for and so like Mr Picard, when the customer says 'make it so' - you should  do it.  

For total failure or garbage, the form just degraded to let you fill in your address by hand. So there were options for everyone, even those who mistyped it – but that part is completely invisible – we just filter it on submission.

This sort of field level tuning netted out 2.5% higher conversion for the entire funnel. 

It amazes me that when I started, I had a computer with 1 Kilobyte of memory and a TV as a monitor.  The CPU was puny.  And all this time later, over 30 years - we're still rubbish at harnessing that power properly.  We've seen Moores law and the growth of processors and the internet - but we're still not smart enough to code good forms validation? <shakes head sadly>.

Using common sense at a field level, you can get a lot more people through a form.  It's observation and data you need - the triangulation of the two is the start of your journey to removing friction for a lot of people.  I'll never get to be batman but at least I can make a form less fiddly for heaps of people.  A wee bit less grrrr in their day.

I must collect all my forms resources sometime as I'm sure some of you would find them useful.  Do let me know if this is of interest - help spur me on to curate them,

Source: http://designmodo.com/ux-form-validation/
5
7
Paras Chopra's profile photoTim Leighton-Boyce's profile photoAndré Mafei's profile photo
2 comments
 
What a great post. Thanks, Craig. I've just shared this with the GA community because of your point about instrumenting the fields.

Are you compiling all this advice somewhere? G+ is a lot better that Twitter for length of content (!), but it still feels ephemeral.
Add a comment...
In his circles
172 people
Have him in circles
644 people

Craig Sullivan

Shared publicly  - 
 
WTF do Wheeled suitcases have to do with Design?

“I hate those effing wheelie trolleys – they’re the spawn of the devil” recently spat a friend.  I then got a rundown of how much of a social menace these little lucifers of luggage presented  for us regular travellers  And they’re not alone in this rich seam of haters gonna hate – but there’s a serious story behind this that came to me today.  

I got stuck in a crazy duel with another wheelie suitcase for the racing line into Piccadilly Station in Manchester.  At full tilt, there was only going to be one outcome – yes, unbelievable - two little wheelie suitcases humping each other in the station entrance.  I wasn’t sure whether I was angry or laughing my ass off – both I guess.

And then I thought of the two sides here to the argument.  Those who hate them, those who love them (and those who are on both sides).  So if you regularly use one, I’d like you to try and think about why people get angry with their ‘drivers’.  And if you regularly curse the day they were invented on this planet, I’d like you to consider the flip side too.

Let’s cover the wheelie hate stuff first.  More fun this bit of course (I can’t help myself):

(1) Some of you forget that you’re dragging this long thing behind you, like a small crazy dog on wheels.  It’s like a car trailer – you can’t ignore it being attached to your body as it will smack into things as you corner.  Oblivious and about as spatially aware as a house brick, the wheelie trails carnage: clipped heels, cursing and sometimes even little jumps over the payload happen regularly.

(2) If you make sudden manoeuvres, you’re forcing all the pedestrian traffic around you to adapt – like a school of grumpy fish, the shoal adapts but not without subvocally bitching about it.  You see none of this as you stride blissfully ahead.

(3) If you cross another traffic stream, it’s like dragging a dead killer whale across a busy street on a chain – you part everything but with a huge freaking interruption to the traffic flow.

(4) If the case is low profile, people may miss it in busy foot traffic and trip over it even more.  

(5) Most people buy the big versions of these cases – so they eat much more space in smaller planes – making space harder to find in overheads.

(6) Havoc is caused on escalators due to the mode change between wheeling/pulling to standing.  Expect pirouettes and more.

These are the main things I have observed.  The mosh pit of terror has to be the London Underground ticket hall at London Bridge Station – the designers were dumb enough to cause a terrible problem.  The different entrances and exits lead to at least four traffic streams bisecting each other, at different speeds and points in the hall.  As there’s virtually no overt or even subtly designed flow – it’s freakin open season in there.

Suitcases, strollers, packages, work bags, umbrellas, people, boxes, children are whizzed in a blender by the lack of design and flow control.  People are bumping into each other all the time and constantly forming new and different traffic streams that jostle for space.  Just watch it sometime – fascinatingly dumb.  And into this mix comes tada! – the wheelie suitcase.  

As hilarious as it is being an observer, I’ve been on the end of the terror too.  Makes you angry - it’s like someone ‘driving’ irresponsibly.  But one question why then do THEY have a look on their face like it’s your fault?  Ever wondered why?

So let’s cover the other side of the story.  I wonder how many of you own a suitcase as well as dislike people who use them (or drive them badly).  How many of you that also see both sides but never reconcile them.

At a conference, I asked an audience of senior marketing execs a question.  I got them to raise their hand if they really hated when they called a phone number on a website and the on hold message said “Please visit our website at blahblah.com”.  

Most of them thought about how annoying that is – after all you’ve just come from the effing website haven’t you?  Why would you be calling if the website didn’t have what you needed eh?

I then asked the audience to drop their hands ONLY if they knew 100% that their web phone number did NOT have this type of cruddy message.  Very few hands dropped – it’s a case of hate it for myself but sod you, millions of customers.  Meh.

So the flipside of the story is this.

(1) Wheelie suitcases are really hard to manoeuvre.  A lot of the designs have really long handles and if you’re unfortunate enough to be shorter in stature, it sits lower in profile to the ground making it harder to see and extend further back.

(2) If you’ve got a heavy case, they’re like dragging a rock around.  Really tough to move unless you apply a lot of torque with your poor tired hands.

(3) It’s pretty hard to remember the bit sticking out behind you.  And since the brain is focused on processing the front view, the carnage behind you (or being able to avoid causing it) is kinda hard to manage.

(4) Your hand feels it’s pulling something but doesn’t always mentally extend that into the full length of the wheelie.  Depending on how you hold it, these things go waaay back.  It’s actually hard to know where it is spatially by recalling your movement history..  The problem is the suitcase is never following you – it’s on a curve based on your movements and a completely different path.

(5) And the most compelling reason of all?  Compared to how luggage used to be (remember lugging a suitcase around by hand? – these are a joy to use, especially for those without the hand strength or stamina.  People love wheelie suitcases and owners that remember the old luggage designs swear this is the best design invention for luggage, ever.

So what does this examination tell us – well that it’s a bummer to have one of these, never mind have a gymnastics lesson with one as a pedestrian.  And that both sides are probably right – maybe we should rethink this design to save the misery it causes both.  

The downsides seem less for prounounced for the owners out on the road – compared to the hapless victims – but it isn’t a one sided story.  There are similar things in our behaviour in cars – our expectations of what we want are different to what we wish (charitably or not) would happen with other drivers.  Our standards for their behaviour are different from how we behave ourselves.  And we all have this bias when we design websites too – an ability to take our own thinking or viewpoint or personal experience and use it in place of others. 

Designing products takes a type of empathy that involves you taking the other side, even fighting in several corners, inhabiting alternate realities and trying to think like other people.  

It isn’t copying and you can’t just mind read people – it involves taking contradictory viewpoints, challenging your own opinions and ways of doing things.   It also reminds me a lot of software destruction testing – finding every possible way to break a product, destroy it, make it fail.   Except this is a different killing field – that of cherished notions.

I’m not recommending this as some new technique or in place of proper user research, (I’m a big fan of rapid, iterative, UCD, by the way).   I’m just saying that taking sides and trying to understand all viewpoints can help at every stage in the design process.  

With well loved and instrumented analytics data, user research and iterative design – it’s easier to do without ego, opinion and cherished notions.  The process you follow frees you to shrug off the stuff you think and simply try to imagine.

And so next time you design a new bit of website, drive your car or carve up London Bridge with your wheelie case – flip your side of the argument for one of the others.
2
Alex Poole's profile photo
 
"Putting yourself in their shoes" covers anything from parenting to international diplomacy. Those cases though - can't you use one when necessary but still abhor them as the least worst of a bad job? Where's our hover-cases? :)
Add a comment...

Craig Sullivan

Shared publicly  - 
 
Elite Camp Estonia, Analytics Hell and Redemption

What a week.  

This time last week, I had arrived in Tallinn, Estonia, for a conference called EliitLaager (http://eliitlaager.ee/) - which sounds brilliant in English.  Like some sort brewery conference in the mountains with suits, powerpoints and a painful hangover - but it wasn't like that at  all.  Apart from the hangovers <grins ferociously>.

It's been going for a few years now and the attendees are an excellent mix  of marketers, techies, analysts, growth hackers, investors, startup folks and curious people.  It's nice to see a wide age range and mixture of backgrounds.  And the companies involved serve markets that I've been  involved with but many countries that I've never worked with.  Very interesting to hear about new projects, brands, startups and their different challenges working across western and eastern european markets through to Russia, China and the USA.  

It's also interesting to see many key decision makers avoid the fate of similar western startup failures - by having customers and data at the bloody heart of everything.  I'm never a fan of copying models - you're always one step behind - but taking, understanding and adapting things to make them better?  This is a nice way of thinking that I heard a lot when I was in Estonia.  

I met some fearsomely smart people - if you're building a new tech business for local markets or for the world - then this is a pretty good place to start. 

I'm always interested in economic differences between the places that I visit and that starts with the cost of things.  I like looking in estate agents, recruitment agencies, corner shops and figuring out the cost of everything - and where odd differences stick out. And I love trying new food -  a pathetically great excuse to eat out.

And my rough conclusion is that:
(a) salary versus life quality is very high in Estonia, at least for tech and web stuff
(b) Talent is very good and a  whole raft of young people (early twenties) is coming through and
(c) They're completely clued up about working across timezones, countries and languages. C'mon - where do you think Skype came from eh?

See this for info on the taxation structure in Estonia, which is broad, low  and involves different handling for capital gains.  Seems like very reasonable employee flat rate taxes and low costs for unemployment and employee pension contributions.  What a riveting read this was:

http://www.slideshare.net/rahamin/estonian-taxes-and-tax-structure-2012-13681611

And in many places in Eastern Europe, conferences, communities and groups of  tech people are finding that there's a local market for their skills.  More  importantly, there's an international market too.

For the Xenophobic amongst you, this doesn't mean hordes of imaginary Estonian people in your head coming to visit you in England.  You've been reading the Daily Mail too much.  

They're actually more than happy to keep a high quality  of life and lower costs by staying where they are!  There is also a sense of building something local - investing in their own future.  I actually quite fancy working there for a while myself, it's so darned nice.

But here is an interesting point - costs begin to factor in here.  An example of an economic shift might be if South Africa  manages to get improved telecoms infrastructure sorted.  If that happened, it might well be the  next call centre growth area, moving business from India.  With a ready  population of english speakers and a european timezone, it should be a natural for locating stuff there.

And for Estonia, I think it's an ace place to meet a growing band of people creating new services and companies.  It's also going to be a bunch of people who can afford to work for less and live better than you too. 

I've been working with virtual teams around the globe for a few years and  I'd certainly try Estonia if I wanted to hire people to help on a project.   

Be aware of macroeconomic trends too - the tools and software we're using these days means it doesn't matter where the team are.  And if you can earn good day rates for remote work and stay and build your own thing, in your own country - isn't that the best of both worlds?  Yes - according to a few people I talked to.  

Memory of Soviet occupation and it's shadow are never far away.  There is a hotel in  Tallinn that hid a secret - an entire floor dedicated to the KGB, now  restyled as a museum.  I missed getting on the tour and am gutted - so this is on my list for the next trip.  

Ironic comments about the past were occasionally muttered sotto voce by those old enough to remember that time.  Estonia lost independence in 1940 and finally regained independence in 1991, after the breakup of the USSR,  joining the EU in 2004.  I spent some time reading about over half a century of occupation (by  
Germany and Russia) and the destruction and horror it caused - pretty sobering stuff.  

The one thing that stuck in my mind was the organised demolition of graveyards, gravestones and also monuments from the first world war -  many of them being blown up and sometimes re-used by the Soviets. A chilling  quote from the orders given out:

Võrumaa Committee, Tamm, No. 101/s to 1st secretary Nikolai Karotamm.  06.04.1945. ERAF Archives 

"In order to carry out demolition works, 15 Party activists and 275 persons  from the Destruction Battalion must be mobilised. 15 workers are needed for  the execution of each demolition and 10 people are needed for protection....  In order to carry out demolition works, 225 kg of TNT, 150 metres of  rope/fuse and 100 primers are needed, since there is no demolition material  on the spot. 11 lorries, which are available but which lack petrol, are  needed for carrying the ruins away."

in the UK, our geography as an island may have spared us from many of the deprivations that befell our continental European cousins during the 20th century wars.  As much as our parents and grandparents sacrificed for our freedoms today we were at least spared from occupation and loss of liberty.

But back to Elite Camp:

There's a good vibe in Estonia and the Camp got excellent feedback from the  participants.  Having a less formal setup where everyone stayed in small cabins nearby was pretty different and much nicer than a hotel - right near the beach and miles away from town.

In terms of the sessions, the formats were pretty interactive and the venue  would shame most places in the UK for having great interior design, light and space.  Props to Peep and the team for having a different and better kind of event, in a high quality venue.

Most people are excited about the future, particularly for tech, analytics, startups, marketing and small agencies - it's a zeal for the freedom and the sheer possibilities of tomorrow that I admire so much.  

Tallinn is a wonderful city to stay in if you can visit - the old town was  fascinating and you can experience a dizzying range of architecture in a short walk.  The food was superb.  Away from the usual tourist haunts you can sample  fine fresh ingredients, top of the line cooking and excellent service.  It's much better value than London and the quality of fish and variety of cuisines adds to the mix.

So - analytics hell:

I don't know if you've experienced this but I began to question the very data I was seeing for the last four months.  The only explanations were that our market had been impacted or that the analytics config or collection had been completely borked in the past.  

I'd looked at it with a few people and we had various theories - none of which solidly checked out.  It was all about a huge flip in traffic patterns which happened around the time of launching a mobile site.

It was a head scratcher and I'll write more later about it.  I was  
determined to work out what it was and although I can't prove it, I have very strong evidence for the reason after much help.  

I was actually feeling pretty good about the week at that point. Then the site broke, we found another bug in our call tracking analytics and another two analytics holes surfaced.

I was ready to implode.  Disappear into a puff of data denial.  The numbers not making sense for 3 weeks of digging finally came to a head.  

I took a break and looked at a charity site instead for a day.  Sometimes this is the best way to put something in perspective but I'm lucky to have this variety.

I recently read an article about someone who deliberately designed their work projects to be more exciting!  How?  Well they always carved a few niches in the product design that were not part of their core work.  So you might be redesigning the overall funnel but on the side, you've picked two small or gnarly little problems to keep on ice.

When things get tough, turn to some puzzles you can fix and volunteer for or include these in your work.  At least you get to have the pleasure of fixing something rather than worrying about HOW to fix something else.  

When you return to the core task later, your heart and spirits may be lifted a little!  Having 10-15% of your time AWAY from your main project can be a psychologically useful tool to keep you fresh on your main gig.

And redemption:

Asking other people is usually a good idea and several did look at my analytics hell, some deeply and completely so that the answers surfaced.  Not only that, they looked at the data differently, tried other tools and segmented it across more  
dimensions.  Evidence was crystallised.  

People sometimes think based on my slide decks that I can solve all these optimisation projects single handedly.  What they don't realise is I get stuck or need a fresh viewpoint just like anyone.  Most of what I've learned has come from other people and most of what I continue to learn comes from looking at smarter people than me figure stuff out.

Having the humility to understand how limited one's knowledge is - in the face of the sheer variety and difference of millions of customers - may come to you from usability testing, split testing or just trying harder to inhabit your customers shoes.  

Without some humble, your overconfidence means you think you know more than you actually do.  Most of us if asked to design a web page would have a high degree of confidence about the improvements we're making.  Testing shows me that this isn't reliable  - even for the best optimisers or split testing genuises out there.

One of my biggest insights from the last 3 years is that there is no expert here.  There isn't a CRO genius out there who can magically fix or know the problems on your site - it all depends on your customers, team, tools and mentality of how you approach the task.  The secret sauce is not in the parts but in the connection between the parts.  Like an Orchestra.

Think of the worlds best Orchestra as a CRO programme you've seen at another company.  You could just buy all the instruments - every Stradivarius in town -  just copy everything they own?  

An Orchestra simply wouldn't work without the right quality of musicians to use the tools (instruments) would it?    So you need the musicians, the instruments and the music and you're done?  No - you still need the orchestration.  

The conductor.  Their job isn't to be the best musician or tool user - their role is to support and weave the constituent parts into a musical cohesive whole.

So when you're building your optimisation plan, think about the outcome - the music you're playing.  Look at the parts and also at the whole machine.  And be humble - acknowledge your lack of knowledge and then act to fill this gap and improve the orchestration as well as every group of instruments.

And if you're acting at your limits or know you're getting stuck?  Phone a friend.  Always ask someone else to look over your shoulder, share a problem, pore over something together. You know who you are.  Thank you.
14
2
Patrick Hathaway's profile photoCharles Meaden's profile photoPriit Kallas's profile photoSteven Macdonald's profile photo
7 comments
 
Great write up Craig! I've been in Estonia now since 2004, and it's a really nice place - A lot of talent!
Add a comment...

Craig Sullivan

Shared publicly  - 
 
Crudely Photoshopped Picture of me in a Veil
1
Add a comment...
 
Behavioural Modification, The Nudge Unit, Thorium Reactors, Nanoncellulose, Menu Psychology and more....

A roundup here I put together this weekend - hope you enjoy:

Wellbeing and Behavioural modification
Brian Cugelman (@Cugelman) has been doing a lot of work with healthcare and wellbeing stuff and may be a source of some good research or insights for you.
This is a good video presentation but he has many resources and articles on a wide ranging set of problem spaces:
http://www.cugelman.com/persuasive-psychology/podcamp-2013-persuasive-psychology-for-interactive-design/

Article on the nudge unit results:
http://www.guardian.co.uk/politics/2013/may/02/nudge-unit-has-it-worked

Hooked : The Psychology of how products engage us (video) by Nir Eyal :
Hooked: The Psychology of How Products Engage Us by Nir Eyal

 @TheBrainLady 
http://www.theteamw.com/books-from-november-24-2012/
http://www.businessinsider.com/100-things-you-should-know-about-people-2010-11#1--you-have-inattention-blindness-1

Designing for behavioural change in health:
http://www.uxbooth.com/articles/designing-for-behavioral-change-in-health/

Popular misconceptions in the media - folk psychology:
http://www.uxbooth.com/articles/designing-for-behavioral-change-in-health/

Startups replacing coaches and the shrink's couch with software:
http://gigaom.com/2013/05/04/web-therapy-4-startups-overcoming-mental-health-taboos-with-technology/

Google hacks employee health through food habits:
http://www.fastcodesign.com/1669355/6-ways-that-google-hacks-its-cafeterias-to-make-employees-eat-healthier

Psychologytoday.com has a raft of articles, some of which are useful.  @DrEscotet also tweets a lot about stuff that has implications in digital crossover.

Materials
Nano crysalline cellulose  - a few articles on promising early work with cheap wonder material made from left over wood pulp.  Yup, a hyper strong and light construction made from the bits of wood we don't use :

CelluForce Nanocrystalline cellulose
http://en.wikipedia.org/wiki/Nanocellulose
http://www.universityaffairs.ca/nanocrystalline-cellulose-is-breathing-new-life-into-Canadas-forest-industry.aspx
http://www.newscientist.com/article/mg21528786.100-why-wood-pulp-is-worlds-new-wonder-material.html

And one I love and have hope for changing our energy problems :

Thorium Reactors :
http://www.telegraph.co.uk/finance/comment/ambroseevans_pritchard/9784044/China-blazes-trail-for-clean-nuclear-power-from-thorium.html
http://www.wired.com/magazine/2009/12/ff_new_nukes/all/1
http://www.wired.co.uk/news/archive/2011-09/16/a-nuclear-future

Automation
Driverless trains, planes and automobiles : An interesting article on automation, micro craft production, wellbeing and data with the changes it may drive :
http://www.guardian.co.uk/technology/2013/may/19/driverless-cars-pilotless-planes-jobs-human

Big data & Psychology
Stephen Haggard and David Stillwell from Cambridge Personality Research presented @ #ChinwagPsych on tuning and personalising experiences based on scraping and analysing opengraph (Facebook) data.  
http://cambridgepersonality.com/

Prof. Karen Pine presented their app - 
www.dsd.me - as a behavioural shifter for various purposes but primarily in creating feedback loops for job wellbeing, productivity, personal development.  Very nicely done I'd say.

All presentations from #ChinwagPsych are here.  
http://psych.chinwag.com/presentations
The pricing Psychology presentation was also very interesting.

Misc

Quantum Computers sold to Google and Nasa -> 3,600 times faster than some current CPU operations : 
http://buff.ly/17OJQyP

Restaurant menu Psychology : 
http://www.guardian.co.uk/lifeandstyle/wordofmouth/2013/may/08/restaurant-menu-psychology-tricks-order-more?CMP=twt_gu Tomorrow's world : A telephone you can carry around with you (ROFL) : 
Tomorrow's World: Mobile Phone 13 September 1979 - BBC

Epigenetics and Brain scans:
How to spot a murderer:
http://www.guardian.co.uk/science/2013/may/12/how-to-spot-a-murderers-brain 3,000 prisoners scanned and therapy (not stick) outlined as positive outcome : http://www.wired.com/wiredscience/2013/03/brain-scans-predict-which-criminals-will-reoffend/
And a warning to check your results carefully :
http://www.wired.com/wiredscience/2009/09/fmrisalmon/
7
Charles Meaden's profile photoAndré Morys's profile photoSebastian Kaye's profile photo
10 comments
 
+Craig Sullivan, Yes Craig, I recently drew a moc Augmented reality HUD for developing consciousness. aulteristic objectives (like Terminators mission hit list) , meditation reminders, layering halo's over peoples head to encourage you to see the best in them.... augmenting our subjective view of the world into a 'pure view' of the world through virtual reality. can virtual reality be used in such a way that it transforms our perception of the world so that we see it as it really is... I'd love to catch up with you some time, maybe we meet in london if your free... June-July?
Add a comment...
People
In his circles
172 people
Have him in circles
644 people
Education
  • Boclair Academy
    1984 - 1986
Basic Information
Gender
Male
Work
Employment
  • Optimal Visit Ltd
    Chief Optimiser, 2012 - present
Places
Map of the places this user has livedMap of the places this user has livedMap of the places this user has lived
Currently
London
Links
Other profiles
The food was very good here but the service was *AWFUL*. I had to try three waitresses - two of whom ignored me - just to get another drink. Then, after I finished my main course - I waited 15 minutes for them to ask me if I wanted anything. Again, I tried to catch the eye of the waitresses but practically had to wave my arms to attract them. For the owners - you either need more staff or you need to train them properly. I normally tip 20% for good service but tonight your staff got nothing. It isn't their fault - it's your fault for letting this happen! My advice - if you like your food and can take a big flag with you to wave then enjoy - otherwise avoid and try somewhere else in Stockholm.
• • •
Atmosphere: GoodDecor: Very goodService: Poor to fair
Public - a year ago
reviewed a year ago
Fantastic local and fresh food, served with style and impeccable service. Simply amongst the best I've had across the world and worth a visit. Fish was superbly cooked and every course was a delight of flavours, presentation and culinary skill. Can't recommend it more!
Public - 2 years ago
reviewed 2 years ago
3 reviews
Map
Map
Map
Excellent barbers near to where I commute from. They are open late during the week and I've had my hair cut there often. The prices are good and a nice view of the open heath.
Public - 4 years ago
reviewed 4 years ago