Search and filtering, cont’d…

Search and filtering has to start appearing in a company’s customer-centered strategy, because it can’t be left to “the internet” anymore. If you’re a company, you’re going to have to start caring about this boring-sounding stuff right now.

Allow me to explain – let’s start with some history…:

Back in the 1990’s, there was a pioneer in the field of selling things via the then-newfangled internet. They started with books, then music, called themselves Amazon – and the most amazing thing about them was their “people also bought” suggestions. Back then it seemed like magic; you were all but certain to find something you didn’t know you wanted in the suggestions. 

DeathtoStock_Meticulous-09-retina

In the early 2000’s there was a music service called Audioscrobbler, later Last.fm (which still exists, in a considerably stunted version), which seemed able to read parts of your music-loving mind even you yourself didn’t know about. Using your listening habits it suggested songs, albums and artists, and it hit you right in the taste with mind-blowing regularity. And of course there was Google, a search engine for the internet which somehow managed to put the one or two sites you needed, out of the millions and millions that existed, on page 1 of the search results, and did so in fractions of a second.

These are examples of “algorithmic filtering”. Sounds geeky, but an algorithm is simply a recipe, a set of instructions: When this happens, do this etc. And for a while it was the magic of the internet.

Lately, however, it’s like that magic has gone… sour. Recommendations don’t knock your socks off anymore – sure, you might get lucky, but you may also very well get the sneaking suspicion they’re showing everyone the same suggestions. Recommendation engines seem to draw from a fraction of the available content (I’m looking at you, Youtube. Also, Spotify). Or, going by “popularity” the service ends up catering to an extremely narrow minority – and, again, narrowing the selection. Google’s results are still impressive, not least considering that there’s now more than a billion websites – and yet, the magic’s gone. As with the other ones, you are less and less likely to be blown away, and more and more likely to get the feeling that what you see is less a result of understanding what you were looking for, and more one of companies fighting for your attention.

The twin reasons for this is scale, and corporate shenanigans. I’ve talked about the scale and popularity issues before here and here – in brief, there’s just so much stuff now that it’s getting really hard for the right stuff to find its way to you even when you search for it, and “popular” becomes a misleading compass at these scales, too. The “corporate shenanigans” part is basically you being right when you suspect somebody’s trying to sell you something. You can think of it like this: If you’re a shop, and someone searches for a certain kind of product – and you have several versions of that product, some of which make you a better profit than others, wouldn’t it be prudent for you to guide the customer to those ones? 

“Leading companies understand that they are in the customer-experience business, and they understand that how an organization delivers for customers is beginning to be as important as what it delivers” – McKinsey Quarterly

I should note that I use the word “shenanigans” in lieu of a better one; it’s not that any maneuvering for business reasons is inherently bad. It can, however, become bad – as in, detrimental to the user experience – very quickly. Say, if you guide your user towards an inferior or overpriced item, because the purveyor of that item pays you money. Or if the search provider is actually a go-between whose revenue stream clashes with the needs of both purveyor and end user. 

And so we reach the reason this is something we should care about as businesses: Anything that’s detrimental to the user or customer experience will come back to hurt the business at some point. the consensus gets stronger every year: – the future belongs to the companies who put their customer’s experience and satisfaction front and center, and lets it drive the business. Even McKinsey, the business-iest of consultancies, has been on board with this for a while.

This means that, as companies existing in a digital world, we need to start addressing the filter and search challenges right away – because the difficulties I’ve described are your customers’ problem right now.

An important way of doing so is starting to recognize that search and filter algorithms aren’t just a coldly logical engineering field. It’s an immensely important part of a digital world, and one that people need, and need to be able to rely on. We need to formulate clear ideas and strategies for what we want our customers to experience when using them – and we need to use these strategies to pose clear questions to the engineering part of it. A certain divide still exists between the technical aspects of digital and the human experience side, but it hurts us, and we need to consciously work to break down that divide.

On another note, we – as businesses – may also consider that, since there’s still going to be some form of scale issues, even with much better search and filtering, there might be other ways of presenting our products to the audience. Curation and human filtering, for example, are already having an impact and will do so more and more. There’s also something to be said for peer-to-peer recommendations – though we’re going to have to do some work, both to rebuild the credibility of this kind of “filtering”, since it’s been abused a lot already, and to imbue it with a genuine human intimacy that lets people understand and trust in their recommenders. We need to emphasize the “peer” part. 

Also, we must recognize that this matters whether we have internal search – say, if we’re a webshop, or have a lot of content – or not. It matters as to how we look at being found, and what to expect and demand of external search engines and services.

In short, it matters. Time to start putting it on the docket.