Product Discovery is About Goals, Not Queries
Presented at Activate Product Discovery 2021. When we go beyond the lexical search and deterministic browse paradigms that ecommerce has relied on since it all started in the 90s we can address business problems that have otherwise required tedious curation and data work. In this session we will explore the realms of lexical and semantic information retrieval through the lens of customer experience outcomes.
Speaker:
Peter Curran, Lucidworks General Manager, Digital Commerce
Transcript:
Peter Curran: Hello and welcome to Activate, the virtual search and AI conference on product discovery presented by Lucidworks. My name is Peter Curran, and I’m the general manager of digital and commerce here at Lucidworks, and I’m thrilled that you would join us today.
We have a ton of great content for you, and I’d like to show you very quickly the agenda for today. We have a little something for everybody. If you’re interested in platform migrations, we have several talks that would be of interest to you. If you’re interested in personalization, we have two that you could focus on. If you are the business user or data scientist who works with business users and want to become more productive, we have two sessions that we think you’d find interesting. And then finally, if you’re interested in the latest in deep learning and dense vector search, we have three exciting sessions, including the announcement of our new deep learning based dense vector search solution called Never Null.
If you think about all of the great things that happen from the search bar, it’s really an unlimited promise behind that blinking cursor.You have the ability to go somewhere fun like on a vacation, to buy something new, to achieve a goal, to fix something that was broken that you loved. Search boxes have unlimited promise and it all comes from a user’s goal, and not necessarily from the words that they enter into the search box.
But unfortunately, the experience of being a user is that the search box has betrayed us. We find nothing. We get stuck halfway through the project we thought we had everything we needed to complete. We have to return things, because we bought something that doesn’t fit. In other words, we have to keep re-querying and re-querying over and over again until a system finally understands our goal, triple and double and quadruple checking, to make sure that the system did in fact, completely understand us. This is the opposite of understanding a user’s goal, and it means that we have trained users to have to work with the search box in the language of the search engine. I’m not just talking about users typing out complete sentences, I’m not just talking about them speaking as if they would speak to a digital assistant, I’m just talking about the search box understanding what someone is looking for and what their goal is at a given time.
We can understand this concept of achieving a goal in an e-commerce context, in terms of the funnel. And if we look at it in terms of the funnel, we can see some places where there are conflicts in the user journey in a sample site. So for example, the funnel that we’re all aware of and there are many different formulations of this, looks something like this. Starts with awareness, making sure that people are aware of brands and products. And this is often the focus of SEO, how do I get someone aware of something so that I can educate them and drive them through to conversion. Educating is explaining the value of a product or a brand to a person who’s now aware of it. Conversion is getting them to pull out their credit card and buy it. And then of course, retention is getting them to come back and buy from you again, maybe to buy the next iteration of the thing they already bought before.
So this purchasing funnel is something that we all understand, yet for some reason in the search box, we always are trying in e-commerce to divert people to either the conversion or the education part of the funnel. And sometimes the user’s query is either implicitly telling us or explicitly telling us that they don’t want to go to that place just yet.
So let’s look at an example. Saint Patrick’s Day was yesterday and I like to cook around holidays.And so I decided I was going to make key lime pie because it’s green. I think my kids will eat it, it looks delicious. And so I’m gonna make key lime pie. I usually like to go to places like the New York Times cooking app or Saveur magazine when I’m looking for a recipe. This time I decided to go to a place where I know they have recipes which is Williams-Sonoma.
I go to Williams-Sonoma because some of the recipes there sort of feel a little bit more high-end, a little bit more appropriate for a holiday. So I start by typing key lime into the search box. And immediately you can see that Williams-Sonoma is rightly taking me to the conversion part of the funnel.They’re showing me a product and type ahead that if I click on it, I go straight to a PDP page where I can add it to the cart and buy it, short-circuiting the search results page, which is a good thing. This is exactly what they should do.
However, when I actually hit search, that key lime pie curd which actually could probably help me make the pie that I’m looking to make isn’t there, but check it out. There’s a pie. I didn’t know that but I can actually buy pies from Williams-Sonoma and they have one right there for me for 70 bucks.So one thing I really love about this website, and about this experience on this page, is that they have these delivery filters. So I can say I want something that arrives by Easter or that can ship tomorrow. Saint Patrick’s Day was coming up when I took this screenshot. So said, let me see what’s available for next day shipping, key lime pie’s not available. So now I’m gonna go back to the process of finding a recipe. And I know they’ve got them here and look there’s the tab up at the top.
And so now I’m about to leave the conversion part of the website and go to the awareness and education part of this website. When I click through onto recipes, I see that in fact, they’ve got three very relevant looking key lime pie recipes, and I like the classic. I click the first one, it looks delicious, I can’t wait to make it. And I see as I read through the directions that it’s a good fit for what I’m looking to do. It only takes 30 minutes and further down the page, there are some great reviews that say kids love it. That’s gonna be perfect for me.
As I’m scrolling through there, I see that they’re making a product recommendation. They’re trying to take me back into the conversion part of the funnel, which is a good thing for them to do. But look at the recommendations that they’re making here. They’re asking me if I might consider a Cuisinart for $279.95 or a bottle of vanilla. And I got both of those things already. While they’re both used or ingredients within this key lime pie recipe, they’re particularly related to key lime pie. They don’t have anything really special about them
that ties them back to key lime pie. But interestingly, if I were to go back a screen to the search results screen, there was a set of recommendations that Williams-Sonoma was making which look very relevant.
There’s a pie lifter there, there is the key lime pie that I couldn’t get next day, as well as the curd and a few other things. And actually, if I keep scrolling, there are all kinds of other things like pie weights and different things that I might need to be successful. And if I’m going to make this for Saint Patrick’s Day tomorrow, then I might very well want to go to Williams-Sonoma, buy some of these things, maybe stop at a grocery, get a couple of ingredients that I don’t have, and go home and make this key lime pie. The recommendations that are on that key lime pie landing page are not particularly relevant to me and to my goal. And so what does this mean?
What it means is that while we have been persistently pushed towards the conversion end of the funnel, at certain times it feels like although we’ve left and gone into the education part of the funnel, we’re not being as aggressively pushed back as we probably should be. And I don’t know the reason why. Williams-Sonoma is not a customer of ours and I don’t know how they build their website, but I suspect that those recommendations down at the bottom of the screen are probably some kind of a manually curated relationship between this recipe and those products. And as a result, there are only two recommendations there rather than a full compliment that I might get from an algorithm and they don’t feel particularly relevant.
Whereas if I had a signals based system,that is looking at what I am doing and realizing through the friction that I have a goal of actually cooking something, and not necessarily just buying something, maybe it could make some more relevant recommendations to make me successful and actually get me to transact.
What’s interesting is if I, very aggressively and very explicitly try to look for key lime pie, I go to the recipes tab of this website and then I go up to the top and type key lime pie recipes, I get to the same search results page with slightly different results but look, there’s no recipes tab. It’s as if they don’t understand what the word recipes means, even though the name of the tab itself is recipes, and this is puzzling for a user. Now as a search person, I know why this is happening, but for an ordinary end user, this could be very puzzling indeed. And what they would want you to do in this website as I looked at it very carefully is to click this recipe search button. And if I do click that a second search bar shows up and that search bar is against recipes only.
And this is what I mean by we are trying to force the user, not only into the UI paradigm that we can work with. In other words, if they don’t enter the search in this box, then we’re not really sure how to handle the query for products or recipes. We’re forcing them into a certain UI paradigm because we need them to work the way that our search engine works. And I’d like to propose that that’s not good enough. We can do a lot better in this day and age.
This is where semantic search comes in. Now, semantic search is a term that’s been used for a long time. I think you’ll see references back to it in the early 2000s. And what people were talking about then is not the same thing as what we’re talking about now. But the two broad categories of search I’d like to outline for you are lexical search, which is searching by keywords and semantic search, which is basically just searching by meaning or as the title of my presentation says, searching by goals.
Inside of semantic search there are a set of emergent technologies, which are built around deep learning and a way of organizing content in a database that’s called vector space. Vector space is not a two dimensional or three dimensional space, but a 50 or 100 or 300 dimension space where I can plot ideas, products, users, segments, documents, all kinds of different content and then go and retrieve all of the documents and products that exist in one location inside of that vector space.
And when I do that, I can do really startling things like I can search for a brand that’s not in your catalog and still find relevant results that are similar to that brand. In fact, I can search in the language that your catalog is not translated to, and find results that are relevant in the language of your catalog. It’s absolutely astonishing what you can do when you describe products and documents across 40, 50, 100, 200 dimensions. Things like language start to matter a lot less.
And this is incredibly promising. It means that we can solve a ton of really gnarly problems that we’ve suffered with for a long time.
Let’s look at a different kind of an example. Here’s another brand like Williams-Sonoma that I like a lot. This is Starbucks. And here’s the case where I just wanna transact. There is no web content or anything I’m looking for. I need caffeine and I need it now. And I don’t know about you, but I have pretty much one drink that I drink at that Starbucks and that’s a doppio espresso macchiato with one pump of hazelnut. And so here, I’m gonna start to type it in the search box that they have in the app. That it is a screenshot from my wife’s a Starbucks app, which she uses quite a lot. I start to type doppio and it’s got Tree Top apple juice, that’s not right. I keep going.Now it’s thinking about glaze donuts and cookie dough pops, which are absolutely delicious. I keep going, adding the word espresso, making sure to spell espresso with an S and now I’m getting a bagel. I keep going and start to type macchiato and now I’m getting a plastic hot cup, which I guess if you eat, it is 240 calories.And then for those of you who know how to spell the right way, doppio is actually spelled with two Ps not one. I didn’t know that. You can see that even if I use the exact right language, in fact, language that Starbucks has taught me to use, I would say a double espresso macchiato if I were ordering it with a barista or at a drive in and what usually is said back to me is a doppio espresso macchiato. So I’m using their language, I’m still not able to find the thing that I’m looking for.
The way that I find the thing that I’m looking for at Starbucks, the way that I’m trained as a user, forced as a user through a failed search experiment is to go through the browse path. I click hot coffee, I find the picture of espresso macchiato, I go to flavors, I add the hazelnut, I buy it and pick it up and enjoy it. There are a few things here that are interesting. One is that while my main drink might always be espresso macchiato, I am open to trying different syrups.
So one time I was in Hawaii, I had coconut syrup and it was absolutely great. You just can’t get it in most places. You can always get it in Hawaii. So here’s an opportunity for Starbucks to try to give me the thing that I want, my hard parameters, but then soften up one of the parameters, for example, the hazelnut syrup, and maybe offer me something a little bit different.
Let me give you another example of that. Here’s a pair of sneakers that I’ve been lusting for forever. This is a collaboration between Nike and a designer named Sacai. It looks like two shoes combined into one. Unfortunately, it’s 1200 bucks. And I don’t know about you, but I don’t drop 1200 bucks on a pair of sneakers. So I go to the Nike website and I search for Sacai sneakers, and unfortunately I don’t find it. I try searching for Sacai, and I get a lot of clothes and things like that, but no filters for sneakers. I assume that they don’t have a landing page for this really hot collaboration, which has produced a whole bunch of sneakers, not just this one. And so I’ve gotta go looking for it somewhere else.
I decide I’m going to be really persistent though. I’m not just gonna give up on that zero results page. I’m going to keep going and I’m going to browse for that kind of sneaker, the Daybreak sneaker. And ultimately I find it and look here, that’s a pretty good facsimile and it’s 95% less money than the other one. And so that’s the kind of consumer I am. I’m gonna go for that one.
Some people will know the difference, but my feet will be just as happy. Unfortunately, they don’t have it in size 12. So that’s a bummer. So I do something naughty. I go over to Amazon and I check for Dbreak type. I use the same language as Nike used and that’s a completely irrelevant result. I see at the top, they’re saying Daybreak type and also that there are two Nike brand facets, that’s weird. So I click the Nike Daybreak type link at the top and it takes me to a page that looks like this where I get some pretty relevant results, but in order to see the size, the one thing I cannot compromise on is the size. I can’t cram my foot into a size 10 no matter how hard I try. I have to pick between these two confusing choices of men’s fashion sneakers and men’s road running shoes. I picked fashion, because I’m not running in them and I get this stuff, which looks totally wrong. And not only is it totally wrong, notice that on this screen before I click that department that the Nike brand, there’s only one of them. So I’m feeling like, yeah I’m honing in on this thing that I’m looking for. Notice now even though I’ve
made that filter selection, all of a sudden I have two Nike brand facets and the results look really wacky. So I don’t know about you, but when I now see that size 12 is not available, I’m done. I’m not gonna go back to the other screen and click road racing shoes, and so I’m done. I’m gonna go back to Nike. And I bought two different colorways of that same sneaker that was pretty close. And I guess that they’re Sacai like and that I can wear one on my foot and one on my right foot and I’m wearing them right now. I think they’re awesome.
So this is the result of having a weird and very confusing user experience in search, one that doesn’t understand my goal, my language, or what’s really important to me and what’s not. The irony is if I search men’s Nike dbreak type size 12, this really long and weird query, Amazon seems to understand it pretty well. They come back with a set of results. You can see in the size section down in the lower left that my size is available.
And the only criticism I would have here is that maybe there’s a New Balance shoe, not a Nike shoe, but a new balance shoe that they could show which looks sort of similar to the Nike Dbreak, and that would be available in my size.
This is an idea that vector search allows us to do. It allows us to be really specific about certain things, the way a lexical search engine would be, and be a little bit looser about other things, try to get at the main idea of the look or the goal of the consumer.
So there’s a moral to these stories. The first is that relevancy is the most important user experience consideration in search, full stop. It is the lens through which you should look at all product discovery experiences – is this relevant for the consumer? You should remember that the user’s goal is not always inherently understandable with keywords alone. In other words, the things that they say and what they mean may be very different. And that’s why getting away from keyword based searching is so important.
Third, even great sites like Amazon and Williams-Sonoma and Starbucks, great websites with lots of budget to do great things sometimes struggle to understand where the user is in the buying journey. And then finally relevancy is subjective and contextual. But when we work with vector space, we can detect ambiguity in a set of search results and build a UI that gives users options to break up that ambiguity. This is the promise of deep learning and semantic search, and I couldn’t be more thrilled about the future.
I hope you’ll join us in some more sessions today where we talk about these exciting concepts. Thank you so much for your time and attention today. I hope you enjoy our conference.