Bridge Organizational Gaps to Maximize Machine Learning

This year at our annual Search and AI conference, Activate, Lucidworks General Manager of Digital Commerce, Peter Curran, shared how to use machine learning to scale ecommerce business operations and unify the organization. Curran explained that there’s a critical issue for many ML tools: they silo business users who make strategic decisions from technical experts who fix issues. In order to reach the full potential of ML in your ecommerce operation, business users and the people who manage the ML models need to work together.

Curran takes us through the search maturity curve to understand the common progression of ML models for most companies over the last 20 years. You can view the full presentation below.

Stage Zero: Basic Search

Have you ever searched for something and the website comes back with the wrong item and then you try to figure out how to fix it? At an organization in Stage Zero, a typical approach to an issue like this would be to pull a null results report to figure out why users aren’t finding what they need.

These reports are an okay place to start, but they only confirm that you’re leaving a large set of users out in the dark. At the end of the day, we want organizations to be proactive, solving problems like null results before people experience them.

A common approach to proactive problem-solving is creating a bunch of rules to anticipate possible issues. Keeping track of a decade’s worth of manually created rules creates a messy web for search teams. Tuning searches manually not only takes a long time to set up, but once you’ve put in a manual curation rule, you have to maintain it as time goes on. When creating rules, they need to be able to scale and adapt to ever changing terms and queries.

In Stage Zero, most platforms focus on head queries, terms that define a broad category like “pants.” The major issue we come across is that those people who submit such queries have low intent to purchase. The solution? Direct your platform to cater towards tail queries, searches that are more specific such as “blue corduroy pants.”

Stage One: User Driven

Many companies who are trying to grow out of Stage Zero will replatform with black box AI technologies. There are tons of these options out on the market, some being pretty good and others being very cheap. While you may get some lift at first with your black box platform, do not be fooled. This platform will eventually plateau and fail to continue learning and improving over time.

Many businesses‘ AI projects fail because they do not understand the underlying logic being applied by the algorithms. If AI is trapped in a black box, users don’t feel empowered to make the changes they need and don’t understand how to achieve business value with it. With explainable AI, our Fusion customers have control over the algorithms that go into training and the data that’s being used to drive better business reulsts.

Stage Two: Experiment Driven

Galileo once said, “Maths are the language of nature.”

Curran tells us that “signals are the language of users.”

Any time a user conducts a search, clicks (or doesn’t click) a link, visit various pages or make a purchase, they give signals about their intent. These digital interactions, taken at scale and acted upon in real time, form the basis of personalized experiences for shoppers and employees.

In addition to capturing signals, to achieve Stage Two of the search maturity curve, organizations must have the following capabilities:

• Run 1K Experiments Per Year
• Curate Synonyms
• AB Testing
• Machine Learned Ranking
• Query Intent Detection
• Natural Language Processing (NLP)
• Recommendations
• Omnichannel Personalization

Gathering good quality data, organizing it, and putting it into a product discovery platform is the way to start to understand your users at a mathematical or signal level. Organizations need to connect the merchandiser with customer motivations to build ML models for continuous improvement.

Stage 3: Horizontal Integration

People tend to say, “Machine learning? That’s for technical people, the business people don’t need to worry about that.” Curran believes that statement is fundamentally false. If the business user understands the products, the shared knowledge between the ML models and insight from customer signals can work in concert to continuously improve the digital experience. Here are some ways organizations can align across their teams:

Organize Work Across Teams

Organize Your People

• Organize Your People
• Stop organizing as “IT-side” and “Business-side”
• Organize into agile, cross-functional teams
• Communication over process
• Clear RACI for key activities
• Customer/business-first mindset
• Deconstruct heroism. Teams > heroes

Prioritize Your Time

You have to start somewhere. Take it little by little, execute that. Halve. Then repeat it. Ask yourself, what is the maximum number of experiments I can see myself doing in a year? As for data wrangling, remember to be patient. It’s okay to come up with numbers on an annual or quarterly basis.

Bridging the gap between business and IT users will help set your organization up for success. Check out Peter Currans’s full Activate session here.

If you are interested in learning more about how to leverage your ecommerce operations check out our product discovery page here for more information.