Impedance Matching for DevRev

 

New, innovative products like DevRev are fascinating. They involve immeasurable quantities of hard work by lots and lots of people to get to market. But once you get there, how do you make it Super Easy(™) to communicate and make your audience understand the fruits of all that work? That. Is. Not. Easy.

This past week I have spoken to so many people, friends and contacts old and new, about this fascinating new adventure that I have embarked on. And I have felt like I really had to iterate multiple times to better tune the message of what it is that we provide to our customers. Communicate. Fail. Rinse and repeat. Until it works. Until it clicks.

In order to find that “click”, I was thinking of the idea of the Impedance Matching. Those of you that have an engineering background immediately understand: you need to match your messages to the audience that will be receiving it, or else … stuff will get lost :) … Too little detail and people will be frustrated - too much detail and they will be overwhelmed.

So that’s why I started to think about different “levels of communication” for different “levels of audiences” that  would understand different “levels of messages” for our different DevRev offerings. Here’s what I came up with.

Industry level - We want to make work matter. We want to connect builders to customers. We want to help build the world’s most customer centric organisations.

These may sound like different objectives - but they aren’t. Especially for people that have seen the complexities of building digital products in today’s day and age, it will probably ring true. How many software engineers never see the fruits of their work in the hands of a customer? How many of them have actually never seen or heard the voice of their customer, literally? That’s not a very satisfying place to be. What if we could shrink that distance between builders and customers? What if we could give builders and buyers, dev’s and rev’s, a true voice in the conversation?

Company level - We want to solve the problem of Information Asymmetry in digital product building organisations: different teams have different access to different information. This problem is the root cause for many Customer Experience problems: siloed teams lead to a frustrating client experience that effectively limits growth.

Great companies excel at customer focus. They are obsessed with their customers’ success, with the value that they derive from the product - and will walk through fire to help the customer get there. There is no substitute for that - but there are lots of barriers to get there. Information siloes are real, in fact they have gotten worse since the moment SaaS 1.0 made it dead easy for every department to automate their departmental processes with yet-another-cloud-platform. Where did the holistic view of the customer go? That’s right - it disappeared. And with it, so did the truly exceptional customer delight. 

CxO level - We want to offer new growth opportunities, by enhancing the customer experience at a lower cost. This means breaking down silos between tools and teams, bringing the data together, and using the latest Agentic AI technology to automate the automatable.

At DevRev, we make this a reality, today, by integrating the different tools in your different departments in a comprehensive Knowledge Graph that connects all the dots. Using that data, we can offer holistic search that reduces the information asymmetry, automated workflows and analytical capabilities on top of that. Using AI, we automate the time-consuming tasks, and make the cross-cutting information accessible through conversational interfaces. 

Customer Support - we want you to be able to help more customers quickly and efficiently, using the full information that is needed to do so, and leveraging AI assistance whenever possible. 

Leveraging DevRev, customers have seen significant drops in resolution times, much higher call deflection rates, faster customer service and as a consequence, a higher net promoter score. As a result, the company can turn support from a cost into a revenue generator.

Product Management - we want to break down the barriers between devs and revs, and make sure that you have all the information to better tune your development and support resources to your most valuable product parts. 

Understanding what is wanted and needed by your customers is not trivial, especially when you have layers of Chinese whispers standing between the engineers and their customers. With DevRev’s knowledge graph, a holistic customer view becomes accessible and actionable. With AI, we can aggregate requirements and align your resources. We can tune in to the customer voice, and foster long term success.

Head of data - as digital product organisations become successful, as their departments grow, they become more complex. To deal with that complexity, many organisations have implemented departmental tools to optimize departmental processes - and by doing so we have lost the overall picture. SaaS 1.0 has created data silos - we now face a real data integration challenge.

Using patented “Airdrop” technology, DevRev has successfully implemented a bidirectional syncing system for most sources of enterprise data in the cloud. CRM data from Hubspot or SalesForce, Customer Support data from Zendesk, Freshdesk or ServiceNow, Product data from Jira / Github, it all comes together in a fully synced up Knowledge Graph. This repository is searchable and actionable, and can drive new business processes in real time using AI and AI Agents. This will allow us to lever the holistic view on the  data as additional context for better human and AI decision making.

Head of AI - leveraging the potential of AI is on everyone’s radar. Not doing AI is not an option - you do NOT want to fall behind. But how does one operationalise this amazing technology, without spending an arm and a leg and months/years of development time? How do you limit the risk, and ensure compliance? How do you prevent hallucinations and reputational damage? 

Turns out you don’t have to do it all yourself. DevRev has spent hundreds of person-years in design and engineering time to build a product offering that does it for you, fast, and at a much lower cost. Leverage the benefits, but don’t run the risks. We help you implement AI efficiently and effectively, and together we will unlock its potential for your organization.

I am hoping that these messages are a bit clearer. We have an incredible story to tell, but it’s like so many beautiful stories: there is more than one storyline. By tuning the story to the listener, by matching the impedance, I have been trying to make it easier to understand - whatever your background.

Looking forward to many more discussions in the next couple of days, weeks, months to come. It’s going to be an incredible journey.

Rik

Pattern Recognition: The Powerhouse Behind LLMs and all Real-Time AI

Real-time applications of Artificial Intelligence (AI) and Machine Learning (ML) are transforming industries, from finance to entertainment. One of the great benefits of this age of Generative AI (large language models and the likes), is that it has opened up people’s imagination. People can now see that many things, almost anything, is and are possible with today’s real-time AI capabilities.

In this article, I would like to show the similarities and therefore analogies between generative AI and two of the most popular applications for real-time AI: real-time fraud detection and real-time recommender systems. While these applications seem distinct, they share some fundamental aspects: recognizing patterns, and then acting on these patterns. Some would joke that fraud detection systems only “recommend that the fraudster is put in jail”. Therefore it is useful to reflect on what that fundamental shared core of these, and many other, use cases actually consists of. In this article, we will argue that pattern recognition is that core.

Pattern recognition is how AI and ML are used to identify trends in historical data, understand these patterns, and then utilize these insights to forecast future behavior. This approach is crucial in both fraud detection and recommender systems, enabling them to deliver real-time, insightful and actionable results. Sometimes the action may be to offer a new previously unknown product to a returning customer (in the case of a recommender system). And sometimes the action may be that all the alarms go off and authorities are notified to forcefully lead the bad guys to a safe place (in the case of a fraud detection system). 

Large language models use Pattern Recognition

Large language models (LLMs) leverage advanced pattern recognition to understand, learn, and generate language. Trained on vast amounts of text data, these models analyze patterns in word usage, sentence structure, context, and relationships between concepts. This training enables them to develop a probabilistic understanding of language, identifying how words and phrases typically interact. 

When faced with a question or prompt, an LLM uses this knowledge to predict the most contextually relevant and coherent response by evaluating patterns similar to those it has encountered during training. By iterating on this process across diverse contexts, LLMs excel at producing nuanced, human-like answers that align with the input’s meaning and intent. In effect, the recognised language pattern is used to predict the most qualitative and accurate responses.

Fighting Fraud with Pattern Recognition

Fraud detection aims to identify and prevent fraudulent transactions or activities. This requires analyzing large historical datasets to spot subtle but repetitive patterns that indicate fraudulent behavior. For instance, an e-commerce platform might analyze user behavior, transaction details, network activity - or even a combination of all of the above - to identify suspicious patterns.

Consider a sudden surge in purchases from a new account using multiple credit cards. This pattern deviates from normal user behavior and raises a red flag for potential fraud. Real-time fraud detection systems leverage pattern recognition to detect such patterns and make instantaneous decisions about blocking new incoming suspicious transactions that display similar patterns as the fraudulent ones that were seen before.

Building Robust Fraud Detection Models Requires:

• Data Quality: High-quality data is essential for training accurate fraud detection models. This data should accurately reflect user preferences and behaviors.
• Feature Engineering: Identifying and selecting relevant features that capture fraudulent patterns is crucial. For example, analyzing ratings and their positive/negative rating distributions can help identify suspicious users.
• Robust Algorithms: Fraud detection models need to be robust to adversarial attacks, where fraudsters try to manipulate the system. Graph representations of the interactions between fraudsters and systems, like a Graph Convolutional Network (GCN) for example, offer a promising approach to learning robust user representations for fraud detection.

Recommending the Perfect Choice with Pattern Recognition

Recommender systems aim to predict user preferences and interests, and will suggest items they might enjoy. This would promote more and more profitable interactions with the provided systems - which could be shopping cart systems, media portals, or other systems that would benefit from a more intimate relationship between the provider and the user. These systems learn from past user interactions, such as purchases, ratings, or browsing history, to identify patterns that indicate user interests.

Imagine a user frequently purchases books in the kids reading book genre and leaves positive reviews for authors with a specific writing style. A simple recommender system can recognize this pattern and recommend other science kids reading books by similar authors. Real-time recommender systems utilize predictive pattern recognition to provide up-to-date suggestions based on the latest user interactions. A sophisticated pattern-based recommender system would learn how specific times and days of the week (eg. mornings just before going to kindergarten, or evenings just before bed), specific computers from which the system would be accessed (eg. home vs. work computers), and real-time stock keeping availability would matter in making the best possible decisions and recommendations.

Effective Recommender Systems Depend on:

• Understanding User Behavior: Accurately modeling user preferences and interests from historical data is essential.
• Capturing Contextual Information: Incorporating contextual data, such as time, location, and device, can improve recommendation relevance. For instance, a travel recommender system can use location and weather data to suggest suitable destinations.
• Exploiting Multimodal Data: Utilizing multimodal data, like text reviews and images, provides a richer understanding of user preferences. Deep learning techniques, particularly Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs), have proven effective in handling multimodal data.

The Similarities between LLMs, Fraud Detection and Recommender Systems

At the core of both LLMs, real-time fraud detection and recommender systems lies predictive pattern recognition. Both applications rely on analyzing past data, understanding patterns, and leveraging insights to predict future behavior and take action based on that prediction. This shared foundation highlights the versatility and power of this approach.

Consider the similarities:

• Pattern Recognition: Both applications aim to detect and understand patterns in user behavior and data.
• Predictive Modeling: Both utilize historical data to predict future actions, whether fraudulent transactions or preferred items.
• Real-time Analysis: Both operate in real time, analyzing incoming data streams and generating immediate results.

The Hopsworks AI Lakehouse: The Foundation for Real-time Pattern Recognition Systems

Real-time AI pattern recognition applications, like LLMs, fraud detection and recommender systems, thrive on high-quality, readily accessible data. The Hopsworks AI Lakehouse emerges as a powerful solution, enabling organizations to build and deploy these applications efficiently. The Hopsworks AI Lakehouse is the centralized repository for storing, managing, and analyzing data from diverse sources. It integrates the capabilities of a data lake and a machine learning operations (MLOps) platform , providing a unified platform for data-driven AI initiatives.

It seems obvious at this point, but the Hopsworks AI Lakehouse offers significant benefits for Predictive Pattern Recognition applications of all kinds. It offers:

• Feature Engineering and Model Training: the AI Lakehouse facilitates feature engineering and model training by providing tools for data transformation, feature extraction, and model development.
• Centralized Data Management: the AI Lakehouse provides a single source of truth for all data, simplifying data access and management for all teams that are developing predictive models
• Scalability and Performance: the AI Lakehouse is engineered to handle massive data volumes and supports real-time data processing, essential for real-time AI applications.
• Unified governance: the AI Lakehouse will allow for governance on source data, and provide the required explainability and transparency on the end-result, the predictive pattern recognition system.

Wrapping up

Predictive pattern recognition is a transformative force driving real-time AI applications like LLMs, fraud detection and recommender systems. The Hopsworks AI Lakehouse solution empowers organizations to leverage this power effectively, providing a robust foundation for building and deploying real-time AI solutions.

Hopsworks simplifies the process of:

• Data Ingestion and Management: Streamline the process of ingesting data from multiple sources and managing it centrally.
• Feature Engineering: Provide tools for efficient feature extraction and transformation, enabling the creation of powerful predictive models.
• Model Training and Deployment: Facilitate model training and deployment, making it easier to build and operationalize real-time AI applications.

With the combination of powerful AI algorithms and robust infrastructure, businesses can unlock the full potential of predictive pattern recognition, leading to enhanced security, less fraud, improved user experiences, and increased business value.

I hope this was a useful clarification of how different AI use cases share specific characteristics that are all facilitated by the AI Lakehouse.

Let me know if you would like to discuss!

Cheers

Rik

Can you elevate your pitch with AI?

 

Working at Hopsworks has been a great experience for many reasons, but one of the main attractions for me personally has been and still is the proximity that it offers me to some of the most exciting IT developments in our lifetime: the rise of Artificial Intelligence in an innumerable number of business use cases.

Of course, much of that interest and the fascination for it is fueled by the impressive achievements offered by Large Language Models (LLM) and their applications:  LLMs are such powerful tools, when used in capable hands, of course, that they can really offer massive productivity enhancements and therefore, new fields of application. 

In my own daily work, I use LLMs (either Google's Gemini or OpenAI 's different ChatGPT based systems) very regularly - increasingly so. I have found it to be a superbly useful tool for writing, summarizing, coding and just in general, learning. And recently I had a couple of amazing experiences that have simply been too good not to share. One of them I already wrote about: using ChatGPT as an interactive role-playing agent to practice objection handling. It is a baffling experience.

But here's another one. I recently tried to generate a short "Elevator Pitch" for Hopsworks, which goes something like this:

The Hopsworks AI Lakehouse is unique: it provides organisations like yours with the data infrastructure for your Machine Learning systems, allowing you to streamline all your MLOps tasks, teams and processes quickly and efficiently. With the AI Lakehouse, all your stakeholders benefit. First, your individual data scientist, data engineer, or machine learning engineer benefits, because they will be able to work with the same consistent operational infrastructure for all of their tasks. They will save precious time by not having to integrate the infrastructure themselves, and spending more time with their actual day jobs. Second, your data science or machine learning team leader will win because the AI lakehouse will make the team more efficient, and therefore they will be able to do more with less, and contribute more and better end results back to the business. Thirdly and lastly, your governance team will win, because the centralized infrastructure will be much easier to govern, making compliance with the latest and upcoming AI regulations much easier. This is how Hopsworks makes the booming AI application space much more valuable and attainable for your organisation. 

I wanted to figure out a way to customize this "Pitch" for different potential prospects, and see if I could use AI tools to do so. So I tried a bunch of tools, and found that they all have their different strengths and weaknesses. I found that the voice synthesis of ElevenLabs was clearly the best and most flexible around, but then also found that Google Vids offered some amazing capabilities, and could get me some crazy nice results super easily.

So: let me show you some of the results. Here's a Youtube playlist with some of the videos that I generated:

 

I thought that was pretty cool, but... I was also pretty underwhelmed with the lack of intonation and variation that was delivered by these AI voices. They are good - way better than the robo-voices of yesteryear, but they are nowhere near the quality of a real, human voice. To try and prove that - with my limited acting / voiceover skills, here's how I would deliver the same pitch:

There you go. I think it was amazing to see how far the technology has gotten already, and how easy it has become to make custom pitches for specific environments in a fairly automated way. But it's also pretty clear that we still have a way to go and that for now, personal and human content will stand out pretty clearly.

Hope that was a useful experiment. As always, I look forward to your comments and reactions!

Cheers

Rik

Training yourself with ChatGPT

Here's something I want to share. I have been using OpenAI's chatgpt for some personal training, and I have also been sharing this with our Hopsworks team. One of the unbelievably cool things you can do with it, is that you can role play specific topics with it. For example: Objection Handling - ie. to get better at dealing with some of the objections that a prospect might throw at you. Let me give you an example - completely hypothetical. I am going to try to handle the objections that a salesperson for a solar panel company might get from one of his/her prospects.

Role-playing part 1: Setting the scene

In ChatGPT, what you can do is you can set the scene by explaining the type of situation that you are in: the product that you are selling, and the prospect that you are dealing with.

It will then revert with some very detailed guidance on the objections that you may find.

You can find the entire overview of all the objections in this chat transcript. The key is that at the end of the overview of all possible objections and counter-arguments, ChatGPT basically says "Would you like to role play this?"

Role-playing part 2: going back and forth

The coolest thing then, is that the role play is not WRITTEN - it's ORAL. You literally talk to ChatGPT, and it will act out the role of the prospect, and you can practice your handling of the prospect's objections as a tried and tested salesperson. Afterwards, you get a nice little transcript of the entire conversation, of course, for review.

Here's a little clip of the way I was addressing one particular concern:

I have found this method extremely useful and interesting. It's like having an endlessly patient, unemotional and always available teacher at your fingertips. I liked it a lot!

Hope you thought this was a useful article - looking forward to your feedback.

Cheers

Rik

Happy Hoppaversary!

Today, November 6th 2024, it’s been a full year (!) that I have been working for Hopsworks. As with any youthful startups, the road to world domination is a bumpy one, and the journey with our team has been no different, so far. But one thing’s for sure: after one lap around the sun, my interest in and enthusiasm for the work that our team produces on a daily basis, is as radiant as ever. In this article, I would like to explore what I personally find so interesting about Hopsworks, and why I find the company and its products so interesting and motivating.

Hopsworks’ underlying tech is super solid

At the lowest level Hopsworks has unique technology that originated in years of research: HopsFS and RonDB. At some level, the origin story of Hopsworks is actually firmly tied to the bringing together of these two technologies in one consolidated data infrastructure platform. This project is grounded in years of practical experience, and the basic realization that these two components enable some really interesting capabilities in Machine Learning and AI. Bringing these components together, and integrating them into the coherent architecture that is the Hopsworks Feature Store in its essence, is the secret sauce of what Hopsworks does. It’s super unique because it can be deployed anywhere, and boasts crazy good performance stats - if only because the underlying storage components (HopsFS and RonDB) have been optimized for the feature store workload.

Hopsworks takes a broader view

Ok - so the tech is great. But why does that matter? Well, to me, it seems like the “Feature Store story” is “just” the start. A feature store is more than just a technology component: it is an enabler for building better AI and ML systems, by applying all the lessons learned from DevOps, Agile software development and FTI pipeline architectures. Today, Hopsworks is an MLOps platform, that brings the people and the processes that we want to apply to professional AI and ML systems, together around a feature store. This is not trivial, because MLOps is about more than just the tech: bringing people and processes together is hard, as anyone who has ever worked on a complex project will know. We have found that the feature store can be a fantastic forcing function for MLOps: the data foundation will lead the way, and it will bring the people and processes together.

Hopsworks’ unique value proposition, on technical AND non-technical levels

Sometimes people think that an infrastructure product like Hopsworks will only be used by very technical ML engineers, data engineers or data scientists, and that they are the only persona that stand to benefit from this kind of implementation. This is a complicated message, because of course it is true and not true at the same time.

It’s true because data engineers and data scientists stand to gain a massive amount of productivity and professional satisfaction, because the Hopsworks infrastructure will simplify their infrastructure related tasks. Research has shown that technical engineers spend 30-40% of their time doing non-productive, infrastructure-related tasks. That number needs to come down if we want to have any kind of productivity in building these systems, and that is Hopsworks’ objective. By providing a unified platform for AI and ML, technical stakeholders can make their lives easier and more productive.

But it’s also not true, because Hopsworks serves two other, important stakeholders, and it does so quite significantly.

First, we serve the technical team managers, IT managers, project managers and budget holders to much more efficiently allocate their budgets. We do that by reusing artifacts (features, pipelines, models), but also by offering deployment flexibility (on-prem and in the cloud) that allows you to choose the right platform for the right workload. Cost savings of 100+% are not unrealistic there, on an annual basis. That is NOT small change!

And secondly, we facilitate ML and AI governance at a very profound level. By managing the data that is used in models and tracking all the different manipulations that are run on it as we prepare, create and deploy our models, we can work towards the required explainability and FAIR principles that our regulators are going to require, in all kinds of industries. Before too long, any organization, public or private, that wants to use AI/ML for business purposes, will need to demonstrate proper governance - and the MLOps infrastructure around the feature store that Hopsworks provides will be super useful for this. As such, it will enable EU AI Act, or any other regulatory initiative out there, compliance for our customers.

So let there be bread cake

These are the main reasons why I think Hopsworks is just the best thing since sliced bread, and why it has been a fantastic personal and professional challenge to work with this team for the past year. It’s not always easy to get your head around complex platform products like this, but after this year in the trenches, I feel like I have seen a lot, learned a lot, and that we are supremely well positioned to provide amazing value for our clients. Onwards, and upwards!

The benefits of Openness and Modularity in an AI Lakehouse

I have written before about some of the important paradoxical tendencies in the Machine Learning (ML) and Intelligence (AI) industries. Let’s look at it from two perspectives:

• The Developer: the top engineer that is trying to help his or her organization move forward, by using ML and AI techniques.
• The CTO: the engineering manager that is trying to oversee these efforts, and who wants to ensure that they are pursued in a technologically and economically/budgetary sound way - while complying with all the new rules and regulations that are now forthcoming.

It should be clear that these are two important, decisive, even, perspectives that can truly determine the success or failure of AI/ML projects. Therefore, we’d like to zoom into these two perspectives in a little more detail, and then explore what the impact of openness and modularity could be on the context of these two personas.

The Developer's Context

For developers working on these types of AI/ML software projects, I think we can state some universal truths:

• developers don't like architectural overhauls
• developers are opinionated about the tools that they like and use: they want to be productive, and because of this they usually prefer to stick to the tools that they know.

At the same time: we know that data scientists and machine learning engineers spend a lot of time on setting up and managing the tools in their technical work environment. See the 2022 Kaggle survey: data scientists spend 50-80% of their time on their environment, data engineers spend 30-40% of their time on maintaining their pipelines, ML Engineers spend 20-30% of their time on this. All of this time is being spent on If we can cut the time spent on non-core ML and AI tasks by 10%, we will make a massive win for the organization and the individual contributor.

The CTO's Context

For CTO’s that are working on AI/ML software projects, we can also consider some universal considerations:

• The CTO is responsible for making sure that his organization is constantly innovating and adding value to the business. AI and ML seem to offer massive potential for innovation for many organizations: we want to seize the opportunity, before our competitors do so. If the CTO does not act proactively, there will be a CxO on the management team that will start asking questions about the potential of AI shortly - you can bet your house on that.
• Exceptionally, this is done with "big bang" solutions that will radically change an entire architecture from scratch, in a greenfield.
• However, most people don't work in a greenfield, they work in a brownfield. Therefore, big bang approaches are not possible, and most people and organizations turn to innovate in different small incremental steps.

We know that all this work will have to be done in an economically sound and feasible way. See the SpringerLink Study and the McKinsey State of AI report - both stressing the need for cost efficiency in AI and ML systems.

So: in assessing both of these perspectives, we are clearly presented with an important question: how can we figure out a way to incrementally innovate using AI/ML, provide value to our organization in an economically sound manner, while respecting the productivity requirements and current tool choices of our engineers as much as possible?

The importance of Openness and Modularity

As I have tried to explain how both the individual engineer and their CTO management have at least slightly conflicting interests in the implementation of AI and ML projects. How can we make this work then? Here’s where I think we can take a look at the software industry in general, and look at some of the technological and methodological approaches that have made software engineering so much more productive in recent years.

To do so, let’s think back to where we came from: as little as two or three decades ago, waterfall-based software engineering methodologies were the standard for how you would tackle large software projects. It took books like the Mythical Man-month and the Agile Manifesto to emerge for people to start changing their approach, and to start developing and then using iterative software engineering methodologies, DevOps practices, Infrastructure-as-Code and cloud computing as critical enablers of a more modern approach. This is what we need to apply in the world of the very specific domain of AI and ML now: we need to learn how to apply these techniques in the world of AI and ML, and that will require a specific architecture to enable this.

This architecture should therefore

• NOT be a closed, all-or-nothing environment,
• NOT limit the engineers to a specific toolset,
• NOT bound you to a specific deployment environment, offered by this that or the other cloud vendor
• NOT dictate the specific framework that you should or should not use for your AI and ML

On the contrary - our vision here should be for an open and modular architecture that would allow for us to leverage the power of AI and ML in an incremental, balanced way. Like in general software development best practices, this architecture will enable a combination of tools and processes that will propel AI and ML projects forward, and resolve the paradox that we have seen emerge.

Let’s discuss this vision in a bit more detail. We’ll call this the Incremental AI Architecture (AIAI), enabled by the AI Lakehouse.

The Incremental AI Architecture (IAIA)

Today, many AI and ML projects are characterized by structures that are very different from what we want in an innovative and incremental AI Architecture. Systems are developed with bespoke tools and processes that are specific to the project in which they are implemented, and do not allow for reuse and automation in a way that an IAIA enabled by an AI Lakehouse would. Therefore, we propose a combination of best practices and tools that will allow you to move in this direction.

Let’s walk through the different steps in a few succinct paragraphs.

Start with modularization of pipelines:

At Hopsworks, we have argued many times before that one of the first steps that one should take when embarking on an AI/ML project, is to conceptually break up the AI/ML system into three parts. We refer to these three parts as the “F-T-I” pipelines, where we would distinguish three distinct phases and processes in the development and productionalization of these systems. At a high level, we would consider

• The Feature Pipeline, where the different data sources of the input data of the AI/ML system are wrangled into the right format(s), and then eventually persisted in a way that would enable the next “T” phase of the system to proceed.
• The Training Pipeline, where the input features would be used to train and test a an AI/ML model that would be used for making the predictions that we need for our organization.
• The Inference Pipeline, where we deploy the model that was generated by the training pipeline, and start using it to allow systems and applications to use it’s capabilities and serve the predictions of the model to the outside world. In this pipeline, we would also want to monitor for any impactful changes inside or outside the system that would warrant us to revisit the process.

Once we have split the AI/ML system into these three functional and modular parts, we can proceed to take the next step to make the system into a true production-ready AI system. This will require some education of our team, and that would be the important second step in this journey.

Educate the team on the importance of MLOps for production AI systems

In order to maximize the benefits of AI systems in production, we will need to automate as many of the FTI pipeline steps as possible. This is what we call MLOps, short for “Machine Learning Operations” and analogous to DevOps.

These automations will offer

• Significant Engineering and Technical benefits that will allow us to write better software systems
• Process benefits that will allow these improved systems to be seized and maintained
• Budgetary and Financial benefits: both our operational expenditures (OpEx) and our capital expenditures (CapEx) can be reduced, enabling more innovation at a lower cost.

Once we have the team on board, we can start looking at an implementation of a professional MLOps platform - and to choose one that allows for a gradual, incremental implementation to accommodate the concerns of the Developer and the CTO.

Do a gradual implementation of an MLOps platform

Over the years, Hopsworks has gained a lot of experience with the implementation of AI/ML systems on top of an MLOps platform, and as a consequence we have developed and released the 4.0 release of the Hopsworks Enterprise platform. We have called this the “AI Lakehouse” release, as we believe that it allows for a comprehensive but gradual implementation of everything an AI/ML project needs to become successful.

Based on this experience, and using the AI Lakehouse infrastructure, we recommend that you consider the following steps in sequence:

• Implement a feature store, using the appropriate taxonomy of feature transformations for offline, batch use cases, as well as for online, real time use cases. As I have written before and elsewhere, a Feature Store is a critical piece of software infrastructure that will serve as a forcing function for MLOps.

• Implement and automate the FTI pipelines on the Feature-store platform. As you can see from the graphic above, all of the pipelines of the recommended F-T-I approach will leverage this central infrastructure, and therefore drive all users and stakeholders of the AI/ML system towards a centralized approach that will be much easier for the Alpha Developer, manageable for the Empowered CTO, and governable for the architects and compliance teams that are to ensure that rules and regulations are appropriately followed.

• Implement a model registry, thereby accurately keeping track of all the impacts and changes that are required of our production AI/ML systems. If necessary, you could combine the registry with an experiment tracking system - but we have found this to be largely unnecessary if the above steps are duly implemented. Once you have the FTI pipelines connected to a feature store, tracking experiments becomes pure overhead.
• Implement centralized model deployment for offline, batch use cases, as well as for online, real time use cases.
• Finalize with centralized feature and model monitoring, iterating back to the FTI pipelines whenever such monitoring demonstrates the need for updating and adaptation.

Last but not least, and in order to ensure solid future implementation of additional AI and ML systems in your organization, you should leverage the above architecture to measure the business results of the AI system implementation. If you do so, chances are that this will be fueling additional investments into the technology, bringing better prediction services to our organizations and a more competitive market position as a result.

Wrapping up

With that, we hope to have made the case for open, modular AI and ML infrastructure more clearly. At Hopsworks, we live and breathe this philosophy day in and day out, and we would love to help you see its power for yourself and your business. Please reach out to us if you have any comments or questions - we will be happy to help out.

Part 3: An AI Lakehouse can be a forcing function for MLOps

In the first and second part of this article series, we explored the paradox that organizations face when considering the adoption of AI systems. On the one hand, AI systems offer significant potential benefits, such as real-time operational input and improved decision-making. On the other hand, organizations are confronted with significant challenges, primarily related to the perceived complexity of AI systems.

Part 1 of the series identified three main categories of complexity: ecosystem integration complexity, engineering complexity, and operational complexity. Ecosystem integration complexity arises from the need to integrate AI systems with various input and output systems, which can involve multiple data sources, targets, cadences, and types. Engineering complexity stems from the extensive pre-processing, efficiency requirements, and diverse frameworks and languages used in AI systems. Operational complexity involves ensuring the availability and uptime of AI systems, maintaining and evolving them, and implementing and ensuring data governance.

Part 2 of the series introduced MLOps as a solution to the paradox of AI complexity. MLOps is a set of practices and tools that help organizations manage the lifecycle of machine learning (ML) models. By implementing MLOps practices, organizations can improve the quality and reliability of their ML models, reduce the risk of model failures, and accelerate the time it takes to bring ML models to production.

In Part 3, we will delve deeper into the benefits of MLOps and explore how it can help organizations overcome the challenges of AI complexity. MLOps has implications on people and processes that are used to build AI systems, and in this third article we will discuss how we can maximize the chances of such a system to be successfully implemented. We will consider the key components of an MLOps platform, articulate how to use these components as forcing functions and provide practical tips and best practices for implementing MLOps in organizations.

At Hopsworks , we have worked with many clients that have successfully used and implemented an MLOps system - and they have done so using the Hopsworks AI Lakehouse.

The Hopsworks AI Lakehouse acts as a centralized repository around which all the different process steps in a machine learning system can be built. This will then help people, and force them to some degree, to adhere to the principles of MLOps by leveraging the AI Lakehouse:

• It brings together and integrates all the moving parts of an ML system, so that the individual engineer does not need to duct tape a system together with him- or herself.
• It integrates online and offline machine learning data into one coherent metadata-based system.
• It provides easy-api based access to the right data that is needed for the ML processes to take place. For example, it ensures point-in-time-correct joins that prevent leakage of future data into the predictive model that we want to use.
• It provides automated deployment procedures that allow devops best practices to be adopted by machine learning experts
• It guarantees reproducibility and auditability that is (or will be) requirement by internal or external AI rules and regulations

The core idea of the AI Lakehouse, therefore, becomes that of using centralized data as the forcing function for implementing all the required tools, processes and team interactions that we need to make AI a much more predictable, productive, and efficient endeavour.

Just to make sure we all understand: a forcing function is a factor or event that compels or influences a change or action. It is often used in project management, strategy development, and organizational change to create a sense of urgency and drive progress. The goal is to encourage individuals or teams to take action by creating a situation where they feel compelled to do so. Forcing functions can be internal or external. Internal forcing functions are typically related to the organization's goals, objectives, or values. External forcing functions are often driven by market conditions, technological advancements, or regulatory changes. Forcing functions can be powerful tools for driving change and innovation. By creating a sense of urgency and compelling action, they can help organizations overcome inertia and achieve their goals more quickly.

And that is exactly what the AI Lakehouse can provide in the context of the AI Paradox and MLOps as a solution strategy to that paradox: all different stakeholders will be organizing their processes around the AI Lakehouse, offering a central point of control for MLOps best practices to be implemented. This makes MLOps a reality, and allows us to use the power of these principles to resolve the AI paradox that we addressed in the first part of these three articles. Effectively, AI Lakehouses make AI systems easier to implement, faster to deploy, more economically viable and structurally compliant with your organizational context’s regulations. That is a pretty great thing!

Thank you for taking the time to read these three articles - I hope they will allow you to shape your thinking around AI projects.