Revolutionizing ML Performance with LumaWarp (Alexandra Pasi)

Full transcript

Welcome back to the podcast, guys. Today we are joined by Dr. Alexandra Paci, CEO at Lucidity Sciences, the team behind Lumaworp, a machine learning engine focused on performance. Alexandra, welcome to the podcast. >> Thanks for having me. >> To start, could you share a bit about your background and what you and the team are building at Lucidity Sciences? >> Yeah, so um I'm a mathematician by background. I've been working in machine learning and AI for a long time about 15 years. Um, and that's true of many of the people on our team. So, we've seen, you know, many issues in terms of machine learning and AI model performance uh that kind of distill into either accuracy issues or efficiency issues or both. Um, and as mathematicians, we were able to identify that some of these issues are really bottlenecked by the underlying math. Lumawarp is a new approach to numerical AI.

That's where our focus is. Uh that's able to perform with better accuracy, so at a significantly lower error rate and better efficiency. So when you kind of quantify it across some of the standard benchmarks, we get like a 43% lower errors, 382 times speed up in inference. >> That's impressive. And what kind of bottlenecks, if I may ask? Yeah, it's a good question. So to get a little bit technical, all of the other AI systems out there, you know, most of us are familiar with kind of the application layer. So we're familiar with interfacing with an AI tool that's maybe created for a specialized workflow or a specialized purpose, but then some of us will go a little deeper into the foundation models. What many of us don't realize is that the foundation models are actually an application of a different kind of algorithm to a large body of language data. Right? So beneath all of this, beneath all of these models, there's a couple of different algorithms. And what do those algorithms do?

They learn from training data that they've seen. Well, how does a machine learn? In many cases, it turns out that these models are more or less memorizing their training points, drawing a line, connect the dot style between the training points that they've seen. The downside of that is that when the model sees something new that it hasn't encountered before, it'll often have less reliable performance. And because you're essentially memorizing, you have to see and encode large amounts of that training data. So the models get very large and they take a while to compute. So the only solution forward to the accuracy and efficiency problem is really to create machines that learn the whole pattern. So you want them to learn these global patterns from the training data. And if you're able to do that, then they're able to adapt to new circumstances that they haven't seen before better. And they're also able to have a more compact representation of those patterns, which means that they're much quicker and much cheaper to run and train. >> I'm interested in the shifts that you are seeing in machine learning since that is your expertise. Are there any?

>> There's certainly a lot of shifts, right? So I think you know in AI and machine learning in general that whole space has undergone a lot of change within the last 10 years and I I think that this is kind of where it's settled right now but machine learning is typically reserved for more numerical data maybe a more specially trained model unless it's image based or language based and then it's often AI there's a little bit of blurring of the edges around there but I think some of the most important parts of the cultural shift that I'm seeing is that over the Last year, people were really exclusively focused on standalone LLMs, right? And then as those kind of showed some of their rough edges, it was all about rag. So, how do we get external data into these LLMs? And then that kind of started breaking down. And so people were like, okay, how do we do agents or how do we actually integrate these using these, you know, agentic systems? And then people started hitting up against the edges of these old analytics problems. How do we deal with numerical data?

How do we get better reliability? And so there have been a few different responses there. But there's been a resurgence of interest, I think, in these more specially specifically trained models for specific use cases, especially where the reliability and the privacy needs are a little bit higher. Many of the companies that we work with are in areas such as pharmaceuticals, financial trading, legal tech. So all of these areas where you have a higher desire for accuracy and to be able to keep the data private and contained. >> You are also a AI specialist. What are your predictions for the future in AI space? >> It's a good question. I really think that you have to separate out to large extent the market conditions and the technological progress that's being made. you would expect in many ways that those are on related trajectories, but I think that we've seen that hasn't necessarily been the case. There's just so much going on within the markets more broadly and much of that is driven by AI, but much of that is not that a lot of the investment into AI that we saw over the last year was very much on the capex side, right? So, it was spending in data centers and data warehouses and hardware assets.

Well, those hardware assets depreciate and the value of the data centers, the land, the powered land themselves is really contingent on continuing demand for the GPUs for AI. So I think that the question kind of remains if we do find more effective ways of doing inference or doing training or or doing AI or as businesses move towards these more kind of specialized models as they're trying to get sort of higher more reliable performance there will probably always be a place for those assets but what does that look like as the actual GPUs degrade? It's hard to say. So financially a lot of the debt that backed that expansion is backed by the hard assets of the GPUs and those have a lifespan of maybe three years. So there's some kind of economic and financial things happening that are linked in some ways to technological progress to making machine learning models and AI these large AI models more efficient.

But those things might interact in kind of volatile ways that have ripple effects for the entire market. >> I'm curious about the deployment. I know that a lot of teams get strong results in training and then their accuracy drops post the deployment or just the costs go to the stratosphere. Where do you see the biggest failure modes and how does your approach address them? >> I love this question. The answer is a little bit spicier than I think most people realize it'll be. When it comes to validating the performance of a model, you'll often use something like an accuracy metric or a metric. And if you're trying to be careful, you'll have a hold out test set. So you'll say you're not, you know, machine learning algorithm, you're not going to see this data. You're going to learn from this data. This is the data that you train on. But then I'm going to hold some back so that I can make sure that you're actually learning and get a sense of what your actual performance is on this new data.

So then you'll go and you'll look at the accuracy the test set and you'll get some idea of what you can expect from the model in production. It's very common at that point for you to be like great I have 95% accuracy let's go deploy the model and then like you said the accuracy will often tink in production. I've seen drops or heard of from my peers 95% to like 65% accuracy in production and there's a lot of fingerpointing that happen. It's like okay well maybe DevOps did something crazy when they implemented it or maybe the sales data is changing in some way we need to go retrain. Part of what has happened here is that a lot of the providers of these machine learning solutions especially these larger ensemble models have sort of hidden a meta overfitting mechanism. So they're still overfitting the model even though their validation metrics look good on a hold out set because they're sort of trying it on all of these different models and then they're selecting the best one but that active selection is a training in of itself.

So that's where you get a lot of this hidden generalizability issue. Up until this point the only answer available to people managing these models was just to retrain which came at enormous expense. If I'm the provider of a machine learning or AI cloud platform and that's where most of my revenue comes from, I love that you're retraining your model on a larger data set. The answer to this problem comes in the form of better underlying algorithms. We need algorithms that actually learn, not just memorize. But if most of the revenue just comes from cloud compute, maybe the incentive of market differentiation, which there's enough stickiness that they didn't really need it to develop those algorithms. So that's kind of the why has nobody developed these better algorithms from the incentive side. But then the other piece of the question of why nobody's developed better algorithms is that it's really hard to do the math out and then it's really hard to implement that into something like CUDA so that it scales across multiple GPUs. So that's one of the things that we've spent significant amount of investment in research and development accomplishing.

>> I would like to also touch up more on Luma warp as you bring this to market. What's your go to market focus first and what types of teams do you think are the best fit to try Luma warp? >> Good question. So we tend to work primarily with teams and companies that are working on providing a better AI or ML solution to their customers. So these teams are highly motivated to have the competitive edge and the customer experience of a better more reliable AI or ML model. And so they'll often come to us looking for that competitive edge, whether that's competitive edge in terms of the reliable performance, competitive edge in terms of the greater efficiency, which lets them have higher margins, lower cost on providing their product, or whether they're working in one of these industries where they need on premise compute and they're limited in terms of the availability of AI solutions for them because most of them require cloud compute.

So that's to say that overall the teams that we tend to work with are teams that are actually building AI and ML solutions or products for their own customers. You know these teams range in terms of technical sophistication. So the level of technical sophistication doesn't necessarily have to be super high because the actual deployment of Lumaworp is very simple, right? There's no external dependencies. You just put it on the computer, you feed in CSV or a binary file and it runs. So the integration of Lumaw warp into a system from a technical point of view is very simple. But what it does require is domain expertise in terms of knowing what high-v value targets are going to look like, what the model has to be able to do, what kind of data you want to be able to feed the model in order to better train it, and then of course just that knowledge of how to provide it to your customer in a way that's going to meet their needs. So we leverage the alpha so to speak that these companies have within their specific domains and then we empower them using this really nextgen core machine learning engine to take that knowledge to the next.

>> I would like to understand what are your plans with luma warp as it seems to be a very successful project. Can you touch up more on the >> you know I think over the next year we're largely focused in this sort of numerical AI machine learning space. So working with those customers that are trying to build these models for their customers that need the higher accuracy that are looking for lower cost better efficiency but we do have some really interesting R&D projects that we're working on right now right so expanding that into something called self-supervised learning and sort of different forms of AI that may be a little bit less structured than the applications that we're using it within now. So that's really exciting frontier research over the next year that going to tie in with a lot of the momentum in the larger AI space around world models around self-supervised learning that's a little bit more on the R&D side but over the next year we do expect Luma work to start to come to market via our customers that are going to be providing it to their customers at pretty significant scale. So I think that because of our distribution, you're going to see Luma work touching on many different industries and sort of relieving those bottlenecks across all of those different domains.

We have our paid beta program that we're running right now and then we launch our sort of more public version in April. >> Sweet. Alexandra, thanks again. I will add links in the description so people can check you out. Yeah, thank you so much.