AI/ML, MongoDB in DevOps and More with Richmond Alake

[00:00:00] Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability for some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain Podcast and my goal is to help you create DevOps toolchain awesomeness.

[00:00:19] Hey, do you want to learn more about AI machine learning and MongoDB and all the things while you're in for a treat? Cause we have a specialist with us today. We have Richmond, who is a developer advocate at MongoDB and an AI machine learning practitioner with an academic background in computer vision, robotics, and machine learning. Really cool stuff. He's worked in software development and machine learning roles since 2016. And he's taught many, I think a few online AI/ML data courses written over 200 technical articles with over 1 million views. Really knows his stuff. He's really passionate. I've checked out some of his oldest stuff. Really you're in for a treat. You don't want to miss it. Check it out.

[00:00:56] Hey, before we get into this episode, I want to quickly talk about the silent killer of most DevOps efforts. That is poor user experience. If your app is slow, it's worse than your typical bug. It's frustrating. And in my experience, and many others I talked to on this podcast, frustrated users don't last long, but since slow performance is a sudden, it's hard for standard error monitoring tools to catch. That's why I really dig SmartBear is Insight Hub. It's an all in one observability solution that offers front end performance monitoring and distributed tracing. Your developers can easily detect, fix, and prevent performance bottlenecks before it affects your users. Sounds cool, right? Don't rely anymore on frustrated user feedback, but, I always say try it for yourself. Go to smartbear.com or use our special link down below and try it for free. No credit card required.

[00:01:53] Joe Colantonio Hey Richmond, welcome to the Guild.

[00:01:57] Richmond Alake Hi there, Joe. Thank you for having me. I'm looking forward to talking about MongoDB, talking about AI with you and your audience.

[00:02:04] Joe Colantonio Yeah, love it. Love it. Really excited about this because there's been a lot of buzz around AI machine learning. But as I mentioned in your bio, you've been here before I think GenAI really took off. So maybe a little bit like, how did you see the vision of being AI machine learning, because it seems like you really got into the game before it became more of a buzzword or really a hot, hot, hot in the industry.

[00:02:24] Richmond Alake Yeah. So very quickly, before I got into AI, I used to be a web developer, so I used to do a lot of, I was a full staff web developer for a few years. And in my perspective, I reached a ceiling in terms of how I can develop. So the next, what seemed like a logical step was to explore intelligent systems and machine learning. And I bought a book. It's a very hefty book, right? And I thought, okay, let me actually learn machine learning from this book. And I opened the first page and there was so many equations. It scared me back to uni. That's when I decided to go to uni and do a master's in what we call AI today. We have a bit of robotics as well. So really I wish it was a calculated full site. I was just curious and I was a bit bored being a web developer, so looking for the next challenge.

[00:03:18] Joe Colantonio Nice. Were you surprised by when Gen .AI really took off? Probably like, what was it? 2020, 2022. I mean, in November, probably when GenAI would really seem to take off. Were you surprised by that?

[00:03:29] Richmond Alake I was surprised at the uptake from the mainstream folks, right? ChatGPT really became a very, very prominent consumer AI product. There've been AI products in the past as in generative AI is not new. Natural language processing is not new. There've been a few. When I was in uni, we did a bunch of image text generation from image with a recurrent neural networks and whatnot, trying to caption Instagram images automatically, but ChatGPT came out time when it was needed. Cause with building software and startup ecosystem is all about timing, right? Timing and execution. And it came out at the right time. We're in November right now, ChatGPT came out 2 years ago. 2 years ago, which is amazing, right? Cause it feels like age, been out for ages. And in 2 years, the space has just developed so much for a long story short, no, I wasn't surprised. I wasn't surprised, but I wasn't skeptical as well. I wasn't just putting it aside like with other stuff we've added in the past, such as like internet of things or maybe big data in some respect.

[00:04:39] Joe Colantonio Absolutely. Kind of off topic, we're about to head into 2025. GenAI has been a thing. Do you see other forms of AI or machine learning being neglected that you see maybe is about to pop off as well, maybe in 2025?

[00:04:53] Richmond Alake I think really the whole space is moving towards a convergence in terms of the capabilities or modalities of AI. You're going to have Generative AI is mainly focused on text and at the moment with text generation, but you're seeing image generation as well with a lot of diffusion models, but really there's going to be a convergence of functionalities that comes out from this model. I don't think any capability or modality is going to get left behind.

[00:05:19] Joe Colantonio Nice. I know one thing this year seems to be a trend is multimodal AI. Before it was more like text, now it could work with images and things. I think your background is also in computer vision. Is that the same thing, multimodal AI? Cause I've seen like Claude just came up with something where I think it could automate a lot of business tasks. It's not very successful right now, but is that the same thing as computer vision?

[00:05:42] Richmond Alake Well, it's using some computer vision elements. I'm guessing, I don't know how the team over at Anthropic or what they're doing behind the scene, but they've been similar tools that have come out in the past where we're taking screenshots of the actual computer or the system. I'm passing that into a model for understanding or interpretation of next steps. They're doing this obviously in real time are very, very high speeds, high latency, low latency, I mean. But one thing is what Anthropic puts out with, I guess you're referring to computer use.

[00:06:16] Joe Colantonio Yes.

[00:06:18] Richmond Alake Yeah. It's pretty much the promise of AI, right? Which is automation and freedom for manual labor. And what we're seeing that, and again, what they put out is not new because Microsoft have done, they did something similar. Maybe a few years ago, they had a bit of a backlash in terms of privacy and sending up your data from your computer over to a big tech. But one thing I would say is we're going to see more systems like that, and we can categorize them as agentic system where you have this systems are able to act on your behalf and actually use the components of your system infrastructure to actually achieve an objective. And over at MongoDB, we're seeing a lot of this conversation popping up and we're getting involved.

[00:07:07] Joe Colantonio What do you mean by you're getting involved then? I guess let's bring in MongoDB into the equation here. MongoDB, what does it and how does AI machine learning play into it and how are you building it out to meet these needs like you just mentioned?

[00:07:19] Richmond Alake Yeah. So I'll talk about what we're looking at, what we've seen in the past 2 years, right? The form factor. Just to give a very quick background, we saw ChatGPT, came out around 2 years ago. And what we could do is just query and send in the text and we got some response back. It blew our minds because this little model was a world compressor and had most of the internet inside of it. We can interact with it. The next form factor, we brought in our own data, supplemented it with the user query. Now we're getting really good responses that are relevant to us, that are personalized. Now, we're seeing more agentic system where the systems can actually act on our behalf when we give them objective. So MongoDB is, we've been at the game of storing data and retrieving data for over a decade. One thing that we know is how to retrieve relevant data and supplement this data over to this agentic system or to this LLM application, mostly RAG pipelines and AI agents and agentic system. When MongoDB really plays in is, and just to give a bit of background, when we're talking about supplementing data, one of the key things that's happened in the 2 years is, I guess, the proliferation of vector embeddings. You can imagine a data object. Let's take like a piece of text. We could actually pass it into this embedding models and out comes a numerical representation of this data object. But this numerical representation has captured the semantics and meaning of the data object. And what that allows us to do is, and we can actually start to calculate the similarity, the vector similarity between various data objects in various vector embeddings in a high dimensional space. Now what I've been to say is these things are not new. Vector embeddings have been in the world of computer vision for years and NLP for years, but now they're more accessible. The models are lot more capable. They have several billions parameters, which means that there is more performance that we can get out of them. Now, what MongoDB allows you to do as a developer, when you're building this application is actually store this vector embeddings in a database, alongside all your usual operational data. And then we allow developers and our customers to use this data to build LLM application in whatever form factor they take. When I'm talking to customers building, right, I tell them MongoDB is your operational database and your vector database. But when I'm talking to developers building agentic system, I get a bit more higher level and I tell them MongoDB is the memory provider for your agentic system, right? So if you think about agents and you think about humans, humans need memory to actually perform really well, right? We need different forms of memory. MongoDB is the memory provider for your agentic system. I'll put a pause there. I've said a lot, I bet you have some questions.

[00:10:23] Joe Colantonio Yeah. I'm a little bit older, I assume a lot older than you. And so MongoDB is, it's been around for a bit, like you mentioned, but so when I think of, and also I'm a novice with AI, so I never thought about where AI, the LLM is storing the data. Is it always storing in a database or are you saying MongoDB is in order to have an LLM, it needs to store information in the database and MongoDB has been created to be optimized to deal with all kinds of LLMs for that.

[00:10:51] Richmond Alake Yes, exactly that. But one thing I need to mention is a caveat, right? LLMs, they have something called a context window where they could hold data temporarily for maybe for the conversation. And so we have this context window, you put your input, you could put a bunch of data there, but if you have another conversation or another session, that data is lost. We need to store that data. We need to persist the data and retrieve it whenever we're engaging in or picking up from a session. So essentially you have more, you can use MongoDB as a conversational store, but also with MongoDB, you can actually start to build something that we call retrieval augmented generation, which means that you're using domain specific data as input to the LLM to get more better output. You might have some data on your laptop about yourself and now you can have like a local LLM where you can just have this LLM, get access to your data. But what MongoDB can do is store information about the data, such as the vector embedding, and enable you to retrieve data for semantic search and supplement that to any query to an LLM. This is what we call RAG, retrieval augmented generation.

[00:12:05] Joe Colantonio Nice. So give me an example of a real world application that would benefit from this.

[00:12:09] Richmond Alake A very simple one is a chatbot. We all know chatbots, we've used them for, like you said, you're much older than me. I don't know when chatbots came about, but I feel like for me, chatbots have been around since I could use the internet.

[00:12:27] Joe Colantonio Ouch!

[00:12:27] Richmond Alake You brought your age up, I did it. But really chatbots have been about, right? So for several years, but one thing that chatbots have been lacking is a bit of personalization, which is the first thing. The second thing is the ability for this chatbot to retrieve information behind the scene that is relevant to the current conversation, be it the customer or the problem domain. What we have today is a system or mechanism of retrieving information by semantics and meaning. And that is also supplemented by the incredible power of LLM to generate response that are just, that sounds so natural language like. And with this, you can bring this capability into chatbots and start to create customer service chatbots that sound human-like, that can use all the data within your organization to help troubleshoot problems and could help people get to a resolution much faster. This is within the customer service industry. And I'm using that example because it's common, right? Most companies have chatbots or have a customer call center, have a customer service, a function to help solve problems. That is one of the common form factors and use cases we see.

[00:13:48] Joe Colantonio Gotcha. Awesome. All right. So once again, I'm going to show my age. I know back in the day, having a database, just with stored procedures, sometimes it cause a lot of performance issues. Just stuffing in this with all this other data now, are there performance issues, has this been a solved thing that I missed out on? Like how does this handle, it sounds like you're using large data sets, so how does it impact performance at all?

[00:14:10] Richmond Alake Yeah. So MongoDB has been at the game of storing data and retrieving data for over a decade. And what we actually do for our customer is bring up all of our learnings on how we handle data, unstructured data, and the way that we actually structure data within MongoDB is so familiar to developers. We bring all of this learning directly to our customers. What that means is we know what it takes to optimize data retrieval. And we know the applications that our customers are building and the use cases. MongoDB is very well built for performance and actually low latency for AI application, especially if you have the application running in real time. So we have a bunch of customers that are building conversational AI that can operate in real time or chat bot that can actually solve problem and retrieve problem, retrieve information in real time. And then we have more features that allows performance to be balanced across different data types, which is another key learning. Imagine you have a chatbot that uses vector data more heavily than the operational data. We have features that allow you to actually balance the performance or the allocation of compute to a certain operation or a certain node. This is one of a very key feature that we learnt across the years. And we'll bring into our customers that are building the future essentially.

[00:15:39] Joe Colantonio And so good point there is once again, we used to have like a database admin. You own the database. You knew everything about it. You come from a background where you started off as a web developer. So nowadays is it more like everyone is a full stack. So they need to be a database expert as well. And MongoDB also makes it easier to stay up to speed on what they need to know.

[00:15:57] Richmond Alake Yeah. So I will tell you, I work in MongoDB, but I've been a fan of MongoDB for years since I was in university. And that's because MongoDB for developers did this thing. It unified the application stack when I was a web developer. So I didn't start off as a full stack web developer. I started off as a front end developer, but out came MongoDB and Node.js. And it used similar paradigms that I was familiar with as a front end developer within the data layer and with Node.js and a bunch of ORMs like Mongoose were able to connect the entire stack. And then I could transition literally because MongoDB, I could transition into being a full stack developer. We're doing the same for AI. We have an AI stack and what MongoDB does is it applies the same paradigm, the same framework of operating with LLMs or other components of the stack. What I mean is LLMs understand JSON structured data. In fact, for it to enable to use in LLMs, the schema you provide to the LLMs is in JSON. And we've had some customers where we actually enable scalability by storing these JSON schemas in MongoDB. It feels like such a natural fit. And you see that with the amount of code you have to write, which is quite short. To basically answer the question, that's what we're doing. We're just shrinking the space. And it's not, one thing I find is most developers don't want to become database admins, right? So we want to do the cool stuff, which is build code application. So I don't want to spend a lot of time thinking about optimizing my database. And so what MongoDB does is it's a very developer-friendly database. We have a developer-friendly platform where you can start to see performance insights on the interface, query optimization on the interface. And it tells you key information. We make developers develop as opposed to maintain database systems.

[00:18:02] Joe Colantonio Love it. Yeah. So that's a big selling point. So also you are a developer advocate. How much education do you need to do in order to let them know? Yeah, MongoDB has been around for a while, but it has these AI/ML capabilities that's going to enhance your future development as well.

[00:18:16] Richmond Alake Yeah. So because MongoDB is done its job really well, we're actually quite fortunate in the fact that within the developer space, we are known as the NoSQL database, right? So again, right from when I was in university, I was introduced to MongoDB up until now. So we're very popular within the space of application development. Now AI today presents an opportunity for developers to create new intuitive tools and applications that they couldn't before. MongoDB has always been with the developers. We're almost like a natural fit for developers that are willing to build innovative products and do it very quickly. We really focus on developer productivity. And we take that to heart because we partner with most of the application, most of the key technology provided on the application stack and enable a developer to spin up a database and create an application, an LLM application within minutes and even a more complicated application within a day or so. And that's because we focus on making our developers productive.

[00:19:23] Joe Colantonio Nice. So you made the point that developers don't want to be DB admins. Also with LM AI, the thing lately has been security. So I'm sure they don't want to be security people either. Has there anything built into MongoDB to make that aspect a little less daunting for developers?

[00:19:41] Richmond Alake Yes, security is nothing new when it comes to application development and because we've been in the game of storing data securely, safely, MongoDB is a trusted platform for holding your data securely. So we have different features, various level of securing and storing your data. We have role-based access control. We have queryable encryption. We have field level encryption, document level redaction as well. So there are different features and tool sets at various level for developers to actually enforce enterprise level security and governance within their application. And the one thing we do is we don't make you a database admin, right? All of these features are very easy to understand, very easy to use, and just get the developer more productive and doing what they do best, which is develop.

[00:20:36] Joe Colantonio All right. Developers listen to this like, Oh, I used MongoDB with the AI/ML, I'm going to create something really quick. What do they need to know? So you've started as a web developer and then you taught yourself AI/ML. A lot of people always ask me, what do I need to know in order to be successful as a developer with the AI/ML? Any first steps or things you think are critical?

[00:20:54] Richmond Alake The first thing is don't do what I did, which is buy a master's book and scare yourself to get a master's. I've seen a lot of YouTube videos that probably covered a few of the courses I did at university and I paid a hefty price for a university degree. I would say the need to go to university to become really knowledgeable in what we define AI as today. And that's very key. What we define AI as today is reduced. We have the internet, there's YouTube, Stanford lecture videos are available online and top university videos are available online. Deep learning AI has some very good resources, including a resource sharing MongoDB for AI applications. So if you're a developer today and you want to get started with AI and machine learning, I would say, always start with what languages do you know? What programming language do you know and you don't know? Right? Python is pretty much the most dominant language when it comes to AI application development or data science. So if you need to learn a new language or you don't know Python, you need to learn Python or even JavaScript, maybe that is the gap you need to fill. You could start looking at your environment and start to think, what can I solve with AI today? That's how I started. I looked for a problem and I tried to solve it with AI. And that sent me down a path of exploring different tools, different form factors. And you start to build up knowledge through curiosity, which is how I got into AI anyway. I was just curious. We have loads of resources over at MongoDB that help developers with backgrounds in Python, in JavaScript, in Java, get started with what we define as AI today, right? RAG and agentic system. So in the show notes below, there's going to be the links to actually get you into a GitHub repo. So if you're a very hardcore developer, you just want to show me the code, right? So it'll be a GitHub repo that actually gives you a bunch of resources, a bunch of Google core lab notebooks to show you how you can build AI applications.

[00:23:04] Joe Colantonio Love it. All right. In my industry with software testing, a lot of testers are saying we're going to be replaced by AI machine learning. Our role is going to change. As a developer, AI machine learning expert, where do you see it? Because I was speaking with someone that came up with me at the same time. He's a developer. Now he's a CTO and he said he was using Cursor to create an app. And he said it was incredible. Like he didn't have to do a lot of things to get a working application. Do you see development being replaced by AI machine learning and what skills will people need in order to keep up with maybe the advancements that you see coming down the road?

[00:23:39] Richmond Alake I'll be lying to you if I say I don't see some development being replaced by AI tools today. But to say, to go to the extent of saying developers will be replaced. I think, it's too early to call, right? But one thing that there is some development that will be replaced by AI. And that's because when you're building a software and you're building tools, there's a lot of boilerplate code that is just repetition, right? Those things AI can probably help you with. Oh, create a button in this image. Okay. I want a button to maybe send a HTTP REST call to the back end. Okay. We create a backend and all of these are all sort of like paradigms that are standard, but there becomes a point, there always comes a point where you actually need an actual knowledge of building applications and system design and debugging and AI can't help you. AI might help you get to the 80% which is very useful for becoming productive. But there is 20%, which is just experience and expertise. And again, that 20%, I don't bet on language models or on AI ever. So that 20 % will eventually become maybe 5 % in a year. It may be two or five years time, but the job and the function of a developer will only evolve along with it, right? So this is not the first time humans have gone through of evolution, right? So we went through the agricultural evolution, industrial. There are different things that happened within mankind history that forces us to rethink the nature of work. AI is a forcing function and it's an accelerating function, which means we have to rethink what we define as work and start to rethink how we start to maybe define our purpose and find fulfillment in the things we do. But it's not going to replace. If there was an AI that could replace me, I would use it in a heartbeat.

[00:25:44] Joe Colantonio Really?

[00:25:44] Richmond Alake I've got a lot of educational code materials to be right and say, if I have an AI, like an AI version of Richmond just coding away, I would love it while I'm doing the podcast. There's just an AI version of me just coding away. That'll be lovely.

[00:25:57] Joe Colantonio Nice. So you also have a background in robotics. Where do you see robotics going? This is off the wall, but you know, Tesla released something. It was, I forgot the name of the robots are, but someone said it wasn't they were kind of being remote controlled. So it really isn't where people think it is. Do you see robotics take being the next level from AI?

[00:26:16] Richmond Alake Very quickly, my background in robotics was actually in space robotics, so we have rovers that do space exploration and we were just calculating the freehand motion in space, which was just inverse kinematics and kinematics, which is basically mechanics, right. We did a bunch of path planning and working with slam and all of the good stuff, which is all fun, but where do I see AI in terms of robotics, right, going today with the stuff we're seeing from Tesla up to this in-Boston dynamics.

[00:26:51] Joe Colantonio Yes.

[00:26:52] Richmond Alake Very quickly. We're not going to see iRobot anytime soon. I don't know if you've watched the movie iRobot, but that's not going to happen next year, but again, I don't bet on the innovation on the speed of innovation we have in AI. And also the determination of us as a collective, when we set our eyes on solving problems, especially when there is huge capital involved. Maybe in 10 years, I would say in 10 years, we might be having household humanoid robots, and that is not even far fetched because we have toy robots that kids can buy and play with, but again, the definition of useful robots or humanoid robots is going to be a spectrum. Do you have one in your house that can just maybe lift his arm? Or you have one that can just actually do all your manual labor and all your laundry and whatnot. I think we're going to get there. That's the objective. How quickly is anyone's guess? But there is a lot of very determined and educated people working on this problem.

[00:28:03] Joe Colantonio Love it. Yeah. And as populations get older, I'm looking forward to a robotic caregiver for sure, so hopefully 15 years, it'll be there.

[00:28:11] Richmond Alake You're always bringing up your age and you look very young. So you'll be there to see our robot overloads.

[00:28:21] Joe Colantonio All right, Richmond, I'm just thinking of people that are listening to this like, well, MongoDB sounds awesome. I assume all of them are using MongoDB, but for those that aren't, they're working on a more legacy application, they want to move to MongoDB, how developer-friendly is it? What does it take to migrate from one to the other? Is that something that's common?

[00:28:38] Richmond Alake So we actually built internally a product that helps people move away from relational to MongoDB databases. I'll get a link below to our relational migration. And we have a bunch of programs that help modernize legacy systems in enterprise organization. So we have the app modernization program, and we also have the relational migration tool that helps people migrate. And one thing we also see people doing in when they're building this AI application is using maybe specialized databases, right? They go into this space thinking, okay, I need to use vector data because I've been told I need to search by meaning. So, okay, now I need a vector databases, but what most people then find out is I need to store more than just vector data. I need to store my metadata. So they're pretty much invested in a database solution. Then they plug in another database solution to host their metadata operational data. Now you have two database solutions, right? You have to hire two developers or maybe more. You have to deal with two development systems. I have to sync data between them. And I already said developers don't want to become database administrators and definitely not for two database systems. What we do in MongoDB is unify the functionalities that we need today to build AI application. Within MongoDB database, you can store your vector data and your operational data. And this makes MongoDB one of the most intuitive developer-friendly databases for most folks are building out there. So that's my two cents on that.

[00:30:24] Joe Colantonio Okay, Richmond, before we go, is there one piece of actual advice you can give to someone to help them with their MongoDB, AI, machine learning efforts? And what's the best way to find or contact you?

[00:30:35] Richmond Alake One piece of advice is there is a lot of noise in the space. You need to figure out how to block out the noise. And one way to do that is to follow people such as Joe, listen to podcasts such as this, to get some ideas of what, at least at a high level, what people that are in the space are thinking. The second thing you should do is look for learning resources. Learning is not as expensive as it used to be, except for university education, the prices are going up here in the UK. But learning is, there's so many learning resources online. And amongst that, in MongoDB, we are in the art of structuring and creating educational resources for developers that makes them productive as quickly as possible. Not just for using MongoDB, but for actually building and understanding RAG systems, agentic systems, and LLM application. And what that means is we have a portal called the developer center that has tons of tutorials that could teach you how we build these systems using MongoDB. We have a GitHub resource that could give you the code that can show you how to develop this system and implement them. But even higher level, we have several articles, our thought leadership pieces that actually show leaders and developers, what to think, what questions should they be asking when they're thinking about RAG, the AI stat or agentic systems. So we care about the developers, we care about the people building these systems, and we provide educational content for various audiences. So all of where we do this are going to be in the show notes.

[00:32:25] Joe Colantonio And you can find it down below, so definitely check it out. All right, before we wrap it up, remember, frustrated users quit apps. Don't rely on bad app store reviews. Use SmartBear's insight hub to catch, fix and prevent performance bottlenecks and crashes from affecting your users. Go to SmartBear.com or use the link down below and try for free for 14 days. No credit card required.

[00:32:49] And for links of everything in value we covered in this DevOps toolchain show, head on over to testguild.com/P176. So that's it for this episode of the DevOps toolchain show. I'm Joe, my mission is to help you succeed in creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good. Cheers.

[00:33:12] Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation, software testing, and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit TestGuild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

[00:33:58] Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.