Raj Kapadia

Hi, this is Raj Kapadia, and I'm the team lead in the AI and ML section at Lutz and Kendall in Ahmedabad. I lead a team of five developers, and we build products around large language models and everything related to AI. We have developed couple of SaaS products in this company as well. I am an electrical engineer, and I have a master's degree in electrical power systems. However, since my interest in programming began during my master's program, I switched my career while working. I learned programming and did freelancing work in the space of three major areas. First, I developed APIs using Python and Flask, Node.js and Express, both JavaScript and TypeScript. Then, I built applications around AI and ML, such as image classification, object detection, object segmentation, natural language processing, including text classification, summarization, and question and answer on a document, among other things. The third area is chatbot development on Google Dialogflow. There are two platforms: the essential version and the CX version. I have experience with both platforms and have built over 125 JetBots on both of them. I have recently acquired knowledge in the field of generative AI. I have experience building retrieval-augmented generation, image generation, or using the OpenAI assistant API or tool calling functionality, and I can build applications in the LLM space as well. Recently, we are building a SaaS platform in our current organization, using agents and tool calling functionality, and indexing large amounts of data. So, this is my brief background.

Decorators in Python are a kind of middleware. When you provide a decorator on top of a function, that function will first run the decorator and then Python will first run the decorator and then the actual function. I haven't used decorators for a logging mechanism in my previous work. What we usually do is create a separate file, for example, log.py, in the source folder and write all my logging functionality into that file. This file usually handles both standard output and logging into a log file. Then, I import the logger into all the files where I need it and use logging.info, logging.error, or whatever I want to log. This is what I have experience doing with decorators.

Okay, how will you deploy and manage machine learning? So, machine learning models are usually big in size. If the model is big in size, then creating a container using Docker takes time. So, usually, what we do is keep the model in a separate space, like shared storage, and then provide the path of that model into our goal environment variable. AWS SageMaker and Google Cloud AI platform both provide hosted models as well. You build the model on SageMaker or on Google Cloud Machine Learning Platform. You create your model, host the model; both provide a facility to download the model as well. But if it's feasible to use the API version, that's a better solution because we don't have to manage anything on our own. But if we don't want to go on that route, we can download the model, put it on a cell storage, and get a part of that into our application through an environment file, build the Docker image, and run the Docker image. We can do load balancing and everything that ops can do so that our instance can be up in a fraction of seconds. So, we can do this.

You explain also caching and how it is used in Python? So, you also explain how we can use Radish to cache our information. All the routes in all the routes, we can cache the body and the response for that body. And we can look if the information is there, still there. If it is there, we can send it back directly. Otherwise, we can create that information. And we can use any queue-based mechanism like BullMQ or RabbitMQ to maintain a queue. And, there's another kind of caching as well. Python has lru_cache, which is a caching mechanism. So, you can put a decorator on top of your function, and Python will cache the function input and output. Basically, hash it and store it in cache so that, for example, if you use a search API, and you search for a term, like Apple iPhone 15 Max Pro, and you get a bunch of results. And if you have used the caching decorator, what happens is that when you search the same term again, you'll get a response in, like, 20 milliseconds. So, we can use these two caching mechanisms.

Oh, can you explain the concept of async IO and how it differs from traditional setting in Python? I'm not 100% sure on this because I haven't basically used async IO in any of my application till now. But asynchronous code, basically, certain functions take time to complete. For example, API routes where you want to upload a document, and when you upload the document, it takes time to upload on S3 or cloud storage. And then you want to process it. Let's say you want to create vectors of the information inside the document, then it takes time. You chunk the information, get the embeddings from OpenAI embeddings, and store the vectors into quadrant and reply back. This whole process can take time, and it also depends on the size of the document and the information inside the document as well. So for this kind of thing, you cannot let the front end wait the whole time. So what you can do is immediately send the status back that, hey, we received the request, and this is the ID of the document that you have uploaded. And you can retrieve the ID. Using threading, you can create a new thread, do all the stuff I mentioned, and change the status of the document at every stage. This scenario explains threading, and asynchronous IO is a kind of asynchronous code. For example, let's say you are building a Telegram bot and inside the Telegram bot, you have some process that takes 1 to 2 seconds or 1 to 2 minutes to complete the process. So what you do is use async IO to let the process complete. Once the process is complete, you send a response back to the user. Async IO stays in the same function, while threading takes the process to a different thread or different thread that is going to be invoked by a function as well. So it basically takes the process into a separate path, completes the process, does its own work, while async IO stays on the same thread.

Transactions in a database is a concept that ensures database actions are successful. It's a way to make sure that a database action, like creating a new row in a table, is either fully completed or rolled back to its previous state if it fails. For example, if you want to insert a new row into a table, you can use a transaction to commit the action if it's successful. If the action fails, you can roll back to the previous state of the database. This way, transactions help ensure that either the action is fully completed or it's not done at all. In SQL, you can use session rollback or commit based on the outcome of the action, which helps handle the correctness of database operations using transactions.

Have you used any MLOps tools such as TensorFlow and? Can you give an example on how you use each? I'm not sure. I've actually I guess, I don't understand the question because MLOps is related to logging all the actions that are being performed in our application or in our script, basically, from data pipeline to model pipeline to evolution pipeline to serving pipeline. Like, you get the data, clean the data, make it available for the model, train the model, evaluate the model. You know, let's say, train a couple of models, evaluate all the models, and choose the best one to put into production. And you can do this for multiple pipelines. So I guess MLOps is related to that, but TensorFlow, PyTorch, and scikit-learn are the frameworks to train, build, and infer machine learning models. So I have experience with all three frameworks, TensorFlow, PyTorch, and scikit-learn. I have experience building, for example, CNNs or image classification or object detection models using TensorFlow and PyTorch, and I understand the whole framework of scikit-learn and how it works, from normalizing the data to encoding categorical data, filling in the data, splitting the data, training a model, such as a random forest classifier or logistic regression, and then getting the scores. I have experience with all three frameworks.

When writing unit and integration tests for a Python machine learning pipeline, I follow several best practices. The first thing I do is ensure I have the correct model by loading it so that I don't face issues afterwards. Then, I write all the test cases for that model, covering all possible cases. I also integrate the tests into the deployment process. Until all the test cases are successfully checked, we don't move to the further step. Additionally, I look into the availability of objects, such as scaling the input coming via the request, to make sure that all things are there before we do the predictions.

I guess I haven't used Kubernetes, but I have used Docker, and I have been using Docker to deploy all the projects on Python or on Node.js Express as well. So, basically, what we do with Docker is that we write a Dockerfile. And if we need multiple images, we write a Docker Compose file. But often, the case is that your work is done with a Dockerfile. You write a Dockerfile, pull the Python image, upgrade pip, copy the code, install the dependencies, and start the application using Flask or FastAPI. And then you run the Docker image using the Docker run command, and you can use -d to use it in detached mode. And you can see the logs using Docker logs -f to follow the trail, as well as provide the container ID. And I guess, with these things, you can easily manage the production of a particular application. And, obviously, you push the code to GitHub, pull the latest code, build the image, and run the application again. Also, this whole process can be automated using GitLab as well. So in our company, there is a DevOps guy who does the GitLab thing. So we push the code, merge the code into the main branch. And when we create a new job, that job is created, first goes to the sandbox, then goes to the testing phase. If everything works well, then it replaces the production environment. So I guess we can use, but I actually haven't used Kubernetes ever. That's it for this question. Thank you.