Rakesh Kumar Mallik

Hi. Uh, I'm Rakesh. Currently, I am working as a senior machine learning engineer at Reliance Jio Platform in India. I have been I have been working in various machine learning model, especially in different domains, basically in production, in supply chain, in HR. And now in Reliance, I'm working for industry safety optimization where we do industrial safety using deployed model in in the factories to detect people, to detect restriction zone entry, to detect some detect fall, and this kind of understanding will help to overall design a robust industrial setting. So I have been working in different technology. I I've been introduced with mass master learning, especially deep learning, uh, since my basic life sorry, since my MTech life in Indian Institute of Science almost 5 plus years ago. I have been I've worked in TensorFlow, PyTorch, and also a little bit of Keras and other other different technology. I am very good, compatible at Python and, uh, PyTorch. I am deploying end to end missile learning model, training from scratch, designing optimal object detection system, and training data, preparing data, deploying model, and finally, evaluating the model and these matrices and generating report end to end.

So, uh, the complexity is is a subjective word. In order to release the complexity of model, we need to understand if is the model, uh, supposed to do what it's supposed to do. For example, if you give give me a task where detects a person just or sorry, if you give me a task of classifying a person, then instead of going to very huge and heavy model like object detection, segmentation, key point, and yellow, whatever yellow kind of model, these are very heavy parameter wise. I can simply go to a ResNet and do a deduction. Sorry. Do a classification. So, uh, to in order to understand the model, understand the to in order to redo the complexity, we need to understand the what the model is doing. And given that, okay, we understand now. This is the model, and we're already deploying a model what industry is doing. Now to reduce the complexity that so many thing we can do. 1, we can reduce the parameters of the model. We can we can we can see what are the different layers which are without that, also, the model can perform as good as the actual model. We can go and check other different cost function, the loss function that's being used. Is it too much competitional expensive? We can we can have a loss which can solve our problem and which can also do the work with less cost. So, uh, last, the computation. We can review the computation this way. We can do another way way to look is if we want to reduce the complexity of the model, we can also do a better preprocessing where, like, we take care of some of the aspect where the model don't have to do lot of preprocessing and post processing. So everything we we don't need to give to the model. For example, let's say we do object detection where we predict some mounting box. Now we can have a simple NMS algorithm which is non max suppression, which actually give me a non, uh, interaction bounding box with some threshold confidence. So preprocessing, post processing, that will also help to kind of take away some of the complexity that we do in the modeling. So these are the, uh, modeling aspect I I I believe that can help to decrease the complexity of the deep deep learning model while preserving the predictive powers. Thank you.

Okay. So so to assess adversary attack, we need to understand, uh, what kind of data, what kind of, uh, what kind of distribution the model was trained on? We we try training a model on a particular set of dataset and, uh, with a train. But if you we, uh, infer an dataset which is total unknown, then the model will not perform well. So to to to assess, we need to keep monitoring the model performance. So we need to keep monitoring the model performance. Let's say we have a model deployed with some 80% accuracy or 85% accuracy, and we need to at least monitor the model performance so that it doesn't fall below very drastically to some point so that it's like, tomorrow, it's probably 30%. That means, uh, something has totally gone wrong. Now how to mitigate this? So to mitigate this, uh, as I said, we need to, uh, we need to train model on very generic robust features, and we need to keep monitoring model performance to understand if there is a chance of total distribution or trend to their model performing not not able to perform any given task.

Okay. Well, uh, I do not have much experience in l LLM, but I do have, uh, an I do have, uh, experienced some learning, uh, GPT based, uh, pre training a d p GPT based model, uh, to to to a given book, uh, book of corpus and, uh, try to query them and deploy them. So I I believe, to to to deploy your LMM based model, we need to first understand what is the problem we are solving. And and the model we need to select the model. What kind of model do we want? Do we want a very generic, open ended model which has been trained on various topic and which can query on various thing? Second thing is we need to understand the size parameters. How much are the size of the model? How much, uh, token it has been trained and what how much parameter it contain? Then we need to understand uh, the inference time, how much it take to inference. We we also need to kind cut we need to have a constraint on the model, uh, response. We cannot, um, let model response on anything and everything purely lost. If we want, it should be a limited rate. And finally, deployment. So the the ML pipeline should be such that we we detect, uh, in advance any lag in any down downtime in the model deployed server. So for that, there are different tool that can actually detect if model is down. So this kind of, uh, analysis will help to safely deploy a model and reduce the downtime.

Interesting. So, uh, there is always a trade between costs and accuracy. Computational cost competition and accuracy. Yes. So many times, we especially in financial prediction where accuracy is, uh, a Chris matters a lot. So in order to in order to but but that doesn't mean we should we can have a very huge competition that will slow the product that will slow down the total, uh, prediction. So high a high competition can lead to high accuracy model, but a high competition will lead to slower the model. So to to do this, we need to have a proper data between, uh, uh, speed and accuracy. We need to find out the sweet spot that is acceptable by the business so that that will not hamper our prediction while maintaining a threshold of the a minimal decrease in the performance that is that is tolerable. For example, let's say we are predicting the stock price. So to predict the stock price, there are different way to do. We can have, uh, time series analysis like classical models. We can have LLM. We can also train a a deep neural network with long, you know, huge number of parameters. So having a, you know, edge neural network theoretically says, uh, a infinite number of neurons can solve infinite, uh, complexity problem. So in that theory, a neural network can solve very huge complex prediction prediction. But if you want to deploy that model, the compression cost is very high. And also not only that, it will the interest time will be more. On the other hand, if you have a simple regression like random forests or or time series analysis, which let's say random forest which, you know, as you random forest, basically, it's they are highly overfitted model which learn based on the trends and then let you quickly make a prediction, but where we can have some simple even equivalent simple model like time series, which will understand that that trend of few few of the seasons, and they'll try to predict based on that. And that can solve our problem. So, uh, it will imperative to understand the sweet spot, as I said, between the complexity and the performance. We can have, if it's required, a very high, uh, very if it require a very, very, very fast response, then we can have I can go as simple as a regression model or a random forest model or a decision tree model to give a prediction. But if we need to up to 99% 9% accuracy, then we can we can go to as as good as as fast as any of the latest model.

Well, uh, okay. So if we are having a kind of overloaded model, it definitely means we're using we're having more number of parameters, more than what required. And it's it's, of course, uh, all it need to consume the memory will need to consume by all the parameters to do the inference. So we what we can do is we can quantize the model and we can reduce the number of neurons based on, uh, based on the training, based on the performance variation so that we can get as much as performance without actually reduce the we can we can get as much as performance maybe with some threshold of 5 to 10% of lag dip in the performance, but it can definitely decrease the number of computation we do. We can decrease the number of computation, and we can yeah. That will help quantizing the model using tensoriety. Uh, these are the already available tools which we which we can use nowadays to, uh, decrease the, uh, model and quantize on the top of that to to to make it more faster. Yes.

Well, strictly speaking, I I'm a Python person. Have literally never coded in JavaScript, but still let me give it a try. Okay. So since it's just, uh, array of strings, there's a chance it may contain null value or empty empty string. So file a tech to upper class, it'll fail. Like, I hope, uh, that makes sense because we are kind of every item in the string area, we are trying to convert to upper class. It's empty or null value, null array, then it will fulfill.

So it looks like you've fully connected, dense network that we have been getting it's a input layer and one is output layer. And we are So the best size kind of looks suspicious to me. We're giving 32 32 data points and, uh, having only 10 input in neural, that is that is something that's going to create problem.

I'm going to. Well, uh, the simple rule to avoid overfitting, it's to have more data. And to, yeah, to have more data, and that can be achieved in different way, we can apply augmentation to generate more data. And yeah. So and, uh, and if model it's not fixed, then we can choose a model, smaller model so that, you know, we have could not have, uh, choose model to to solve a small problem. So one thing is we can generate more data, and second thing will be we can have a to this model that's appropriate for our for our task for for our for our market prediction.

So the continuous model and innovation. The con interview. Some kind of test. You're not with somebody? Some test. What is? Uh, some kind of in the kind of recording. Some people are there No. No. It's a kind of screening video screening. Who is that in the? We'll allow you. Well, uh, so the AI field is continuously evolving, and since it's continuously evolving, uh, like, one way to keep track of it is keep reading. And, uh, I will suggest to focus on one area rather than focusing on all, but, uh, attend more and more conference. And, uh, so focusing on one particular stream and having a broader view of the other stream, that will help.

Both. I'm on the laptop, Bruce. The phone. Last person. Extract an analysis up inside from them. So to extract analysis, yes. See, uh, when you generate from the data from LLM, uh, there will be noise. There will be garbage. Like, no, as you know, if LLM doesn't have a contact, we'll have to hallucinate. They have to give, like, now some hypothetical data. So if we are using NSA data for decision making, then we should have a bounce and check so that it shouldn't go go away from reality. That's that's that's something we should do.