Nevil Shah

Hi, I'm Neville. I did my master's in physics from the Indian Institute of Science Education and Research, Mysore. During my five years of bachelor's and master's, I had several internships all over India. I had esteemed astronomy institutes where I did hands-on projects. I processed a lot of image data and used a lot of data preprocessing techniques, post-processing, and caught up with Python knowledge. And it really got me excited about the field of computer science and deep learning. As I moved forward, I once I graduated, I was pretty much excited about building things. I wanted to join a startup because I started I joined Lightspeed Labs. It's my first company. There, I learned most of my AI skills. I started working on hands-on projects. So, and I started off with building two algorithms, really. There's a face mask detector algorithm for Bosch. And secondly, we also worked on a fraud detection project, which allowed me to keep, which bagged up the second prize, and we won a pool for $2,000. It's a big achievement that helped the company get started without finding the right people. And then I worked on different software engineering works, applying these models to and fro. That's when I decided once I felt like it was time, so then soon I decided to switch to Radius AI. I started my journey at Radius AI in 2022. Early in 2023. That's when, there, I did a lot of deep learning algorithms starting from post-processing, 3D. I started training models. And I've been optimizing models for edge AI deployment. So, that's where I learned frameworks like Nvidia, Tensor, or like deployment on Tensorflow servers and also building custom AI models to cater to the needs. So, there were two major projects there. One was in post-processing where I was able to create a custom model architecture and reduce the latency of the model by at least 15 times. After that, I worked on a project as you're working with tracking between multiple cameras, like three cameras, and it tracks you would track across them. So, after that stage, as the AI field was evolving and there was a boom in chain AI. I'm trying to start up with my ex-colleague to start Hedwig AI. And Hedwig AI is responsible for building the platform from scratch. I was building the search and rank pipelines for image, video, and audio, and document retrieval and augmentation strategies. I worked with small elements to generate text outputs as well as I worked on different pipelines for creating from single images to multi-view images using diffusion models. So, all I have worked on, I have five years of experience in this field.

Machine learning model with CICD practice. To do machine learning model with CICD practice, we can have two Docker containers. One Docker container which runs the inference script. The inference script will contain the DockerGuard team. We will have access to the GPU. Now this DockerGuard team, we can and we can deploy the custom Docker script. The script will contain steps for the deployment as well as running the script. The script will basically load the model in the GPU and run the inference. We can also have two different servers. In the inference script, we can coordinate with the second Docker container, which is a Trident inference server. If the machine learning model is all converted to TorchScript, then Webex, and then TensorRT, we can deploy it on the GPU in a more optimized fashion. Since the TensorRT model is a very hardware-optimized framework, it basically works at the layer of bare bones, like matrix multiplication, and simplifies the whole model weight into a graph layer, which can directly be mapped to the GPU of the hardware. Since the hardware contains multiple NVIDIA cores, like floating-point 16 intent, based on the quantization scheme, we can have the model loaded.

Meaning transfer model for a real-time translation service. Okay, for a real-time translation service, we can use a predictive transformer architecture, which is a decoder-only architecture. This model basically takes as input the source language and converts it using a pre-trained tokenizer converted into tokens. These tokens are mapped to a vector embedding space using the tokenizer. And once we have the vector embeddings, we can pass these vector embeddings along with the position embeddings, which give the position of the token or the word in the input sequence. Now, this position in input encoding is very important because it explains the context of the tokens. We have a self-attention mechanism here, which maps the attention or how correlated each word or how dependent each word is in respect to each other. Now, if we pass this to the transformer architecture, which consists of a feed-forward network, which can serve multiple decoder blocks and a softmax layer, and a token predictor, and a post-processing tokenizer. Now, the two major parts of the transformer are multiple decoder blocks, but each decoder block contains two kinds of layers: a feed-forward network and a self-attention layer. So once we pass the input, it will pass through the feed-forward network for the input vectors. Let's say we start with keywords and tokenize into k tokens, and the k tokens are further added with the position embeddings. We have k tokens and k vectors, each with 768 dimensions. This will pass through the feed-forward network and then to the attention head, and we remove the position embeddings. And then, once we have the output from the first decoder, we add the position embeddings again. It passes further into the multiple decoder layers. And then, finally, once we have the output layer, we pass it through a softmax to get the output tokens. Once we have the output tokens, we convert the output tokens to the post-processing tokenizer, which is the output tokens in the other language. So, that's how we can implement a transfer model for real-time translation service.

Based on the feature set, we can do our TF-IDF based modeling and estimate for each content. What is the frequency of the force present in this? Now, given us the content of either the document, we can again estimate the TF-IDF scores. This TF-IDF based model can be converted to a one-shot encoded vector sequence. Now, given another content source, we can model the same four parameters for this content doc, video, view, and then compare and get the vector embedding. Now, we can do a vector cosine similarity between the input vector and the preprocessed vectors of the content, and calculate to retrieve the top k vectors and recommend those. That will be a content-based recommended system.

I'll do training on the show. Okay. To handle imbalance in data sets when training a machine learning model, we can use custom learning rates. So, a cosine learning rate or a step function based learning rate will allow us to reduce the impact on the loss function due to the imbalance in the data. Secondly, if we have imbalanced data, we can balance out the data or we can generate more data based on the distribution of the data where we don't have data. We can do synthetic data to add to the dataset to remove the imbalance. Or if the dataset size is large, we can subsample the input dataset and the pre-trained model, and use that to train the machine learning model.

In one, reducing the inference time of a deep learning model is significantly affecting its accuracy. Yes. So to reduce the inference time of a deep learning model, something like if we start with PyTorch weights, the inference time is very high. But if we move, we can convert these weights to Onnx and then to TensorRT. TensorRT optimized weights will be much faster because it's more hardware optimized and can be directly loaded into the GPU. So that's one approach. And while doing the conversion, we can always utilize the pre-training dataset and use a quantized-aware training, which will allow us to do so that's one approach where the model accuracy also doesn't take a hit and the inference time can be reduced. The second approach could be a quantized-aware training where if the weights are in floating-point 32 to a floating-point 16, we can convert them into int8 weights using a quantized-aware training approach. A quantized-aware training usually takes a subset of the training data, and while doing the quantization, this is the way the floating-point 32 weights are converted into int8 weights. It basically recalibrates the weights based on the subset of the training data. This allows us to convert the model into int8, and int8 operations are at least 8 times faster than floating-point 32 operations because the number of arithmetic manipulation operations reduce significantly. Therefore, using either one of these techniques, that is, converting the PyTorch or TensorFlow model to TensorRT or converting the model using a quantized-aware training approach can significantly reduce the inference time without affecting the accuracy.

Just pardon function. So first thing is, the category variables are converted to numerical embeddings using a unique identifier. It's a good strategy, but we can see the categories present in it or the categories could have a lot of overlap as well. So rather than using simple, we can use a sentence embedding model like InstructEmbedding, which allows you to cluster the embeddings based on the task you had. Let's say if you are doing a clustering task or category discovery or acquisition task, based on that, we can specify the instruction in the InstructEmbedding model. The InstructEmbedding model is a simple text-to-vector model, which allows us to embed this in a much more efficient and maintainable way. However, the second drawback is if the order of the categories changes or the categories change, the encoding will change immediately, and we'll have to retrain the model to account for the change in the categories as the encoding of the order of the categories would have changed. So it's always better to use a standard text embedding, text-to-vector embedding model for categorical variables like this.

So the important issue is if the max and the minimum values are the same or if there are already NAND values in this feature scaling function, then once we do this feature scaling, the data frame column values will explode. And rather than it won't be a feature scaling. So we need to make sure to filter out the NAND values as well as ensure that the max value and there's a testing max value and the mid value before applying this maximum scale.

So using TensorFlow, we can write the full architecture by hand, like defining the CNN layers, and then having x number of CNN layers which helps in decoding, or encoding the image. And then at the end, we have a softmax layer which predicts the classification accuracy. So basically, it predicts the category in which each image belongs to. So based on the number of classes, we can define the softmax layer. Or the second approach could be starting from a pre-trained model, something like a ResNet 50, which is a good embedded imaging coding model, and train it on images. Then we can add a classifier layer on top of it to predict the classes. We also fine-tune that data to fine-tune the model on a custom dataset. So we can use a transfer learning approach to produce the desired outcome, since the model is trained on images, we can use transfer learning to train the model on a custom dataset to improve image classification accuracy.

How might it have left? Could you show the uncommented study? This audio time series. So the audio data can, audio data is basically the time period versus amplitude kind of data. Now we can convert any audio data to a spectrogram. Now what is a spectrogram? A spectrogram is basically a 2D image plot where frequency versus time period, and each pixel represents the amplitude at that particular time and for that particular frequency. So if we take the fast Fourier transform of an audio signal, we can get the amplitude for that signal at that particular frequency. Now using that particular frequency amplitude and the time period, we can define a spectrogram, which is a 2D image map. This image map can be passed to a standard convolutional neural network, and it can be used for embedding the audio signal as well.

Machine learning models for edge devices. By developing machine learning models for edge devices, we need to consider several factors. One is the infrared state. The second is the model size. The third is, what are the required input image size and the RAM requirements for processing the input and loading that input in the modem and the GPU. So I think these three factors are extremely critical because when we deploy machine learning models on the edge, we need to be very careful about the inference time. At edge devices, you want an inference time or latency of a few milliseconds. But if the model is extremely heavy, you might not be able to achieve that low inference time. Second, if the model size is too large and it's not able to fit in the GPU core, then that's also something we need to consider. Third, as I mentioned, if the image size is too large, creating an HD-quality image is extremely difficult to fit on an edge device. So we need to make sure the model takes in a smaller image size, which can fit in the GPU or the RAM of the edge device.