Thuvaraga Krishnarajah

I'd be happy to help you understand more about my background by giving a brief introduction. Okay. I have been passed out in 2019. I'm from a BTech Mechanical background. Initially, I worked as a freelance developer. There, I used Node.js and MongoDB, and also Python to automate WhatsApp messages and other tasks. Then, after I joined Arithmetic Solution, I worked in embedded systems. So, there, I used ESP32, STM32, and PCA boards. There, I worked on ICT protocol and CAN protocols. After that, I joined IndianDynamics. There, I worked on ROS. There, I worked on SLAM, G mapping, navigations, and hardware and software integration using it. After that, I joined The Limited. There, my work involved converting a semi-automatic Jeep Compass vehicle to a fully autonomous vehicle. There, I worked on driver systems. There, I used Prostoke. And, also, I worked on auto wear to simulate sensor data and create sound mapping. Then, after that, I joined Polio and Infotech. There, I worked on simulations. There, I used Gazebo to plot mapping using a sonar sensor. And, also, I worked on Ignition Gazebo simulation to set up an environment and navigate the vehicle there, and read radar camera and GPS data, and process that for deep learning. Also, I worked on reinforcement learning for self-driving. There, I used CARLA, and I used PPO. I have worked in Python, ROS, and also C++ and deep learning, and reinforcement learning. This is my main focus.

Python based approaches I will use computer visions, deep learning, and reinforcement learning, and also we can use Python for automation also. Using computer vision if we want to detect any object. For example, in Collaborative robots we will mainly be used in mechanical manufacturing and production environment. Therein quality testing we have to test the product is in proper quality. For that we can use deep learning and computer visions to detect that effect of the robot. And also in the environment if a human faces any accidents we can detect. And if humans are following all the safety procedures like wearing a helmet and having all the safety requirements with them, we can use any deep learning or computer vision model to detect all those things. And machine learning and data analysis from starting to end how the production plant and supply chain is working we can analyze in the data and we can calculate how the loss is going to be and how can we eliminate those losses? We can use a loss algorithm to process all those things. That we can use Python to analyze all these.

Can you share how you have approached the incorporation of artificial intelligence models into robot presentation for improved decision making. While I am working in reinforcement learning, we only use camera data to gather evidence of obstacles, lanes following, and intersections in the route, as well as any directed objects. If the robot is in a warehouse, it needs to navigate to its designated position and perform its assigned task. While navigating, the number of turns, lane following, and other factors are crucial. So, we can use RL algorithms or deep learning algorithms to teach the robot how to follow roads, avoid obstacles, and perform specific tasks. If it's performing a particular task, we have to guide the robot on how to do it. We can use artificial intelligence to train the robot to perform specific tasks.

In what way can reinforcement learning within a simulation framework like Isaac Sim in robotics? Yes, of course. There are lots of algorithms in reinforcement learning, DQN, PPO, and DDPG. So first, we have to decide what we are going to give as input. We are going to give camera input or Lidar input or any other data as input to our reinforcement learning model. And before this data is analyzed, we have to define what is the reward, what is the penalty, and what are the termination conditions. Everything we have to define and analyze within Isaac Sim itself, which has lots of APIs, camera APIs, Lidar APIs, and also obstacle detection APIs, and collision detection APIs. Everything will be available in Isaac Sim itself. And also, the object's dynamics and obstacles. So now we have to write a reward policy or penalty policy or termination policy based on all these conditions. And when the robot completes the task, we will give a reward. And when the robot disobeys, we will give a penalty. And when the robot does collisions or makes a big mistake, it is unacceptable. For that time, we will give a termination condition to the robot. So now we need to give an input to the reinforcement learning model. What is the input this robot is going to receive? So using Isaac Sim's APIs, we will give input from the simulator to the reinforcement learning model and analyze how it is behaving in the simulation. Based on that, we will give a policy, and we will make a reward, penalty, and termination. Based on that, now we have to consider what is the action space. There are two types of actions: discrete actions or continuous actions. That is very much important in reinforcement learning because based on this action category only, we will decide what kind of distribution we are going to use to take the decision. This is a very much important concept in the reinforcement learning. It is a continuous space or discrete space. So in Isaac Sim, what kind of action is available, which is suitable for Isaac Sim simulations. Both continuous and discrete actions are suitable. So based on that, we can make the algorithm. It can be a Gaussian distribution or a beta distribution. Gaussian means discrete, and the term is as continuous maximum space based on that. We can write an algorithm to automate the robot in the Isaac Sim. It's not only Isaac Sim. We did that in the Carla simulator also.

What step you should take to ensure a simulation scenario created using LLMs accurately references the given real-world problem? LLM is a language learning model. So, if it's an example is a collaborative robot, it has to collaborate with humans involved. If a human gives some command to the robot, that human can give the command in their language. Now we can use LLM models to determine the words that the human is using, and we will vectorize those words and pass this, or the words or sentence that the human language command of the robot. That's where we will use LLM models. And to make that LLM models, we need to have lots of data. So, examples, an example in the while in the term, we are typing lots of words that Google will automatically autocomplete all those words. Right? All those are happening because of the LLM model. Previously, we may use the words in social media based on that, we will collect all the tasks and train the LLM models after we are deploying that, and they are using that LLM model to predict your questions and answer your questions. And while we are searching something, they are giving lots of website links. These are the solutions for your questions. This all happens using LLM models only. So, the question is what steps would you take to ensure a simulation scenario created using LLMs accurately references the given real-world problem. To achieve this, we should ask the robot to perform a particular work or task that is similar to the real-world problem. We should also give the robot some real-time problems to solve in the simulation environment. In a production environment, we may ask lots of questions, like how many production things happen, what is the quality result, and everything. So, in the simulation, we make the robot respond like it's in a production environment, and we can ask this kind of question, "How will humans or our managers work," and we will try to resolve those problems in the simulation environment. We can approach these things in the simulation itself, and we can check how it is working in real-time and how it is working in the simulation environment.

Yeah. As in the white function is like a function, where we will call a function. And until that function will give a return, or suppose any delay will happen, we have to use the await command to wait there until that function gives a return. Then only after that, other things will be called. If any function or any other process depends on this result of that async function, it will wait until that it is completed. Then other process is started. So it won't block the call. Yes. It will wait. This is not only in Python. If you can see, async and await functions are in Node JS also. Suppose APIs, won't be, is an example. We did, we are extracting data from some database. The data is far away, so sometimes the delay may happen to get the data from there, but our process depends on the data. So the async function will defend the async function, and we will use the await command until the return is coming. So until our query is connected with the database, until we get the response, the function will wait for that response. After it will get that response, only other process will be started. If suppose if we didn't use an async function, we are connecting with some HTTP protocol, WebSocket protocol. But some Internet delay, we didn't get the response on time. The function will be terminated or exit, or it will block other process. But here, the blocking won't be happening. It will just wait for the process.

Given this period of code for robotic simulation where a robot must navigate through a series of waypoints, identify and expand the bug that will cause a robot to skip every other waypoint. This function is similar, it's just a for loop only. In the first line, the waypoints are defined in an array, and in the second line, there's a for loop from 0 to the length of waypoints minus 1. In the for loop, from 0 to length of waypoints minus 1 is nothing, but the first array is the first item of the array, which we represent as s, the syntax is 0 and the second index is 1, and another one is 2. So waypoint 0 is nothing but waypoint 1, so waypoints array number 1 is nothing but waypoint 2. This loop will happen, and waypoint 0 means the first waypoint, 1 means the second waypoint, and 2 means the third waypoint. That's why it will require how many waypoints were defined, and it will request the robot to navigate to all those waypoints. This is the function that is defined for this purpose.

How assuming you are using Isaac simulation for developing a realistic robotics simulation, the following is a snippet of a loop that's mean to execute as long as the simulation is running. However, due to a logical error, the loop runs infinitely even after the simulation should have ended. Identify the logical error in this x similar step. If check simulation in condition, simulations assuming you are using my initially, simulation running is slow. So singleness is running also. So while simulation running means they are creating a while loop. This while loop is only running if simulation running is true only because while loop will run if there is true, while loop will stop if the condition is false. Okay. Simulate set. So that function is doing some simulation step and there is 1 if condition check simulation and conditions so this condition is checking the simulation and its conditions if it is true, then simulation training will be false. So and simulation cleanup means all the things set up in the simulation will be cleaned up and end y. The y loop will be ended. See, that simulation cleanup should be outside the while loop. Yes. Because if the diff condition check simulation end condition if true, then simulation running will be false. If that is true only, the simulation will be stopped. Apart from that, this while loop will be running continuously. So while simulation is running, you should not clean up the simulation. So the simulation cleanup should be outside the white loop. This is the logic x. Otherwise, if you are having the simulation cleanup inside the while loop main, every time the while loop is running, it seems like the simulation is restarting everything, so it will be a big issue. we are necessary cleaning up the simulation all the time. We are running x

Principle of concurrency to manage robotic systems in relation with NVIDIA Omnibus involves using the principle of concurrency to manage robotic counts. To check how its joints and links are working in every concurrent moment. This allows us to determine if the robot can reach its target or perform its task. Concurrency checking is very important to monitor robotic moments in simulations. Not only in simulations, we can deploy that in the real world.