Venkatesh Mamillapalli

Would you help me to understand more about the background by giving a brief introduction of yourself? Yeah, myself Venkatesh. I have been working in the IT industry for the past 11 plus years. I worked in different technologies. I have experience in talent for the past 11 years. I have experience in AWS and I have exposure to Azure. And also I have experience on Java also. I work on advanced Java. I work on different tools in talent. I work for talent BI, talent big data, talent cloud, talent ESP. All these things I work and I have better understanding about this REST APIs and also I deal with so many APIs and how to read those APIs in the talent, how to expose any of the services. And I have very good amount of understanding on CAML. Like I migrated talent jobs to CAML based code and it was successful. I was able to, I mean I have very good amount of experience in DevOps technologies also. Like I worked on Docker, I worked on Kubernetes. I have exposure to repository systems like GitHub, Stash, Bitbucket. All these things I have experience and also I work on Linux, Linux, Windows systems. And I have good amount of experience in real SQL and SQL queries. I have experience on databases like I worked on more SQL databases and SQL databases. Most recently I worked on Snowflake also. Like I wrote SQL clusters, I fixed some of the bugs in the production environment. Like I worked for L1, L2 support also. And I have more understanding on these technical bugs. Like I was fixing those bugs with less amount of time. Like we have P1, P2, P3, P4 bugs. Once the P1 bug is raised, then I was fixing it in half an hour or one hour. And if there is any additional code change required, we raise the change request. And also if there is any small configuration change, we are raising service request for it. I worked in this support, production support, development and maintenance projects. Yeah, I think I worked in all of the environments and I worked for bank domain. And I worked for banking and finance and retail and insurance. All these domains I worked on. And I am a quick learner. I am able to pick up any of the technologies and also I don't have any issues with technology perspective. Because I worked for NuViz. I worked for Python in some of the projects. And I don't have any problem with any of the new technologies. Because I delivered some of the technologies which I don't have any idea. But I learned that and I delivered it in less amount of time. Because I worked for C++ and I have on-site experience also.

So what would be your approach to the rendering speed and stream processing system, if you can talk a little bit about that. See this one, I have worked on AWS and Kafka related stuff I used in Talend ESP because we have some of the Kafka components, Kafka this kind of components are there where we were using all this Kafka related things in Talend and this stream processing everything I have done it in the Talend only because AWS it's like you need to have all those setups and everything in the AWS that's why it was I have done it in Talend ESP where like it is having like more freedom to do and also like how the partitioning is done and how the processing is done all these things I have done it in Talend using the Kafka components and also I used some of the AWS components in Talend like I used AWS Gatebook and all these data retrieval parts in AWS and I have connected some of the Spark related stuff using the Talend ESP that is somewhat we have done it and for stream processing like as it is I mean it's a continuous process right we have to use the Talend ESP only because Talend ESP has a routing system where like you have 24 by 7 routes where you can retrieve any kind of data and if it is a file based system or if it is a Kafka related all these things like you can retrieve the data at any point of time if there is a queue mechanism also then it will be used in Talend ESP is more useful in that case that's why like you can use the Talend ESP for any of the streaming process if it is like a 24 by 7 where you are getting the data then you can go with that you don't have any problem with that that is one good thing about the Talend ESP compared to Talend DA because Talend DA you don't have I mean this routing mechanism is there in the Talend ESP and also like where you have hierarchical mapper where you can do the transmission from XML to JSON, JSON to XML all these things are available in the hierarchical mapper and also you have different conditions where you can set there in the Talend ESP and when you connect to a straight up ESI then definitely like if you use the Kafka components and Spark related stuff all these things you can implement it in the Talend ESP that is possible in Talend using those Kafka related components and also Spark related components which you can root in Talend ESP and AWS components you have I mean those components where you can utilize that also to connect to AWS and to retrieve the data.

How to integrate a python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database? Python based machine learning model with immediate python on database?

How to design talent job to not only process the data transfer but also handle error logging and recovery. So first thing is, when we are using the talent job, we have pre-job, main job and post-job. And also, when we are using the talent USB, we have this catch mechanism there like whatever the exceptions which have occurred. Like you can use the try-catch blocks. I mean in the Java, how we use the try-catch block in the same way, talent also can handle this try-catch blocks. We can use try-catch blocks so that like you can get the exceptions and also how you want to go for the logins. Once you catch the exceptions or suppose you get any errors. Suppose we have HTTP errors, we are doing different HTTP errors on it. You want to log that like you have 500 internal server error, you have 404 markdown errors. All these things you want to log it. We can have some logging table at the end of the, I mean at the below of the job. Whenever any exception comes, never any, like if you have like predefined errors you want to throw, that also we can catch. Like we have checkbox where like we can catch the exceptions. Like you can throw the warnings also. You can kill the job at any point of time. Those kind of things it is possible in the talent. I mean talent. And also like whenever any error comes, you can use this tlog catcher where it will catch all the exceptions, all the errors, everything will catch it and that we can store it in any of the tables. And when the first job got completed without any error, then if there is any successive job is there, then it will go to the table and check whether the data is loaded properly or not. If it is like full load or incremental load or any kind of load. But the next day when we are running the same job, then we can have this kind of check whether the previous job got run successfully or not. If there is a full load, the load is completed successfully. Or if there is a partial load, whether it is an incremental load, all these things like we can check it in that login table so that like we can run the job based on that. Whether we want to run it like before where the data is not successful. Those kind of things we can do. Context is important in a talent job. Like whenever you use the tag or tmc, you can specify the context that from where to where the talent job should run. Like if there is any data or some particular data. Suppose if you want to run for only one day, you can run the job. If you want to run for two days, three days, all these things like you can mention in the context so that whenever any failure occurs from that point of time, you can just check the login table and see like from where to where you want to run the talent job. All these things can be possible in the talent. But only thing is we should have this try catch blocks and also we should have log catches and we should have warnings, key dials should be put in the job in the correct place so that it will be very easy to log it and get the status of the job very easily in the talent.

Describe how you leveraged EC2 support instances within your database and how does it cost you? EC2, I didn't work in Pakistan with EC2 because it's completely because I worked on this talent developer position where I worked for mostly database and components but I didn't get much chance to work on EC2. I think that is where like I don't have much experience on this data processing workflows and like I had so many times tried to work on this EC2 technologies like AWS and also Lambda functions. On ES3 I have worked so many years and for EC2 I think I got, I mean I didn't get a chance to work on EC2 till now but I am very much eager to work on it. Data processing flows, when you create the talent job and data processing flows, like I told talent ESP where like any other file system, like if you see file component where like you can get the file data at any point of time irrespective of like, I mean the talent DA jobs where you schedule it, once the file is there it will take that to process. If you see the C file, it is kind of like focus on the job which will deploy to any other system where like you can have like data processing for that particular instance like whenever we put the file there, we can easily get the file. I think in the same point of time, sorry I think in the same way like in the EC2 also we have different, I think we have different instances where like you can see, I mean you can see the file size and also like how the data is on the process, all these things like you can do it using the EC2 instances and it depends on like whatever size, if the size is more, actually like the processing will be more and also like cost optimization, I think it depends on the data size. Whenever the data size is more, then maybe like the processing time will be more and also like if the data size is less, then definitely the processing will be much faster and that time I think the cost will be very less and EC2 instance which we are using, I think we can have very less amount of cost when there is any size is less. The size is more, I think that time definitely the EC2 instance will take time to process the entire workload. If there is any like processing workload, we have different, different like if you want to call different system to get the data for validation and after that you want to load it, all these things I think will take time. I think that time much more in the cost.

Yeah, I think you can have one log table where this all the talent jobs got an executing so that like you can check whether this particular and before running any of the jobs, we can have like one free job concept where like it will check that particular log table and that particular log table to see whether the talent job got executed successfully or not. The only thing is you need to pass that particular talent job or else like whenever we run it right, it can take the talent job, I mean talent job name also, based on the talent job name, it will check that particular log table whether the talent job executed successfully or not. If it is not executed successfully, then I mean we can have different condition that suppose you got warning based on the error, if it is like having the data error or I mean this kind of thing, I think we can use this please send email component also where like you can send the mail at the end of any of the talent job which is like executed successfully and also if there is any error in the talent job, like we can have mailing mechanism where like it will check whether the job is successful or not, whether it have any error, right. All these things, it can log it in the same way and at the same time, it can send the email or it can just call the talent job but before talent job, it can check the talent job execution whether it is successful or not and after the talent job, we can update the same table that this talent job got success at some point of time and what is the duration of the talent job, when it is started, when it is ended, all these things we can have one log table so that like before running the job, we can check the talent execution like previous one or how it is done and after the execution, how much time it took, all these things and if it is like any timeout or whenever you are running the APA jobs, like suppose if there is any timeout errors, like you can have like sleeps, sleep additions and from there like you can get the logs and how much time that talent job, whether it is successfully updated or maybe like it failed with any of the timeout errors, all these things like we can have that kind of thing but that is what I told right, you can use the pre-job and you can use the post-job, pre-job execution and post-job, it will update the log table where then the job is started and when the job is ended, all these things it can capture, I mean it can capture that entire thing, I mean even for if you are using for AWS in different regions, that is also like we can have like one, I mean database where like it will check the executions, I mean you should have like some place, some database where it can capture all the data, the pre-job and post-job where you can utilize that, that can be possible using this.

The job involves transferring data from an RDS instance to a STU, how does it work? What steps do you take to debug this primitive issue and ensure that it is successful? When there is no clear pattern, first we need to see whether the STU connection is successful or not. Because when AWS connection is successful, then we can have a different kind of approach. First and foremost thing is to check the STU connection from the talent server, if it is correct or not. Then, as mentioned, first thing is AWS connection, then it will go to RDS input. Whether the RDS input, whether you are getting the data or not, we need to see whether you logged it or not. If you have not logged in, then we need to put some TLABRO there and see what kind of data is coming from that RDS input. From there, you need to see the Tmap, how the mapping is done. If there is any nullporter exception, if there is a class cast exception, if there is any runtime exception, we need to see it in the Tmap. But the thing is, we need to make sure that all the connections and also the data which is coming from the jobs, all those things should be having, I mean we need to print that and see the data. Also, when any connection issue comes, immediately I think, if you use any other SME main component, we need to have multiple, if there is any connection failure, we need to wait for some sleep and also after that we can have multiple connection options. It's like for every one minute you can try a connection like that, you can try three times. So that if the connection is established, then you can go for this. Like you can have AWS connection and after that you can go for this RDS input, Tmap, all these things. But make sure that you use the TLABRO to understand what is happening. When the connection is established, definitely there is an issue with the data. But if it is a data issue, definitely I think it will be there in the backlogs. You can have a runtime server where you can see the logs there and you can check. Like if the log is enabled with the info, then you can activate the debug mode so that in which component the exact error comes, then you can easily find it out if you are running the job in a debug mode. That is entirely up to the tag configuration. You need to ask if any of the person is handling these tags or not, then we can enable that in debug mode. All these jobs, you can enable that in debug mode and see at which point this error comes and which component this error comes. So we can enable that in the tag mode.

Not selecting the disc options. Because in the team app, like, you have, uh, in that front, left zone, and you have, like, options for, uh, this, uh, these options. First thing is whenever you are doing the joinings, whether the joinings are correct or not. Like, whenever, uh, maybe if it is a cost approach, cost, I mean, this cost match, cost match, unique match, all these things, uh, we need to see. And also, like, whether you want to reload at each home because when the suppose if you have, like, 2 inputs, one of the input is keep on changing it, then you can go for this redirecting. So otherwise, uh, you can have the same option. Like, you can just select the data, and then you can use this disk option. There, like, uh, when you select the display, definitely, this, uh, auto memory errors will not come. Mostly, it will not come. And if you are still getting, like, uh, auto memory errors, then what you need to do is, like, you need to increase the size of this, uh, XMS and XMS. Like, we'll be having, like, uh, options where, like, uh, you can increase the, uh, talent process size also so that, like, you won't get the out of memory errors. But, uh, most thing is that whenever any of the, uh, input or output comes in, Make sure that you need to use the proper joints and proper conditions. Like, you can use the proper filters, like, whenever you want to have any, uh, filter kind of thing. Uh, I mean, in my case that, uh, I prefer that if you are using 1 to 1 mapping, don't, uh, use the t map because, uh, t map is somewhat large compared to t Java. You can use the t Java row component for 1 to 1 mappings. Other than that, if you are having any of the join condition, then you can use the t join. Suppose if you don't want to have, like, a unique match, all these kind of conditions, 1 instead of t map. And, uh, if you are specifically using the t map, then you can have, like, the expression I mean, you have expression filters. You can have, uh, different joins and also you can have the different matches. And, uh, whenever, like, uh, you have to catch, uh, I mean, and, uh, catch this, uh, say I mean, filter whenever any of the, uh, filter this, uh, work from it. Whenever any of the expression, uh, filter is digits data also, you can have it. All these kind of things, we can do it in a, uh, uh, team. But the main thing is that if they are using the team app, I mean, we need to make sure that, uh, we can have this this option, but this option when it goes selected. And also, like, make sure that if it is, like, increasing more of the thing, then make to increase the XMS and the XMS, uh, size of the JVM sort of like, uh, and, uh, that talent process which Those are the best options which, uh, we, uh, can do to the d map. Uh, so that's like it. It will not have any failures.

This, uh, number tool I used to, uh, for this, uh, tracking purpose because it was We can use some tool there. Yeah. Yeah. There's, uh, tracking can be tracked. I mean, every server's there as well. The tracking can be done. I think both of these, uh, you can try that. But, uh, ladies are through the, you know, Uh, definitely, that we do have, like, all the servers are done also, the, uh, beginning of the API down or beginning of the, uh,

First thing is, uh, uh, when we are running the sprint priority site, we have the data available. So, uh, any other part of the tables, it is easy to go, then we can use the temporary tables. Otherwise, you can have the CPUs where you can have this. CTS. It sounds like adding, like, bulk queries. Like, suppose, if you are doing, uh, so many joins, like, there are joins, like, you, uh, should not put it in the, uh, inner files. You can have that city options where you can have those files in the city, and that city will be proper. You won't only to I mean, get the queries on again and again. You can have this or maybe, like, issue of scoring some data already. Temporary data structures where you can have the data. And also, you can even partition tables also. Yeah. That is also cool. If you are doing the partitioning papers, then yeah. That is also wonderful. But the thing is I mean, the bigger guys, AS SQL and all these things. Right? Then I use the CTEs also this, uh, temporary things temporary, uh, coils. Sorry. Sorry. Temporary things. Then I think that time, we can have those kind of things. I think I think based on that, and I think we can get all the. And also, like, what the suggestion that when you're running any of the queries, then no need to put all those things in the queries because and whenever you are running those queries using another tools like. Then there's a condition for while all these, uh, groups are supported by. Then now this connection, we can, uh, use it to use your timing. That we can any kind of, like, large. So let me know. We can We can just use the or we can just use the temporary tables where we can get the data and we can just we'll reuse it so that, like, you know, need to run the queries again.

What strategies can you utilize for reducing the time it takes to various competitors? The first thing is when you were when you are having the power be added, you need to get it from the data. You should not hit the you are you are I think I think, um, in my product also, they have used the view, and, uh, the view running time is very, very much, uh, not effective. Sometimes the view takes a lot of time to run so that, like, to load the particular dataset takes a lot of time. That's why always we need to use the table in the back end configuration where the data is instead of, like, putting it from the loop. That is the best option which I suggest for the power delay as we already loaded the data. We can just take it from the table and put it in a code. And, also, like, um, what other things are done, like, any of the conditions are acting that we can implement that in a view on instead of writing everything through our way, all the calculations and everything, then we can have all those things in the view. What about our, uh, calculations, what are the things we are looking. As it is only select query, right, we are not updating or we are not doing anything with the view, then you can do any other transformation things that you want to show other than a table, then those things you can put it in a view, then that you can separate it from the Power BI. That can be possible. But if you don't have, like I mean, you is having, like, so much of queries, then I suggest that all the data printing up table that, uh, you you will be running it for any of the group. Um, that data, we can load it so that it will be easy because it just take the data and, uh, put it in the body. Like, you can have, like, different kind of things. You can you can have tables. You can have charts. All these things, you can have enough of it. Look. My version is how to get the data from the teams instead of, like, calling the view. If it is, like, very big SQL queries to load the dataset itself, it will take a lot of time because in Power BI, we have different options to get the data get the data set. That, uh, we can do it, and it will reduce the time. The loading time to to load the dashboard. Right? Power by dashboard. That will take very less amount of time. Yeah. That can be possible. I think that strategy, I think it'll be useful so that, like, uh, we can have no. Data will be loaded with.