Manish Choudhary

Hi, I'm Manish. I have been working in the IT industry as a Python developer for the past 6 years and 5 months. So far, I have worked on data science, machine learning, and I am currently working on back-end development. In my recent project, I'm working on API development, maintenance, and doing some data transformation, creating installations.

So as we all know that with this statement, we usually use it to operate or open file operations. Like that, we can use database interactions as well. Whenever we connect to a database, the statement automatically closes the connections after the database operation, commit, or query execution.

So Python ETL, usually, we can create APIs on top of like Spark. Spark will be very helpful and useful to deal with large datasets. Apart from the Pandas library, Python can be used to transform data in chunks, and Dash is also nowadays popular. So these libraries can be used. Apart from that, we can use caching mechanisms of Python, and we can do multiprocessing or multithreading as well to deal with multiple or big datasets. So that's

So, in a skill, we can write objects directly or we can do data selections and deselections, and we can use those objects whenever we need them. SQL can be very helpful to use objects directly from databases.

So AWS Lambda will be very helpful as this is a serverless service. So we don't need any servers for that. We just need to run our scripts on a schedule basis, and it is very much helpful to run the pipelines.

So when it comes to database operations or connecting with databases, usually we have integrity errors, which we can use in exception handling. Sorry, not integrity. Integrity errors or some formatting issues with the query or some data type errors. So that's what we can handle.

We are not using any exception handling. So in line number four after line number four in for loop, whenever we invoke the client, Lambda client, or both of the clients, we would say, there we can add exception handling so that we are sure that we are able to connect through the Lambda client to the Lambda function.

This is a security snippet, which is meant to select all the calls from sales data where the revenue is higher than the previous month's revenue. So it looks like we are directly using and revenue in where condition, and we're adding our result. Instead of doing this, we can use a self join, and we can get the desired output.

So the best way to debug Python applications is over SQL transactions or transformations. We can add as much exception handling and do the proper logging of those exceptions, like integrate error or anything else, or if you're not able to connect to a database. So the best way is to use exception handling.

I'm not sure about that.

Can you discuss an approach to manage your state effectively in DX application working with stream date based Python backend. Not sure again