Tanisha Medewala

Hi, everyone. This is Tanisha Medawala, currently working as an AI engineer in CTS Tech Health Care Technology, a private company. So here, this is a healthcare domain company where we, as AI engineers, basically work towards designing, building, developing, and maintaining machine learning applications. So when I started my journey as an AI engineer here, I started with MLOps assessment projects, which dealt with AWS as well as Azure cloud, where the client would come up asking for these solutions to be architecture workflows so that we could identify security process gaps and regulatory compliances that need to be put into the architecture. And, basically, we analyze the current architecture and provide them the correct way of implementation. Then, I worked on MLOps assessment projects. Following that, we also did something like MLOps implementation projects where one of the projects required the MLOps registry module as well as the MLOps monitoring module. We needed to track all the models that were being created. And once we tracked all of them together, we evaluated metrics based on that. One model was selected and stored in a central repository. After that, we basically pushed it to production and came up with different visualization capabilities to make business stakeholders, data scientists aware of the model performance metrics, data drift, business KPIs, outlier detection, and so on. So, this was the work that we did using AWS SageMaker, QuickSight, TimeStream DB, an S3 bucket, and so on. The next piece that I worked on was connecting calls. The calls connected to customer service centers, we were intelligently finding out the call intent summarization and the purpose of the call and who the caller was by doing AI capabilities such as using routing models, artificial intelligence models, like reinforcement learning models, intent models, and so on. So, by using these kinds of models, we would get different values, key-value pairs so that they could be sent to the response. And, basically, we got all the decision-making steps during the call, which call had to be classified, which call had to go towards the concerned department. So, intelligently, we're trying to do it at the backend service. This was the work that we did, and I worked towards setting up the entire project in the cloud, that is Azure, by using Terraform, GitHub Actions, and basically working towards Azure Machine Learning Studio Workspace where we leveraged notebooks, data experiments, and deploying it as a web service endpoint. So, this was the work that I've done so far in machine learning. Apart from this, I have also worked on one of the POCs, which is nothing but brain tumor segmentation where we have leveraged a 3D unit architecture. And, using the 3D unit architecture, we thought of early detecting the tumor cells within the brain tumor in different regions. I was annotating them and recommending them to the surgeon. The surgeon would use the dashboard. Through that, it would recommend the patients that they have to go to chemotherapy or radiotherapy and so on. So, these are the use cases that we have worked on so far in AI and machine learning. And apart from that, I also started my journey with Java development. Basically, I worked 2.5 years in Java development as a backend developer in Zoom in, which is a product-based company, and then the Zoho company. So, these are the companies that I have worked with as a Java developer.

I have experience with databases in writing queries, specifically where I have worked. Okay, so now coming to databases, I have worked with TimeStream DB, which was used within the AWS cloud. It helps you to track each and every event, which are basically depending on the timely dimension. So timely identifiers are used within all the rules and observations. The second piece I've worked with is an RDS instance, which basically uses Aurora or Postgres or MySQL engines. So that was used for transaction-based data for storing data in a tablet format. The third piece I have worked with is MongoDB. And with the use of MongoDB, we basically saved key-value pairs as documentation, and this was done during back-end application development. So MongoDB was used. Similarly, I have also used some Postgres for developing Java applications. With regards to cloud, I've used storage accounts. In storage accounts, basically, I went for blob storage, maintaining all file systems, hierarchy of folders, and so on. The fourth piece I have observed is an S3 bucket. Definitely, using it for transaction logs, keeping them in a proper format and modern artifacts.

Okay. So when it comes to experience in ops concepts, yes. So I have done the ops concept. I basically use object-oriented programming, which is based on classes. A class is a blueprint, and an object is an instantiation of the class. So by using these two properties, we try to encapsulate the entire data unit into variables and methods, which are kept as a class. We also use polymorphism. So, majorly, I've used a lot of polymorphism in Java. In Python, you cannot go for method overloading, or we prefer to have class arguments done or the class methods written. So we used to have class methods, static methods using class variables or static variables, and instance variables. So, we utilize them as per the needs. Now when it comes to SOLID principles, so, with regards to SOLID principles, we try to use recursive functions. A function should be defined with documented strings and within the comment section, what exactly that function is used for. It should be reusable. It should not have redundant logic. The first thing is dependency inversion should be there. So, when we are creating inheritance, the base class has been created. All the properties of the base class will be inherited by the child classes. We give implementation in the child classes. Similarly, when we use interfaces, we define them as abstract methods in the parent class, and the abstract methods have to be implemented in the child class. So, these are the things that we do in ops concepts. We basically modularize the code, create different Python utilities for different purposes. Let's say, if you're working with machine learning, so creating a separate utility for data processing, model training, model evaluation, hyperparameter tuning during model building, and so on. So, this will help us come up with different designs and concepts.

Okay. Describe your experience in writing unit case test cases in Python. So, basically, when it comes to unit test cases, I have used Pytest as a framework where I have leveraged Pytest's library version. So, we installed it and then basically write different functions there, test functions. With the test functions, we use assertion commands. By using assertion commands, whatever are the major functions that you've written and have a very big service logic, we'll try to evaluate it by giving inputs as well as the corresponding output and basically maintain a quality condition over the assert command so that all the function parts, whichever output they are trying to give, will be noted down, and then we are checking it through Boolean conditions. Similarly, we also perform some test functions using negative cases and see what kind of an error should come up if it is handled very well. So, these are the things that I've done in test cases.

So let's talk about advanced Python concepts. Definitely, the removal of all loops is being done by using list comprehensions. List comprehensions are the ones that use optimized methods. So you can basically work around with them. The second piece is Lambda functions. If you have a single task to perform, you can basically write lambda functions. That is also a way to introduce a proper way of writing the code. The third piece is using inheritance, creating interfaces, and basically going for the implementation in child classes so that you can handle it. Then there are different forms of inheritance. You can basically try to see according to the use case, which will be the parent class, which will be the base classes. And abstract classes can be done by importing them and getting them inherited, then that is what there is in advanced Python concepts. Apart from that, using the built-in methods such as map, filter, and reduce, this will also help you to map or transform data. The second piece is filtering all conditions over data frames. Reduce means when you do some cumulative operation or you do cumulative subtraction, and you can easily do it by using Python concepts. So these are the things that I've done so far.

And this is how we will utilize object-oriented programming (OOP) in Python to create extendable machine learning models within it. So when it comes to working with machine learning models, basically, creating a project folder with all the Python utilities, we use when we create data preprocessing or let's say we create loggers, when we create model training, we basically try to define it using a class function. Within that class function, all the predefined methods have been written there. And then, basically, we use this to create an object in the class in Python and extend it in the AWS services. So when it comes to model development, I would say a project should be created properly. The main.py file should be there where all the initiation of the objects have been done using different workflows. So data processing separately, data visualization separately, model query, model building, and hyperparameter tuning and leading to the scoring or evaluation of the models. So we do it through the OOP concept. Utilize the classes, utilize the inheritance, utilize polymorphism, utilize the abstract method whenever needed. Then, instance variables which we keep on initiating through the methods, class variables where the values seem to be the same throughout or should be shared among all the instances of the object should be written there. So that is how we have done it through OOP.

So taking a look at this Python code submitted to the finance checklist. The issues with the template methods and for tools and data. Okay. And okay. So if you see this from ABC import ABC, abstract method, we are basically inheriting ABC as the class. Now when we're inheriting this, we are not annotating the method with abstract method here. I'm basically not writing the execution. It should have the execution here. The first thing we do is that something should be there with some code or let's say even if you have passed, that sometimes works. So an abstract method should be used for the I would say implementation here. So either you if you're using abstract methods, it should be basically used for defining or declaring it. Now if you wanted to use it or give some implementation code, then you have to use another class for the same.

What are some of the challenges you have faced while implementing your microservices in Python on AWS? How do you overcome them? Yes. So when it comes to microservices in Python, the main thing is, you know, making our calls as synchronous calls to different services that we basically implement. So let's say if you are making an application and you have lots of services being made – one is the payment service, another is used for some other core logic services, and the other is used to keep utility services. So here, when we are implementing all of these models together, interacting with all the modules, keeping separation of concerns, and making the parts asynchronous – that is the first issue or the challenges that you will face. The second piece is working with API gateways. So you have to make sure that whenever you are connecting with different endpoints or different services, the URL hits the pattern and routes the direction as per the pattern given there. And basically, when we are creating so many services together as a microservice architecture, we need to make sure that it is purely fault-tolerant. It is scalable and elastic in nature. Whenever required instances will be created for the services in the target groups, and a load balancer can be used. A load balancer basically behaves in such a way where you can use weighted methods. If it is not properly being configured, then whenever you will get a high request rate, you will not be able to overcome this particular issue. And the customer will basically feel bad experiences. So these are all the challenges that will come. The first is integration with all services, talking with all the services, and keeping loose coupling among all the services – that is a must to have. Then connecting with the databases. So when we're using this, we basically let's say, if you're going for a Java application, we'll be having Spring JPA repositories to connect with the database. So it should be updated, whether we should go for asset compliance, whether we should go for base compliance status – that should be decided when we are implementing the same. And the infrastructure, whenever we are using a simple application, then we should have separation of concern in the web tier, app tier, as well as the database server tier. And to overcome this, I would say you can do it by using loose coupling techniques, by using inheritance, polymorphism, by creating interfaces as well as the implementation in different other classes – that is nothing but the child classes. The second piece is load balancing. The API gateway should be given out patterns. We should have a plan for canary release deployment as well as the blue-green deployment so that you can basically give the weighted methods. Whenever a new feature is coming up, it should not give a downtime to our customer experiences, and these are the things that will help us to overcome it.

When we talk about designing a Python-based MLOps pipeline, I would say let's say you have created a project folder. In that project folder, you have different utilities, data preprocessing, model evaluation, model training, scores, deployment, and configuration. Within the configuration folder, you can have environment YAML files. You can have settings. You can have JSON files to configure all the parameters and so on. And basically, you can utilize, if you're going for Azure, you can utilize Azure services and implement the entire code through it as your machine learning services or as your MS SDKs. You can utilize them for the same. Now when we create this project folder, you can also create GitHub actions workflows so that you can push the code to the Azure machine learning workspace. We can perform testing. We can also have test files, spend time for testing for, and basically, when you have YAML workflows, you can go for checkout. You can go for unit testing. You can go for reviewing it through pilot and then triggering the data preprocessing step followed by the dependency on training. Then through the training py file, once the model has been trained, you can see the evaluation metric. Then basically, we can deploy the model and create a web service endpoint. So for this, we can basically utilize the YAML workflows and utilize the Azure CLI commands to perform the entire activity.

When it comes to unit test cases, I would say the major functions which are highly weighted, we create a Pytest framework. And through the Pytest framework, we create Pytest functions. We keep on finding out how many test cases have been passed, whether we can fail a test case, whether we cannot fail it, and all should be jotted down in a separate file, a Python utility. When it comes to integration testing, I would say that whenever we are integrating any services, we need to look at those pages which are integrating with that system and note it down. We create an Excel sheet of positive cases, negative scenarios, and jot down comments for the same. We also create bugs and fixes in the sprint planning used in Jira or project management tools. So these are the ways that we can basically follow to write unit and integration testing.