Vishal Daga

I was working as a lead consultant, as a full stack developer. I worked on JavaScript to develop applications, as I worked as a full stack JavaScript developer. And, I am also an AWS certified developer. I have two certificates in AWS. So, I have been involved in multiple projects, developing those applications from ground zero to its completion. And, I have been responsible for the entire application performance, from technical selection to guiding the team members. And, I have been leading teams and have been able to bring full projects to completion.

Okay, so caching is one of the most difficult things in tech in computer science. There is a lot of code changes required, and it has to be done very thoughtfully. I have been involved in implementing caching and I have also invented a smart caching mechanism. I'll come to that later. But when it comes to caching, we have to decide what kind of mechanism we'll be using for forecasting. I don't know. My previous statement was cut off. So, we can have our lazy loading mechanism. So first, we'll look to load data. And if it's not there, we'll go into the database and get the data, or any data source will get the data. Then, we'll store that result in CAS. After the data is in the CAS. Another way is, we proactively create the cache beforehand so that all of our parameters match. When our parameters in the query match, the result is automatically returned because it has been pre-rolled up or pre-calculated beforehand. Another caching mechanism, which I actually invented, was based on parameter values. A lot of the time we observe that if we have, let's say, three or four parameters, and when we have multiple queries, only one parameter changes. But the other two or three parameters are not changed. And that one parameter just affects a few lines of code in the computation logic. So in that case, we may not have to run all the logic. All we may not have to run through all the statements in the logic. We may need to only run the computation that is needed, leaving aside the parameters which are not changed and the logic which is depending on those parameters. So, I call this smart guessing. I implemented this in one of my previous jobs.

So we can have a 7/4 mechanism. The same database, right? Same Postgres table. So in that case, we make sure that there is some kind of mechanism or new text, which is making sure that there is no multiple write operation happening into the same reason of the table or accessing the same table at the same time. And then, whichever process requires to modify the table can own this mutex. And then do the operation and then release the mutex. This is one mechanism. With databases, like Postgres, there are some algorithmic mechanisms as well, which can be used to prevent multiple processes accessing the same reason of the database at the same time or same kind of modifications being done at the same time. In any case, in any scenario like this condition or any situation, there are other ways to handle it. We can have a big transactional query and make sure that the entirety of the query is run in one go. And during that time, no other queries are executed. Or if the query fails, then the entire changes which have been made are rolled back.

Yes. So Docker can be useful for Node.js because we can ensure that we can completely package our application with all the required runtime and environmental requirements and create an image and then deploy the image, ensuring that we are not bound to a specific platform, like when we're using Vocker images. So we don't have to configure a lot. And also, we can deploy our application to run in a predefined environment, such that the application when it is tested or when it is being developed is run in the same way when we are deploying it to the external server or in production. So we can ensure that the consistency of the environment is being maintained. And again, when it comes to distributed deployment of a type low latency, then we can have a lot of benefit. We can gain a lot of benefit by actually using Docker to create our instances of the application. Apart from that, since Node.js is a single-threaded language, we can use Docker with PM2 to effectively use the resources of the host machine. But with Docker, we can easily load balance by responding to multiple nodes. And ensuring that all the node instances are able to run in the same kind of environment.

So the efficacy strategy can be, we can have the same ability zone as well as different zones separated by a certain distance to avoid any situation hitting our server location or within the server host itself, that is the impact is minimal if the dishwasher is quite apart from it by a certain amount of distance. So this is where we are going to replicate the post. And when it comes to replication, what we can do is we can have synchronous replication, which will make things slow. So to ensure that we are doing consistent read and write, then we want to make sure that all the replicated instances are in the same active state, then we can go with the synchronous replication. Again, this will be making things slow because it has to wait for the replicate, for the application to be completed as well to complete. And then there is asynchronous replication. And we can have active passive configuration as well, where we can just maintain a read replica of our instance, of our primary instance. So that is our read replica. There is so with three replicas, we can have two benefits. 1, we can have multiple read instances, which are just using the select script to just read intensive workload. And we can also think of having the read replica as our backup database in case of a disaster. So we can have active passive as well as active kind of configuration with synchronous replication and asynchronous.

I have not seen or I have not, I don't know much about print as you can see here. I know that we can set radius, we can set record with key and value. But then we have a third parameter, radius.print. And I'm not sure what this radius.print actually does. I've not used it so far. Since I don't know about radius.print, I am just assuming the radius.print operation result. But this is a very rough case. I'm not sure about radius.print, it's two words: radius print. Looks okay. There's no issue. We are creating that client. And then when we are creating the Redis client and then we are connecting to it. But with the Redis create client, we are not passing any credentials to which client we are going to create to which client we are going to connect. So on line number 2, ready to create client is not correct. Looks not correct because it does not say which instance we are going to connect. Since I'm not aware of Redis print, I cannot fully conclude what the exact issue is. I need to know what Redis print is first. One issue which I can see is that we are not passing any

Request.params.it. Because, IT is just an arbitrary name for the parameter, and we need to use the same name when we are retrieving that value from request.params. So it should have been request.params.it, not I. Yeah. This is. The issue is, as I told you, the ID is just an arbitrary name for the parameter, and we need to use the same name when we are retrieving that value from request.params. So it should have been request.params.

So, I can profile the query. There are some mechanisms that we can use within the SQL engine itself that help us to profile the query. How the query is being planned, where we can see the inefficiencies in the plan and how the query is structured. One is by looking into the query plan, and other diagnoses which I can think of is the data structure in the table also plays a role in the performance of the query. For example, a very simple example is that let's say we want to store a gender which is just male and female, we should use a data type like char, not because it is with, let's say, 10 size of string vectors, that's unnecessary. So, how we are restricting the table also helps in improving the performance. But the first thing which I look into is the query plan. And from there, I'll see where things are being redundant. Are we doing any redundant calculations in the query? And how we can avoid that. Apart from that, obviously we can, I mean, mostly it comes with not well-structured queries. Because retrieving data can be done in multiple ways. Either you scan the entire database and then do calculations, or you can filter out within the database itself. Ideally, all the computation should be handed over to the database, not computation that's done within the code. That should be the approach. Most of the computation should be done by the database itself. In that way, the overall performance improves.

So the CICD pipeline, whenever a code is committed to a predefined branch, then a deploying strategy is offered for each commit or for each branch. Whenever there is a need for a build and deployment, we are going to build it, create an executable, and create the build. So when it comes to Node and Docker, we'll be creating the image and pushing the image to the Docker registry, whether it's private or public. And then we'll be deploying that image to the deployment group or the servers where the application is going to be deployed. After that, we'll monitor whether the deployment was successful and we can use tools to manage automatic rollbacks. Or even do blue green deployments to ensure that we do some kind of testing before we're actually moving it to production, which is not easily needed when we're doing development. So, the approach I was thinking of is creating the build image. When it comes to production environment deployment, we need to have some kind of manual approval. And we should be deploying to a staging environment and let the tester do their testing before the final deployment. And we need to ensure that the build artifact we have tested and marked as successful is the same one that's deployed. This should not be the case that while doing that testing, another build artifact got created later on, and we mistakenly deployed the untested artifacts. Our untested artifacts should be strongly considered.

So, an API does have two parts. One is a computer-simulated API. And there are some database operations which are not in millisecond latency. They take a few seconds. So, we need to establish a new connection to the database and then disconnect, close the connection as well. So, Redis, whatever we did, all the result of the computation. We can put into Redis with a key set as a parameter, which is a function of the parameters that are passed through the API. So the key is a function of the parameters and the value is the final computed value, which has been sent as a response to the user. So next time onwards, when we are making the same API request with the same parameters, we can get the result without going into the Node.js API calculation logic or connecting to the database and getting the data and doing all the computation. So the response time improves drastically. This requires a lot of port changes. That's the part. But that's how we can improve the performance of an application significantly using a cache in between the back end and the front end.