Jatin Choudhary

What if we see notes here? Can prevent callback error when managing the synchronous code? Well, we can use promises. We can use async await. So callback hell happens when, like, when there's a function which is being called, and after response, we get a callback. Like, we we get a, uh, we get a response in a callback function. And then in that function, we again call some processes, and then we again call a callback function inside of that. So there's a nested callback, uh, callback functions are being called. This is ideally considered as callback. These can be prevented by converting, like, you know, uh, wrapping the entire thing either in in a promises and then wait for these promises to resolve. Once these promises promises are resolved, we can just go to the next function. This gives a better look for coding as well. Like, it's more manageable compared to, you know, handling but compared to other operations, like, you know, doing the callback health things. Right? We can also use async of it. We can mark a function, uh, asynchronous function. Uh, if the function is asynchronous, we can just add async keyboard in front of the function. And then if they if inside of the function, there's another function which will take, you know, uh, of the function, there is another function which will take, you know, uh, which is a blocking code. Right? So we can just use await to wait for this code to resolve. And then once that is resolved, we can then proceed to the next line.

How to properly structure a Node. Js back end project that utilizes MongoDB for database operations for my time. Okay. What we can do is right? So what we follow is what we usually initially, we were following was MVP architecture. So, uh, MVC, model view controller. So that was our, uh, thing. But now we are thinking about moving to clean architecture, but I'll explain what we are using. So that is but what we have is we have a model, right, where we keep the track of all of the database models. The, uh, we use Prisma, standard, and we also use NEXT for database query as well. So for Prisma models as well, we just keep all of those models, uh, we keep because we we think, like, uh, Prisma gives us a lot a better understanding of, like, how we can manage the models. But, again, for querying something, we don't need Prisma because sometimes Prisma query takes a lot of time. It's hard to, like, you know, modify, uh, like, run the raw queries that we want to do. So we can sometimes use Next. It's it's still a query builder, but we what we do is we, uh, yeah, we use Next sometimes to do some hard joining and nesting things. So these can be done. So what we do is, like, we have this model, and then another thing is then we have a controller. Right? And we have a routing file where we had we have, like, routing folder inside of that for each model or a specific function, we define some routes, right, which links to their specific controller, which is inside the controller. Right? Uh, we just call those we use there, we use, like, models to access the data, uh, perform whatever option you want to do, and then we, uh, we send the response to the view. Uh, apart from that, what we also do is, uh, we also like, uh, for any operations which are being reused, we like, let's say, if if you are doing an integration let's say we are doing close integration, stripe integration, we create, uh, classes or kind of wrap around those functions, and then we store all of the methods inside that class. And then then we initiate it, and then we call it. But, also, what we do is now we are thinking of moving to clean architecture where we are, like, slowly transitioning. Like, not everything is there, but we are halfway there. So what we're doing is we're creating domains. Like, we are also creating infrastructure layer. We are also creating then the, uh, view layer and the UI presentation layer in which we we are going to show all of the UI layer, and we want to abstract the entire the entire business logic, entire the domain layer because now the things are actually expanding a lot, and then we want to we want a more manageable scenario. So we are moving towards clean architecture from the MVC. But yeah.

Data models when using JavaScript model and well, we create, uh, we create models. Like, for MongoDB, we use mongoose to create, uh, models, and we stick with that models. And what we do is we use TypeScript. So there's very strong type checking of every data that we are entering the database and we are trying to access. At the same time, we ensure that like, I'm speaking in terms of MongoDB, but we also use other databases as well. So when for one of the part of the product, we also use MongoDB. So what we do is we make sure that every every database, every, uh, collection is properly defined, all the types, all the all the types, all the data types, everything is defined. If we are if we if there's a relation, then we, uh, define that as well with using the keys of other collection as well, the foreign keys. And, uh, yeah, we we strongly type check all of these things to make sure that, uh, the consistency remains as it should. Also, like, in terms of MongoDB, we, again, we define the proper SQL, uh, uh, SQL columns and everything. And what we do is we use Prisma in that case for handling, uh, models. So we believe, like, um, Prisma gives us more option to to handle this compared to the raw way. So, like, there's a better way of doing in Prisma. But, again, when it comes to querying, we feel like we can do raw query more freely on other solution as well. So these are the combination of things that we use to make sure that everything like, you know, the integrity remains, the consistency is maintained properly.

For code quality. Right? So we sometimes, like, in some of the code ways, right, we we started using a lot of, you know, unit test as well, And we are sticking a lot with solid principles, and we are trying to, like initially, it was everything was very raw. We were doing how we were, like, coming up with things because, initially, the other focus was not that much on code quality, but more on the business. But now what we do is, uh, I use, uh, Jenkins, uh, to actually do the unique tests, make sure that the code is covering every every edge cases. And then we also, like, in, uh, we also have, uh, we also follow a solid principle, which means, like, we can only have a single responsibility function, and everything is properly abstracted. The interfaces are properly defined before writing any code. Uh, that should be, like, you know, prediscuss the domain of the problem. Like, if if you are using any domain, then we have that should be predefined. The functions would be predefined, and it's it should be interfaced at some at some place, and then then only it is implemented in some in infrastructure layer. So that's these are the ways, uh, these are very intensive, like, you know, code review process as well. Once we once we build something, we make sure that we actually review it again before committing it to the main branch. There's a lot of manual end to end testing as well before we commit anything. And these are the process that we we employ, and we can also, like, uh, automate a lot of this. But, yeah, we this is how we ensure the poor quality.

Uh, Azure gives us a lot of tools to actually, uh, monitor the Node. Js application. So for Azure app services, you can use, uh, I think it's called Azure monitors. Uh, the namings are pretty straightforward on Azure, but we still forget it. So Azure Monitors act what, uh, in Azure Monitors, there's I think there's called, uh, another thing called application insight, which you can, you know, which gives a lot of features built out or, like, you know, you can just implement it inside your node application, which can just undo a lot of tracking of any any errors that that is happening. You can also implement central login process so that all the logs that you that you receive can be, uh, accessed in a wall center place in Azure, uh, it's called Azure logs or something. But, uh, yeah, along with that, you can implement automated health checks, uh, with Azure monitor itself and, uh, make sure that nothing is breaking that much. So, yeah, these are some processes that you can use.

Callback structure. Okay. This is, uh, okay. This is an example of, uh, callback hell. So so as you can see, like, when the fresh data function itself is expecting a callback, uh, when you are calling a function, we just we are calling the callback, uh, on the on the bottom. And, again, then we are, like, uh, for errors and what whatever happens, we are just expecting the errors to be handled. But, again, uh, you can see, uh, this is this is one of the callback because then, again, if you want to call any function and, uh, the fresh data callback function can also call another callback, it can have any callback as well, and then we can implement that as well. And this would create an ideal scenario of, uh, this is a this itself is a callback problem. What we can do is if we can, uh, instead of, you know, having, uh, database query and then based on the errors and everything, and database query function itself is calling a callback function, which again is, uh, returning something to callback function. We can just, you know, in instead, we can just what you can do is we can call, uh, rapid use either use promises or use async of it, make that fetch function, uh, on your market as a async. Uh, based on that, like, the database dot query function could be awaited for, like, we or either we type it around the promise and then we await for it. And then based on that, we return the output, or we just mark it, uh, the entire database query inside the try catch block. And if it's in the catch block, then we can return the header as well. We can throw the header as well, and then we can simply just call the first data function. It doesn't needs to have a callback function, which will just log it. We can just handle it either inside. Or if we want to do it, we based on the return that we are getting from first data, we can just check it and then, uh, log it. If if it was, like, you know if it was if it got an error, then we can just enter the error or if it's through through an error, we can just log an error. Or if if we get the data, then we can just, uh, console data received.

It is a not sharing. It's okay. I see. Okay. This is something here. Thanks. Right. So anytime someone calls the API route, it will first check for XAuth token. Right? And then it will just proceed there. The first thing what we can do is we can wrap this entire thing inside, like, uh, the entire callback function of I've got to use into a try catch block. Based on the try catch, we can we can, uh, return appropriate responses. Right? I or, yeah, that's the best way. And then another thing is inside the requested hard disk function, uh, we see that if it is if it if it's found, yeah, we have to, like, validate it. We can do g w to get verify. And if it's it's a g w two token or other things as well to actually validate and verify the token as well. And then in the else, yeah, in the else statement, I see 403 is required. We can we can just send 401 unauthorized. Uh, Yeah. That that should be a good enough response as well because that's a yeah. That could be good enough. And, uh, yeah. Yeah. So the model is just we can just add it inside a try catch, and then we can proceed. Yeah.

Millions of concurrent database operations on MongoDB. Okay. If we are having a millions of operations, concurrent operations on a single database, The wiser thing to do would be that break it into microservice architecture wherever he can. Uh, the microservice can itself take care of like, it will have their its own database instance. Uh, so it will have a smaller load, and then, um, we can what we can also do is implement like, on MongoDB, what we can do is implement sharding. We can implement master slave thing where we that, you know, we can create read or write replicas of the database. And based on the operation, we can just, uh, transfer it. We can implement load balancing. Also, to handle, like, you know, concurrent database operations, like, if you want to make sure the database is properly hand handling the you know, uh, what it's called. Yeah. The it's called, I think, the race condition, I think. Uh, What we can do is, like, you know, we can implement, uh, some sort of atomic operations. Like, if if we are, uh, performing any operations on on a on a document, we can we can use the atomic operator instead of the usual set operation. So I think I think some examples are dollar set, dollar push. These are the examples of, you know, atomic operators, which will make sure that the operation is, uh, successful on the or, also, we can now implement versioning as well. So yeah. That's it.

Good to have questions. Uh, not yet. Not yet. Not yet. Not yet. Not yet. Not yet. Not yet. Well, for memory leaks, uh, what we can do is, uh, we can implement monitoring tools. Like, even on Azure, we can use Azure monitors, uh, to see a CPU spikes, uh, or or the RAM uses. Like, we can implement the manual, uh, garbage collector as well to make sure, uh, there is no garbage values being stored. Uh, We can we can implement I think we can use profilers as well to see which functions are are, like, uh, yeah, which which functions are causing issue, which functions are actually leaking memories or are using a memory in the inappropriate amount, and then we can just optimize it. Uh, we can do code optimization at, uh, every place we can. That's where that's where, uh, unit test comes into play. Like, we can just make sure that there is no process memory which is being leaked, uh, which is, you know, which is being stored in a global, um, memory instead, like, just being a, uh, just being a block scoped so that, like, it's it's clear when the when the function is called back. So all of these things, they can also be, like, you know, implemented.

Yes. Uh, so in our product, we do end to end testing. It's it's mostly manual, but when when we are doing a big feature launch, what we we either reuse is Cypress or any other automated test, uh, testing libraries to actually test for, like, you know, end to end. So it we try to mimic the user behavior. Uh, we program it to, like, you know, click on buttons and all of these things and then perform and see the API responses as well. Uh, we do all of these things. So, uh, compared to unit test, the difference is, like, unit test is, uh, more of an isolated test. So it test the functions, and it tries to, like, you know, enter all of the possible combination. We'll try to enter all of the possible combination, like, parameters as well and see what the responses are and if it's if it's handling it properly or not. But but end to end testing is more like an integration testing instead of, like, being, like, extensively testing a single function. So what does it means is, like, uh, when we are performing a login function, and let's say, uh, login function let's say we are testing register function, and what, uh, what happens is when you registers, uh, you also send an email to a user. Right? And then there's a call like, it can be called as a callback, but, usually, it's a side effect function that we are calling as well. So when you're writing sign up, uh, sign up function, the register function, you will check the register function, and it will work or not. Right? But, uh, now this register function is being accessed by an API. This API is being called by the front end. So instead of testing all of these, like, you know, unit by unit, you just test the entire thing end to end. So which means, like, you trigger the entire login function and see if everything is happening properly or not. Uh, this is very this is actually more of a satisfying way to test the function because this way you can, uh, be sure, like, okay. The feature works instead of just being the function works. Like, we can be sure with unit test, like, this function actually works properly. But there might be something which we missed when we are testing, uh, unit testing it, and the this will be easily caught up with end to end testing. I prefer doing a combination of both. Like, I I believe unit test are very, very important, but at the same time, uh, end to end testing also needs to be done. So I I I prefer, like when we are writing code, we prefer, like, to have the unit test ready. But when we are launching and before the launch, uh, we make a list of all the all the, like you know, we do the impact analysis first, right, which functions are being impacted by this, and we make sure we test all of those functions either manually or if if not, then we automate it via some libraries as well. But we make sure that we test it properly. Yeah.