Jatin Choudhary

What if we see notes here? Can prevent callback error when managing the synchronous code? Well, we can use promises. We can use async/await. So callback hell happens when there's a function which is being called, and after response, we get a callback. Like, we get a response in a callback function. And then in that function, we again call some processes, and then we again call a callback function inside of that. So there's a nested callback. Callback functions are being called. This is ideally considered as callback hell. These can be prevented by converting, like, you know, wrapping the entire thing either in a promise and then wait for these promises to resolve. Once these promises are resolved, we can just go to the next function. This gives a better look for coding as well. Like, it's more manageable compared to handling but compared to other operations, like, doing other things. Right? We can also use async of it. We can mark a function as an asynchronous function. If the function is asynchronous, we can just add async in front of the function. And then if there's another function inside of it which will take which is a blocking code. So we can just use await to wait for this code to resolve. And then once that is resolved, we can then proceed to the next line.

How to properly structure a Node.js back end project that utilizes MongoDB for database operations for my time. What we can do is right? So, what we follow is what we usually initially follow was MVP architecture. So, MVC, model view controller. That was our initial thing. But now we are thinking about moving to clean architecture, but I'll explain what we are using. So, that is but what we have is we have a model, right, where we keep track of all of the database models. We use Prisma, standard, and we also use Next for database queries as well. So for Prisma models as well, we just keep all of those models because we think Prisma gives us a better understanding of how we can manage the models. But, again, for querying something, we don't need Prisma because sometimes Prisma queries take a lot of time. It's hard to modify, like, run the raw queries that we want to do. So we can sometimes use Next. It's still a query builder, but we use Next sometimes to do some hard joining and nesting things. So, these can be done. What we do is, like, we have this model, and then another thing is then we have a controller. Right? And we have a routing file where we have a routing folder inside of that for each model or a specific function, we define some routes, right, which links to their specific controller, which is inside the controller. We just call those we use there, we use models to access the data, perform whatever option you want to do, and then we send the response to the view. Apart from that, what we also do is, for any operations which are being reused, we let's say, if you are doing an integration, let's say we are doing a close integration, stripe integration, we create classes or kind of wrap around those functions, and then we store all of the methods inside that class. And then we initiate it, and then we call it. But, also, what we do is now we are thinking of moving to clean architecture where we are, like, slowly transitioning. Like, not everything is there, but we are halfway there. So, what we're doing is we're creating domains. We are also creating an infrastructure layer. We are also creating the view layer and the UI presentation layer in which we are going to show all of the UI layer, and we want to abstract the entire business logic, the entire domain layer because now the things are actually expanding a lot, and then we want a more manageable scenario. So we are moving towards clean architecture from the MVC. But yeah.

Data models when using JavaScript models, we create models. For MongoDB, we use Mongoose to create models, and we stick with those models. We use TypeScript, which provides very strong type checking for every data we enter into the database and access. We ensure that, like, speaking in terms of MongoDB, we also use other databases. When a part of the product uses MongoDB, we make sure every database and every collection is properly defined, with all types and data types defined. If there's a relation, we define that as well using the keys of other collections as foreign keys. We strongly type-check all these things to ensure consistency remains as it should. In terms of MongoDB, we define proper schema columns. We use Prisma to handle models in this case. We believe Prisma gives us more options to handle this compared to the raw way. There's a better way of doing things in Prisma. However, when it comes to querying, we feel like we can do raw queries more freely on other solutions as well. These are the combination of things we use to ensure everything, like, the integrity remains, and consistency is maintained properly.

For code quality. Right? So we sometimes, in some of the code, ways we started using a lot of unit tests as well. And we stick to solid principles, and we're trying to, initially, it was everything was very raw. We were doing things because, initially, the other focus was not that much on code quality, but more on the business. But now, I use Jenkins to actually do unit tests, make sure the code is covering every edge case. And then we also follow a solid principle, which means we can only have a single responsibility function, and everything is properly abstracted. The interfaces are properly defined before writing any code. We should, like, discuss the domain of the problem beforehand. If you're using any domain, then it should be predefined. The functions would be predefined, and it's interfaced at some place, and then it's implemented in some infrastructure layer. So, these are the ways, these are very intensive code review processes as well. Once we build something, we make sure that we actually review it again before committing it to the main branch. There's a lot of manual end-to-end testing as well before we commit anything. And these are the processes we employ, and we can also automate a lot of this. But, yeah, this is how we ensure good quality.

Azure gives us a lot of tools to actually monitor the Node.js application. For Azure app services, you can use Azure Monitor. The naming conventions are pretty straightforward on Azure, but we still forget it. So Azure Monitor acts as a monitoring tool in Azure Monitor, there's another thing called Application Insight, which gives a lot of features built out or you can just implement it inside your Node application, which can track any errors that are happening. You can also implement a centralized login process so that all the logs you receive can be accessed in a central place in Azure, it's called Azure Logs. Along with that, you can implement automated health checks with Azure Monitor itself and make sure that nothing is breaking. So, these are some processes you can use.

Callback structure. Okay. This is okay. This is an example of callback hell. So, as you can see, when the fresh data function itself is expecting a callback, when you are calling a function, we just call the callback on the bottom. And, again, we are expecting the errors to be handled. But, again, you can see, this is one of the callback issues because then, again, if you want to call any function and the fresh data callback function can also call another callback, it can have any callback as well, and then we can implement that as well. And this would create an ideal scenario of this itself is a callback problem. What we can do is instead of having a database query and then based on the errors and everything, and database query function itself is calling a callback function, which again is returning something to the callback function. We can just make the fetch function, use either promises or use async, make that fetch function on your market as async. Based on that, the database dot query function could be awaited, like we type it around the promise and then we await for it. And then based on that, we return the output, or we just mark the entire database query inside the try catch block. And if it's in the catch block, then we can throw the error and then we can simply just call the first data function. It doesn't need to have a callback function, which will just log it. We can just handle it either inside. Or if we want to do it, based on the return that we are getting from the first data, we can just check it and then log it. If it got an error, then we can just log an error. Or if we get the data, then we can just console log "data received".

It's not sharing. It's okay. I see. Okay. This is something here. Thanks. Right. So anytime someone calls the API route, it will first check for an XAuth token. Right? And then it will proceed there. The first thing we can do is wrap this entire thing inside the callback function of I've got to use into a try-catch block. Based on the try-catch, we can return appropriate responses. Right? I think that's the best way. And then another thing is inside the requested hard disk function, we see that if it is found, we have to validate it. We can use a library to get and verify. And if it's a JWT token or other things as well to actually validate and verify the token as well. And then in the else statement, I see 403 is required. We can just send a 401 unauthorized. That should be a good enough response because that's what's expected. And the model is just we can add it inside a try-catch, and then we can proceed.

Millions of concurrent database operations on MongoDB. Okay. If we have millions of operations, concurrent operations on a single database, the wiser thing to do would be to break it into a microservice architecture wherever possible. The microservice can itself take care of its own database instance. So it will have a smaller load, and we can also implement sharding on MongoDB. We can implement a master-slave setup where we create read or write replicas of the database. And based on the operation, we can transfer it. We can implement load balancing. To handle concurrent database operations, we can ensure the database is properly handling the race condition. We can implement some atomic operations. If we're performing operations on a document, we can use the atomic operator instead of the usual set operation. Examples of atomic operators include dollar set and dollar push. These will make sure the operation is successful or, also, we can implement versioning as well. So that's it.

Good to have questions. Not yet. Not yet. Not yet. Not yet. Not yet. Not yet. Not yet. Well, for memory leaks, what we can do is implement monitoring tools. Like, even on Azure, we can use Azure monitors to see CPU spikes or the RAM usage. Like, we can implement the manual garbage collector as well to make sure there are no garbage values being stored. We can implement profilers as well to see which functions are causing issues, which functions are actually leaking memories or using memory inappropriately, and then we can optimize it. We can do code optimization at every place we can. That's where unit testing comes into play. Like, we can just make sure there is no process memory being leaked, which is being stored in a global memory instead of being block-scoped, so it's clear when the function is called back. So all of these things can also be implemented.

Yes. So in our product, we do end-to-end testing. It's mostly manual, but when we're doing a big feature launch, we either reuse Cypress or any other automated test libraries to actually test for end-to-end functionality. So we try to mimic user behavior. We program it to click on buttons and all of these things, and then perform and see the API responses as well. We do all of these things. So compared to unit tests, the difference is that unit tests are more isolated tests. They test functions and try to enter all possible combinations of parameters to see what the responses are and if they're handling it properly or not. But end-to-end testing is more like integration testing instead of extensively testing a single function. So what does it mean is when we're performing a login function, and let's say the login function is testing the register function, and what happens is when you register, you also send an email to a user. Right? And then there's a call that can be called as a callback, but usually, it's a side effect function that we're calling as well. So when you're writing a sign-up function, the register function, you will check the register function, and it will work or not. Right? But now this register function is being accessed by an API. This API is being called by the front end. So instead of testing all of these unit by unit, you just test the entire thing end-to-end. So you trigger the entire login function and see if everything is happening properly or not. This is actually a more satisfying way to test the function because this way you can be sure the feature works instead of just being the function works. Like, we can be sure with unit tests that this function actually works properly. But there might be something we missed when we're testing it unit by unit, and this will be easily caught up with end-to-end testing. I prefer doing a combination of both. Like, I believe unit tests are very important, but at the same time, end-to-end testing also needs to be done. So I prefer to have unit tests ready when we're writing code, but when we're launching and before the launch, we make a list of all the functions that are being impacted by this, and we make sure we test all of those functions either manually or if not, then we automate it via some libraries as well. But we make sure that we test it properly.