You don't truly understand the concept or technology until you get your hands dirty while implementing it!

I have read many blogs and watched many YouTube videos about nginx, http caching, load balancing, api gateway, reverse proxies, ssl certificate, ssh, port forwarding, url redirection, etc but never have I ever implemented these concepts in a real project.

Few days ago, I have to host a node.js backend written in TypeScript on my client's own server (no aws & no other hosting provider). I have to setup everything from scratch by myself and it took me almost 2 days to host it properly.

What's for you

• Mindset of a junior engineer
• What mistakes we make as a junior engineer
• Running from the challenges and then finally accepting to face it
• Learning something new
• Typescript with node.js can be challenging if not configured properly
• Debugging is one of the most prominent engineering skill

How I started

I started with what most of the engineers will do at the very first, by reading some blogs and going through some YouTube videos to get a basic idea of how to host a node.js app on server (I was mainly focusing on aws and digital ocean).

What I learned from those articles/videos:

• Make a production build of your node.js app
• Resolve TypeScript issue (If they come up. Mine was import alias)
• Install pm2 (Advanced, production process manager for node.js)
• Run node.js app using pm2 for better management of your app
• Use Nginx as a reverse proxy (But your app can run without it. So I neglected it!)

So I made the build of my app locally using npm run build, then got some typescript issue regarding import alias (fixed it then!). I got the build of my app in a different folder named dist.

pm2 was a new thing for me, so I decided to learn more about it. Like what is pm2, its uses, benefit and some basic commands. In short, you can run many node.js app using pm2, manage them, stop them, monitor their memory consumption and performance, see logs, setup autostart on crash, etc.

Time to host (My first try)

I choose to go with windows OS on the production server (Who doesn't like GUI).

This was my very first mistakes. The articles I read were using linux OS but I thought GUI will be more easy and convenient for me (trying to choose the easiest way).

I build and ran the production build of my node.js app locally, so I just did these steps:

• Deleted node_modules folder (project size will be decreased & will be copied quickly on server)
• Setup cors origin to allow FrontEnd url to hit the node.js app
• Changed .env file values accordingly
• Copied the project to the production server
• Opened the project root folder in terminal
• Installed all deleted packages using npm install
• Made a production build using npm run build
• Installed pm2 globally using npm install pm2 -g
• Ran the build using pm2 command pm2 start dist/index.js
• The App started successfully!
• Ran few commands to restart the node.js app after crashing or even when the server reboots.

We are good to test!

The first observation & SSL requirement

The app was running fine (assume, http://www.xyz.com:8000) but the problem came when I wanted it run on HTTPS. I had a mobile app (built on react-native & expo) and a web FrontEnd which I wanted to use with node.js app without HTTPS the production build of mobile app was not able to make api calls to the backend (obviously, I found a work around using a package expo-build-properties but didn't wanted to go with this approach). Frontend was running on HTTPS but axios was showing some errors too (I don't exactly remember the error but all I understood, it was like frontend is on HTTPS and cannot connect to HTTP backend).

Nginx and SSL installation

Again I've gone through some articles which guided me how to install the Nginx server, SSL certificate, SSL validity and renewal setup on windows server.

Nginx: A web server that can we used as a load balancer, reverse proxy and SSL management, api gateway etc.

Then I installed the Nginx server and some tools to generate the SSL certificate. The only problem was that it was getting more and more complex to find tools and do things on windows server in comparison to ubuntu.

I got tired to solve issues while setting up the nginx config file and also the SSL auto-renewal process was quite difficult and not reliable.

Since, most of the articles were using linux based server so after spending one whole day I decided to go with ubuntu server instead (Because it was easier to do these all things on ubuntu and I've already read many articles on it).

Transition to ubuntu server

After installation of ubuntu server, I follow the the earlier steps like copy pasting the project, installing nvm or node, installing pm2 and all those other things.

The project was running properly on the Ubuntu server now I just have to setup the Nginx and SSL. So,I did just these things:

• Installed nginx using sudo apt install nginx
• Installed certbot sudo apt install certbot python3-certbot-nginx
• Generated SSL certificate
• Setup the config file
  • By default nginx was running on port 80 (http://www.xyz.com)
  • Redirect the request coming to port 80 to port 443 (default https port) (https://www.xyz.com)
  • SSL was not yet installed but I had setup the redirection
  • Then I redirected all the request coming to port 443 to my node.js server port (From https://www.xyz.com to http://www.xyz.com:8000)
  • So, now I was able to indirectly hit my node.js server without exposing its real port to the users i.e. 8000
  • Added SSL certificate path in port 443
  • Reloaded the nginx server using sudo systemctl reload nginx

Nginx config file (Demo):

server { 
    listen 80; 
    server_name api.xyz.com; 
    return 301 https://$host$request_uri; 
}

server { 
    listen 443 ssl; 
    server_name ANY_SERVER_NAME;
    
    ssl_certificate /etc/letsencrypt/live/api.xyz.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.xyz.com/privkey.pem;

    location / {
        proxy_pass http://localhost:8000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

All done! The project was successfully running!

What I learnt

Now matter how complicated things you learn about, you really don't understand it without implementing it. I've already learnt about load balancing, reverse proxy, forward proxy, port forwarding, api gateway, ssl, etc while learning system design but it was just the concepts. The day I hosted this project, all those concepts started really making sense.

Choosing the GUI over CLI is not always a good idea as an engineer. If you want to grow as an engineer, make sure you try things that you haven't tried earlier.

Debugging is a skill not found in all engineers! Reading the error, going through out the web to understand it and figure out its solution is something that keeps us in the market.

Always use TypeScript if you are going to work on JavaScript either on frontend or on backend. I won't tell you the reason because you can feel the difference by yourself for the first time when you'll be working with TypeScript.