Tag: web hosting

Categories
Technically Speaking

UPchieve remote tutoring platform deployment

With the ongoing Covid19 situation, platforms that specialize in remote learning and education distribution, have become invaluable. UPchieve is such an open source platform. This platform connects a volunteering tutor and a student to connect and learn. The communication methods also include an interactive whiteboard and audio calling.

UPchieve also has iOS and Android apps, built on the React Native platform.

In a previous Technically Speaking installment, we outlined the steps needed to upload a Node.JS web app onto the Google Compute Engine. The NodeJS app used as the sample is the UPchieve/web app. The procedures for deploying the web app can be found in a previous Technically Speaking installment. In this article, we talk about how the server portion of the platform can be deployed on the Google Compute Engine.

This NodeJS stack comes with a twist – we will be using nginx as a reverse proxy server for NodeJS. This extra server setup has an advantage – NodeJS itself will run without requiring root permissions. Instead, nginx will handle http/s access for us and route requests locally to the NodeJS server.

As requirements, we are assuming that you have already created a new virtual instance on the Google Compute Engine.

Procedure

Install Applications

Initially, a VM would not have any software. We need to populate it with the software that fits our needs. For our scenario, we are going to need these software:

  • git
  • NodeJS
  • MongoDB
  • make
  • certbot (for SSL connectivity)
  • nginx

The commands to install these tools on a Debian 9 system are given below:

# Install git
sudo apt-get install git

# To install nodeJS, we need to install curl
# https://tecadmin.net/install-latest-nodejs-npm-on-debian/
sudo apt-get install curl software-properties-common
curl -sL https://deb.nodesource.com/setup_13.x | sudo bash -
sudo apt-get install nodejs

# Install mongodb
#Update 02Jul20 : Run `sudo apt-get install wget` if wget is not there
wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -
echo "deb http://repo.mongodb.org/apt/debian stretch/mongodb-org/4.2 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list
sudo apt-get update
sudo apt-get install -y mongodb-org

# Install make
sudo apt-get update
sudo apt-get install build-essential

# Install nginx
sudo apt update
sudo apt install nginx
systemctl enable nginx # start the nginx service

To install some of the tools on macOS (maybe as a development environment):

brew tap mongodb/brew
brew install [email protected]

Clone the Repository

We need to pull the source from the remote GitHub server and store it on our server locally. The open source UPchieve server source can be pulled from its GitHub repository.

# Clones the repository onto a folder named 'server'
git clone https://github.com/UPchieve/server.git server
cd server

Starting Applications

Next, we will start the servers necessary for our UPchieve server to perform its first run/ setup. First, we will start the MongoDB background service.

For Linux systems:

sudo systemctl daemon-reload
sudo systemctl start mongod
systemctl status mongod # check if the service works

If the service has been set up successfully, you will see an output similar to this:

UPchieve requires MongoDB service to run properly

For macOS:

brew services stop [email protected]
brew services start [email protected]

mongod --config /usr/local/etc/mongod.conf --fork

# check if MongoDB is running
ps aux | grep -v grep | grep mongod

Next, we need to set up the UPchieve server and databases.

# setup database and install dependencies
cd server
bash bin/setup # if there is an error, run npm rebuild
node init
npm run dev # start upchieve server
# if you get a New Relic error, run
# cp node_modules/newrelic/newrelic.js newrelic.js
# if you get a bcrypt error, run `npm rebuild`
# if you still get the bcrypt error, run `npm install bcrypt`

You should be able to check if the server is working at this point. Open your browser and open the page at

http://<VM IP Address>:3000/eligibility/school/search?q=test

If it works, you might want to open a new shell (the current shell will be running the node server) to execute the other commands.

Production-ready!

We need a few more changes to make the application server ready for production. At the moment, we need to type in the IP address and the port number to access the server. Although this should also be fine since the consumers of the application would not have to access the server directly, this is not recommended. Besides, there is no SSL facilities on the server.

To set up free SSL using Let’s Encrypt SSL, you can refer to our previous article that outlines the procedure also for the Google Compute Engine.

Configure nginx

We will use nginx as a reverse proxy for our NodeJS server. Assuming that nginx is already installed, we need to configure it.

sudo nano /etc/nginx/nginx.conf

Add a server in this file to listen on port 80 (http port). We do this by adding an entry inside the http block (make sure to replace server.example.com by your sever name):

	server {
		listen 80;
		server_name server.example.com;

		location / {
			proxy_set_header   X-Forwarded-For $remote_addr;
			proxy_set_header   Host $http_host;
			proxy_pass         http://localhost:3000;
		}
	}

Now you can test the server using the domain name instead of the IP address and port number

http://<DOMAIN NAME>/eligibility/school/search?q=test

Configure SSL

Before starting configuration of SSL, we might have to stop both our NodeJS and nginx servers

sudo systemctl stop nginx
ps aux | grep -i node # to find our node processes and PIDs
kill -9 <PID> # here PID is the ID of the node process

Since we have nginx running as a proxy, the usage of slightly different and tailored for an nginx environment:

sudo apt-get install certbot python-certbot-nginx
sudo certbot --nginx # automates the editing of nginx configuration file

sudo systemctl start nginx # start nginx service
cd server
npm run dev # start our NodeJS server

Depending on the selections you made during the SSL configuration, you would be able to access the server on both http and https at this point.

http://<DOMAIN NAME>/eligibility/school/search?q=test
https://<DOMAIN NAME>/eligibility/school/search?q=test

Configure Socket.io

The socket.io protocol is used by the server to trigger request notifications on the volunteer dashboard and the session chat system. This should not be confused with WebSockets. These are two different protocols.We will be configuring WebSockets separately.

By default, the NodeJS server is listening to socket.io based requests port 3001. But we need to route it through our nginx server if we are to enable SSL for socket.io requests.

Our game plan to cover all these grounds is to:

  1. Add a destination server on port 3001 to our NodeJS server (which will be http://localhost:3001)
  2. Add a reverse proxy for the location /socket.io/ – this specific location is defined by the socket.io protocol. This proxy will take care of other socket.io requirements such as http upgrade
  3. Publicize a SSL supported port, 3002, that can be accessed externally by our web app

We cannot use 3001 or 3000 in place of the 3002 without changing any NodeJS config code as this is what the NodeJS server itself is listening on. Instead, we define an unused port 3002.

http {
  upstream upstream-nodejs { # NodeJS socket.io destination
    server  127.0.0.1:3001; # OR localhost:3001
  }
  # other stuff...
  # Add SSL support to port 3002 which will be publicized
  listen 443 ssl; listen 3002 ssl; # managed by Certbot
  # Other SSL properties...
  location /socket.io/ {
    # listen for this location on port 3002
    proxy_pass              http://upstream-nodejs;
    proxy_redirect off;
    
    proxy_http_version      1.1;
    
    proxy_set_header        Upgrade                 $http_upgrade;
    proxy_set_header        Connection              "upgrade";
    
    proxy_set_header        Host                    $host;
    proxy_set_header        X-Real-IP               $remote_addr;
    proxy_set_header        X-Forwarded-For         $proxy_add_x_forwarded_for;
  }

Configure Websockets

WebSockets also requires the http upgrade method. In contrast to the socket.io method we followed, we use a map symbol to describe this.

Next, we specify the URI that we should listen for WebSocket requests is /whiteboard/. Requests to this URI will be automatically referred to the back-end WebSockets server on NodeJS.

The specialty with this NodeJS server is that it is listening on port 3000. If you remember, this is also the port NodeJS processes normal HTTP requests. The difference is the URI. Only requests sent to /whiteboard/ will be upgraded to WebSocket protocol. This is quite sensible as WebSockets operates on the same ports as HTTP/S, 80 and 443.

http {
  map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
  }
  
  upstream websocket {
    server localhost:3000;
  }
  
  # other stuff...
  
  location /socket.io/ {
    # socket.io stuff
  }
  # add right after socket.io location definition
  location /whiteboard/ { # this URI is where WebSockets is used
      proxy_pass http://websocket;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "Upgrade";
      proxy_set_header Host $host;
  }
}

Configure Database Backups

A method to backup and restore is crucial when it comes to production environments. MongoDB has built-in functions do carry out these procedures.

To backup the entire database:

# Install `zip` utility to compress the exported folder for download
sudo apt install zip unzip

sudo mongodump -d upchieve -o home/backups/
# this will create a backup directory (upchieve) with JSON or BSON of the collections
cd /home/backups
sudo zip -r db_29Jul20.zip upchieve

The resultant zip file can be downloaded from the SSH terminal by clicking on ‘Download File’ in the cog icon.

To restore the entire database, the zip file should be extracted to show the upchieve folder. Then;

mongorestore -d upchieve upchieve

UPchieve Server Gotchas

Git ignores certain configuration files like config.js. Therefore if you need to update them, you have to edit the copy on the production server. Updating on the git repository will have no effect.

That has been all for today’s Technically Speaking discussion. Although we focused specifically on UPchieve, we hope this document will act as a summary of setting up any server based on a similar stack (nginx and NodeJS). Hope you will use your newly learnt knowledge for improving the accessibility of education in the current times. Don’t hesitate to leave any comments or suggestions below!