Previously on Technically Speaking, we discussed about setting up the UPchieve platform and making fundamental configurations to both the web frontend and the server. In this article, we discuss about other configuration options
Before we can add categories, we have to remove the existing categories. The system determines categories using the categories the questions are added under.
The easiest way to do is to use the mongo command line.
mongo
use upchieve
db.question.remove({})
Or we can use the Edu Admindashboard by going to
http://<server>.example.org/edu
UPchieve Edu admin – add a question view
But make sure you have logged in as an Admin user to the UPchieve web client first.
After we have cleared the question collection, we can add our own questions using the Edu Admin dashboard. However, this procedure on its own will not update all mentions of the categories on the site. Some mentions we do have to update ourselves.
Changes to the web client
Firstly, we need to update the categories that appear in the subject selection area of the student dashboard. We need to update the topics.js file at;
/src/utils/topics.js
When creating the topics and subtopics in topic.js, make sure you make the displayName and the key name the same. (This might be a bug)
Changes to the server-side
We need to manually update the questions model at;
/models/Question.js (line 24 onwards)
The categories and subcategories here are not referring to the category and subcategory on the student dashboard! These subcategories refer to certain areas of the the subject. For example: if there is a a super category on the student dashboard called “IAL”, with a subcategory called “Maths”, the Question.js category should be “science” and subcategory should be, let’s say, “calculus”.
With the ongoing Covid19 situation, platforms that specialize in remote learning and education distribution, have become invaluable. UPchieve is such an open source platform. This platform connects a volunteering tutor and a student to connect and learn. The communication methods also include an interactive whiteboard and audio calling.
UPchieve also has iOS and Android apps, built on the React Native platform.
In a previous Technically Speaking installment, we outlined the steps needed to upload a Node.JS web app onto the Google Compute Engine. The NodeJS app used as the sample is the UPchieve/web app. The procedures for deploying the web app can be found in a previous Technically Speaking installment. In this article, we talk about how the server portion of the platform can be deployed on the Google Compute Engine.
This NodeJS stack comes with a twist – we will be using nginx as a reverse proxy server for NodeJS. This extra server setup has an advantage – NodeJS itself will run without requiring root permissions. Instead, nginx will handle http/s access for us and route requests locally to the NodeJS server.
As requirements, we are assuming that you have already created a new virtual instance on the Google Compute Engine.
Procedure
Install Applications
Initially, a VM would not have any software. We need to populate it with the software that fits our needs. For our scenario, we are going to need these software:
git
NodeJS
MongoDB
make
certbot (for SSL connectivity)
nginx
The commands to install these tools on a Debian 9 system are given below:
# Install git
sudo apt-get install git
# To install nodeJS, we need to install curl
# https://tecadmin.net/install-latest-nodejs-npm-on-debian/
sudo apt-get install curl software-properties-common
curl -sL https://deb.nodesource.com/setup_13.x | sudo bash -
sudo apt-get install nodejs
# Install mongodb
#Update 02Jul20 : Run `sudo apt-get install wget` if wget is not there
wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -
echo "deb http://repo.mongodb.org/apt/debian stretch/mongodb-org/4.2 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list
sudo apt-get update
sudo apt-get install -y mongodb-org
# Install make
sudo apt-get update
sudo apt-get install build-essential
# Install nginx
sudo apt update
sudo apt install nginx
systemctl enable nginx # start the nginx service
To install some of the tools on macOS (maybe as a development environment):
We need to pull the source from the remote GitHub server and store it on our server locally. The open source UPchieve server source can be pulled from its GitHub repository.
# Clones the repository onto a folder named 'server'
git clone https://github.com/UPchieve/server.git server
cd server
Starting Applications
Next, we will start the servers necessary for our UPchieve server to perform its first run/ setup. First, we will start the MongoDB background service.
For Linux systems:
sudo systemctl daemon-reload
sudo systemctl start mongod
systemctl status mongod # check if the service works
If the service has been set up successfully, you will see an output similar to this:
For macOS:
brew services stop [email protected]
brew services start [email protected]
mongod --config /usr/local/etc/mongod.conf --fork
# check if MongoDB is running
ps aux | grep -v grep | grep mongod
Next, we need to set up the UPchieve server and databases.
# setup database and install dependencies
cd server
bash bin/setup # if there is an error, run npm rebuild
node init
npm run dev # start upchieve server
# if you get a New Relic error, run
# cp node_modules/newrelic/newrelic.js newrelic.js
# if you get a bcrypt error, run `npm rebuild`
# if you still get the bcrypt error, run `npm install bcrypt`
You should be able to check if the server is working at this point. Open your browser and open the page at
http://<VM IP Address>:3000/eligibility/school/search?q=test
If it works, you might want to open a new shell (the current shell will be running the node server) to execute the other commands.
Production-ready!
We need a few more changes to make the application server ready for production. At the moment, we need to type in the IP address and the port number to access the server. Although this should also be fine since the consumers of the application would not have to access the server directly, this is not recommended. Besides, there is no SSL facilities on the server.
We will use nginx as a reverse proxy for our NodeJS server. Assuming that nginx is already installed, we need to configure it.
sudo nano /etc/nginx/nginx.conf
Add a server in this file to listen on port 80 (http port). We do this by adding an entry inside the http block (make sure to replace server.example.com by your sever name):
Before starting configuration of SSL, we might have to stop both our NodeJS and nginx servers
sudo systemctl stop nginx
ps aux | grep -i node # to find our node processes and PIDs
kill -9 <PID> # here PID is the ID of the node process
Since we have nginx running as a proxy, the usage of slightly different and tailored for an nginx environment:
sudo apt-get install certbot python-certbot-nginx
sudo certbot --nginx # automates the editing of nginx configuration file
sudo systemctl start nginx # start nginx service
cd server
npm run dev # start our NodeJS server
Depending on the selections you made during the SSL configuration, you would be able to access the server on both http and https at this point.
The socket.io protocol is used by the server to trigger request notifications on the volunteer dashboard and the session chat system. This should not be confused with WebSockets. These are two different protocols.We will be configuring WebSockets separately.
By default, the NodeJS server is listening to socket.io based requests port 3001. But we need to route it through our nginx server if we are to enable SSL for socket.io requests.
Our game plan to cover all these grounds is to:
Add a destination server on port 3001 to our NodeJS server (which will be http://localhost:3001)
Add a reverse proxy for the location /socket.io/ – this specific location is defined by the socket.io protocol. This proxy will take care of other socket.io requirements such as http upgrade
Publicize a SSL supported port, 3002, that can be accessed externally by our web app
We cannot use 3001 or 3000 in place of the 3002 without changing any NodeJS config code as this is what the NodeJS server itself is listening on. Instead, we define an unused port 3002.
http {
upstream upstream-nodejs { # NodeJS socket.io destination
server 127.0.0.1:3001; # OR localhost:3001
}
# other stuff...
# Add SSL support to port 3002 which will be publicized
listen 443 ssl; listen 3002 ssl; # managed by Certbot
# Other SSL properties...
location /socket.io/ {
# listen for this location on port 3002
proxy_pass http://upstream-nodejs;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
Configure Websockets
WebSockets also requires the http upgrade method. In contrast to the socket.io method we followed, we use a map symbol to describe this.
Next, we specify the URI that we should listen for WebSocket requests is /whiteboard/. Requests to this URI will be automatically referred to the back-end WebSockets server on NodeJS.
The specialty with this NodeJS server is that it is listening on port 3000. If you remember, this is also the port NodeJS processes normal HTTP requests. The difference is the URI. Only requests sent to /whiteboard/ will be upgraded to WebSocket protocol. This is quite sensible as WebSockets operates on the same ports as HTTP/S, 80 and 443.
http {
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream websocket {
server localhost:3000;
}
# other stuff...
location /socket.io/ {
# socket.io stuff
}
# add right after socket.io location definition
location /whiteboard/ { # this URI is where WebSockets is used
proxy_pass http://websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
}
}
Configure Database Backups
A method to backup and restore is crucial when it comes to production environments. MongoDB has built-in functions do carry out these procedures.
To backup the entire database:
# Install `zip` utility to compress the exported folder for download
sudo apt install zip unzip
sudo mongodump -d upchieve -o home/backups/
# this will create a backup directory (upchieve) with JSON or BSON of the collections
cd /home/backups
sudo zip -r db_29Jul20.zip upchieve
The resultant zip file can be downloaded from the SSH terminal by clicking on ‘Download File’ in the cog icon.
To restore the entire database, the zip file should be extracted to show the upchieve folder. Then;
mongorestore -d upchieve upchieve
UPchieve Server Gotchas
Git ignores certain configuration files like config.js. Therefore if you need to update them, you have to edit the copy on the production server. Updating on the git repository will have no effect.
That has been all for today’s Technically Speaking discussion. Although we focused specifically on UPchieve, we hope this document will act as a summary of setting up any server based on a similar stack (nginx and NodeJS). Hope you will use your newly learnt knowledge for improving the accessibility of education in the current times. Don’t hesitate to leave any comments or suggestions below!
