1 of 1

Standalone Installation

Useful for development or custom implementations

Prerequisites

In the following, we will introduce the software prerequisites to be able to run the JeMPI client registry on your machine.

Docker

Please refer to the official installation guide in order to install docker on your machine. It is best to follow the post installation process so that you grant docker sudo access.

[for Windows users] WSL2

Installing WSL2 is required to be able to develop and test the project. It is recommended to limit the memory usage of WSL2.

Build Utilities

Follow the steps to install sdk sdkmanager, check if you already have it by running sdk. To install it, run the two following commands:

curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"

To check the installation, you can check the version by running: sdk version

We should install the following:

Maven: Command: sdk install maven Version: 3.9.8
Scala Build Tool: Command: sdk install sbt
Java: Command: sdk install java 21.0.3-tem Version: Temerin 21.0.3-tem (See list by running: sdk list java)

Note: when installing with a non-root user set java directory

source "/home/${USER}/.sdkman/candidates/java/current"

Check the version of java by running: java --version. We should get: Temurin-21.0.3+9.

Starting JeMPI

In the following section, we will discuss the steps for running JeMPI on you machine, start by cloning the JeMPI repository on your machine and navigate to JeMPI by running the following command in you terminal of choice

git clone https://github.com/jembi/JeMPI.git && cd JeMPI/

#1 Semi-Automated - Local Setup

Overview

This Bash script is designed for deploying JeMPI locally with various options. It performs tasks such as installing Docker, SDKMAN, Java, Maven, and SBT, setting up the environment configuration, creating a Docker registry, pulling and pushing Docker images, initializing the Docker Swarm, building the entire stack, rebooting, restarting, tearing down, Backup & Restore Databases and destroying JeMPI.

Usage

Location of file - JeMPI/devops/linux/docker/deployment File Name - local-deployment.sh

Set following variables JAVA_VERSION=21.0.3-tem JEMPI_ENV_CONFIGURATION=create-env-linux-low-1.sh

Installation Process

This script must be run from the following path and will not work if executed from a different location

Location of file - JeMPI/devops/linux/docker/deployment

./local-deployment.sh

Option 1: Deploy JeMPI (For Fresh Start) This Option used to install JeMPI from Scratch or Fresh setup

Set up hostname and IP address in the Hosts file.
Docker Swarm Initialization.
Creates a Docker registry, pulls Docker images from the hub, and pushes them to the local registry.
Builds and reboots the entire JeMPI stack

Option 2: Build and Reboot

Builds and reboots the entire JeMPI stack.

Option 3: Restart JeMPI

Reboots the entire JeMPI stack

Option 4: Down JeMPI

Stop entire stack

Option 5: Backup Postgres & Dgraph

Postgres backup process creates a folder with a timestamp, and inside it, SQL files are generated for each postgres database.
Backup Directory: JeMPI/devops/linux/docker/docker_data/data/backups/postgres
Dgraph backup process creates a folder with a timestamp, and inside it generates the Json file of data.
Backup Directory: JeMPI/devops/linux/docker/docker_data/data/backups/dgraph

Option 6: Restore Postgres & Dgraph

Users need to confirm with “ctr + Y” for restore.
This process will wipe all existing data from both Postgres and Draph DB’s and restore new from backup.
Users need to enter the folder name of the backup directory to initiate the restore process.

Option 7: Re-Deploy JeMPI

Updates environment configuration settings
Update HAProxy settings
Pulls Docker images from the hub, and pushes them to the local registry.
Builds and reboots the entire JeMPI stack

Option 8: Install Prerequisites

Install SDKMAN - SDK Manager
Install Docker
Install Java, Maven, and SBT using SDKMAN

Option 9: Destroy JeMPI (This process will wipe all data)

This process will remove all stack from swarm and leave the swarm.
Remove all data and volumes

Notes

The script prompts for user input to select an option.
Confirmations are requested for critical actions.
Use Ctrl+Y for "Yes" confirmation to Destroy all systems and Restore DB.
Customize the script as needed for your specific deployment requirements.

#2 Manual - Local Setup

Setup an IP address Before starting the process of running JeMPI, you will need to setup an IP address for your machine.

On your terminal of choice, run the ip a command and retrieve the ip address from your wi-fi or ethernet interfaces

skunk@skunks-server:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:bf:f5:e2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.137/24 metric 100 brd 192.168.1.255 scope global dynamic enp0s3
       valid_lft 86346sec preferred_lft 86346sec
    inet6 fdc2:75c1:449e:9200:a00:27ff:febf:f5e2/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 7145sec preferred_lft 3545sec
    inet6 fe80::a00:27ff:febf:f5e2/64 scope link
       valid_lft forever preferred_lft forever

In our case, we are using the enp0s3 interface, the IP address that we will need is 192.168.1.137.

Next, you will need to set up the hostname for your machine. to do so, run the command bellow, it will open the hosts file under /etc/ directory using the nano text editor (you can use any other editor e.g. VIM, VI, Emacs, Helix, etc.) :

sudo vim /etc/hosts

Keep the localhost IP and Comment any other IP address, follow the screenshot bellow.

Initialize the environment variables In the JeMPI directory, navigate to: docker/conf/env/ directory.

cd /devops/linux/docker/conf/env

if you have less than 32Gbs of ram run the ./create-env-linux-low-1.sh. If you have 32Gb or more, run the ./create-env-linux-high-1.sh. both those script will create conf.env file that we will need.

Note: for server installations, manually set SERVER_IP for environment variable before executing the script

export REACT_APP_JEMPI_BASE_API_HOST=http://${SERVER_IP} eg. http://192.168.10.90

# If you have less than 32Gb ram, run the following script

./create-env-linux-low-1.sh

# If you have 32Gb ram or more, run the following script

./create-env-linux-high-1.sh

Pull the latest images

Pull the latest image versions form docker hub using the a-images-1-pull-from-hub.sh

Make sure you have a clean docker swarm

It is fine to keep the images, you can either remove all the services, containers, volumes, configs and secrets.

Run the b-swarm-2-leave.sh to leave your current swarm

./b-swarm-2-leave.sh

After running the previous script, initialize a new swarm by running the b-swarm-1-init-node1.sh script locacted in the JeMPI/docker/ directory.

./b-swarm-1-init-node1.sh

Add the ability to use local registries

Now, we need to tell docker that it is okay to run on the local registry because it is http and not https.

To read more about it check this link: Test an insecure registry.

Go to devops/linux/docker/helper/scripts.

Run ./x-swarm-a-set-insecure-registries.sh (you need to grant it executable access first by running: chmod +x ./x-swarm-o-set-insecure-registries.sh), it will edit the file /etc/docker/daemon.json and will restart docker to make changes take effect.

NB: The script will edit the access grants of the /etc/docker/daemon.json file.

./x-swarm-a-set-insecure-registries.sh

Create a local registry Now that you can use local Docker registries, run the c-registry-1-create.sh script to create a registry service. This service will host the docker images that we will use in our stack.

./c-registry-1-create.sh

Push the images to the local registry We will need to pull images from docker hub then push them to the local registry :

./a-images-1-pull-from-hub.sh
./c-registry-2-push-hub-images.sh

Run the stack

After pushing the images into the local registry, we are ready to run the app, we have several options, we can run the whole stack (UI + Backend) by running the d-stack-1-build-all-reboot.sh, Or run each of the backend (d-stack-1-build-java-reboot.sh) and the UI (d-stack-1-build-ui-reboot) seperatly.

# build, push and run the whole stack (backend + ui)
./d-stack-1-build-all-reboot

# build, push and run the backend services only
./d-stack-1-build-java-reboot

# build, push and run the UI
./d-stack-1-build-ui-reboot

Other scripts

d-stack-2-build-java.sh: This script will build and push the backend services to the local docker registry.
d-stack-3-down.sh: This script will remove all services from the stack.
d-stack-3-reboot.sh: This script will only remove everything and start again.

That's it 🚀

Testing

Check the deployment sanity

To check for running containers you can run: docker container ls To check for running services you can run: docker service ls To list all the containers you can run: docker ps -a

Or you can go to devops/linux/docker/helper/scripts and run d-stack-ps.sh and it will run: docker stack ps <NAME_STACK>

Example of the stack when running with local docker registry: (docker stack ps jempi)

Example of the stack when running with docker hub: (docker service ls)

Stop or remove the stack

To remove everything in the swarm: you can go to devops/linux/docker/ and run b-swarm-2-leave.sh

To shut down the stack: you can go to devops/linux/docker/ and run: d-stack-3-down.sh