Standalone Installation
Useful for development or custom implementations
Prerequisites
In the following, we will introduce the software prerequisites to be able to run the JeMPI client registry on your machine.
Docker
Please refer to the official installation guide in order to install docker on your machine. It is best to follow the post installation process so that you grant docker sudo access.
[for Windows users] WSL2
Installing WSL2 is required to be able to develop and test the project. It is recommended to limit the memory usage of WSL2.
Build Utilities
Follow the steps to install sdk sdkmanager, check if you already have it by running sdk. To install it, run the two following commands:
curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"To check the installation, you can check the version by running: sdk version
We should install the following:
Maven: Command: sdk install maven Version: 3.9.8
Scala Build Tool: Command: sdk install sbt
Java: Command: sdk install java 21.0.3-tem Version: Temerin 21.0.3-tem (See list by running: sdk list java)
Note: when installing with a non-root user set java directory
Check the version of java by running: java --version. We should get: Temurin-21.0.3+9.
Starting JeMPI
In the following section, we will discuss the steps for running JeMPI on you machine, start by cloning the JeMPI repository on your machine and navigate to JeMPI by running the following command in you terminal of choice
#1 Semi-Automated - Local Setup
Overview
This Bash script is designed for deploying JeMPI locally with various options. It performs tasks such as installing Docker, SDKMAN, Java, Maven, and SBT, setting up the environment configuration, creating a Docker registry, pulling and pushing Docker images, initializing the Docker Swarm, building the entire stack, rebooting, restarting, tearing down, Backup & Restore Databases and destroying JeMPI.
Usage
Location of file - JeMPI/devops/linux/docker/deployment File Name - local-deployment.sh
Set following variables JAVA_VERSION=21.0.3-tem JEMPI_ENV_CONFIGURATION=create-env-linux-low-1.sh
Installation Process
This script must be run from the following path and will not work if executed from a different location
Location of file - JeMPI/devops/linux/docker/deployment
Option 1: Deploy JeMPI (For Fresh Start) This Option used to install JeMPI from Scratch or Fresh setup
Set up hostname and IP address in the Hosts file.
Docker Swarm Initialization.
Creates a Docker registry, pulls Docker images from the hub, and pushes them to the local registry.
Builds and reboots the entire JeMPI stack
Option 2: Build and Reboot
Builds and reboots the entire JeMPI stack.
Option 3: Restart JeMPI
Reboots the entire JeMPI stack
Option 4: Down JeMPI
Stop entire stack
Option 5: Backup Postgres & Dgraph
Postgres backup process creates a folder with a timestamp, and inside it, SQL files are generated for each postgres database.
Backup Directory: JeMPI/devops/linux/docker/docker_data/data/backups/postgres
Dgraph backup process creates a folder with a timestamp, and inside it generates the Json file of data.
Backup Directory: JeMPI/devops/linux/docker/docker_data/data/backups/dgraph
Option 6: Restore Postgres & Dgraph
Users need to confirm with “ctr + Y” for restore.
This process will wipe all existing data from both Postgres and Draph DB’s and restore new from backup.
Users need to enter the folder name of the backup directory to initiate the restore process.
Option 7: Re-Deploy JeMPI
Updates environment configuration settings
Update HAProxy settings
Pulls Docker images from the hub, and pushes them to the local registry.
Builds and reboots the entire JeMPI stack
Option 8: Install Prerequisites
Install SDKMAN - SDK Manager
Install Docker
Install Java, Maven, and SBT using SDKMAN
Option 9: Destroy JeMPI (This process will wipe all data)
This process will remove all stack from swarm and leave the swarm.
Remove all data and volumes
Notes
The script prompts for user input to select an option.
Confirmations are requested for critical actions.
Use Ctrl+Y for "Yes" confirmation to Destroy all systems and Restore DB.
Customize the script as needed for your specific deployment requirements.
#2 Manual - Local Setup
Setup an IP address Before starting the process of running JeMPI, you will need to setup an IP address for your machine.
On your terminal of choice, run the ip a command and retrieve the ip address from your wi-fi or ethernet interfaces
In our case, we are using the enp0s3 interface, the IP address that we will need is 192.168.1.137.
Next, you will need to set up the hostname for your machine. to do so, run the command bellow, it will open the hosts file under /etc/ directory using the nano text editor (you can use any other editor e.g. VIM, VI, Emacs, Helix, etc.) :
Keep the localhost IP and Comment any other IP address, follow the screenshot bellow.
Initialize the environment variables In the JeMPI directory, navigate to: docker/conf/env/ directory.
if you have less than 32Gbs of ram run the ./create-env-linux-low-1.sh. If you have 32Gb or more, run the ./create-env-linux-high-1.sh. both those script will create conf.env file that we will need.
Note: for server installations, manually set SERVER_IP for environment variable before executing the script
Pull the latest images
Pull the latest image versions form docker hub using the a-images-1-pull-from-hub.sh
Make sure you have a clean docker swarm
It is fine to keep the images, you can either remove all the services, containers, volumes, configs and secrets.
Run the b-swarm-2-leave.sh to leave your current swarm
After running the previous script, initialize a new swarm by running the b-swarm-1-init-node1.sh script locacted in the JeMPI/docker/ directory.
Add the ability to use local registries
Now, we need to tell docker that it is okay to run on the local registry because it is http and not https.
To read more about it check this link: Test an insecure registry.
Go to devops/linux/docker/helper/scripts.
Run ./x-swarm-a-set-insecure-registries.sh (you need to grant it executable access first by running: chmod +x ./x-swarm-o-set-insecure-registries.sh), it will edit the file /etc/docker/daemon.json and will restart docker to make changes take effect.
NB: The script will edit the access grants of the /etc/docker/daemon.json file.
Create a local registry Now that you can use local Docker registries, run the c-registry-1-create.sh script to create a registry service. This service will host the docker images that we will use in our stack.
Push the images to the local registry We will need to pull images from docker hub then push them to the local registry :
Run the stack
After pushing the images into the local registry, we are ready to run the app, we have several options, we can run the whole stack (UI + Backend) by running the d-stack-1-build-all-reboot.sh, Or run each of the backend (d-stack-1-build-java-reboot.sh) and the UI (d-stack-1-build-ui-reboot) seperatly.
Other scripts
d-stack-2-build-java.sh: This script will build and push the backend services to the local docker registry.
d-stack-3-down.sh: This script will remove all services from the stack.
d-stack-3-reboot.sh: This script will only remove everything and start again.
That's it 🚀
Testing
Check the deployment sanity
To check for running containers you can run: docker container ls To check for running services you can run: docker service ls To list all the containers you can run: docker ps -a
Or you can go to devops/linux/docker/helper/scripts and run d-stack-ps.sh and it will run: docker stack ps <NAME_STACK>
Example of the stack when running with local docker registry: (docker stack ps jempi)
Example of the stack when running with docker hub: (docker service ls)
Stop or remove the stack
To remove everything in the swarm: you can go to devops/linux/docker/ and run b-swarm-2-leave.sh
To shut down the stack: you can go to devops/linux/docker/ and run: d-stack-3-down.sh
Last updated