How to easily setup mongodb database with github actions workflow

Any real world application would most often use some form of database for storing master and transactional data. When we build our CI pipeline on Github actions, we are often required to setup a database in our test environment in order to execute test cases.
In this post, we will setup a mongodb database in a test environment for a java web app.
If you want to learn the basics of how to setup a build and test pipeline for a java web app with maven, you can refer to – GitHub Actions workflow to enable a simple CI pipeline for a java web app with maven and junit

Steps to create mongodb database with test data

The steps that we will follow to setup our database are mentioned below :
1. First we will export data required to run our test cases into a file from mongodb database collections.

2. Next, from within our github actions workflow file, we will

a. Setup a mongodb database using the latest image from docker hub.

b. Import data from the file in our mongodb database on new environment using github actions

We will use a simple docker command to setup mongodb in our github actions environment, which we will discuss in detail as we move forward. You do not need to be an expert at docker to be able to use this command. Though it will be helpful to understand containers as a concept so that you can follow what we are doing here.

We will not need any additional server resource for our workflow as Github itself will provide the required resources.

Scenario Walkthrough to setup mongodb database

Let us consider the following example. We have a database named appDB which for the sake of simplicity contains just 2 collections. First is the userColl which contains the user data and the second collection is appData which contains transactional data for the application . Here we do not need all the data present in our collections but only the test data which we can use for running our CI pipeline. 

We will follow the following steps:

1. Export the test data from mongodb

First we will need files containing the data that we want to import in our database. For this we will use a mongodb command line utility called mongoexport. The mongoexport tool is part of the MongoDB Database Tools package. Follow the Database Tools Installation Guide to install mongoexport if not already installed.The mongoexport command has the following form.

mongoexport –collection=<coll> <options> <connection-string>

Run mongoexport from the system command line, not the mongo shell.

You need to specify the collection which you need to export. If you do not specify an output file, mongoexport writes to the standard output (e.g. stdout).
For our purpose, we will select the output file as .json file. To connect to a local MongoDB instance running on port 27017, you do not have to specify the host or port. If your database is on a remote server, you need to specify the uri/host. You can check the syntax from mongodb documentation for mongoexport.

We will run following commands to export data from our 2 collections from mongodb database installed locally. Goto the bin directory of your mongodb installation and run the following commands

mongoexport –collection=userColl –db=appDB –out=userColl.json

mongoexport –collection=appData –db=appDB –out=appData.json

We will get 2 .json files as output – appData.json and userColl.json

2. Edit the test data

After this we can edit the json file to ensure that it contains only the required data and add/delete data as required. 

3. Upload the test data in GitHub repo

Once we have the required data in the .json files, we will upload these files in the github repo that contains our code. 

4. Create mongodb container

Next step is to create the mongodb database from our workflow file (. yaml file).

If you want to learn the basics of how to setup a build and test pipeline for a java web app with maven you can refer to this post.

Once you are familiar with the fundamentals of workflow creation, you can proceed from here. 
Following code will need to be included in the workflow or the .yml file. 

  – name: Create mongoDB Docker container      run: sudo docker run -d -p 27017:27017 mongo:latest

 With the above command, we will install a docker container with the latest version of mongodb available on docker hub which can be accessed by our web application. This command will run in the default shell of the OS of our test environment. We are using – d flag to run this container in a detached mode, so that the container will run in the background. We use the – p or the – – publish flag to create a firewall rule which maps the docker container port on which mongodb is installed to a port on the docker host from where it can be accessed by the outside world. 

5. Import the data

Once we have our mongodb database in place, all we have to do is import the test data to our database. Below is the sample code on how to achieve it.      

– name: Add data to userColl collection 

run: mongoimport –collection=userColl –db=appDB –file=userColl.json   

– name: Add data to appData Collection     

run: mongoimport –collection=appData –db=appDB –file=appData.json

Conclusion

To sum it all, let us put all the code together in a workflow file. For this, let us fetch the workflow file from our example GitHub Actions workflow to enable a simple CI pipeline for a java web app with maven and junit and insert the above code in that file. We will have the following updated workflow. Mongodb database specific code has been highlighted in blue text color.

# This workflow will build a Java project with Maven and mongodb

# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven
name: Java CI with Maven
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Set up JDK 1.8
      uses: actions/setup-java@v1
      with:
        java-version: 1.8

    - name: Create mongoDB Docker container      
      run: sudo docker run -d -p 27017:27017 mongo:latest
    - name: Add data to userColl collection        
      run: mongoimport --collection=userColl --db=appDB --file=userColl.json    
    - name: Add data to appData Collection      
      run: mongoimport --collection=appData --db=appDB --file=appData.json

    - name: Build with Maven
      run: mvn -B package --file pom.xml

Let us know if you found this post helpful.