How to integrate a GitHub repository with a Jenkins pipeline

It is a common requirement in any project to fetch the code from source control and run continuous integration and delivery cycle for building, reviewing, testing, packaging and deploying. Jenkins is one of the most popular orchestration tool used for continuous integration and delivery. GitHub is also a popular tool used for version control. It is a common scenario where a GitHub repository is required to be cloned on a location from where Jenkins can run the continuous integration cycle.

Jenkins can integrate with a large number of applications due to availability of extensive number of community contributed plugins which can aid in building, deploying and automating the projects. Jenkins can be easily integrated with GitHub. For our continuous integration, our first step will be to fetch the source code from GitHub. We will focus on this step in this post. Once we are able to fetch the source code, we should be able to build, test, review, package and deploy the source code during subsequent pipeline stages.

What is Cloning?

When we will integrate GitHub with Jenkins for the first time, we will need to clone the repository, that means we will create a replica of the GitHub repository on the system from where Jenkins can easily access the code.

Pre requisites

• Jenkins should be installed on the system with recommended plugins.
• JDK and Git should be configured on Jenkins.

Implementation

Lets go through the steps for implementation.

Create a new Jenkins Pipeline

First of all, we will create a new Jenkins pipeline project. For this click on New Item from the left side toolbar in the Jenkins Dashboard. We will provide a pipeline name and click OK.

Then we’ll be directed to pipeline configuration screen. We can also reach this screen from the project dashboard from Configure link in the left toolbar.

We will go to the pipeline section on this screen, where we will write the pipeline script. Inside the pipeline, we will create a stage to clone the github repository. Once the project is built, we should be able to clone the GitHub repository.

Pipeline script

We will create the pipeline script using declarative pipeline syntax. For this let us generate the script to clone our GitHub repository.

Note: If you are using a scripted pipeline, then just the structure of the pipeline script changes but overall implementation remains same. You can go through the Structure of a Scripted pipeline to understand the same.

We will next generate script for GitHub connectivity by using pipeline syntax generator. The reason for using pipeline syntax generator for generating the script is that it will not just generate the script but also let us know if there are issues with the integration between GitHub repository and Jenkins.

The above link will take us to the pipeline snippet generator page as shown in the screenshot below

Create personal access token in GitHub

GitHub does not allow to simply use the GitHub password for integration for security concerns.

For Jenkins and GitHub integration to work, we will require a personal access token from GitHub with all the required permissions. For our purpose, we will use fine grained token to achieve connectivity over http between GitHub and Jenkins so that we are able to access the GitHub repository. Fine grained tokens provide specific access for a specific period as compared to classic tokens which are more generic. We should provide the minimum required permissions to the token, which will fulfill our requirement. If our aim is to just clone and write to a specific repository, then we can only provide code read and write permission for that specific repository. To create a fine grained PAT (personal access token), you can follow the detailed instructions provided in this post: Generate fine grained personal access token in GitHub

Once we have created the personal access token we can move forward to the next section for adding the credentials in Jenkins.

Add GitHub personal access token in Jenkins

We can add new credentials in Jenkins via Add button in the Credentials section in the pipeline syntax generator screen.

Click Add button to open Add Credentials screen.

Create a new credential with Kind as as username and password. Provide personal access token number in the password section. Provide a description to identify the credential. You may provide the same in the Id field. Save the credential. Then select the credential from the select box under Credentials in the pipeline syntax screen.

Generate the script

Once the credentials are added, we will go to Generate Pipeline Script section to generate the script.

We will copy the pipeline script snippet and go back to the Pipeline section on the project Configure screen

We will then paste the script in our SCM stage.

node(label:'master') { 
    stage('SCM) {
        <Add script here>
   	echo 'SCM stage complete' 
    } 
}

Build the pipeline

If you like, you can add additional stages to the pipeline which will use the code cloned from the GitHub repository. Then we will save the Configure screen and return to the project dashboard. We will now run the Build on this screen.

When we build the job, Jenkins will clone the GitHub repository. This code can be used in the subsequent stages e.g. for building, testing and deploying.