Continuous Integration of Github Releases with Jenkins
Introduction
Delivering the products or applications at high velocity is an important aspect of any company or organization. This is where DevOps comes into the traditional software development process. Specially DevOps is much more important in agile software development where many teams work on an iterative development process to deliver the product incrementally. In the same way, Continuous integration is a DevOps software development practice that often refers to the build or integration stage of a software release. Usually, the codebase needs to be built and run the relevant tests whenever the development teams merge new changes into their central repository. And at some point, if we want to deliver the project to the end-user, we can create a release that bundles the code base and relevant artifacts. This manual process can be done more efficiently by integrating GitHub with Jenkins which is an open-source automation server. Today we will look into how we can set up a Continous Integration workflow that can automate the build, test, and creation of releases using the power of Jenkins.
Overview
Following is the overview of the procedure that we have to follow in order to automate the workflow. There are multiple ways to implement the pipeline which has been discussed later in this article.
Stage 1: When creating the tag locally, we can use an annotated tag instead of a lightweight tag if we want to add something to the release description.
- Lightweight Tag: git tag v1.0.0
- Annotated Tag: git tag -a v1.0.0 -m “Github Release Description”
- After that, push the tag into Github: git origin push v1.0.0
Stage 2: Triggering the Jenkins job can be done in two ways. An appropriate method can be used according to the Jenkins development environment.
- Using GitHub WebHook (recommended)
If your Jenkins Server can be accessed from outside, then this is the best option you can use to trigger the Jenkins Job. A Github Webhook sends an HTTP POST payload to the Jenkins Server when something is pushed into the repository. Configuring the Github Webhook to successfully trigger a Jenkins job has been discussed later in this article. - Using SCM Polling
In some cases, Jenkins Server will be hosted inside a private network (depending on the company or organization). In that case, the GitHub Webhook will not operate as the Jenkins server is behind a secure firewall. Most probably these firewalls are one way which means the Jenkins side can connect to anything outside its network, but things outside cannot send any requests to Jenkins. Because of that, Github Webhooks will not work if the Jenkins Server is behind a secure firewall.
— One solution for this is to use SCM polling. Poll SCM periodically polls the SCM(Source Code Management) to detect any changes and if changes are found(if new commits were pushed since the last build), it will build the project. We can adjust the polling time to suit our requirements. But the main drawback is that this method of polling doesn't get the changes in real-time. So it’s not much efficient compared to Github Webhooks.
— Another approach is to use a webhook forwarding service. Smee is a webhook forwarding service that can be used in this case. Here, GitHub pushes an event (via HTTPS/JSON in this case) to Smee.io, and Jenkins in turn subscribes to Smee with an outgoing connection from a client. Jenkins only makes an outbound connection and this will work as long as the firewall is one way.
Stage 3: The project can be built and tested before creating a release in Github. Although the whole project is tested at the time of release creation, this step can be used optionally to mitigate any falsehoods that will happen before creating a release. A Jenkins pipeline job would be ideal for this workflow if the project is going to be tested again before assembling a release. A special file called Jenkinsfile which is written in Groovy will be used by the Jenkins Pipeline job to execute the workflow. All kinds of tests that the project should undergo before creating the release can be specified under separate stages in this Jenkinsfile.
Stage 4/ Stage 5: Creating a new release from the latest tag and uploading built assets to Github using the GitHub REST API can be done in several ways. If you are using a Jenkins Pipeline project this can be accomplished using a Shell Script or a Python script which will be defined under the post-action section of the Jenkinsfile. But both ways have their own ups and downs which will be discussed later in this article.
Prerequisites
- Github Account — With a repository that can be built and tested. (I’m using an android project in this article to explain the workflow)
- Jenkins Server —In most cases, this could be public or sometimes it’s hosted behind a secure firewall.
Procedure
Github Configuration
- Setting up a Personal Access Token
A personal access token(PAT) is an alternative to using passwords for authentication to GitHub when creating a release and uploading build artifacts using Github API. Follow the below procedure to configure a PAT.
— Verify the email associated with the Github account, if it hasn't been verified yet.
— Click Profile Photo → Settings → Developer settings →Personal access tokens →Generate new token
— Give the token a descriptive name and an expiration date
— Select the scopes or permissions, you’d like to grant to this token.
(I would suggest only to give the most needed permissions to get the job done)
— Then Generate the token and save it somewhere as it won’t show again after the popup is closed.
— To use your token to authenticate to an organization that uses SAML single sign-on, authorize the token. For more info: Authorizing a personal access token for use with SAML single sign-on
- Setting up a GitHub Webhook
This method only works if the Jenkins Server can be accessed from outside. (If it’s under a secure firewall, use SCM Polling instead of WebHooks). Follow the below procedure to configure a Github Webhook.
— Go to the Github repository → Settings → Webhooks →Add webhook
— Payload URL: Use the IP or domain name of your Jenkins host as the Jenkins_environment_URL
— Secret: Leave this field empty
Note: The Jenkins and Git Plugin only triggers the jobs if you select “Just the push event” option. Although you can select the “Let me select individual events” option as an event type to trigger the webhook, that will not trigger the Jenkins Job when a new tag is pushed. So make sure to select the “Just the push event” option. Even though we only need to build the Jenkins job for new tag pushes only, the Webhook will trigger the Jenkins job for any kind of pushes to the repository. But later, Jenkins job itself can be configured to build only for new tag pushes.
Jenkins Configuration
- Setting up Jenkins
Install the Github plugin if it’s not installed by following the below procedure.
— Go to Manage Jenkins →Manage Plugins →Available
— Then search for “Github plugin”, install the plugin and restart - Setting up a Jenkins Job
This part will be trickier if we are going to use SCM Polling to build the Jenkins job and also pipeline job as the project type. If we can use Webhooks, the configuration part for the Jenkins job will be pretty easy. Below are some important things we need to consider when following different approaches to configure the Jenkins Jobs. Also, the script to create a release and upload built artifacts to Jenkins will be discussed later in this article.
Using Github Webhook to trigger Jenkins and Freestyle Project as the job type
- Configure the Github Webhook as mentioned above.
- Under the “General” section select GitHub project and provide the Github Project URL
- Under the “Source Code Management” section do the configuration as follows
Repository URL: git@github.com:<Organization>/<Repository>.git
or https://github.com/<Organization>/<Repository>
Credentials: Provide the credentials used to check out sources.
In order to make sure that the Job is only building for new tags, do the following configurations further in the SCM section. ( click “Advanced” in the SCM section)
Refspec: +refs/tags/*:refs/remotes/origin/tags/*
A refspec controls the remote refs to be retrieved and how they map to local refs and if left blank, it will default to the normal behavior of git fetch, which retrieves all the branches. Here, we only need tags so we set the refspec to tags only.
Branch Specifier: refs/tags/*
This specifier tells Jenkins to only build on new tags, no matter how the tag is named. So this tracks/checks out the specified tag.
- Next, under the “Build Triggers” Select the “GitHub hook trigger for GITScm polling”
- Finally provide the script in the “Build” area of the Freestyle Project to build the application, create a release and send the built artifacts to Github using Github API. (This script is available in the final section of the article)
Using Github Webhook to trigger Jenkins and Pipeline Project as the job type
- Configure the Github Webhook as mentioned above.
- Do the same configuration to the “General” and “Source Code Management” sections as mentioned above
- Under the “Build Triggers” Select the “GitHub hook trigger for GITScm polling”
- Then write the script in the Jenskinsfile (A Jenkinsfile is a text file that contains the definition of a Jenkins Pipeline and is checked into source control). The building and testing of the project can be done inside separate stages of the Jenkinsfile. Finally, the release creation and uploading of the built artifact to Github can be done in the post-action area of the Jenkinsfiles using a shell script or a python script. Below is a sample of the Jenkinsfile which is written in Groovy.
SCM Polling to build the Jenkins Job and using a Freestyle Project as the job type
- Do the configuration as in “Using Github Webhook to trigger Jenkins and Freestyle Project as the job type”
- Additionally under the “Build Triggers” Select the “Poll SCM”
This field follows the syntax of cron (with minor differences).
Specifically, each line consists of 5 fields separated by TAB or whitespace:MINUTE HOUR DOM MONTH DOWE. g.H/15 * * * *: poll every fifteen minutes
But here we will configure this job to poll once a minute.* * * * *: poll once a minute
SCM Polling to build the Jenkins Job and using Pipeline Project as the job type
- This part is somewhat trickier to implement. As there is no “Source Code Management” Section, “Poll SCM” option doesn’t work in a Pipeline project. Instead we can use a Freestyle Job to do the polling and finally execute the Pipeline project from its “Build after other projects are built” option under “Build Triggers”.
- First, configure a freestyle project(Job 1) as in “Using Github Webhook to trigger Jenkins and Pipeline Project as the job type”. And in the “Build” section execute a normal shell command just to build the job.
- Next, create a Pipeline job similar to “Using Github Webhook to trigger Jenkins and Pipeline Project as the job type”
- Under the “Build Triggers” of Pipeline job, select the “Build after other projects are built” option and provide Job 1 as the Project to watch. Also, select “Trigger only if build is stable”.
- This will successfully trigger the pipeline job when a new tag is created through SCM Polling.
Build Script
Following are different scripts that you can use to create a release and upload the built artifacts to Github using Github Rest API. You can specify this script directly in the Jenkins Job if you are using a Freestyle job. If you want more maintenance on the scripts for the developers, you can add the script to the Github Repository and run the below python script in the “Build” area to fetch the script and execute it. For better security, you can save the Github Credentials as Secret Text inside Jenkins Credentials Manager and specify the Credentials under the “Build Environment” of the job. Select the “Use secret text(s) or file(s)” and provide the necessary bindings that the script in the “Build” section needed. (In this case, provide the Github Personal Access Token and Branch of the Github repository where the actual script is located). Also, you can fetch these secrets using os.getenv('Name_of_the_secret')
import requests
import base64
import json
import os# Github Specifics
GITHUB_ACCESS_TOKEN = os.getenv('Personal_Access_Token')BRANCH = os.getenv('BRANCH')PATH =
FILE_URL = "https://api.github.com/repos/<Organization>/Repository/contents/<Scrip_Path>?ref={}".format(BRANCH)# Function to fetch get the python script and read it
def github_read_file(url):
r = requests.get(url, headers={'Authorization': 'token %s' %GITHUB_ACCESS_TOKEN})
data = r.json()
file_content = data['content']
file_content_encoding = data.get('encoding')
if file_content_encoding == 'base64':
file_content = base64.b64decode(file_content).decode()
return file_content# Execute the python script returning from the above funtion
exec(github_read_file(FILE_URL))
Otherwise if you are using a Pipeline job, specify the script under the post-build area of the Jenkinsfile. Also if you are using a python script, execute the python script under the “post” section of the Jenkinsfile as follows. (Make sure the python script is in the same directory as the Jenkinsfile. Otherwise provide the relative path to the script)
post {
success {
sh 'python <Script_Name>.py'
}
}Note: If you are using a shell script, aware that some of the shell commands are not working properly inside Jenkinsfile. There will be json parsing problems and also other kinds of problems that may occur when executing shell commands inside a Jenkinsfile. If you are going to use a pipeline job, execute a python script in the post-action area as it will mitigate most of the problems.
— If you are using a shell script in the Build section
echo "Script starting...";//Github Personal Access Token
token="<YOUR GITHUB ACCESS TOKEN>";//Get the latest git tag
tag=$(git describe --tags); //Get git message associate with the tag message="$(git for-each-ref refs/tags/$tag --format='%(contents)')"; //Get the name
name=$(echo "$message" | head -n1);//Get the description
description=$(echo "$message" | tail -n +3)
description=$(echo "$description" | sed -z 's/\n/\\n/g') //Create the release from tag(put the github default branch as the target_commitish)
release=$(curl -X POST -H "Authorization:token $tag" --data '{\"tag_name\":\"$tag\",\"target_commitish\":\"develop\",\"body\":\"$description\",\"draft\":false}' https://api.github.com/repos/<Organization>/<Repository>/releases);
//Extract the id of the release from the creation response
id=$(echo "$release" | sed -n -e 's/"id":\ \([0-9]\+\),/\1/p' | head -n 1 | sed 's/[[:blank:]]//g')//Get the built artifact from Jenkins
file = */app/build/outputs/apk/release-*.apk;//Upload the artifact to Github
curl -X POST -H "Authorization:token $token" -H "Content-Type:application/octet-stream" --data-binary @file https://uploads.github.com/repos/<Organization>/<Repository>/$id/assets?name=release.apk
— If you are using a python script in the Build section
import subprocess
import requests
import json
from subprocess import Popen, PIPE
import glob, os
print("Python Script Starting..")#Github Personal Access Token
token="<YOUR GITHUB ACCESS TOKEN>"#Get the latest tag
process = subprocess.Popen(["git", "describe", "--tags"], stdout=subprocess.PIPE)#Get the tag name
tag_info = process.communicate()[0]
tag = tag_info.split('-')[0]#Creating the release
endpoint ="https://api.github.com/repos/<Organization>/<Repository>" data = {"tag_name":tag,"name":"SWMS Mobile - Release - %s" % tag}
release = requests.post(endpoint,headers={'Authorization': 'token %s' %token},data=json.dumps(data))
id = release.json().get('id')#Get the Jenkins directory location to get the built artifact
directory = os.getcwd()#Find the artifat using a wildcard
filespec="*/app/build/outputs/apk/release-*.apk"
release_apk = glob.glob(os.path.join(directory, filespec))[0]#Read the file as binary data
with open(release_apk, 'rb') as f:
data = f.read()#Uploading the artifact to Github
upload_asset = requests.post("https://uploads.github.com/repos/<Organization>/<Repository>/releases/%s/assets?name=release-%s.apk" %(id,tag),headers={'Authorization': 'token %s' %token, "Content-Type":"application/octet-stream" },data=data)
But in some cases, the script is not executing successfully when the artifact size is somewhat large (>20mb+). In that case, you can either send the file in small chunks or use a python library to upload the artifact using the GitHub REST API. Below is a script that is written with the help of the PyGithub Python library. But before proceeding, you have to install PyGithub to the Jenkins Environment. Or else if you have installed “pip”, you can install the PyGithub library within the script as follows.
import glob
import os
import pip
import site
import sys
import requests# Install PyGithub package
if not os.path.exists(site.USER_SITE):
os.makedirs(site.USER_SITE)
sys.path.insert(0, site.USER_SITE)
pip.main(["install", "--user", "PyGithub"])
# Create release and upload assets
from github import Github
print("Python Script for CI workflow")
g = Github(<GITHUB_PERSONAL_ACCESS_TOKEN>)
repo = g.get_organization(organization).get_repo(repo_name)
# Get all tags at page 0
tags = repo.get_tags()
# Sort tags by date to get the latest tag
tags = sorted(tags, key=lambda tag: tag.commit.commit.author.date, reverse=True)
latest_tag = tags[0].name
# Create a release from the latest tag
release = repo.create_git_release(
tag=latest_tag,
name="Release",
message="",
)
#Get the Jenkins directory location to get the built artifact
directory = os.getcwd()#Find the artifat using a wildcard
filespec="*/app/build/outputs/apk/release-*.apk"
release_apk = glob.glob(os.path.join(directory, filespec))[0]# Upload the built artifact to the created release
release.upload_asset(
path=release_apk,
content_type='application/vnd.android.package-archive',
name='release-%s.apk' %latest_tag
)
Note: If you want to add some documentation URL to the Github Release, you can use an annotated tag instead of a lightweight tag and specify the Documentation URL along with the tag as follows
E.g. git tag v1.0.0 “https://documentation_url.com”
You can get this annotated tag message and add it as a hyperlink to the release description.
# Get the reference of latest tag in order to get the tag sha of annotated tag itself. (Important)
tag_ref_command = 'git show-ref ' + latest_tag
tag_ref_stream = os.popen(tag_ref_command)
tag_ref_output = tag_ref_stream.read()
tag_sha = tag_ref_output.split()[0]# Get details of the reference
tag_response = requests.get("https://api.github.com/repos/<organization>/<repo_name>/git/tags/$tag_sha",headers={'Authorization': 'token %s' %GITHUB_ACCESS_TOKEN, "Content-Type":"application/octet-stream" }) # Get the tag message from the reference details
document_url = tag_response.json()["message"].strip()# Make the hyperlink body
message = "Release Note : [Release - %s](%s)" %(latest_tag, document_url)
# Create the release from the latest tag and add a hyperlink of the Documentation URL as the release body
release = repo.create_git_release(
tag=latest_tag,
name="SWMS Mobile - Release - %s" % (latest_tag),
message=message
)
Conclusion
This article provides various ways that you can implement the Continous Integration workflow for release automation with Jenkins which I have researched and developed. I have mentioned some important points that you need to know when publishing releases on Github when a tag(annotated/lightweight) is pushed. Although this workflow is not that much hard to implement, it’s not possible to natively use the Jenkins Github plugin to implement this workflow. So, I hope this guide helped you to customize and create your own automated workflow for release creation.
References
[1]: SystemGlitch. February 11 2019). Continuous integration with Jenkins and Github Release https://medium.com/@arinbasu/you-can-use-footnotes-thus-babus%C2%B9-6c485c4eff1e
