Jenkins backup to Github/Gitlab/Git

 Hello Guys,

Its been a long time after which i am writing a blog on a automation recently i got a requirement about backup the Jenkins into a git and since after a quick search on internet i find it out there is no such plugin available i have to do it the hard way 

Requirement : Backup all Jenkins important config files (xml) in git on daily basis

Now there are two ways to do it 

•  Write a script and schedule it in cron to execute it on midnight 
• create a Jenkins job for this and let the backup script get executed using jenkins

I like the 2nd approach as it will give a transparency I did not need to login to a box just to check if my script is working or not also i can send an email if i wanted to if some thing goes wrong and its easy to keep an eye on if some thing goes wrong

So lets get started

create a jenkins job its a free style jenkins job

in the General config you will find the option

Restrict where this project can be run : in the label expression write master . 

Source code management : None or you can have it if you want to  check out your script from git every time you want to execute the job

Build Trigger : Build Periodically in the schedule write 0 0 * * * while means you need to execute it on daily basis

Build : in the build section select Execute shell 

copy paste the below shell script

#!/bin/bash

# Setup

#

# - Create a new Jenkins Job

# - Mark "None" for Source Control Management

# - Select the "Build Periodically" build trigger

#   - configure to run as frequently as you like

# - Add a new "Execute Shell" build step

#   - Paste the contents of this file as the command

# - Save

#  

# NOTE: before this job will work, you'll need to manually navigate to the $JENKINS_HOME directory 

# and do the initial set up of the git repository.  

# Make sure the appropriate remote is added and the default remote/branch set up.

#  

# Jenkins Configuraitons Directory

cd $JENKINS_HOME

# Add general configurations, job configurations, and user content

git add -- *.xml jobs/*/*.xml userContent/*

# only add user configurations if they exist

if [ -d users ]; then

    user_configs=`ls users/*/config.xml`

    if [ -n "$user_configs" ]; then

        git add $user_configs

    fi

fi

# mark as deleted anything that's been, well, deleted

to_remove=`git status | grep "deleted" | awk '{print $3}'`

if [ -n "$to_remove" ]; then

    git rm --ignore-unmatch $to_remove

fi

git commit -m "Automated Jenkins commit"

git push -q -u origin master

save the job 

Login to jenkins server from shell Now Go to the Jenkins Home directory usually its a /var/lib/jenkins

also make you the public key(/var/lib/jenkins/.ssh/id_rsa.pub) for jenkins user is added to the github so that it will be able to push it

# git init

# git add remote <github repo url where you need to backup your jenkins>

once done we are all set to test it. Let me know how it works for you 

GitLab to Github Migration in Easy steps

 Hello Guys,

Its been a long time but recently i have tasked with migrating some of the gitlab repo to github and our objective is to migrate all the branches from gitlab repo to github and we have active it below steps

1. clone the repo which we need to migrate

   git clone <gitlab repo URL>

2. using the below custom git command we are scanning all the remote repos available on the gitlab

git branch -r | grep -v '\->' | while read remote; do git branch --track "${remote#origin/}" "$remote"; done

3. using get fetch and git pull we are pulling all the repos to the local check

  git fetch --all

  git pull --all

4. Add the Remote of github repo to this repository 

    git remote add github <github URL>

5.  Push the changes on the GitHub 

     git push --mirror github

In the above 5 easy steps we have migrate the all branches with history intact to github.

Get the list of most popular users on Github by Location and Language

Hello Guys,

recently i have been asked to get the list of most popular users from github and get the github handle and  retrieve the email. and get the output in CSV
many of you think its a typical developers stuff in which we are going to take response and extract the data etc.etc but I have solve the same proble with shell script which i want to share .

#!/bin/bash
read -p "Enter Your language: "  lang
read -p "Enter Your location: "  location
query="language:$lang+location:$location+type:user+repos:>5"
# for debugging..
#echo $query
#curl "https://api.github.com/search/users?q=$query" |grep -v total_count | grep -v incomplete_results |jq -r '.items[] | .login'
echo "GithubHandle, Email"
for i in `curl --silent  "https://api.github.com/search/users?q=$query" |grep -v total_count | grep -v incomplete_results |jq -r '.items[] | .login'`
do
arr=(`curl --silent https://api.github.com/users/$i/events/public|grep '"email":' | sort |uniq | awk -F ':' '{print$2}'|sed 's/,$//'`)
echo "$i,${arr[*]}" 
done

This will do the magic