Setting up the HashiCorp Vault and Ansible AAP AWX integration Part-4

 In this part we will setup a project in AAP and sync that from (github/gitlab) and using that project create job template execute the same.

So lets quickly go to github and create a repo and add a file name password.yml

---

- name: Genrate and set random password on Remote Servers

  hosts: all

  gather_facts: no

  tasks:

    - name: Check if server are reachable or not

      ansible.builtin.ping:

      register: ping_result

    - debug:

        msg: "{{ ping_result }}"

     

    - name: Generate Complex randome password

      set_fact:

        generated_password: "{{ lookup('community.general.random_string', length=12, min_lower=1, min_upper=1, min_numeric=1, min_special=1, override_special='-_=+!#$()[]') }}"

    - name: Write password to Vault using key value V2 engine

      delegate_to: 127.0.0.1

      community.hashi_vault.vault_write:

        path: secret/data/dev/{{inventory_hostname}}

#        auth_method: approle

#        role_id: ''

#        secret_id: ''

        data:

          data:

            password: "{{ generated_password }}"

    - name: Setting password for user

      ansible.builtin.user:

        name: "{{ ansible_user | trim }}"

        password: "{{ generated_password | password_hash('sha512', 'mysecretsalt') }}"

So in the  This file will help to setup the password on remote server also update it in the password vault so this way if required we can rapidly change the password without disrupting existing automation.

Also we will write one more playbook to test if password change worked and we can execute the play become and after the password change.This play will show you the output of ifconfig and hostname command.point being we can using the updated password from vault and able to connect 

---

- name: Debug AAP

  hosts: all

  tasks:

    - name: Running Hostname command to confirm and no funny bussness

      ansible.builtin.shell: hostname

      register: hostname

    - name: Show debug output

      ansible.builtin.debug:

        msg: "{{hostname.stdout}}"

    - name: Running ifconfig command to confirm and no funny bussness

      ansible.builtin.shell: ifconfig

      register: ifg

    - name: Show debug output

      ansible.builtin.debug:

        msg: "{{ifg.stdout}}"

lets we need to create a job template out in one job template select password.yml and in other select the test.yml also the execution environment will be custom-ee which we have created in the part 3 and Enjoy

we can see in the screen shot as below the execution

Changing/Rotating the Password

Updated Random pass for one of the server 

Able to connect even after changing the pass so no impact to existing automation

We are able to create a zero trust environment .In which we can rotate the password every 60 days if requireed to keep our system safe.

Let me know if on which ansible you want to know more and i am happy to help

 

Setting up the HashiCorp Vault and Ansible AAP AWX integration Part-3

 So Now lets start with building the required execution environment to use the collections which we need to execute our play we will required below collections

1.    community.general --> To Generate the Random Password

2. community.hashi_vault --> To interact with the HashiCorp vault

So lets get into the action lets Install the ansible builder don't go for the latest one as you may encounter some issues lets go for the one stable release as of writing the blog the the release version i know id 3.0.1 so i am installing the same 

We are installing pip if already not present and then using pip we are installing the ansible builder

# dnf install python3-pip

# pip install ansible-builder==3.0.1

Once the ansible builder is installed lets create some files like 

Note: EE stands for execution environment

Create a directory call EE and go to that directory

# mkdir EE and cd EE

create a first file execution-environment.yml and add following content:

cat <<EOT >> execution-environment.yml

---

version: 1

dependencies:

  galaxy: requirements.yml

  python: requirements.txt

  system: bindep.txt

additional_build_steps:

  prepend: |

    RUN whoami

    RUN cat /etc/os-release

  append:

    - RUN echo This is a post-install command!

    - RUN ls -la /etc

EOT

Now lets start create the dependencies which we have specified 

cat <<EOT >> requirements.yml

---

collections:

  - name: community.general

  - name: community.hashi_vault

EOT

We have some dependencies for the collections which will specify in the requirements.txt

cat <<EOT >> requirements.txt

gcp-cli

ncclient

netaddr

paramiko

hvac

EOT

If We have some binary dependencies then specify it bindep.txt

cat <<EOT >> bindep.txt

findutils [compile platform:centos-8 platform:rhel-8]

gcc [compile platform:centos-8 platform:rhel-8]

make [compile platform:centos-8 platform:rhel-8]

python39-devel [compile platform:centos-8 platform:rhel-8]

python39-cffi [platform:centos-8 platform:rhel-8]

python39-cryptography [platform:centos-8 platform:rhel-8]

python39-pycparser [platform:centos-8 platform:rhel-8]

EOT 

One we have added everything we will start building the image using command 

ansible-builder build -v3 -t custom-ee 

Once the image is build we can tag the image 

podman tag custom-ee  aap2.example.com/custom-ee

podman push aap2.example.com/custom-ee

Once we have done with this 

Once we are done with this We can start with the building playbook

Setting up the HashiCorp Vault and Ansible AAP AWX integration Part-2

 In the Second part lets Start working with the AAP login to AAP and login as a Admin user and go to the Credential Type and click on Add

Create a New Credential called HashiCorp

Input Configuration:

fields:

  - id: vault_server

    type: string

    label: URL for Vault Server

  - id: vault_token_id

    type: string

    label: Vault token ID

    secret: true

required:

  - vault_server

  - vault_token_id

And Injector Configuration 

env:

  VAULT_ADDR: '{{ vault_server }}'

  VAULT_TOKEN: '{{ vault_token_id }}'

Its Looks like This

Now Go to Credentials and Create a Credential call hashicorp_token

Enter the values which are associated with the vault 

Now go to inventory and create a Inventory i have created a inventory name hashi and add a host which will look like this just make sure you add the below line in variables

ansible_password: "{{ lookup('hashi_vault', 'secret=secret/data/dev/{{ inventory_hostname }}:password')}}"

We are almost done from host setup stand point in the in AAP all which is remaining is writting a playbook to change the password.

Now lets take a look at setting up the initial password in the hashicorp vault 

login to vault with the taken available in the init file present on path /etc/vault/init.file

Login to UI of Vault and go to Secrets Engine Go to generic inside select the KV

Path : secret 

After the Engine is enable the screen will look like this 

Click on secret and create a paths as per the environment for me its a development environment so i have label it dev and the FQDN or the IP address which we have specify in the inventory

and create a password secret once you build the secrets it may look like this 

As we are done with this part we are almost done We will create the execution environment in the AAP or AWX

Setting up the HashiCorp Vault and Ansible AAP AWX integration Part-1

 Hello Guys ,

Welcome back to my block couple of weeks back i need to work on a specific client request when they want to change the passwords of all of the server for the automation_user without interrupting the automation and daily operation so they ask me how one can achieve this. They came to me with below problem statement

 So the problem statement like this 

Business own the around 1000 server and

 they want to change the password for the automation user

they want to rotate the password every 90 days and 

there should not be any interruption in the operations 

So I came up with an idea if we have to go through the exercise every 90 days so we have to do it again and again so its best we should automate it so they we don't face the problem again and again and it will keep on doing it in the background 

 So i recommended the we should setup a vault and let the vault manage the passwords(store the passwords) and since the HashiCorp vault is the community driven and open source its of my choice below are the steps i have taken to set it up

Download and Unzip the vault binary on the server

cd /opt/
sudo wget https://releases.hashicorp.com/vault/0.10.3/vault_0.10.3_linux_amd64.zip && sudo unzip vault_0.10.3_linux_amd64.zip -d .   

Copy the binary to appropriate paths on the linux system

sudo cp vault /usr/bin/

Create the config , data, and logs directory so that vault can store the data

sudo mkdir /etc/vault
sudo mkdir /vault-data
sudo mkdir -p /var/log/vault/

 Create the config file with below configs

sudo vi /etc/vault/config.json

Below are the config

Note: here 192.168.1.18 is the ip address of my server

{
"listener": [{
"tcp": {
"address" : "0.0.0.0:8200",
"tls_disable" : 1
}
}],
"api_addr": "http://192.168.1.18:8200",
"storage": {
    "file": {
    "path" : "/vault-data"
    }
 },
"max_lease_ttl": "10h",
"default_lease_ttl": "10h",
"ui":true
}

Create a service file to start,stop and restart the service

sudo vi /etc/systemd/system/vault.service

Below are the configs

[Unit]
Description=vault service
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/vault/config.json

[Service]
EnvironmentFile=-/etc/sysconfig/vault
Environment=GOMAXPROCS=2
Restart=on-failure
ExecStart=/usr/bin/vault server -config=/etc/vault/config.json
StandardOutput=/var/log/vault/output.log
StandardError=/var/log/vault/error.log
LimitMEMLOCK=infinity
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target

Start the service

sudo systemctl start vault.service

Check the status

sudo systemctl status vault.service

In order to connect from command line make the below changes in the bashrc config

export VAULT_ADDR=http://192.168.1.18:8200
echo "export VAULT_ADDR=http://192.168.1.18:8200" >> ~/.bashrc

Check the vault status

vault status

Initialise the vault

vault operator init > /etc/vault/init.file

Access the vault and unseal the same from UI using the keys available in the init.file which is available at /etc/vault/init.file

http://IPADDRESS:8200/ui

In the part 2 will cover the integration with AAP and how we can do it...

Ldap integrauon with Redhat Ansible AAP or Ansible Community Tower Part 2

 Now Lets start with the Ansible IT part you need to have working AAP setup

Go to settings and the LDAP settings in that Enter the below values

Ldap server URI :  ldap://192.168.1.17:389

ldap bind dn:  cn=admin,dc=example,dc=org

ldap bind password : admin

ldap group type : PosixGroup Type

Ldap User Search : 

[

  "OU=users,dc=example,dc=org",

  "SCOPE_SUBTREE",

  "(uid=%(user)s)"

]

Ldap Group Search:

[

  "dc=example,dc=org",

  "SCOPE_SUBTREE",

  "(objectClass=group)"

]

Ldap User Attribute map:

{

  "email": "mail",

  "first_name": "givenName",

  "last_name": "sn"

}

Ldap Group Type Parameters:

{

  "name_attr": "cn"

}

Ldap User Flag By Group:

{

  "is_superuser": [

    "cn=superusers,ou=users,dc=example,dc=org"

  ],

  "is_system_auditor": [

    "cn=auditors,ou=groups,dc=example,dc=org"

  ]

}

save all the settings and try login using nrathi ,kjha and lrathi

if you login using nrathi it will be System Administrator

if you login using kjha it will be Nornal User

if you login using lrathi it will be system auditor

And thats how its done...! Enjoy

Ldap integrauon with Redhat Ansible AAP or Ansible Community Tower Part 1

 In order to achieve this in POC we need to have Below prerequisites we need at least two VM's 

1  Ansible Controler / Ansible Tower

OS: Redhat 9  

 IP : 192.168.1.11

2 Ldap Server

OS: redhat 9 

IP : 192.168.1.17

    On the ldap server there is podman installed by default if not install podman its a alternative to docker

launch a podman container with ldap with below command

 # podman run -p 389:389 -p 636:636 --name my-openldap-container osixia/openldap:1.5.0 --env LDAP_ADMIN_PASSWORD="admin"

if you are not an expert with ldap and required GUI then launch another podman container

#  podman run -p 443:443  --env PHPLDAPADMIN_LDAP_HOSTS=192.168.1.17         --detach osixia/phpldapadmin:0.9.0

Once you done with launching two containers you need to test that they are working/ in running state 

[root@localhost ~]# podman ps

CONTAINER ID  IMAGE                                COMMAND     CREATED       STATUS       PORTS                                       NAMES

d560d0d2d420  docker.io/osixia/phpldapadmin:0.9.0              10 hours ago  Up 10 hours  0.0.0.0:443->443/tcp                        suspicious_albattani

116f874f9da6  docker.io/osixia/openldap:1.5.0                  10 hours ago  Up 10 hours  0.0.0.0:389->389/tcp, 0.0.0.0:636->636/tcp  my-openldap-container

[root@localhost ~]# 

  open a brower and hit https://192.168.1.17/phpldapadmin

in the username cn=admin,dc=example,dc=org and in password admin  these are the default values for ldap server to login 

you need to create bunch of groups and users in the ldap to keep the things simple i have shared the export here

# LDIF Export for dc=example,dc=org
# Server: 192.168.1.17 (192.168.1.17)
# Search Scope: sub
# Search Filter: (objectClass=*)
# Total Entries: 9
#
# Generated by phpLDAPadmin (http://phpldapadmin.sourceforge.net) on March 7, 2024 11:09 pm
# Version: 1.2.5

version: 1

# Entry 1: dc=example,dc=org
dn: dc=example,dc=org
dc: example
o: Example Inc.
objectclass: top
objectclass: dcObject
objectclass: organization

# Entry 2: ou=groups,dc=example,dc=org
dn: ou=groups,dc=example,dc=org
objectclass: organizationalUnit
objectclass: top
ou: groups

# Entry 3: cn=Admin,ou=groups,dc=example,dc=org
dn: cn=Admin,ou=groups,dc=example,dc=org
cn: Admin
gidnumber: 500
objectclass: posixGroup
objectclass: top

# Entry 4: cn=auditors,ou=groups,dc=example,dc=org
dn: cn=auditors,ou=groups,dc=example,dc=org
cn: auditors
gidnumber: 502
objectclass: posixGroup
objectclass: top

# Entry 5: cn=superusers,ou=groups,dc=example,dc=org
dn: cn=superusers,ou=groups,dc=example,dc=org
cn: superusers
gidnumber: 501
memberuid: nrathi
objectclass: posixGroup
objectclass: top

# Entry 6: ou=users,dc=example,dc=org
dn: ou=users,dc=example,dc=org
objectclass: organizationalUnit
objectclass: top
ou: users

# Entry 7: cn=Kishor jha,ou=users,dc=example,dc=org
dn: cn=Kishor jha,ou=users,dc=example,dc=org
cn: Kishor jha
gidnumber: 500
givenname: Kishor
homedirectory: /home/users/kjha
objectclass: inetOrgPerson
objectclass: posixAccount
objectclass: top
sn: jha
uid: kjha
uidnumber: 1001
userpassword: {MD5}ISMvKXpXpadDiUoOSoAfww==

# Entry 8: cn=Leena Rathi,ou=users,dc=example,dc=org
dn: cn=Leena Rathi,ou=users,dc=example,dc=org
cn: Leena Rathi
gidnumber: 502
givenname: Leena
homedirectory: /home/users/lrathi
loginshell: /bin/bash
objectclass: inetOrgPerson
objectclass: posixAccount
objectclass: top
sn: Rathi
uid: lrathi
uidnumber: 1002
userpassword: {MD5}ISMvKXpXpadDiUoOSoAfww==

# Entry 9: cn=Navneet Rathi,ou=users,dc=example,dc=org
dn: cn=Navneet Rathi,ou=users,dc=example,dc=org
cn: Navneet Rathi
gidnumber: 500
givenname: Navneet
homedirectory: /home/users/nrathi
loginshell: /bin/bash
objectclass: inetOrgPerson
objectclass: posixAccount
objectclass: top
sn: Rathi
uid: nrathi
uidnumber: 1000
userpassword: {MD5}ISMvKXpXpadDiUoOSoAfww==

I have created 5 groups

1. groups
2. users
3. superusers
4. Admin
5.  auditors

I have created 3 users 

1.  Navneet Rathi --> nrathi (system Administrator)  Memberof  superusers group 
2.  Kishor jha --> kjha          (Normal user)                Memberof  Admin group
3. Leena Rathi --> lrathi       (Syste, Auditors)           Memberof auditors group

Once done with this we need to look at the redhat Ansible aap /Ansible tower where we need to configure the RedHat aap /Ansible tower

ref : https://github.com/osixia/docker-openldap?tab=readme-ov-file#defaultstartupyaml

ref : https://github.com/osixia/docker-phpLDAPadmin

ref : https://docs.ansible.com/automation-controller/latest/html/administration/ldap_auth.html

MS Teams and Ansible Integration

 in order to Achive this we need to create a webhook in MS teams the steps to create the webhook can be found at the URL

https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook?tabs=classicteams%2Cdotnet

once done we will get a web hook URL 

---

- name: Sending msg to ms teams

hosts: localhost

gather_facts: no

vars:

# Use your own webhook URL here

ms_webhook_url: "https://m365x221219.webhook.office.com/webhookb2/12345678-40cc-4c20-9228-f94320b65a82@945c199a..."

tasks:

### your logic goes of automation goes here

##

####

- name: Send a notification to Teams Channel

uri:

url: "{{ ms_webhook_url }}"

method: POST

body_format: json

body:

title: "Active Completed in job: {{ job_id }}"

text: "your Activity is completed below are the detsils"

sections:

- facts:

- name: "{{ item1 }}"

value: "{{ value1 }}"

and we can run it in the redhat ansible AAP and if you want to tun it from command line then define some variable like job_id 

# ansible-playbook msteams.yml -e job_id=123

and check you will received the msg in the ms teams 

Sending Html Email using Ansible Mail module

 Hello Guys,

Recently while working on one of the projects, I have to send an Email notification .Sending an Email is an Easy in Ansible one can use mail (community.general.mail) module to do it but what if we have to send an formatted Email or lets say an Html email

    We can chive the same with the help of  jinja template , file lookup plugin and mail module in ansible

the code for which will look like this

create a file called  alert_email.html.j2

[root@aap1 Email]# cat alert_email.html.j2 

<!DOCTYPE html>

<html lang="en">

<head>

    <meta charset="UTF-8">

    <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <title>Failed {{ job_id }}</title>

</head>

<body>

    <p>Dear Team,</p>

    <p>This is an automated alert to inform you about the following issue:</p>

    <ul>

        <li><strong>Failing Job: </strong> {{ job_id }} </li>

        <li><strong>Details: </strong> {{  issue_description }} </li>

    </ul>

    <p>Please take necessary actions to address this issue promptly.</p>

    <p>Best regards,<br/>{{ user_email }}</p>

</body>

</html>

sendmail.yml

---

- name: Send HTML email alert

  hosts: localhost

  vars_files:

    - var1.yml

  vars:

    job_id: 115

    user_email: "nrathi@example.com"

    issue_description: "The server is down and requires immediate attention."

  tasks:

    - name: Include Jinja template for email body

      template:

        src: alert_email.html.j2

        dest: /tmp/alert_email.html

    - name: Send email Alert

      mail:

        host: smtp.gmail.com

        port: 587

        subtype: html

        to:

        - "{{ to }}"

        subject: "Alert: {{ job_id }}"

        subtype: html

        body: "{{ lookup('file', '/tmp/alert_email.html') }}"

        username: "{{ uname }}"

        password: "{{ pass }}"

and in var1.yml file contains your smtp username, password and recipient list 

This is the final outcome

That should do it

Building Dynamic inventory in Ansible

 Hello Guys,

I have recently worked on a project which involved fetching a data from multiple server at the run time and manipulating it and displaying it Grafana dashboard. the most interesting part of it was generating the  inventory at runtime and using it in the same playbook

let us say we have a mysql db with table contain the information of all the servers 

currently showing 3 column which are of our interest  hostname,ip and some var value

building the dynamic inventory the playbook look like below

---

- hosts: localhost

connection: local

tasks:

- set_fact:

db_database: inventorydb

db_host: inventorydbhost

- name: Query Datacenter hosts from DB

mysql_query:

login_host: "{{ db_host }}"

login_db: "{{ db_database }}"

login_user: "{{ db_user }}"

login_password: "{{ db_password }}"

query:

- select distinct(hostname) from inventory ;

single_transaction: yes

register: dc_db_query

- name: Generate hosts file

template:

src: ./templates/hosts.j2

dest: hosts

- meta: refresh_inventory

 The host.j2 file present in the templates look like this

[all]

{% for item in dc_db_query.query_result[0] %}

{{ item.hostname }}

{% endfor %}

This will build the inventory at the run time as well as  the meta flag at the bottom will help us to refresh the inventory to use it on all the host which it will return 

let me know if you want to know on any other topics 

Jenkins backup to Github/Gitlab/Git

 Hello Guys,

Its been a long time after which i am writing a blog on a automation recently i got a requirement about backup the Jenkins into a git and since after a quick search on internet i find it out there is no such plugin available i have to do it the hard way 

Requirement : Backup all Jenkins important config files (xml) in git on daily basis

Now there are two ways to do it 

•  Write a script and schedule it in cron to execute it on midnight 
• create a Jenkins job for this and let the backup script get executed using jenkins

I like the 2nd approach as it will give a transparency I did not need to login to a box just to check if my script is working or not also i can send an email if i wanted to if some thing goes wrong and its easy to keep an eye on if some thing goes wrong

So lets get started

create a jenkins job its a free style jenkins job

in the General config you will find the option

Restrict where this project can be run : in the label expression write master . 

Source code management : None or you can have it if you want to  check out your script from git every time you want to execute the job

Build Trigger : Build Periodically in the schedule write 0 0 * * * while means you need to execute it on daily basis

Build : in the build section select Execute shell 

copy paste the below shell script

#!/bin/bash

# Setup

#

# - Create a new Jenkins Job

# - Mark "None" for Source Control Management

# - Select the "Build Periodically" build trigger

#   - configure to run as frequently as you like

# - Add a new "Execute Shell" build step

#   - Paste the contents of this file as the command

# - Save

#  

# NOTE: before this job will work, you'll need to manually navigate to the $JENKINS_HOME directory 

# and do the initial set up of the git repository.  

# Make sure the appropriate remote is added and the default remote/branch set up.

#  

# Jenkins Configuraitons Directory

cd $JENKINS_HOME

# Add general configurations, job configurations, and user content

git add -- *.xml jobs/*/*.xml userContent/*

# only add user configurations if they exist

if [ -d users ]; then

    user_configs=`ls users/*/config.xml`

    if [ -n "$user_configs" ]; then

        git add $user_configs

    fi

fi

# mark as deleted anything that's been, well, deleted

to_remove=`git status | grep "deleted" | awk '{print $3}'`

if [ -n "$to_remove" ]; then

    git rm --ignore-unmatch $to_remove

fi

git commit -m "Automated Jenkins commit"

git push -q -u origin master

save the job 

Login to jenkins server from shell Now Go to the Jenkins Home directory usually its a /var/lib/jenkins

also make you the public key(/var/lib/jenkins/.ssh/id_rsa.pub) for jenkins user is added to the github so that it will be able to push it

# git init

# git add remote <github repo url where you need to backup your jenkins>

once done we are all set to test it. Let me know how it works for you