TRANSPARENT DYNAMIC REVERSE PROXY WITH NGINX

Hello Guys welcome back again on my blog again.
Recently I have change my company and in the new office i got this new unique task of creating a new dynamic transparent dynamic proxy the group of developers working with me.
      so here what I have done to full fill the requirement with nginx.

Here is the sit­u­a­tion. You have a sin­gle pin­hole into your pri­vate net­work. You have a sin­gle ip at your gate­way. You want to serve mul­ti­ple web­sites on your lan that may be run­ning on mul­ti­ple phys­i­cal servers. Rather than open­ing up mul­ti­ple ports and pin­holling to all the dif­fer­ent spots you want to serve, or get­ting more exter­nal ips and doing 1to1 NAT you can use a reverse proxy to be your sin­gle entrance point. The reverse proxy will fetch the con­tent from the back­end server and serve it up.

nginx is a HTTP server and mail proxy server. One of its fea­tures basic HTTPfea­tures is accel­er­ated reverse proxying.

nginx should be avail­able through your pack­age man­ager so just apti­tude (or what­ever your pack­age man­ager is yum, emerge, pac­man) install it.

The con­fig file paths shown are Debian spe­cific but the con­fig itself should work on any distro.

Edit /etc/nginx/sites-available/default and make it look like this

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

server {      listen  :80;      server_name  _;      access_log  /var/log/nginx/proxy.access.log;      location / {      resolver        127.0.0.1;      proxy_pass      http://$host$uri;      proxy_redirect off;      proxy_set_header        Host    $host;      proxy_set_header        X-Real-IP $remote_addr;      proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;      }      error_page   500 502 503 504  /50x.html;      location = /50x.html {           root   /var/www/nginx-default;      } }

So this con­fig causes nginx to lis­ten on all interfaces/ips. server_name _; matches on any­thing so essen­tially this is a catchall now. You can tail proxy.access.log in order to see the requests are they come in and are served.

The loca­tion sec­tion is where the actual prox­y­ing hap­pens. Since this is a dynamic con­fig­u­ra­tion you need to set a resolver where the requested names can be looked up (and over­rid­den for the local lan address). dns­masq reads is dns con­fig­u­ra­tion right out of /etc/hosts. It’s easy to install and con­fig­ure so I rec­comend using it. We will install and con­fig­ure it shortly but for now just leave resolver as 127.0.0.1. proxy_pass does the request­ing of the page we are prox­y­ing. Since this is a trans­par­ent dynamic proxy we just have it request the same thing that was requested of the proxy. proxy_redirect should be set to off since we are just pass­ing on the same request. We need to set a few head­ers for log­files on the back­end servers as well as mak­ing sure that Host is set to the request­ing host in case your using name based vir­tual hosts on your back­end servers. I have left the error page in the default con­fig (at least on debian its default). This pro­vides a nice error mes­sage in case your proxy is work­ing but one of the back­end servers is not. It just serves the index.html that is located in /var/www/nginx-default. Feel free to change that path to some­thing else, mod­ify the index.html or omit the error_page and error page loca­tion sec­tion all together as they aren’t needed for this to work.

Now we need to get that local resolver (dns­masq) installed so we can take our reverse proxy for a spin. Go ahead and apti­tude (or what­ever) install dnsmasq.

At least on debian dns­masq comes out want­ing to serve dhcp. You prob­a­bly do not want this behav­ior. There is also the ques­tion of need­ing access to these same ser­vices by the same name on your LAN. If you need this you might need to do some slight adjust­ing of your dns. I might rec­comend point­ing your main dns to this dns­mask proxy or point­ing all of your clients at this dns­masq install since it will look up other requested names other than those in /etc/hosts. For this exam­ple I will assume you will be want­ing to access these same web ser­vices inter­nally with the same names and bypass the proxy. So I will assume you have either changed your pri­mary dns cacher/resolver (think soho router or what­not) to the address of the proxy server (since its run­ning dns­masq as well), or set all of your clients to point directly at the proxy server for dns. We need to edit the dns­masq con­fig to dis­able dhcp.

Edit /etc/dnsmasq.conf and add no-dhcp-interface=ethx. Do that for every inter­face on your sys­tem so that your not acci­den­tally serv­ing out dhcp to any­one. If somone has a more generic way to dis­able dhcp in dns­masq with­out spec­i­fy­ing each inter­face I would love to know but from read­ing the man this was the only way I could find. So you may have some­thing like the fol­low­ing in you /etc/dnsmasq.conf.

?

1 2	no-dhcp-interface=eth0 no-dhcp-interface=eth1

After mak­ing the change you should be ready to add entries to the proxy servers /etc/hosts for dns­masq to use and then test your reverse proxy.

Lets say you have www.test.com served off of a machine with the ip 192.168.1.2 and you have tickets.office.test.com served off of 192.168.1.3. Lets also assume that your world route­able ip is 123.123.123.123. You will need to make sure that your author­i­ta­tive dns (the real one that servs for test.com has A records for both www.test.com and tickets.office.test.com point­ing to 123.123.123.123. Now on the machine run­ning dns­masq (in this exam­ple also your proxy server) add the fol­low­ing entries to /etc/hosts.

?

1 2	192.168.1.2 www.test.com 192.168.1.3 tickets.office.test.com

Go ahead and restart dns­masq (from mak­ing changes to the con­fig, sub­se­quent changes to /etc/hosts should not require dns­masq restart to pick up changes) and nginx.

Now tail your proxy.access.log file and start mak­ing requests to www.test.com and tickets.office.test.com from both the inside of your lan as well as out­side against your world ip. It should all mag­i­cally serve up the same content.

This type of con­fig can be use­ful in many sit­u­a­tions. You have a small office and bud­get that reflects that not being able to afford mul­ti­ple ips but need­ing to pro­vide web ser­vices behind the fire­wall. You work in a large cor­po­ra­tion where some­one else man­ages the fire­wall and you would like to bring up more web ser­vices with­out wait­ing for the other per­son to make the nec­es­sary changes to the firewall.

One of the other ben­e­fits this pro­vides is being rel­a­tively self doc­u­ment­ing  with regard to what web ser­vices you host behind the fire­wall. (you should be able to see all of them in /etc/hosts since you have to over­ride the dns)

and in the next blog i will tell you how you can achive the same for https i mean dynamic proxy with ssl  ....

Setup your MAIL Exchange server with zarafa on CentOS 6 Part 3

now the actual setup of zarafa server after all the work we have done in part1 and part2.

Start the mysql server using command

# service mysqld start

Let us create a database called “zarafadb” and database user “zarafauser” with password “centos”. Change these values with your own values.

Log in to mysql server using command:

# mysql -u root -p

Create database “zarafadb” and assign the full permission to the user “zarafauser” over zarafadb.

mysql> create database zarafadb;

mysql> GRANT ALL ON zarafadb.* TO zarafauser@localhost IDENTIFIED BY 'somepass';

mysql> flush privileges;

mysql> exit

Add the database details to the zarafa server configuration file.

Edit file /etc/zarafa/server.cfg,

# vi /etc/zarafa/server.cfg

Find the following lines and Change the zarafa database values.

[...]

# The user under which we connect with MySQL
mysql_user              = zarafauser

# The password for the user (leave empty for no password)
mysql_password          = somepass

# Override the default MySQL socket to access mysql locally
# Works only if the mysql_host value is empty or 'localhost'
mysql_socket            =

# Database to connect to
mysql_database          = zarafadb

[...]

Now start all zarafa services.

# service zarafa-server start
# service zarafa-dagent start
# service zarafa-gateway start
# service zarafa-spooler start
# chkconfig zarafa-server on
# chkconfig zarafa-dagent on
# chkconfig zarafa-gateway on
# chkconfig zarafa-spooler on

Wait, We didn’t finish yet, we have to create public store where all emails stored and mail users.

Create Public store and users

Create public store using command:

# zarafa-admin -s

Then create users. For example, here i am going to create two users called “navneet” and “mohit”.

# useradd navneet
# useradd mohit
# passwd priyanka
# passwd arun

Now let us assign mail id’s to them as shown below.

# zarafa-admin -c navneet-p centos -e navneet@rathi.com -f "navneet" 
# zarafa-admin -c mohit-p centos -e mohit@rathi.com-f "mohit"

Where,

-c – Create user

-p – password

-e – email

-f – full name

To create administrative user, you should use -a parameter with value “1”.

# zarafa-admin -c nrathi-p sompass-e nrathi@rathi.com -f "nrathi" -a 1

Where,

-a – administrative user

1 – describes administrative user, You can use 0(zero) for non-administrative users.

To delete users, use -d parameter.

Ex.

# zarafa-admin -d nrathi

Access Zarafa webmail

We have done with configuration, Let us log in to Zarafa webmail. Navigate to http://ip-address/webaccess orhttp://domainname/webaccess.

Enter the username and password to log in. and you are done

Setup your MAIL Exchange server with zarafa on CentOS 6 Part 2

Before starting with this post you need to follow the steps which we have followed in the part1

SO Guys lets start with the phase 3 

now starting with the phase 3 lets start with the configurations of postfix. 

Add hostname entries in /etc/hosts file as shown below:

# vi /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.101   mail.rathi.com      mail

 I disabled SELinux to reduce complexity in postfix configuration.

If you want to keep SELinux on, enter the following command in Terminal:

# togglesebool httpd_can_network_connect

Allow the Apache default port 80 and port 443 if you are using ssl and 3306 if your mysql is on another server  through your firewall/router:

# vi /etc/sysconfig/iptables

[...]
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
[...]

fowling are optional as they will come in picture if you are using production or more complex setup of more than one server

[...]
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443-j ACCEPT
[...]

and same true for 3306 as well.

Configuring Postfix

Edit /etc/postfix/main.cf,

# vi /etc/postfix/main.cf

find and edit the following lines:

## Line no 75 - Uncomment and set your mail server FQDN ##
myhostname = mail.rathi.com

## Line 83 - Uncomment and Set domain name ##
mydomain = rathi.com

## Line 99 - Uncomment ##
myorigin = $mydomain

## Line 116 - Set ipv4 ##
inet_interfaces = all

## Line 119 - Change to all ##
inet_protocols = all

## Line 164 - Comment ##

#mydestination = $myhostname, localhost.$mydomain, localhost,

## Line 165 - Uncomment ##\
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain

## Line 419 - Uncomment ##
home_mailbox = Maildir/

Save and exit the file. Start/restart Postfix service now:

# service postfix restart
# chkconfig postfix on

Testing Postfix mail server

First, create a test user called “nrathi“.

# useradd nrahi
# passwd nrathi

Access the server via Telnet and enter the commands manually shown in red colored text.

# telnet localhost smtp

Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 mail.rathi.com ESMTP Postfix
ehlo localhost     ## type this command ##
250-mail.rathi.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
mail from:<nrathi>     ## Type this - mail sender address##
250 2.1.0 Ok
rcpt to:<nrathi>     ## Type this - mail receiver address ##
250 2.1.5 Ok
data     ## Type this to input email message ##
354 End data with <CR><LF>.<CR><LF>
welcome to rathi.com mail server     ## Enter the boddy of the email ##.
     ## type dot (.) to complete message ##
250 2.0.0 Ok: queued as B822221522
quit     ## type this to quit from mail ##
221 2.0.0 Bye
Connection closed by foreign host.

Now navigate to the user “nrathi“ mail directory and check for the new mail:

ls /home/nrathi/Maildir/new/

Sample output:

1390215275.Vfd00Ie04f8M357080.mail.rathi.com

A new mail is received to the user “nrathi“. To read the mail, enter the following command:

# cat /home/nrathi/Maildir/new/1390215275.Vfd00Ie04f8M357080.mail.rathi.com

Sample output:

Return-Path: <nrathi@rathi.com>
X-Original-To: nrathi
Delivered-To: nrathi@rathi.coml
Received: from localhost (localhost [IPv6:::1])
    by mail.rathi.com (Postfix) with ESMTP id B822221522
    for <nrathi>; Mon, 20 Jan 2015 16:23:54 +0530 (IST)
Message-Id: <20140120105404.B822221522@mail.rathi.com>
Date: Mon, 20 Jan 2015 16:23:54 +0530 (IST)
From: nrathi@rathi.com
To: undisclosed-recipients:;

welcome to rathi.com mail server

Add the following line at the end.

mailbox_command = /usr/bin/zarafa-dagent “$USER”

Save and close the file. Restart postfix service to take effect the saved changes.

# service postfix restart

The remaining things i will cover in the next part phase 4

Setup your MAIL Exchange server with zarafa on CentOS 6 Part 1

Hello Guys,

Welcome to the blog again I work with different issues so i come across the zarafa its an alternative to Microsoft exchange server and the best thing about it. that's free open source software.and as you know am big fan of open source software.

The Zarafa groupware provides email storage on the server side and offers its own Ajax-based mail client called WebAccess and a HTML5 based, WebApp.
Zarafa is designed to integrate with Microsoft Office Outlook and is intended as an alternative to the Microsoft Exchange Server. Connectivity with Microsoft Outlook is provided via a proprietary client-side plugin. The WebAccess and WebApp have the same “look-and-feel” as the Outlook desktop application. People used to working with Outlook should be able to use the WebAccess/WebApp without any problems.

Now, let talk about how to install and configure Zarafa Email Server on Centos 6.Install the Centos 6 minimal,we all know how we can do it.if you don't know no worry's.

just download the centos from the below URL and install it while installing it make sure you provide the correct domain name instate of localhost.localdomain and the again all options as per your requirement. like dist options and etc. etc. before stating it its for study perpose only not for production use as for production you need to have more restriction on mail server and need to configure more appropriately.you can say it that its for the proof of concept that yes this can be configured and this can be done.

so first thing first. 
    
lets download centos 6 minimal edition if you have already server setup the its ok.

Phase 1
download the ISO form the URL.

Then install the cent os on the virtual or on physical server. the first thing after that is login on server with the credentials which you have provided during the installation. after that start your network interfaces with commands

# ifup ethX 
where X is your interface nunber form 0 to anything 

after that update your centos for latest new softwares and  security patches and new stable version of kernel available at that time.
and we are done with phase1 

Phase 2
Download the MySQL,Apache,php,
By default centos 6  dont support mysl 5.6. and to reduce your admin jobs of maintenance.the idea is to install the mysl using yum from the mysql repo.
 so download the RPM of mysql repo and install in one line as

# rpm -ivh http://repo.mysql.com/mysql-community-release-el6-5.noarch.rpm

This command will downloas and install the mysql 5.6 repo in your cent os.

Then install Installation of EPEL-- Extra Packages for enterprise Linux.

# rpm -ivh  http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

after this

# yum update  -y && yum remove sendmail -y &&yum install mysql mysql-server php httpd zarafa*  postfix -y

After using this command we have done our most of the work of installation and of downloading.
This  is the end of phase 2 as just the configuration part is left behind

In the part 2 we will look the configuration of posfix and zarafa exchange server

Redis Master slave Replication

              

            Hello and welcome.from the past few days my Brother  given me a book and i start reading and started implementing the same on my local environment.The book was about the Radis and how Radis makes a difference, whats are its pros and cons. etc etc and how we can start using it.

 well then lets start with the Topic of interest installing a redis is not a much deal we all know Redhat/Centos and Debians like Ubuntu both supports redis so we can install then using the default installer like 
 
STEP 1 Installation 
 
In Redhat / Centos we need to add EPEL and Remi and then simply update the yum data and then "yum install redis"
 after that 

#service redis start
chkconfig --level 35 redis on
service redis start
 
In Ubuntu just fire "apt-get install redis-server"
  
Note: I am using Centos 6.6 

STEP 2 SETTING UP REPLICATION

Assume we have two IP Address 
192.168.1.101 ----> Masters_ip
192.168.1.102 ----> Slave_ip

Setup on Master

Open the terminal as a root or use sudo -i to gain the access of root

Note:Location of these conf file depends distribution and on type of installation from source or from standard package used 

#vim /etc/redis.conf

By default the redis bind to loopback ip-address we need to change it make it more appropriate.
so change 

bind 127.0.0.1 

to

bind 192.168.1.100

For security prospective need to add security as its start listning on network now. 
so change like from this
#requirepass foobared 

to  

requirepass myn@m31sN@vn33tR@t1v3 #some thing like this more    
                                  #complex 

and then save the file and restart the redis server with command

# service redis restart

On Slave Machine 

open the redis conf file using vim /etc/redis.conf

change 

bind 127.0.0.1 

to

bind 192.168.1.101

also set a password by default on new version of redis the redis in read only mode and it will be on network now so change the authentication setting for it provide it with some complex password 
so change the 


#requirepass foobared 

to  

requirepass myn@m31sN@vn33tR@t1v3 #some thing like this more    
                                  #complex 

Setting up Replication Actually

The below steps will outline setting up replication from the Slave. Replication only needs to be defined on Slave systems, the Master server does not require any special configuration.
Specifying the Master

In the slave configuration file specify the master server to replicate from. Redis has the ability to replicate from a slave, to set this up you would simply specify the first slaves details in place of the master.
  
change the below like from

# slaveof <masterip> <masterport>

to

slaveof 192.168.1.101 6379 # or some thing like that as per                                 #your need 

If you set a password earlier for the master server you will need to specify that password via the masterauth setting.

change

# masterauth <master-password>

to

masterauth  myn@m31sN@vn33tR@t1v3

Start the Service 

Step 3 Testing of your Replication.

To test the replication we will first login to the master server and set a key-value.
Set the key-value on the Redis Master

To login to the Redis server we will use the redis-cli client. The -h flag specifies the host to connect to, by default the redis-cli will connect to the localhost IP.

#redis-cli -h 192.168.1.100


Once logged in use the AUTH command with the password specified earlier.
 
#redis 192.168.1.100:6379> AUTH <password>
 OK

After authenticating we can now set a key-value pair using the SET command or thing you like you can put it in.I have taken key value pair to test it.

 #redis 192.168.1.100:6379> SET replicated:Navneet Rathi
 OK 

 Get the key-value on the Redis Slave

Now that data has been saved to the master we will login to the Redis Slave server using the same redis-cli command and use the GET command to retrieve the replicated data.

# redis-cli -h 192.168.1.101
redis 192.168.1.101:6379> AUTH <password>
 OK

Once logged in use the GET command to retrieve the value from the replicated:test key

 redis 192.168.1.101:6379> GET replicated:Navneet
 "Rathi"


and we are done.
We can also set the moniting for it we will check that out in nex blog till then enjoy... 


 

Tips for Monitoring Linux server Best and Simples Tips

Well welcome back guys to my blog on Tips for Linux server monitoring the activities on Linux server.

The following tips will allow you to monitor the activity on Linux server.Its start with the few assumptions like  

1.  you are using CentOS server 
2. You are also having minimal installation of Centos  

I am assuming minimal install as i can provide you the details
like installation of mail agent etc .

so lets start with it.

First of all you need to have internet connectivity on server if so then by default postfix is installed on you server so you can go for it else if you don't have internet connectivity and you have mail server configured on some other server This can be the code when you are running payment-gateway on server and PCI slandered won't allow you to so so then 

Install mailx as a client to send a mail so that you can use a command mail to send a mail

 #yum install mailx 

Install ssmtp on your Centos server and open the /etc/ssmtp/ssmtp.conf
  
# vim /etc/ssmtp/ssmtp.conf file

the added the following line to the file make sure you change the email address and password as per your mail server configuration. 

for simplicity and to cover more point i am taking gmail as my mailing server not the default postfix.
 

AuthUser=vijay9867206455@gmail.com
AuthPass=xxxxxxxx
FromLineOverride=YES
mailhub=smtp.gmail.com:587
UseSTARTTLS=YES
TLS_CA_File=/etc/pki/tls/certs/ca-bundle.crt

I have added the last line as while using the gmail you will get a typical error that can not connect to smtp.gmail.com  and to avoid the error I have added the below line for the TLS certificate.

then close the file and execute the following commands 

 # service sendmail stop && service postfix stop

 # chkconfig sendmail off

 # mv /usr/sbin/sendmail /usr/sbin/sendmail.old 

 # ln -s /usr/sbin/ssmtp /usr/sbin/sendmail

so good so then strat the postfix service and we will try to send a sample mail by using following command.

   #echo "This is a test" | mail -s "Test" vijay9867206455@gmail.com

and we will receive a mail for it.

Then open /etc/aliases

vim /etc/aliases

and find the like look like this

# Person who should get root's mail
#root: marc

and change this to look like this

# Person who should get root's mail
root: vijay99867206455@gmail.com
  
as written in file dont forget to fire newaliases command

#newaliases
 
 the simply login as user other than root and try to use sudo with wrong password and you will see you start getting alart for then too for any security breach or any unwanted activity you start getting alert.Its a very small tip but will help you alot for handing large servers for more convinces if you server's ip is 192.168.1.100 then change the host name to 192-168-1-10 or some thing like this so you will get the more clue if you are handling large number of servers

Thanks guys letme know if you have any issue you can mail me on my mail id