When you run a webserver behind a reverse proxy or HTTP accelerator such as Varnish, the webserver access logs will display the IP of the proxy (generally 127.0.0.1) instead of the end user’s IP.
This is a problem when you have a software like
awstats or similar log file analysis program, because you lose one of the most important information: “Who is the requestor of a page ?”, also having all the access coming from the same IP (127.0.0.1) you lose information such as “what’s the browsing pattern of visitors ?” “Is someone trying to do something Nasty ?”
In this small how-to I’ll show how to put this information back on your Nginx log files in 2 different ways.
1) Make nginx aware that you are behind a proxy
Nginx is smart enough to have a dedicated module for this work: Real Ip
This module allows to change the client’s IP address to value from request header (e. g. X-Real-IP or X-Forwarded-For), it is useful if nginx works behind some proxy of L7 load balancer, and the request comes from a local IP, but proxy add request header with client’s IP.
The configuration is really simple, you just have to add these 2 lines in your
/etc/nginx/nginx.conf file (section: http) or directly on your Virtual host file (section : server)
set_real_ip_from 127.0.0.1; #Put the Ip of your varnish/proxy here real_ip_header X-Forwarded-For; #Put the Header that your varnish/proxy set
and restart nginx.
You have also to modify your vcl file (usually
/etc/varnish/default.vcl), in the
vcl_recv part add this rule:
set req.http.X-Forwarded-For = client.ip;
And restart Varnish, this will set the header X-Forwarded-For correctly.
This module is usually not enabled by default, you can enable it rebuilding nginx with the configure option:
If you use Nginx from a binary package verify the description of the package, or simply run from the teminal
nginx -V that will give a verbose output like this one:
nginx -V nginx version: nginx/1.2.6 TLS SNI support enabled configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-pcre-jit --with-file-aio --with-http_gzip_static_module --with-http_ssl_module --with-ipv6 --without-http_browser_module --without-http_geo_module --without-http_limit_req_module --without-http_limit_zone_module --without-http_memcached_module --without-http_referer_module --without-http_scgi_module --without-http_split_clients_module --with-http_stub_status_module --without-http_ssi_module --without-http_userid_module --without-http_uwsgi_module --add-module=/usr/src/nginx/source/nginx-1.2.6/debian/modules/nginx-echo
in this example the module is NOT built in the Nginx webserver, so this solution would not work, let’s move to solution 2:
2) Change the format of your Nginx log files
This solution uses the header X-Forwarded-For too, so you have to set it on
varnish as done in the former solution to set it. The idea behind this solution is that Nginx has all the information about the remote IP, just in a different header, so it’s just a matter of making nginx use that variable in its access logs instead of the default variable defining the referring IP.
Edit your nginx.conf file and in the http section add this line:
log_format varnish_log '$http_x_forwarded_for - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent "$http_referer" ' '"$http_user_agent"' ;
You also have to change the access_log directive of you virtual host, to use the varnish_log format:
access_log /var/log/nginx/access.log varnish_log;
Note: log_format controls how nginx logs the requests, in this example we have added a new log format named “varnish_log” where the first field is the header $http_x_forwarded_for and not the standard $remote_addr, in this way you’ll correctly logs the remote IP of your visitors.
if you have many virtual hosts, you have to change for everyone of them the access_log directive to use the new log_format.