A couple of weeks ago I saw someone posted an article on on the LinuxServer discord describing how to send geo data statistics from Nginx to InfluxDB. I have seen similar articles in the past and I always wanted to try it out, and because adding new dashboards to Grafana is always fun ๐Ÿ™‚

The article was a couple of years old so he was using the now deprecatedย  maxmind geoip database.ย  But the rest of the Nginx configs looked just fine. As for the sending of the geo data metrics to InfluxDB, he was using a python2 script.

Having recently started using Roxedus’s Fail2Ban docker mod for the linuxserver letsencrypt container, I thought that would be an excellent way of running it.
So I looked at how Roxedus set up his mod and used that as a template for building mine.

Getting the script to work was simple enough, I just needed to adapt it for python3 and change a couple of small things, like adding environment variables for use with docker.

I also tried to get Telegraf to parse the Nginx logs like he did in his guide, but it only parsed the logs once and didnt continue to send new log lines to InfluxDB. So I decided to make python parse it instead.

How it will look when everything is setup

How it will look when everything is setup

Adding the docker mod to the letsencrypt container

The mod is added using docker environment variables.

The first one you need is -e DOCKER_MODS=gilbn/lsio-docker-mods:geoip2-nginx-stats

Here is a link to the repo if you want to have a look: https://github.com/gilbN/lsio-docker-mods/blob/master/letsencrypt/geoip2-nginx-stats

The python script has some default variables added already, so you only need to add the ones that’s different for your system.

The available and default variables are:
-e NGINX_LOG_PATH=/config/log/nginx/access.log
-e INFLUX_HOST=localhost
-e INFLUX_HOST_PORT=8086
-e INFLUX_DATABASE=geoip2influx
-e INFLUX_USER=root
-e INFLUX_PASS=root
-e GEO_MEASUREMENT=geoip2influx
-e LOG_MEASUREMENT=nginx_access_logs
-e SEND_NGINX_LOGS=true
-e GEOIP2INFLUX_LOG_LEVEL=INFO

Most of them should be self explanatory, so I’ll just add some comments on a couple of them.

-e INFLUX_HOST=localhost
Since you will be setting this up inside the letsencrypt container, the default value will not work unless you are running the container with host networking and InfluxDB is installed on the same host. So change this to your host IP/docker dns name.

GEO_MEASUREMENT=geoip2influx and -e LOG_MEASUREMENT=nginx_access_logs

These are just the measurement namesย  that will be sent to InfluxDB.

-e INFLUX_DATABASE=geoip2influx

The database will be created automatically by the python script, so no need to create one beforehand.

As of 15.05.20 the letsencrypt container natively supports downloading the geoip2 database with the use of a variable.

-e MAXMINDDB_LICENSE_KEY=<license-key>

Add your MaxMind Geoip2 license key and it will automatically download the latest database and update it weekly . The default download location is /config/geoip2db/GeoLite2-City.mmdb.

If you’re unsure how to obtain a licensekey, check out the first part of my geolite2 guide.

-e GEOIP2INFLUX_LOG_LEVEL=INFO

The script will automatically create a log file in the same directory it exists. Set this to DEBUG for troubleshooting.

Nginx log metrics

For python to be able to parse the nginx logs correctly you need to update your nginx configs with a couple of lines.

  • Add the following in the http block in your nginx.conf file:
geoip2 /config/geoip2db/GeoLite2-City.mmdb {
auto_reload 5m;
$geoip2_data_country_code country iso_code;
$geoip2_data_city_name city names en;
}
log_format custom '$remote_addr - $remote_user [$time_local]'
           '"$request" $status $body_bytes_sent'
           '"$http_referer" "$http_user_agent"'
           '"$request_time" "$upstream_connect_time"'
           '"$geoip2_data_city_name" "$geoip2_data_country_code"';
  • Set the access log use the custom log format.
access_log /config/log/nginx/access.log custom;

The first time the scripts starts it will try and parse the log for 1 minute and if the regex doesn’t match it will disable the log metrics. So be sure to refresh your site a couple of times after adding the mod.

Next add a new datasource to Grafana with the name of the database you chose and import the dashboard and you should start seeing the map getting populated!

https://grafana.com/grafana/dashboards/12268

Other statistics

You can also add a Telegraf plugin for Nginx that gets some extra stats from the webserver.

https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nginx

Location Block:

location /nginx_status {
    stub_status on;
        access_log   off;
    allow 192.168.1.0/24; # Add the IP ranges you want to access. 
        deny all;
}

Sources:
https://medium.com/faun/total-nginx-monitoring-with-application-performance-and-a-bit-more-using-8fc6d731051b
https://geoip2.readthedocs.io/en/latest/
https://www.influxdata.com/blog/getting-started-python-influxdb/
https://grafana.com/grafana/dashboards/8522
https://www.geeksforgeeks.org/python-reading-last-n-lines-of-a-file/