NGINX 4xx Error IP Ban Script

This script monitors NGINX access logs for 4xx errors and automatically bans IP addresses that trigger more than 3 consecutive 4xx responses. Banned IPs are added to an nftables set, which drops packets from those IPs for a period of 24 hours. This is particularly useful for mitigating automated bots or brute force attempts that result in repeated client-side errors.

This was particularly built as a quick solution for a single server instance without an LB or WAF in front of it that was subject to a consistent bot 'attack' and I didn't want to write a custom nginx module or have to compile nginx with lua for a custom lua script. This seemed to be the most straight forward approach.

Prerequisites

Before using this script, ensure the following requirements are met:

Software Requirements

1. NGINX:

   • The script monitors access logs, which are generated by NGINX. Ensure NGINX is installed and running.
   • The NGINX access log format should be set to the default or similar (combined log format).

   Example default log format:

   log_format combined '$remote_addr - $remote_user [$time_local] "$request" '
                       '$status $body_bytes_sent "$http_referer" "$http_user_agent"';
   

   The script extracts key information from these logs, such as IP address and status codes.

2. nftables:

   • Ensure nftables is installed to handle IP banning. If you don’t have it installed, you can set it up as follows:

     sudo apt-get install nftables

   • This script creates and manages an nftables set for banned IP addresses.

3. sudo permissions for nftables:

   • Ensure the user running the script (e.g., www-data) has passwordless sudo permissions to execute nftables commands.
   • Add the following line to your /etc/sudoers file (via visudo):

     www-data ALL=(ALL) NOPASSWD: /usr/sbin/nft
     

4. Bash:

   • The script is written in bash, and assumes /bin/bash is available.

How It Works

1. Log Monitoring:

   • The script tails the NGINX access logs and uses a regular expression with awk to match and extract the IP address and the HTTP status code for each log line.

2. 4xx Error Detection:

   • If an IP address generates a 4xx error (e.g., 404, 403), the script tracks the number of occurrences in a temporary file (/tmp/nginx_4xx_ips.txt).
   • When an IP reaches 3 or more 4xx errors, it is added to an nftables set to be banned.

3. nftables Ban:

   • The banned IP is added to an nftables set (addr-set-nginx-ban), which drops packets from the IP for 24 hours.
   • The nftables set and chain are created automatically if they don’t exist.

Setup and Usage

1. Clone the Repository

Clone this repository to your server where NGINX and nftables are running:

git clone https://github.com/coredevel/monitor-nginx-4xx
cd nginx-4xx-ip-ban

2. Set up Permissions

Ensure the script can manage nftables by adding the necessary sudo permissions:

sudo visudo
# Add the following line:
www-data ALL=(ALL) NOPASSWD: /usr/sbin/nft

3. Configure Systemd for Automatic Execution

Create a systemd service to ensure the script runs automatically and survives reboots.

1. Create a new systemd service file:

sudo nano /etc/systemd/system/nginx-ban.service

2. Add the following content to the file:

[Unit]
Description=NGINX 4xx IP Ban Service
After=network.target

[Service]
ExecStart=/path/to/monitor_nginx_4xx.sh
Restart=always
User=www-data

[Install]
WantedBy=multi-user.target

3. Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable nginx-ban.service
sudo systemctl start nginx-ban.service

4. Monitoring the Service

You can monitor the script and check the logs by using:

sudo journalctl -u nginx-ban.service -f

This will show the latest logs from the service.

5. Customization

You can modify the script to:

• Change the number of 4xx errors before banning an IP.
• Adjust the nftables set timeout for how long IPs should remain banned (default is 24 hours).

6. Temporary File

The script keeps track of the number of 4xx errors for each IP in a temporary file located at:

/tmp/nginx_4xx_ips.txt

This file is updated as the script runs and is used to store IP addresses and their current error counts.

Manual Testing

If you want to test the script manually before setting it up as a service:

/path/to/monitor_nginx_4xx.sh

Ensure that NGINX is actively logging and that the script is able to process access logs and ban IPs when they trigger 4xx errors.