Recently some crawlers have been very aggressive and caused a lot of traffic to one of my servers. Luckily nginx provides a rate limit out of the box to control for example API querries. I have used to to limit traffic to about 10 requests per second and it has already shown very possitive results.
The main documentation was very good and can be found here: https://www.nginx.com/blog/rate-limiting-nginx/
Similar to the post I have added the following into my “http” configuration at /etc/nginx/nginx.conf
limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s;
Further I had to add it to the location /api directive and to the PHP part (location ~ \.php$ {)
limit_req zone=mylimit;
After reloading nginx using “nginx -s reload” you can check if the limits are triggering using “grep ‘limiting requests’ /var/log/nginx/error.log”
At this point it might be worth mentioning that you also want to include the following into your “http” directive in /etc/nginx/nginx.conf to add some security to your nginx:
add_header X-XSS-Protection “1; mode=block”;
add_header Strict-Transport-Security ‘max-age=31536000; includeSubDomains; preload’;
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header Content-Security-Policy “default-src ‘self’;”;
add_header X-Permitted-Cross-Domain-Policies master-only;
fastcgi_hide_header X-Powered-By;
server_tokens off;