If you operate a successful site with thousands of pages, take a look at your server and you will be amazed. The level of crap that your server is constantly bombarded with is simply amazing.
There are some friendly bots like Googlebot, Yahoo Slurb, MSN, Cuil etc and then there are all these spammers who send thousands of requests per day for various reasons.
After few months of being at the mercy of spam bots, we recently decided to take action and start blocking them more actively.
For our Apache + Mongrel set up, we did the following -
- Installed APF firewall to make our system more fool proof. See How to setup APF for step by step guide.
- Installed DoS Deflate to throttle any incoming request. For now, we have taken the extreme step and we are blocking pretty much anyone who is sending beyond a reasonable number of requests.
- Configured Apache to block certain request patterns if they are suspicious.
I hope you and your servers have a nice holiday.
Rajat
Filed under: Technical | Tagged: apache, apf, dos deflate, mongrel, security, server | Leave a Comment »