PacketFence - BTS - PacketFence
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0000879PacketFenceperformancepublic2009-12-14 12:432011-01-26 15:38
Reporterobilodeau 
Assigned Toobilodeau 
PriorityhighSeveritymajorReproducibilityN/A
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version1.9.1 
fixed in git revision
fixed in mtn revision2e045af575176e6fbb263037a3dea64bd8cabbb9
Summary0000879: captive portal "denial of service"
DescriptionRight now, we sometimes experience non-maliciously intended denial of service on our captive portal.

This mostly happens because a client connected to an open wifi and (upon successful connection) all of his apps started to reach for various web services and such. iPod touch are especially good at that.

We need to be able to expect ~ 300 hits for a 2 minute period. Let's try to resist to that.

Several things were suggested:
- optimize the path of redir.cgi removing all unneeded stuff to get there (single captive portal hit takes a few seconds, probably suboptimal)
- apache rate limiting based on IP address: http://dominia.org/djao/limitipconn2.html [^]
- apache whitelist / blacklist of useragents (ensure that only human initiated connections reach the portal)
- iptables tcp rate limiting

There might be other alternatives also. A post to the -users mailing list would be interesting.
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0001072closed rbalzard performance optimization regarding user-agents 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2009-12-14 12:43obilodeauNew Issue
2009-12-14 12:43obilodeauStatusnew => assigned
2009-12-14 12:43obilodeauAssigned To => obilodeau
2009-12-14 12:44obilodeauDescription Updated
2009-12-17 14:21obilodeauNote Added: 0001449
2010-02-01 13:14obilodeauNote Added: 0001460
2010-02-01 15:16obilodeauNote Edited: 0001460
2010-05-05 09:35obilodeauNote Added: 0001544
2010-05-05 09:36obilodeauCategoryfuture => 1.9.x
2010-09-21 14:14obilodeauCategory1.9.x => performance
2010-09-21 14:43obilodeauNote Added: 0001681
2010-09-21 14:43obilodeauRelationship addedrelated to 0001072
2010-09-21 15:02obilodeaumtn revision => 2e045af575176e6fbb263037a3dea64bd8cabbb9
2010-09-21 15:02obilodeauNote Added: 0001682
2010-09-21 15:02obilodeauStatusassigned => resolved
2010-09-21 15:02obilodeauFixed in Version => 1.9.1
2010-09-21 15:02obilodeauResolutionopen => fixed
2011-01-26 15:38obilodeauStatusresolved => closed

Notes
(0001449)
obilodeau   
2009-12-17 14:21   
another option: use mod_perl instead of cgi handler
(0001460)
obilodeau   
2010-02-01 13:14   
(edited on: 2010-02-01 15:16)
Today I looked at:

- mod_evasive: http://www.zdziarski.com/projects/mod_evasive/ [^] (found by Ludo)
- mod_security only: http://blog.cherouvim.com/simple-dos-protection-with-mod_security/ [^]
- mod_security + httpd-guardian + blacklist: http://apache-tools.cvs.sourceforge.net/viewvc/apache-tools/apache-tools/ [^]

mod_evasive seems really simple to install but there is a limitation that it's only hosting state within a single child so it might not do the job correctly.
See http://permalink.gmane.org/gmane.comp.apache.mod-security.user/5335 [^]
(0001544)
obilodeau   
2010-05-05 09:35   
Another option is to take the opposite approach at User-Agent blocking: blacklist all agents except the known big desktop browsers. Chances are that a given user will carry at least one of firefox, chrome, opera, safari or IE. Yes, maybe they use Flock or Links but when they have no network access, they'll try the other ones.
(0001681)
obilodeau   
2010-09-21 14:43   
For the purpose of this ticket, the 'denial of service' aspect has been considered a performance problem. The current fix doesn't consider malicious DoS but more misconfigured clients, softwares who we putting a lot of pressure on the portal.

The 'fix' can be divided in two:

- use of mod_perl to get more performance out of the current cgi scripts
Our current portal scripts were used as CGI which means that apache started a perl interpreter, compiled the code and handed over the execution to a new process for each request. Moving under mod_perl where the perl interpreter is part of the apache thread is a huge gain right there. This is phase 1 of several phases where we integrate better with mod_perl's philosophy.

- some rewrite rules to return HTML Forbidden (status 403) to non-browser clients (disabled by default)
A lot of the requests are done by misbehaving clients (ex: iPhone Apps trying very hard to fetch stuff from the web spawning thousands of web requests over a few hours). Since we redirect 'every' request to the captive portal these clients hurt.

By having easy to implement rules that cut all that crap from reaching our captive portal, we can get improved performance.

See 0001072 for more info on this.

The work has been done in a feature branch: org.packetfence.feature.captive-portal-performance
(0001682)
obilodeau   
2010-09-21 15:02   
fix is now in branch 1.9 (feature branch merged)