PacketFence - BTS - PacketFence
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0001423PacketFenceinlinepublic2012-04-16 11:242012-10-24 09:58
Reporterobilodeau 
Assigned Tofgaudreault 
PrioritylowSeverityminorReproducibilityrandom
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Version3.6.0Fixed in Versiondevel 
fixed in git revision
fixed in mtn revision
Summary0001423: Weird behavior with DNS and connection tracking in inline enforcement
DescriptionSometimes DNS is still blackholed even if the mark on the inline node was removed. Looking at the firewall you see no reason why the DNS would be still handled that way but a pcap reveals that it actually still does. This happens for a couple of minutes and then DNS is NATed to the right destination properly again.

We think it's an UDP connection tracking issue and that clients that don't change their source port for every DNS query are more affected than others as they are in the conntrack table. An old CentOS 5.2 client exhibited that behavior while most other OS tested properly changed its source port (OSX, Windows, Fedora, Android, iOS). We had problems with OSX once but weren't able to reproduce reliably.

We are not really 100% sure it's related to connection tracking at this point since it worked fine on some lab servers but didn't on others and they are very similar (same OS but kernel version differs).

Since our latest tests showed it working on all the above mentionned OS, we are giving this a low priority but giving us solid reproducible cases will increase bug's priority. It is too random for us right now to sanely track.

What we think we'll do in the future is either disable or aggressively reduce the UDP tracking timers for DNS traffic (if that's even possible) or integrate with the conntrack-tools to kill any active conntrack session when a node is changed state. This requires an additional dependency (unpackaged in CentOS5 at this point) so it will need to be evaluated or bring enough value to warrant it.
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0001374closed dwuelfrath Inline mode should work as VLAN mode regarding DNS blackholing 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2012-04-16 11:24obilodeauNew Issue
2012-04-16 11:24obilodeauRelationship addedrelated to 0001374
2012-04-17 10:56obilodeauNote Added: 0002649
2012-04-17 10:58obilodeauNote Edited: 0002649
2012-10-19 12:07fgaudreaultTarget Version => investigate
2012-10-19 12:08fgaudreaultNote Added: 0003150
2012-10-19 12:58fdurandNote Added: 0003164
2012-10-24 09:58fgaudreaultNote Added: 0003235
2012-10-24 09:58fgaudreaultStatusnew => resolved
2012-10-24 09:58fgaudreaultFixed in Version => devel
2012-10-24 09:58fgaudreaultResolutionopen => fixed
2012-10-24 09:58fgaudreaultAssigned To => fgaudreault
2012-10-24 09:58fgaudreaultTarget Versioninvestigate => 3.6.0

Notes
(0002649)
obilodeau   
2012-04-17 10:56   
(edited on: 2012-04-17 10:58)
For the record, connection tracking variables for UDP are located here: /proc/sys/net/ipv4/netfilter/ip_conntrack_udp_timeout*

Turning these down to 0 and trying again with a client that re-uses source ports (like the CentOS 5.2 we tried) would be a good way to confirm or deny that the problem is connection tracking related after all.

Unfortunately we know that it's not going to be an acceptable fix since we NAT through in inline enforcement and it would break UDP-based applications (like Skype).
(0003150)
fgaudreault   
2012-10-19 12:08   
Reminder sent to: fdurand

Is this fixed with the ipset feature?
(0003164)
fdurand   
2012-10-19 12:58   
Yes this is fixed with ipset feature
(0003235)
fgaudreault   
2012-10-24 09:58   
Fixed with the ipset feature that will be released in 3.6.0.