TCP Battleships

TCP Battleships is a model I use to visualize socket dispatching on systems where there is a large "local" route (see IPv6 Things and Snippets:Nginx geo local server address).

The example below assumes 2001:db8::/64 as the routed "local" prefix (i.e. ip route add local 2001:db8::/64 dev lo)

(Local) IP address \ Port #    | 22 | 23 | 80 |443 |8000|8001|8002|8003|8004|8005|8006|
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::                     |    |    | X  | D  |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::1                    |    |    | X  | D  |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::2                    | C  | C  | C* | D* | C  | C  | C  | C  | C  | C  | C  |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::3                    |    |    | X  | D  |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::4                    |    |    | X  | D  |    | A  | A  | A  | A  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::5                    |    |    | X  | D  |    | A  | A  | A  | A  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::6                    |    |    | X  | D  |    | A  | A  | A  | A  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::7                    |    | Y  | X  | D  |    | A  | A  | A  | A  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::8                    |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::9                    |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::a                    |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::b                    |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::c                    |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::d                    |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::e                    |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::f                    |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::10                   |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
...
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::ffff:ffff:ffff:fffb  |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::ffff:ffff:ffff:fffc  |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::ffff:ffff:ffff:fffd  | Z  |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::ffff:ffff:ffff:fffe  |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+
2001:db8::ffff:ffff:ffff:ffff  |    |    | X  |    |    |    |    | B  | B  | B  |    |
-------------------------------+----+----+----+----+----+----+----+----+----+----+----+

X: bind("[::]", 80)
Y: bind("[2001:db8::7]", 23) (perhaps with IPV6_FREEBIND)
Z: bind("[2001:db8::ffff:ffff:ffff:fffd]", 22) (perhaps with IPV6_FREEBIND)

A: setsockopt IPV6_TRANSPARENT and ip6tables -t mangle -A PREROUTING \
-d 2001:db8::4/126 -p tcp -m multiport --dports 8001-8004 -j TPROXY --on-ip ...

B: setsockopt IPV6_TRANSPARENT and ip6tables -t mangle -A PREROUTING \
-p tcp multiport --dports 8003-8005 -j TPROXY --on-ip ...

C: setsockopt IPV6_TRANSPARENT and ip6tables -t mangle -A PREROUTING \
-d 2001:db8::2 -p tcp -j TPROXY --on-ip ...

D: setsockopt IPV6_TRANSPARENT and ip6tables -t mangle -A PREROUTING \
-d 2001:db8::/125 -p tcp --dport 443 -j TPROXY --on-ip ...

(*) For [2001:db8::2]:80, socket C will always override the [::]:80 listener.
For [2001:db8::2]:443, if rule D is specified before rule C, then a connection on this
IP/port will dispatch to socket D; otherwise it will dispatch to socket C.

The example above assumes that the iptables rules were added in this order: A, D, C, B.

When a connection from the Internet is made to, say, [2001:db8::d]:80 (i.e. connect("2001:db8::d", 80)) and directed to the server, this "battleship"-like table is checked on the server to see if there is any listening socket associated with [2001:db8::d]:80. There is one, so the connection is accepted and dispatched to socket X.

Similarly, connections to [2001:db8::5]:8003 are accepted and dispatched to socket B.

On the other hand, connections to [2001:db8::f]:443 are refused (i.e. a TCP reset is sent) because there is no socket that accepts that address.

For sockets associated with multiple "cells" in the table above, the original address can still be retrieved using getsockname() on the socket returned by accept(), allowing for even more fine-grained dispatching in user space (as socketbox would do).