Kill a TCP connection
How many ways do you have to kill a TCP connection?
ptrace/GDB
In the old days, people invented ptrace and GDB, which provided us with an option to terminate a TCP connection from outside the process.
Open two terminals and use nc to run a server and client respectively:
# console 1
$ nc -l 127.0.0.1 1215
# console 2
$ nc 127.0.0.1 1215
Open another terminal and check the pid and fd of the process:
# console 3
$ ps aux | grep nc
root 561894 0.0 0.0 34068 4228 pts/0 S+ 11:27 0:00 nc -l 127.0.0.1 1215
root 561905 0.0 0.0 34988 7292 pts/1 S+ 11:27 0:00 nc 127.0.0.1 1215
$ ss -tanp | grep nc
ESTAB 0 0 127.0.0.1:57076 127.0.0.1:1215 users:(("nc",pid=561905,fd=3))
ESTAB 0 0 127.0.0.1:1215 127.0.0.1:57076 users:(("nc",pid=561894,fd=4))
Close this connection using GDB:
# console 3
$ gdb -p 561905
...
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
0x00007fc33a95c94b in select () from /lib64/libc.so.6
(gdb) call (int)close(3)
$1 = 0
(gdb) quit
A debugging session is active.
Inferior 1 [process 561905] will be detached.
Quit anyway? (y or n) y
Detaching from program: /usr/bin/ncat, process 561905
[Inferior 1 (process 561905) detached]
After calling close, we found that the server connection closed normally, but the client reported an error:
# console 1
$ nc -l 127.0.0.1 1215
$ echo $?
0
# console 2
$ nc 127.0.0.1 1215
libnsock select_loop(): nsock_loop error 9: Bad file descriptor
$ echo $?
1
Because fd is registered in select(2), this error is expected:
select(2) System Calls Manual select(2)
...
ERRORS
EBADF An invalid file descriptor was given in one of the sets.
(Perhaps a file descriptor that was already closed, or one
on which an error has occurred.) However, see BUGS.
...
This also tells us that this plan is not perfect.
tcpkill
Later, people invented BPF, here refers to the classic BPF, also known as cBPF. So, we have a new choice: tcpkill
Similarly, open two terminals, use nc to run a server and a client respectively:
# console 1
$ nc -l 127.0.0.1 1215
# console 2
$ nc 127.0.0.1 1215
Restart tcpkill:
$ tcpkill -ilo tcp and port 1215
tcpkill: listening on lo [tcp and port 1215]
It can be seen that the TCP connection is not directly closed. This is because tcpkill needs to capture the network packets on the target connection, obtain the TCP sequence, in order to construct the TCP RST packet to close the connection.
By creating some traffic on the connection, the connection can be closed normally:
# console 1
$ nc -l 127.0.0.1 1215
hello
# console 2
$ nc 127.0.0.1 1215
hello
# console 3
$ tcpkill -ilo tcp and port 1215
tcpkill: listening on lo [tcp and port 1215]
127.0.0.1:42832 > 127.0.0.1:1215: R 3705939593:3705939593(0) win 0
127.0.0.1:42832 > 127.0.0.1:1215: R 3705940105:3705940105(0) win 0
127.0.0.1:42832 > 127.0.0.1:1215: R 3705941129:3705941129(0) win 0
127.0.0.1:1215 > 127.0.0.1:42832: R 813204288:813204288(0) win 0
127.0.0.1:1215 > 127.0.0.1:42832: R 813204800:813204800(0) win 0
127.0.0.1:1215 > 127.0.0.1:42832: R 813205824:813205824(0) win 0
This solution is not perfect either, it needs to connect with traffic. If TCP Keepalive is not turned on, there may be no opportunity to close the connection.
tcp_diag
Time came to 2015, Google engineers added the option CONFIG_INET_DIAG_DESTROY in kernel commit c1e64e298b8c:
From c1e64e298b8cad309091b95d8436a0255c84f54a Mon Sep 17 00:00:00 2001
From: Lorenzo Colitti <lorenzo@google.com>
Date: Wed, 16 Dec 2015 12:30:05 +0900
Subject: [PATCH] net: diag: Support destroying TCP sockets.
This implements SOCK_DESTROY for TCP sockets. It causes all
blocking calls on the socket to fail fast with ECONNABORTED and
causes a protocol close of the socket. It informs the other end
of the connection by sending a RST, i.e., initiating a TCP ABORT
as per RFC 793. ECONNABORTED was chosen for consistency with
FreeBSD.
Signed-off-by: Lorenzo Colitti <lorenzo@google.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
So that we can close a TCP connection by sending a netlink message, we can use the ss tool for testing:
# console 1
$ nc -l 127.0.0.1 1215
# console 2
$ nc 127.0.0.1 1215
# console 3
$ ss -K dport = 1215
Very cool, except that programming with the netlink interface is a bit troublesome, there is no problem.
bpf_iter_tcp
By 2020, the kernel had something even cooler: BPF Iterator
From 52d87d5f6418ba1b8b449ed5eea1532664896851 Mon Sep 17 00:00:00 2001
From: Yonghong Song <yhs@fb.com>
Date: Tue, 23 Jun 2020 16:08:05 -0700
Subject: [PATCH] net: bpf: Implement bpf iterator for tcp
The bpf iterator for tcp is implemented. Both tcp4 and tcp6
sockets will be traversed. It is up to bpf program to
filter for tcp4 or tcp6 only, or both families of sockets.
Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20200623230805.3987959-1-yhs@fb.com
Through a simple BPF program, we can traverse all TCP connections and filter out our target connections:
SEC("iter/tcp")
int tcp_conn(struct bpf_iter__tcp *ctx)
{
struct sock_common *skc = ctx->sk_common;
struct tcpconn t = {};
struct tcp_sock *tp;
if (!skc)
return 0;
tp = bpf_skc_to_tcp_sock(skc);
if (!tp)
return 0;
if (skc->skc_state != TCP_ESTABLISHED)
return 0;
/* Add your filters here */
t.saddr = skc->skc_rcv_saddr;
t.daddr = skc->skc_daddr;
t.sport = skc->skc_num;
t.dport = bpf_ntohs(skc->skc_dport);
t.seq = tp->snd_nxt;
t.ack_seq = tp->rcv_nxt;
bpf_seq_write(ctx->meta->seq, &t, sizeof(t));
return 0;
}
How about it? Quite simple, right? Coupled with the user-mode Raw Socket, we can create a modern tcpkill, which can close a connection without needing to connect to traffic.
bpf_sock_destroy
If your kernel is new enough, we can use the bpf_sock_destroy kfunc, a new feature of Linux v6.5 that entered the kernel mainline about a year ago.
With it, we can discard the above Raw Socket, and directly end a TCP connection in the BPF program.
SEC("iter/tcp")
int tcp_conn(struct bpf_iter__tcp *ctx)
{
struct sock_common *skc = ctx->sk_common;
struct tcpconn t = {};
struct tcp_sock *tp;
if (!skc)
return 0;
tp = bpf_skc_to_tcp_sock(skc);
if (!tp)
return 0;
if (skc->skc_state != TCP_ESTABLISHED)
return 0;
/* Add your filters here */
bpf_sock_destroy(skc);
return 0;
}
Reference: https://mp.weixin.qq.com/s/LY1ectZffu1mgwk87V-HHA