e7d02be19 ("protosw: refactor protosw and domain static declaration and
load") changed the way this function should be invoked.
Link: e7d02be19d
Reported-by: Michael Pro <michael.adm@gmail.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
The sockbuf changes to merge two versions of sbcreatecontrol() into one
occurred in b46667c63eb7 in the FreeBSD src repository. The value of
__FreeBSD_version at the time of that commit was 1400059, so check
for systems with a version stamp less than 1400059.
Ideally __FreeBSD_version would have been bumped in b46667c63eb7,
however it was set to 1400059 53 commits prior in 3a9a9c0ca44e and not
bumped to 1400060 until 374 commits later in 85d7875d4291. Thus, this
is not a perfect solution because CURRENT systems built from 53 possible
commits that require the adjustment will still fail to build the
wireguard kernel module. This is better than checking for systems with
__FreeBSD_version < 1400057. In that case, there are 767 different
commits a CURRENT system may be built from where the module will fail to
build.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
This requires the the recent changes in FreeBSD to support the
Chacha20-Poly1305 AEAD cipher with support for an 8 byte nonce (vs the
12 byte nonce used by TLS and IPsec).
Signed-off-by: John Baldwin <jhb@FreeBSD.org>
54b2075 ("build: only include compat.h for if_wg.c and fix build with an
obj directory") scoped compat.h to if_wg.c, which broke building on
12.3.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Previously the wg_encrypt/decrypt_dispatch routines would wakeup all
of the non-scheduled tasks. This resulted in waking up several tasks
which did no useful work but did add overhead in terms of extra
context switches, etc. In a single-threaded iperf3 TCP test on an
8-CPU system, only ~13% of the scheduled tasks did any work.
Instead, adopt a similar scheme to that used in the Linux wireguard
driver in which each call only wakes up a single task and uses
per-queue CPU iterators to round-robin among CPUs when choosing a task
to schedule. This improved throughput in the iperf3 TCP test by more
than double and ~65% of scheduled tasks did useful work.
Signed-off-by: John Baldwin <jhb@FreeBSD.org>
Send tags are only added on mbufs sent to an interface which
supports if_snd_tag_alloc. Just assert that they are never
present instead.
Signed-off-by: John Baldwin <jhb@FreeBSD.org>
callout_init_rw() happens to compile ok because both are macros that
dereference the passed in lock's "lock_object" member to pass to the
real function.
Signed-off-by: John Baldwin <jhb@FreeBSD.org>
This reduces the edge cases which need handling, and M_WAITOK is safe
to use in this context.
While here, narrow the scope of the sc_lock to the code that interacts
with the softc, but not copyin/copyout, malloc, and nvlist_pack calls
before and after interacting with the softc.
Signed-off-by: John Baldwin <jhb@FreeBSD.org>
In particular, this permits building the module as part of a kernel
build via the LOCAL_MODULES facility.
Signed-off-by: John Baldwin <jhb@FreeBSD.org>
Since 742e7210 ("udp: allow udp_tun_func_t() to indicate it did not eat
the packet"), wg_input must return a boolean. We force a cast for old
kernels. It'd be nicer to work around this in compat.h, but we can't
because FreeBSD's headers have dependencies we can't resolve from there.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
In the process, just don't do v6 allowedips if there's no INET6 support.
Reported-by: Yi <yu@boenyc.us>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
The keepalive timers -- both persistent and mandatory -- are part of the
internal state machine, which needs to be cranked whether or not the
packet was actually sent. A packet might be dropped by the network. Or
the packet might be dropped by the local network stack. The latter case
gives a hint -- which is useful for the data_sent event -- but is
harmful to consider for the keepalive state machine. So, crank those
timers before even calling wg_send.
Incidentally, doing it this way matches exactly what Linux's send.c's
wg_packet_create_data_done and Go's send.go's RoutineSequentialSender do
too.
Suggested-by: Kyle Evans <kevans@freebsd.org>
Reported-by: Ryan Roosa <ryanroosa@gmail.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Since 407b687dfef ("Make sure all tasklets are drained before unloading
the LinuxKPI. Else use-after-free may happen."), stable/13 now has the
taskqueue API that we need.
Reported-by: Herbert J. Skuhra <herbert@gojira.at>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
There were multiple places where a rwlock was used despite never
rlocking, so just change these into mtxs. This was done with the aid of
Coccinelle's spatch, using this input:
#spatch -j 4 --recursive-includes --include-headers-for-types --include-headers --in-place --macro-file <seebelow.h>
virtual after_start
@initialize:ocaml@
@@
let has_write_table = Hashtbl.create 101
let has_read_table = Hashtbl.create 101
let ok i m =
let entry = (i,m) in
Hashtbl.mem has_write_table entry && not(Hashtbl.mem has_read_table entry)
@hasw depends on !after_start@
identifier i,m;
struct i x;
@@
(
rw_wlock(&x.m)
|
rw_wunlock(&x.m)
)
@script:ocaml@
i << hasw.i;
m << hasw.m;
@@
Hashtbl.replace has_write_table (i,m) ()
@hasr depends on !after_start@
identifier i,m;
struct i x;
@@
(
rw_rlock(&x.m)
|
rw_runlock(&x.m)
)
@script:ocaml@
i << hasr.i;
m << hasr.m;
@@
Hashtbl.replace has_read_table (i,m) ()
@finalize:ocaml depends on !after_start@
wt << merge.has_write_table;
rt << merge.has_read_table;
@@
let redo ts dst =
List.iter (Hashtbl.iter (fun k _ -> Hashtbl.add dst k ())) ts in
redo wt has_write_table;
redo rt has_read_table;
let it = new iteration() in
it#add_virtual_rule After_start;
it#register()
(* ----------------------------------------------------------- *)
@depends on after_start@
identifier i;
identifier m : script:ocaml(i) { ok i m };
@@
struct i {
...
- struct rwlock m;
+ struct mtx m;
...
}
@depends on after_start disable fld_to_ptr@
identifier m;
identifier i : script:ocaml(m) { ok i m };
struct i x;
@@
- rw_wlock
+ mtx_lock
(&x.m)
@depends on after_start disable fld_to_ptr@
identifier m;
identifier i : script:ocaml(m) { ok i m };
struct i x;
@@
- rw_wunlock
+ mtx_unlock
(&x.m)
@depends on after_start disable fld_to_ptr@
identifier m;
expression e;
identifier i : script:ocaml(m) { ok i m };
struct i x;
@@
- rw_init(&x.m, e);
+ mtx_init(&x.m, e, NULL, MTX_DEF);
@depends on after_start disable fld_to_ptr@
identifier m;
identifier i : script:ocaml(m) { ok i m };
struct i x;
@@
- rw_destroy
+ mtx_destroy
(&x.m)
@depends on after_start disable fld_to_ptr, ptr_to_array@
identifier m;
identifier i : script:ocaml(m) { ok i m };
struct i *x;
@@
- rw_wlock
+ mtx_lock
(&x->m)
@depends on after_start disable fld_to_ptr, ptr_to_array@
identifier m;
identifier i : script:ocaml(m) { ok i m };
struct i *x;
@@
- rw_wunlock
+ mtx_unlock
(&x->m)
@depends on after_start disable fld_to_ptr, ptr_to_array@
identifier m;
expression e;
identifier i : script:ocaml(m) { ok i m };
struct i *x;
@@
- rw_init(&x->m, e);
+ mtx_init(&x->m, e, NULL, MTX_DEF);
@depends on after_start disable fld_to_ptr, ptr_to_array@
identifier m;
identifier i : script:ocaml(m) { ok i m };
struct i *x;
@@
- rw_destroy
+ mtx_destroy
(&x->m)
A few macros needed to be provided manually for the parser to work:
#define LIST_HEAD(x,y) int
#define TAILQ_HEAD(x,y) int
#define STAILQ_HEAD(x,y) int
#define CK_LIST_HEAD(x,y) int
#define CK_LIST_ENTRY(x) int
#define LIST_ENTRY(x) int
#define TAILQ_ENTRY(x) int
#define STAILQ_ENTRY(x) int
Co-authored-by: Julia Lawall <julia.lawall@inria.fr>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Before, most uses of rwlock and mtx never called the destroy method,
which might cause problems for witness.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
This was added to 12.1 in a security fix, but wasn't really wired up
properly, so this effectively disables it from packet resetting, which
is a bummer, but it's more preferable than hacking this in bad ways.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Hopefully bad tests will cause the module to not insert, so the CI picks
this up. It looks like a failure to insert the module at the moment
actually causes another crash, though:
Kernel page fault with the following non-sleepable locks held:
exclusive sleep mutex if_cloners lock (if_cloners lock) r = 0 (0xffffffff81d9a9b8) locked @ /usr/src/sys/net/if_clone.c:447
stack backtrace:
#0 0xffffffff80c66181 at witness_debugger+0x71
#1 0xffffffff80c6729d at witness_warn+0x40d
#2 0xffffffff8109499e at trap_pfault+0x7e
#3 0xffffffff81093fab at trap+0x2ab
#4 0xffffffff810687f8 at calltrap+0x8
#5 0xffffffff82925610 at wg_module_event_handler+0x120
#6 0xffffffff80bd53c3 at module_register_init+0xd3
#7 0xffffffff80bc5c61 at linker_load_module+0xc01
#8 0xffffffff80bc73b9 at kern_kldload+0xe9
#9 0xffffffff80bc74db at sys_kldload+0x5b
#10 0xffffffff810952f7 at amd64_syscall+0x147
#11 0xffffffff8106911e at fast_syscall_common+0xf8
Fatal trap 12: page fault while in kernel mode
cpuid = 9; apic id = 09
fault virtual address = 0x70
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80d18e37
stack pointer = 0x28:0xfffffe0115fb35a0
frame pointer = 0x28:0xfffffe0115fb35c0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 1587 (kldload)
trap number = 12
panic: page fault
cpuid = 9
time = 1621380034
KDB: stack backtrace:
#0 0xffffffff80c44695 at kdb_backtrace+0x65
#1 0xffffffff80bf9d01 at vpanic+0x181
#2 0xffffffff80bf9ad3 at panic+0x43
#3 0xffffffff81094917 at trap_fatal+0x387
#4 0xffffffff810949b7 at trap_pfault+0x97
#5 0xffffffff81093fab at trap+0x2ab
#6 0xffffffff810687f8 at calltrap+0x8
#7 0xffffffff82925610 at wg_module_event_handler+0x120
#8 0xffffffff80bd53c3 at module_register_init+0xd3
#9 0xffffffff80bc5c61 at linker_load_module+0xc01
#10 0xffffffff80bc73b9 at kern_kldload+0xe9
#11 0xffffffff80bc74db at sys_kldload+0x5b
#12 0xffffffff810952f7 at amd64_syscall+0x147
#13 0xffffffff8106911e at fast_syscall_common+0xf8
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Fixes:
#12 0xffffffff80f20105 in uma_zalloc_arg (zone=<optimized out>,
udata=<optimized out>, udata@entry=0x0, flags=flags@entry=257)
at /usr/src/sys/vm/uma_core.c:3420
#13 0xffffffff82922844 in uma_zalloc (zone=<unavailable>, flags=257)
at /usr/src/sys/vm/uma.h:358
#14 wg_packet_alloc (m=0xfffff801154bfa00) at if_wg.c:1769
#15 wg_send_keepalive (peer=0xfffff800075dd000,
peer@entry=<error reading variable: value is not available>)
at if_wg.c:1291
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Otherwise CK_LIST_REMOVE might be called twice on the same element.
Running the following trigger will reproduce the bug that Manojav
reported:
#!/usr/local/bin/bash
NUM_PEER=50
peer_args=( )
for ((i=0; i<NUM_PEER; i++)); do
port="$RANDOM"
private="$(wg genkey)"
ifconfig "wg$i" create
wg set "wg$i" listen-port "$port" private-key <(echo "$private")
peer_args+=( peer "$(wg pubkey <<<"$private")" endpoint "127.0.0.1:$port" persistent-keepalive 1 )
done
for ((i=0; i<NUM_PEER; i++)); do
wg set "wg$i" "${peer_args[@]}"
ifconfig "wg$i" up
done
wg
while true; do
for ((i=0; i<NUM_PEER; i++)); do
ifconfig "wg$i" down
ifconfig "wg$i" up
done
done
Reported-by: Manojav Sridhar <manojav@manojav.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
The rest of the code uses this, so go with it for now. Maybe later ncon
will want to clean up everything to be this way, but for now keep it
consistent.
This partially reverts commit a1fdf6646b,
but doesn't reintroduce the bug that it had fixed.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Jason A. Donenfeld