Skip to main content

Multithread RDB crash

Author
quanye
Systems software: OS, networking, distributed systems.

This looks like a multithread-only issue — revisit later. I hit this as a new Valkey problem while testing; worth trying first. Reproduction sketch: Build on unstable, start a server without changing valkey.conf; under load an RDB should appear by default. Test on the main thread and see if it fails.

# 解开当前进程的内存锁
sudo prlimit --memlock=unlimited --pid $$
# 创建在内存中工作的虚拟网卡
sudo modprobe dummy
sudo ip link add dummy0 type dummy
sudo ip link set dummy0 up
# 配置本地的IP
sudo ip addr add 10.0.0.1/24 dev dummy0
# 做RXE的绑定
sudo rdma link add rxe_dummy type rxe netdev dummy0
# 停止本来运行的 valkey
sudo systemctl stop valkey
# 编译一遍
make BUILD_RDMA=yes USE_FAST_FLOAT=yes
# 跑server
sudo ./src/valkey-server valkey.conf --rdma-bind 10.0.0.1 --rdma-port 6379

Then start a client for load:

# 解开当前进程的内存锁
sudo prlimit --memlock=unlimited --pid $$
# 直接做暴力压测,还可以尝试比这个参数更高的
./src/valkey-benchmark -h 10.0.0.1 -p 6379 -d 256 --threads 16 -c 100 -P 32 -n 20000000 -t get --rdma

That produced an RDB file: dump.rdb On unstable I could not reproduce; revisit on the multithread branch after fixing multithread issues. So multithread first, then this.