Skip to main content

UAF with multiple BLPOP waiters

Author
quanye
Systems software: OS, networking, distributed systems.
Table of Contents

1. What is BLPOP?
#

Ordinary LPOP mylist: if the list is empty, return (nil) immediately.

BLPOP mylist 0 is Blocking LPOP:

  • List has elements → pop immediately, like LPOP
  • List empty → do not reply yet; this TCP connection enters a blocked state on the server and waits for someone to LPUSH/RPUSH

From the network stack:

客户端 socket ──write──▶ 服务端 query buffer
                         解析出 BLPOP
                    list 空?──yes──▶ 标记 CLIENT_BLOCKED
                              │         可读 handler 仍在(epoll 仍监听)
                              │         但不再 parse/execute 后续命令
                              │         (数据可进 query buf,先不执行)
客户端 read() 阻塞等 RESP  ◀── 暂时没有任何 reply 写出

So BLPOP is: one long-lived connection + a server state machine that turns client-side polling into server-side suspension. Timeout 0 means wait forever.

Multiple clients BLPOP on the same key → multiple waiters in a queue.


2. How does the server track waiters?
#

Each DB has db->blocking_keys:

key "mylist" ──▶ list:  [ client A ] → [ client B ] → ...

On block (blockForKeys), append to the list tail:

        listAddNodeTail(l, c);
        dictSetVal(c->bstate->keys, client_blocked_entry, listLast(l));

In memory:

list 头
┌─────────────┐   next   ┌─────────────┐
│ listNode A  │ ───────▶ │ listNode B  │
│ value = &A  │          │ value = &B  │
└─────────────┘          └─────────────┘
       ▲                        ▲
       │                        │
   client A 还把自己的           client B 同理
   bstate->keys["mylist"]
   指回这个 listNode*
   (方便 O(1) 摘链,而不是扫链表)

Point: listNode is a separate heap object; the client also holds a pointer back to it. Whoever tears down the client typically listUnlink + zfree(listNode).


3. What happens after LPUSH?
#

LPUSH mylist v1 ends with signalKeyAsReady, enqueueing the key into server.ready_keys.

At the end of that command’s processCommand, handleClientsBlockedOnKeyshandleClientsBlockedOnKey:

        listRewind(clients, &li);
        ...
        while ((ln = listNext(&li)) && count--) {
            client *receiver = listNodeValue(ln);
            ...
                    unblockClientOnKey(receiver, rl->key);

For list commands, wake-up is not “just stuff a reply”:

  1. Unlink the client from blocking_keys, clear CLIENT_BLOCKED
  2. Re-execute the original BLPOP (processCommand again)
  3. Now the list has elements; BLPOP pops and addReply

That is the “reexec / pending_command” path.


4. Root cause: listIter caches the successor early
#

adlist’s iterator contract is clear: during iteration you may only delete the current node, not others.

listNode *listNext(listIter *iter) {
    listNode *current = iter->next;

    if (current != NULL) {
        if (iter->direction == AL_START_HEAD)
            iter->next = current->next;   /* 已经把“下一个”指针存进 iter */
        else
            iter->next = current->prev;
    }
    return current;
}

Serving A:

listNext(&li) 得到 nodeA
同时 li.next = nodeB   ← 后继指针已经钉死在栈上的 listIter 里

Then unblockClientOnKey(A). If before that returns someone unlinks and frees nodeB, the outer listNext later reads a freed listNodeheap UAF.

adlist only guarantees: deleting current leaves the cached next valid. It does not guarantee deleting the successor inside a callback.


5. Who kills successor B in the callback? (module-free path)
#

Call stack (from ASan):

handleClientsBlockedOnKey          // 正在 for 链表,li.next == nodeB
  └─ unblockClientOnKey(A)
       └─ processCommand(A)        // 重跑 BLPOP
            └─ evictClients()      // processCommand 开头:客户端内存超限就踢人
                 └─ freeClient(B)  // B 输出缓冲很大,被选中
                      └─ unblockClient / unblockClientWaitingData
                           └─ zfree(nodeB)   // 正是 li.next 指向的那块
  … 回到 while
  └─ listNext(&li)                 // 读已 free 的 nodeB → UAF

Systems view:

Layer What happens
Event loop Still inside the same readprocessCommand(EXEC/LPUSH) stack; never returned to epoll_wait
Client state A: blocked → re-enters command execution; B still on the same blocking_keys list
Memory evictClients synchronously freeClient(B), returns B’s listNode to jemalloc/libc
Iterator Stack listIter.next is a dangling pointer

Not a “multithread race” — single-thread reentrancy: outer frame holds structure, inner same-thread tears it down.

Difference vs module path #4198 is only who triggers freeClient(successor):

  • Module: CLIENT KILL in a reply callback
  • This path: processCommandevictClients

Same list + same dangerous listIter usage.


6. Why MULTI with CONFIG SET + LPUSH?
#

If you just:

CONFIG SET maxmemory-clients 很小
LPUSH mylist v1

then LPUSH’s own processCommand on entry runs evictClients, and B may be killed before handleClientsBlockedOnKey — you miss “free successor mid-iteration”.

MULTI/EXEC puts the lowered limit and LPUSH in one EXEC:

  1. processCommand(EXEC) entry: limit still high → do not kick B
  2. In-txn CONFIG SET lowers the limit (nested call(), not a full outer processCommand)
  3. In-txn LPUSH marks the key ready
  4. Only after EXEC does handleClientsBlockedOnKeys run
  5. Serving A nested processCommand(A) kicks B under the low limit → hits the iteration window

Artificially creates “sync free mid-iteration on the wake path”. Uncommon in day-to-day scripts, but proves the ordinary BLPOP path can hit the hole.


7. One-line takeaway
#

BLPOP = when the list is empty, hang the connection on the blocking_keys[key] waiter list.
Bug = wake-up scans that list with listIter, but the sync callback for the current waiter (reexec → evictClients) frees the successor’s listNode, then outer listNext reads a dangling pointer.

Fix direction: do not rely on a bare iterator that caches successor listNode* across callbacks that may sync-freeClient; snapshot IDs/pointers or collect-then-process so a torn-down successor is never dereferenced.