Implement support for “Learner” or “Observer” non-voting roles in the Raft consensus group.
实现learner 或者 observer.(实际上就是learner).
Non-voting nodes should still replicate the Raft log to receive topology updates so they can serve client requests like CLUSTER SLOTS accurately.
learner应该复制日志状态机,同时接受拓扑更新,这样就能精确serve来自client的请求。(读处理,写转发)
They should not participate in write quorums or elections, preventing performance overhead and scale limitations on the voting core. In a cluster with hundreds of nodes, it’s beneficial if only ~5 nodes are voting members, to minimize the risk of many nodes simultaneously requesting votes for themselves in a leader election, resulting in high risk of split vote.
不参与选举和日志写入的多数派的确认,防止选举的性能变差。
为什么不参与确认?
因为同步这段时间内,新加入节点的log都很少,根本无法参与确认,如果一下加进来太多,会导致很长时间新的日志无法提交。
数百个nodes组成的集群,最多只要有5个nodes来真的做选举就好,减少许多节点同时发起选举导致的脑裂问题(虽然随机选举超时时间已经很大程度上解决了这个问题)
Support potential chaining-style replication of the Raft log where learners pull from other nodes rather than all overloading the leader. (This point seems like a separate feature, but we should think about the future possibility.)
就是如果说一个voting节点需要被上百个learner同时,
网络IO会成为瓶颈,是否使用链式复制(常见的设计思想,见Google File System中的Dataflow,就是链式的流水线传输。)
之后reviewer和我们align了一下:
I think we should add all nodes as learners by default. When a node has caught up with the raft log, it can be promoted to a follower. I believe this is some recommendation some paper. Btw, does any of you know which paper describes Raft learners?
这里的看法是,当一个节点被加入集群的时候,首先应当作为learner,之后完全同步上日志进度了之后(这里的考究就是,到底是必须一模一样还是说不一定完全一样),才能作为follower.
实际上这里的细节还需要考究,需要研究一下论文原文和etcd的源码看看。
1. Yes, a learner can be primary and replica.
在主从复制的特性中,learner可以作为主节点(own a slot),也可以作为从节点。
选举的时候不是主节点来做会存在问题么?
2. I think it’s good to distinguish an entry that affects quorum from an entry that doesn’t, especially if we want to implement config-on-append later (Raft Cluster: Config-on-append for membership changes #3926). If we want to add all new nodes as learners first, then maybe NODE_JOIN should make the new node a learner. Or we add a flag to NODE_JOIN to say whether it’s a voter or not.
Regardless, we’ll need to be able to promote learners to followers and demote followers to learners, so we’ll need new entry types for that. When a node is removed, NODE_FORGET would apply to learners and voters or should we split that one as well, i.e. demote to follower before we append NODE_FORGET?
我认为区分“影响法定人数(quorum)的日志条目”和“不影响法定人数的日志条目”是件好事,特别是考虑到如果我们以后想实现 config-on-append(追加时即生效配置).
如果首先作为learner 添加,那么也许 NODE_JOIN 就应该默认使新节点成为 learner。
或者我们在 NODE_JOIN 中添加一个标志,来指定它是 voter 还是 learner。
无论如何,我们都需要具备将 learner 提升为 follower,以及将 follower 降级为 learner 的能力,因此我们需要为此引入新的日志条目类型。
当移除一个节点时,NODE_FORGET 应该同时适用于 learner 和 voter 呢,
还是我们也应该把它拆分开,比如在追加 NODE_FORGET 之前,先将其降级为 learner.
3. Yeah, I think server.cluster->size should represent the voters only. In gossip cluster, the cluster size is only the primaries with slots, i.e. the voters, so we can keep the same convention in raft cluster.
就是cluster大小仅代表voter的数量。但是这里矛盾的点不是learner也能作为主节点持有slot么?
这个也好像不矛盾,主从和控制平面完全是两码事。
4. Yeah PROMOTE <id> and something similar for DEMOTE <id> – one node at a time to match the current config-on-apply logic for quorum updates. But these words can also mean promote to leader, candidate, so maybe ADD_VOTER <id> and DEL_VOTER <id> is clearer?
修改一下语义?
5. I don’t think this problem applies to this issue. A singleton leader becomes a joiner (just waiting to be added to a cluster) and then reverts to singleton leader again after a timeout. There’s no learner role involved here. Maybe it’s a scenario that only affects Raft Cluster: Merging non-singleton clusters #3868? We can discuss it on that issue.
这个是集群合并中需要考察的问题,就是leader的退化问题。
Then, we’ll also need a flag in nodes.conf to say if a node is a voter or not. We could use one of these:
在集群配置中加一个标志,用于表明一个节点是否是voter。
- node flags 就使用前两个吧
- aux fields- one of the unused columns ping-sent, pong-received
Just a though: (If we ever want to do joint consensus, we could list all the voters on every quorum change, e.g. an entry like CONFIG_VOTERS id1 id2 id3 .... Did we already say we don’t need joint consensus? If we just keep a list of uncommitted voters or something, maybe it’s not really that difficult? 🤔 We can change this later if we want to implement joint consensus though. If we do it, we can just use this entry type every time we add or remove voters. There shouldn’t be more than 5 or 7 voters in a cluster, normally.)
Most membership change algorithms introduce additional mechanism to deal with such problems. This is what we did for Raft initially, but we later discovered a simpler approach, which is to disallow membership changes that could result in disjoint majorities.
大多数成员变更都会引入新的机制来处理。 开始Raft也是这么做的。
之后我们发现了更简单的方法,那就是禁止会导致出现不相交的多数派的成员变更?
上面这个例子,就是新旧的集群中没有出现交集。
only one server can be added or removed from the cluster at a time.
这就是单节点集群成员变更,每次只允许添加或者删除一个节点。