在知识库中专注阅读，并随时返回相关工具与课程
大家好，这里是公众号 DBA学习之路，分享一些学习国产数据库路上的知识和经验。 @TOC目录 记得在金仓公众号看到过一篇文章：重生之我在甲方做DBA，从单机到集群，易如反掌~https://mp.weixin.qq.com/s/Lam25tD6TAX1x7QhiruD4Q，写的比较有意思。 这篇文章介绍的是金仓数据库单机转集群的一些策略以及大致步骤，单机数据库在线扩展成集群，从单打独斗到同舟共“集
大家好，这里是公众号 DBA学习之路，分享一些学习国产数据库路上的知识和经验。
本章目标：掌握本章核心知识点 前置要求：完成前序章节学习 预计时长：60 分钟
记得在金仓公众号看到过一篇文章：重生之我在甲方做DBA，从单机到集群，易如反掌~，写的比较有意思。
这篇文章介绍的是金仓数据库单机转集群的一些策略以及大致步骤，单机数据库在线扩展成集群，从单打独斗到同舟共“集”。告别单机瓶颈，轻松拥抱高可用！
为了避免绩效评定为 D，我打算学习一下！
KingbaseES 提供数据库扩缩容工具进行数据库集群的在线扩缩容。对于不支持图形化（GUI）的服务器，KingbaseES 提供基于命令行操作的集群扩缩容方式，本文主要用于在不支持 GUI 的服务器上的 KingbaseES 版本集群扩缩容工作。
金仓数据库为我们提供了 3 种单机扩集群的方式：
我们可以根据业务需求、资源可用性、成本预算和技术支持等因素进行综合考虑。
我这里演示的是金仓主备集群使用命令行静默一键扩容的方式。
环境信息：
在正式开始集群扩容之前，需要做好以下几个准备：
1、已部署好一主一备集群，可参考文章：金仓 KES RWC（读写分离集群）静默安装部署指南
2、已准备好进行扩容操作的节点，配置好系统即可，不需要安装软件，可参考文章：金仓数据库 KingbaseES V9 单机安装指南
3、待扩容或缩容节点需要从集群主节点上获取如下文件：
/KingbaseES/V9/ClientTools/guitools/DeployTools/zip/ 中的 db.zip/install 下的一键部署脚本 cluster_install.sh、install.conf、trust_cluster.shlicense.dat 授权文件也需要从主备集群中收集以上文件存放到 /soft 目录中：
1[root@kesrwc01 ~]# mkdir /soft2[root@kesrwc01 ~]# cp /KingbaseES/V9/ClientTools/guitools/DeployTools/zip/db.zip /soft/3[root@kesrwc01 ~]# cp /install/cluster_install.sh /soft/4[root@kesrwc01 ~]# cp /install/install.conf /soft/5[root@kesrwc01 ~]# cp /install/trust_cluster.sh /soft/6[root@kesrwc01 ~]# ll /soft/7总用量 3198728-rwxr-xr-x 1 root root 252402 2月 26 16:26 cluster_install.sh9-rw-r--r-- 1 root root 327258132 2月 26 16:23 db.zip10-rw-r--r-- 1 root root 19430 2月 26 16:26 install.conf11-rwxr-xr-x 1 root root 9677 2月 26 16:26 trust_cluster.sh然后拷贝到需要扩容的单机主机上：
1[root@kesrwc01 ~]# scp /soft/* 192.168.6.98:/soft
使用集群安装用户在待扩容节点，将上面从主备集群中获取到的文件拷贝至单机目录 /install 下：
1[root@kesv9 soft]# ll2总用量 3198723-rwxr-xr-x 1 root root 252402 2月 26 16:29 cluster_install.sh4-rw-r--r-- 1 root root 327258132 2月 26 16:29 db.zip5-rw-r--r-- 1 root root 19430 2月 26 16:29 install.conf6-rwxr-xr-x 1 root root 9677 2月 26 16:29 trust_cluster.sh7[root@kesv9 soft]# cp * /install/8[root@kesv9 soft]# chown -R kingbase:kingbase /install/
确保文件权限属主需为集群安装用户即可。
待扩容节点配置 install.conf 文件，首先需要修改 all_ip 选项：
1## 互信需要所有节点 IP，包括扩容的节点2all_ip=(192.168.6.87 192.168.6.88 192.168.6.98)然后填写 expand 标签下各参数信息**：
1## config of create a standby/witness node.2## when the cluster is in quorum or sync mode and expand sync standby node,3## it may automatically adjust synchronous_node and synchronous_standby_count parameters.4[expand]5expand_type="0" # The node type of standby/witness node, which would be add to cluster. 0:standby 1:witness6primary_ip="192.168.6.87" # The ip addr of cluster primary node, which need to expand a standby/witness node.7expand_ip="192.168.6.98" # The ip addr of standby/witness node, which would be add to cluster.8node_id="3" # The node_id of standby/witness node, which would be add to cluster. It does not the same with any one in cluster node9 # for example: node_id="3"10sync_type="" # the sync_type parameter is used to specify the sync type for expand node. 0:sync 1:potential 2:async11 # this parameter is only valid when expand_type="0" and the synchronous parameter of the cluster is set to custom mode.1213## Specific instructions ,see it under [install]14install_dir="/KingbaseES/V9/cluster" # the last layer of directory could not add '/'15zip_package="/install/db.zip"16net_device=(ens33) # if virtual_ip set,it must be set17net_device_ip=(192.168.6.98) # if virtual_ip set,it must be set18license_file=()19deploy_by_sshd="1"20ssh_port="22"21scmd_port="8890"这里需要修改的信息可以参考下图，修改要求可以参考注释即可：
静默安装需要配置免密，使用 root 用户执行 trust_cluster.sh 脚本即可：
1[root@kesv9 ~]# cd /install/2[root@kesv9 install]# sh trust_cluster.sh 3[INFO] set password-free between root and kingbase4Generating public/private rsa key pair.5...6...7...89## 过程中根据提示输入密码即可10root@192.168.6.87's password: 11id_rsa 100% 2590 1.3MB/s 00:00 12id_rsa.pub 100% 564 486.4KB/s 00:00 13authorized_keys 100% 564 590.3KB/s 00:00 14known_hosts 1516...17...18...19connect to "192.168.6.87" from current node by 'ssh' kingbase:0..... OK20connect to "192.168.6.87" from current node by 'ssh' root:0..... OK21connect to "192.168.6.88" from "192.168.6.87" by 'ssh' kingbase->kingbase:0 .... OK22connect to "192.168.6.88" from "192.168.6.87" by 'ssh' root->root:0 root->kingbase:0 kingbase->root:0.... OK23connect to "192.168.6.88" from current node by 'ssh' kingbase:0..... OK24connect to "192.168.6.88" from current node by 'ssh' root:0..... OK25connect to "192.168.6.98" from "192.168.6.88" by 'ssh' kingbase->kingbase:0 .... OK26connect to "192.168.6.98" from "192.168.6.88" by 'ssh' root->root:0 root->kingbase:0 kingbase->root:0.... OK27connect to "192.168.6.98" from current node by 'ssh' kingbase:0..... OK28connect to "192.168.6.98" from current node by 'ssh' root:0..... OK29connect to "192.168.6.87" from "192.168.6.98" by 'ssh' kingbase->kingbase:0 .... OK30connect to "192.168.6.87" from "192.168.6.98" by 'ssh' root->root:0 root->kingbase:0 kingbase->root:0.... OK31check ssh connection success!互信成功即可。
📢 注意：待扩容节点不需要建库，否则在扩容过程中会提示关闭数据库，而且加入集群后使用的也是集群库的数据，所以待扩容节点建库是多此一举：
如果待扩容节点已经建库，那就要手动关闭数据库：
1[kingbase@kesv9:/install]$ sys_ctl stop2等待服务器进程关闭 .... 完成3服务器进程已经关闭如果已经建库并且数据存放目录与集群目录一样，需要先将待扩容节点的数据存放目录备份改名：
1[root@kesv9 ~]# mv /data/ /databak2[root@kesv9 ~]# mkdir /data3[root@kesv9 ~]# chown kingbase:kingbase /data防止扩容过程中提示冲突：
有一些集群的参数与单机配置有区别，所以需要提前修改操作系统参数，否则扩容过程中会有一些告警和报错：
修改参数：
1[root@kesv9 ~]# vi /etc/sysctl.conf23## 修改为如下配置4kernel.sem= 5010 641280 5010 25656## 然后执行生效7[root@kesv9 ~]# sysctl -p确保准备都好了之后就可以开始扩容了。
2、待扩容节点执行扩容 standby 节点操作：
1[kingbase@kesv9:/install]$ sh cluster_install.sh expand2[CONFIG_CHECK] will deploy the cluster of 3[RUNNING] success connect to the target "192.168.6.98" ..... OK4[RUNNING] success connect to "192.168.6.98" from current node by 'ssh' ..... OK5[RUNNING] success connect to the target "192.168.6.87" ..... OK6[RUNNING] success connect to "192.168.6.87" from current node by 'ssh' ..... OK7[RUNNING] Primary node ip is 192.168.6.87 ...8[RUNNING] Primary node ip is 192.168.6.87 ... OK9[CONFIG_CHECK] set install_with_root=110[RUNNING] success connect to the target "192.168.6.98" ..... OK11[RUNNING] success connect to "192.168.6.98" from current node by 'ssh' ..... OK12[RUNNING] success connect to the target "192.168.6.87" ..... OK13[RUNNING] success connect to "192.168.6.87" from current node by 'ssh' ..... OK14[INSTALL] load config from cluster.....15 [INFO] db_user=system16 [INFO] db_port=5432117 [INFO] use_scmd=118 [INFO] data_directory=/data19 [INFO] scmd_port=889020 [INFO] recovery=standby21 [INFO] use_check_disk=off22 [INFO] trusted_servers=192.168.6.25423 [INFO] reconnect_attempts=1024 [INFO] reconnect_interval=625 [INFO] auto_cluster_recovery_level=126 [INFO] synchronous=quorum27[INSTALL] load config from cluster.....OK28[CONFIG_CHECK] file format is correct ... OK29[CONFIG_CHECK] check database connection ... 30[CONFIG_CHECK] check database connection ... OK31[CONFIG_CHECK] expand_ip[192.168.6.98] is not used in the cluster ...32[CONFIG_CHECK] expand_ip[192.168.6.98] is not used in the cluster ...ok33[CONFIG_CHECK] The localhost is expand_ip:[192.168.6.98] ...34[CONFIG_CHECK] The localhost is expand_ip:[192.168.6.98] ...ok35[CONFIG_CHECK] check node_id is in cluster ... 36[CONFIG_CHECK] check node_id is in cluster ...OK37[RUNNING] check the db is running or not...38[RUNNING] the db is not running on "192.168.6.98:54321" ..... OK39[RUNNING] the install dir is not exist on "192.168.6.98" ..... OK40[RUNNING] check the sys_securecmdd is running or not...41[RUNNING] the sys_securecmdd is not running on "192.168.6.98:8890" ..... OK42 [INFO] use_ssl=0432025-02-26 17:27:36 [INFO] start to check system parameters on 192.168.6.98 ...442025-02-26 17:27:36 [WARNING] [GSSAPIAuthentication] yes (should be: no) on 192.168.6.98452025-02-26 17:27:36 [INFO] [UseDNS] is null on 192.168.6.98462025-02-26 17:27:37 [INFO] [UsePAM] yes on 192.168.6.98472025-02-26 17:27:37 [INFO] [ulimit.open files] 65536 on 192.168.6.98482025-02-26 17:27:37 [INFO] [ulimit.open proc] 65536 on 192.168.6.98492025-02-26 17:27:38 [INFO] [ulimit.core size] unlimited on 192.168.6.98502025-02-26 17:27:38 [INFO] [ulimit.mem lock] 64 (less than 50000000) on 192.168.6.98512025-02-26 17:27:38 [INFO] the value of [ulimit.mem lock] is wrong, now will change it on 192.168.6.98 ...522025-02-26 17:27:38 [INFO] change ulimit.mem lock on 192.168.6.98 ...532025-02-26 17:27:39 [INFO] change ulimit.mem lock on 192.168.6.98 ... Done542025-02-26 17:27:39 [INFO] [ulimit.mem lock] 50000000 on 192.168.6.98552025-02-26 17:27:40 [ERROR] [kernel.sem] 250 32000 100 128 (no less than: 5010 641280 5010 256) on 192.168.6.98562025-02-26 17:27:40 [INFO] the value of [kernel.sem] is wrong, now will change it on 192.168.6.98 ...572025-02-26 17:27:40 [INFO] change kernel.sem on 192.168.6.98 ...582025-02-26 17:27:41 [INFO] change kernel.sem on 192.168.6.98 ... Done592025-02-26 17:27:42 [INFO] [kernel.sem] 5010 641280 5010 256 on 192.168.6.98602025-02-26 17:27:43 [INFO] [RemoveIPC] no on 192.168.6.98612025-02-26 17:27:43 [INFO] [DefaultTasksAccounting] is null on 192.168.6.98622025-02-26 17:27:43 [INFO] write file "/etc/udev/rules.d/kingbase.rules" on 192.168.6.98632025-02-26 17:27:45 [INFO] [crontab] chmod /usr/bin/crontab ...642025-02-26 17:27:45 [INFO] [crontab] chmod /usr/bin/crontab ... Done652025-02-26 17:27:45 [INFO] [crontab access] OK662025-02-26 17:27:46 [INFO] [cron.allow] add kingbase to cron.allow ...672025-02-26 17:27:46 [INFO] [cron.allow] add kingbase to cron.allow ... Done682025-02-26 17:27:47 [INFO] [crontab auth] crontab is accessible by kingbase now on 192.168.6.98692025-02-26 17:27:47 [INFO] [SELINUX] disabled on 192.168.6.98702025-02-26 17:27:48 [INFO] [firewall] down on 192.168.6.98712025-02-26 17:27:48 [INFO] [The memory] OK on 192.168.6.98722025-02-26 17:27:49 [INFO] [The hard disk] OK on 192.168.6.98732025-02-26 17:27:49 [INFO] [ping] chmod /bin/ping ...742025-02-26 17:27:49 [INFO] [ping] chmod /bin/ping ... Done752025-02-26 17:27:50 [INFO] [ping access] OK762025-02-26 17:27:50 [INFO] [/bin/cp --version] on 192.168.6.98 OK772025-02-26 17:27:50 [INFO] [Virtual IP] Not configured on 192.168.6.9878[INSTALL] create the install dir "/KingbaseES/V9/cluster/kingbase" on 192.168.6.98 ...79[INSTALL] success to create the install dir "/KingbaseES/V9/cluster/kingbase" on "192.168.6.98" ..... OK80[INSTALL] try to copy the zip package "/install/db.zip" to /KingbaseES/V9/cluster/kingbase of "192.168.6.98" .....81[INSTALL] success to scp the zip package "/install/db.zip" /KingbaseES/V9/cluster/kingbase of to "192.168.6.98" ..... OK82[INSTALL] decompress the "/KingbaseES/V9/cluster/kingbase" to "/KingbaseES/V9/cluster/kingbase" on 192.168.6.9883[INSTALL] success to decompress the "/KingbaseES/V9/cluster/kingbase/db.zip" to "/KingbaseES/V9/cluster/kingbase" on "192.168.6.98"..... OK84[INSTALL] check license_file "default"85[INSTALL] success to access license_file on 192.168.6.98: /KingbaseES/V9/cluster/kingbase/bin/license.dat86[RUNNING] config sys_securecmdd and start it ...87[RUNNING] config the sys_securecmdd port to 8890 ...88[RUNNING] success to config the sys_securecmdd port on 192.168.6.98 ... OK89successfully initialized the sys_securecmdd, please use "/KingbaseES/V9/cluster/kingbase/bin/sys_HAscmdd.sh start" to start the sys_securecmdd90[RUNNING] success to config sys_securecmdd on 192.168.6.98 ... OK91Created symlink /etc/systemd/system/multi-user.target.wants/securecmdd.service → /etc/systemd/system/securecmdd.service.92[RUNNING] success to start sys_securecmdd on 192.168.6.98 ... OK93[INSTALL] success to access file: /KingbaseES/V9/cluster/kingbase/etc/all_nodes_tools.conf94[INSTALL] success to scp the /KingbaseES/V9/cluster/kingbase/etc/repmgr.conf from 192.168.6.87 to "192.168.6.98"..... ok95[INSTALL] success to scp the ~/.encpwd from 192.168.6.87 to "192.168.6.98"..... ok96[INSTALL] success to scp /KingbaseES/V9/cluster/kingbase/etc/all_nodes_tools.conf from "192.168.6.87" to "192.168.6.98" ...ok97[INSTALL] success to chmod 600 the ~/.encpwd on 192.168.6.98..... ok98 [INFO] parameter_name=node_id99 [INFO] parameter_values='3'100 [INFO] [parameter_name] para_exist=1101 [INFO] sed -i "/[#]*node_id[ ]*=/cnode_id='3'" /KingbaseES/V9/cluster/kingbase/etc/repmgr.conf102 [INFO] parameter_name=node_name103 [INFO] parameter_values='node3'104 [INFO] [parameter_name] para_exist=1105 [INFO] sed -i "/[#]*node_name[ ]*=/cnode_name='node3'" /KingbaseES/V9/cluster/kingbase/etc/repmgr.conf106 [INFO] parameter_name=conninfo107 [INFO] parameter_values='host108 [INFO] [parameter_name] para_exist=1109 [INFO] sed -i "/[#]*conninfo[ ]*=/cconninfo='host=192.168.6.98 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=9000'" /KingbaseES/V9/cluster/kingbase/etc/repmgr.conf110 [INFO] parameter_name=ping_path111 [INFO] parameter_values='/bin'112 [INFO] [parameter_name] para_exist=1113 [INFO] sed -i "/[#]*ping_path[ ]*=/cping_path='/bin'" /KingbaseES/V9/cluster/kingbase/etc/repmgr.conf114[RUNNING] standby clone ...115[WARNING] following problems with command line parameters detected:116 -D/--sysdata will be ignored if a repmgr configuration file is provided117[NOTICE] destination directory "/data" provided118[INFO] connecting to source node119[DETAIL] connection string is: host=192.168.6.87 user=esrep port=54321 dbname=esrep120[DETAIL] current installation size is 102 MB121[NOTICE] checking for available walsenders on the source node (2 required)122[NOTICE] checking replication connections can be made to the source server (2 required)123[INFO] checking and correcting permissions on existing directory "/data"124[INFO] creating replication slot as user "esrep"125[NOTICE] starting backup (using sys_basebackup)...126[INFO] executing:127 /KingbaseES/V9/cluster/kingbase/bin/sys_basebackup -l "repmgr base backup" -D /data -h 192.168.6.87 -p 54321 -U esrep -c fast -X stream -S repmgr_slot_3 128[NOTICE] standby clone (using sys_basebackup) complete129[NOTICE] you can now start your Kingbase server130[HINT] for example: sys_ctl -D /data start131[HINT] after starting the server, you need to register this standby with "repmgr standby register"132[RUNNING] standby clone ...OK133[RUNNING] db start ...134waiting for server to start.... done135server started136[RUNNING] db start ...OK137[INFO] connecting to local node "node3" (ID: 3)138[INFO] connecting to primary database139[WARNING] --upstream-node-id not supplied, assuming upstream node is primary (node ID: 1)140[INFO] standby registration complete141[NOTICE] standby node "node3" (ID: 3) successfully registered1422025-02-26 17:28:24 begin to start DB on "[localhost]".1432025-02-26 17:28:25 DB on "[localhost]" already started, connect to check it.1442025-02-26 17:28:26 DB on "[localhost]" start success.1452025-02-26 17:28:26 Ready to start local kbha daemon and repmgrd daemon ...1462025-02-26 17:28:26 begin to start repmgrd on "[localhost]".147[2025-02-26 17:28:27] [NOTICE] using provided configuration file "/KingbaseES/V9/cluster/kingbase/bin/../etc/repmgr.conf"148[2025-02-26 17:28:27] [INFO] creating directory "/KingbaseES/V9/cluster/kingbase/log"...149[2025-02-26 17:28:27] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/hamgr.log"1501512025-02-26 17:28:29 repmgrd on "[localhost]" start success.152[2025-02-26 17:28:32] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/kbha.log"1531542025-02-26 17:28:34 Done.155 ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string 156----+-------+---------+-----------+----------+----------+----------+----------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------157 1 | node1 | primary | * running | | default | 100 | 1 | | host=192.168.6.87 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=9000158 2 | node2 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.6.88 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=9000159 3 | node3 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.6.98 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=9000160[RUNNING] query archive command at 192.168.6.87 ...161[RUNNING] current cluster not config sys_rman,return.这样就代表已经扩容成功。
在集群主节点执行检查：
1[kingbase@kesrwc01:/home/kingbase]$ repmgr cluster show2 ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string 3----+-------+---------+-----------+----------+----------+----------+----------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------4 1 | node1 | primary | * running | | default | 100 | 1 | | host=192.168.6.87 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=90005 2 | node2 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.6.88 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=90006 3 | node3 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.6.98 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=90007[kingbase@kesrwc01:/home/kingbase]$ repmgr service status8 ID | Name | Role | Status | Upstream | repmgrd | PID | Paused? | Upstream last seen9----+-------+---------+-----------+----------+---------+-------+---------+--------------------10 1 | node1 | primary | * running | | running | 4639 | no | n/a 11 2 | node2 | standby | running | node1 | running | 5049 | no | 0 second(s) ago 12 3 | node3 | standby | running | node1 | running | 36501 | no | 1 second(s) ago确认新扩容节点已正确加入集群，集群状态正常。