在知识库中专注阅读，并随时返回相关工具与课程
大家好，这里是公众号 DBA学习之路，分享一些学习国产数据库路上的知识和经验。 @TOC目录 最近在研究金仓数据库的读写分离集群（KingbaseRWC），看着大家有写过不少图形化部署的教程了，我打算试一下静默安装的方式，本文记录一下部署过程！ PS：看着跟 Oracle ADG 有点相似。 KingbaseES 读写分离集群（KingbaseRWC）： KingbaseES 读写分离集群通过物理
大家好，这里是公众号 DBA学习之路，分享一些学习国产数据库路上的知识和经验。
本章目标：掌握本章核心知识点 前置要求：完成前序章节学习 预计时长：60 分钟
最近在研究金仓数据库的读写分离集群（KingbaseRWC），看着大家有写过不少图形化部署的教程了，我打算试一下静默安装的方式，本文记录一下部署过程！
PS：看着跟 Oracle ADG 有点相似。
KingbaseES 读写分离集群（KingbaseRWC）： KingbaseES 读写分离集群通过物理复制保障企业数据的高可用性、数据保护和灾难恢复并具备读请求的负载均衡能力。相比数据守护集群，该类集群中所有备库均可对外提供查询能力，从而减轻了主库的读负载压力，可实现更高的事务吞吐率；该软件支持在多个备库间进行读负载均衡。
KingbaseRWC 读写分离集群架构：
其成员可能包括主节点（primary node）、备节点（standby node）、辅助节点（witness node）、备份节点（repo node）。
KingbaseRWC 功能特点：
官方推荐配置如下：
本文演示部署一套 1 主 1 备的 KingbaseRWC 集群。
本文演示环境：
检查操作系统版本信息：
1## 以节点一为例2[root@kesrwc01 ~]# cat /etc/os-release3NAME="Kylin Linux Advanced Server"4VERSION="V10 (Halberd)"5ID="kylin"6VERSION_ID="V10"7PRETTY_NAME="Kylin Linux Advanced Server V10 (Halberd)"8ANSI_COLOR="0;31"建议配置主机名解析：
1## 以节点一为例2[root@kesrwc01 ~]# cat<<-EOF>>/etc/hosts3192.168.6.87 kesrwc014192.168.6.88 kesrwc025EOF数据库安装均建议关闭防火墙：
1## 以节点一为例2[root@kesrwc01 ~]# systemctl stop firewalld3[root@kesrwc01 ~]# systemctl disable firewalld4Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.5Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.所有节点建议关闭 SELNUX，防止造成一些不必要的问题：
1## 以节点一为例2## 这里使用 setenforce 0 临时生效3[root@kesrwc01 ~]# /usr/sbin/setenforce 04/usr/sbin/setenforce: SELinux is disabled5[root@kesrwc01 ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config6[root@kesrwc01 ~]# sestatus7SELinux status: disabled将 SELINUX 参数设置为 disabled，即 SELINUX=disabled 保存退出后，需要重新启动才能生效。
建议在所有服务器上创建 KES 产品的安装用户 kingbase，而非使用 root 身份执行安装部署：
1## 以节点一为例2[root@kesrwc01 ~]# useradd -u 2000 -d /home/kingbase -m kingbase3[root@kesrwc01 ~]# echo "kingbase:kingbase" | chpasswd4[root@kesrwc01 ~]# id kingbase5用户id=2000(kingbase) 组id=2000(kingbase) 组=2000(kingbase)如果有密码复杂性要求，不想设置密码太复杂的话，可以取消密码复杂度：
1## 以节点一为例2[root@kesrwc01 ~]# sed -i 's/^password\+[[:space:]]\+requisite[[:space:]]\+pam_pwquality.so/#&/g' /etc/pam.d/system-auth3[root@kesrwc01 ~]# sed -i 's/use_authtok$//' /etc/pam.d/system-auth为了利于数据库的日常运维、持续使用、存储扩容等，我们在安装前必须做好选项、存储目录规划。
创建安装所需目录并且授权：
1## 以节点一为例2[root@kesrwc01 ~]# mkdir -p /KingbaseES/V9 /backup /archive /install /data3[root@kesrwc01 ~]# chown -R kingbase:kingbase {/KingbaseES,/backup,/archive,/install,/data}4[root@kesrwc01 ~]# chmod -R 775 {/KingbaseES,/backup,/archive,/install,/data}为了避免在 KingbaseES 安装和使用过程中出现问题，官方建议调整系统内核参数：
注意：这里关于 shmmax 和 shmall 参数值的计算，我们可以参考 Oracle 来设置：
1# 物理内存（KB）2os_memory_total=$(awk '/MemTotal/{print $2}' /proc/meminfo)3# 获取系统页面大小，用于计算内存总量4pagesize=$(getconf PAGE_SIZE)5((shmall = (os_memory_total - 1) * 1024 / pagesize))6((shmmax = os_memory_total * 1024 - 10))7# 如果 shmall 小于 2097152，则将其设为 20971528((shmall < 2097152)) && shmall=20971529# 如果 shmmax 小于 4294967295，则将其设为 429496729510((shmmax < 4294967295)) && shmmax=4294967295复制以上命令，直接执行即可计算得出这两个参数值：
1## 以节点一为例2[root@kesrwc01 ~]# echo $shmall320971524[root@kesrwc01 ~]# echo $shmmax57008071670根据官方建议值，配置系统参数文件：
1## 以节点一为例2[root@kesrwc01 ~]# cat<<-EOF>>/etc/sysctl.conf3kernel.shmall= 20971524kernel.shmmax= 70080716705kernel.sem= 5010 641280 5010 2566net.core.rmem_default= 83886087net.core.rmem_max= 167772168net.core.wmem_default= 83886089net.core.wmem_max= 1677721610net.ipv4.tcp_rmem = 8192 65536 1677721611net.ipv4.tcp_wmem = 8192 87380 1677721612net.core.default_qdisc=fq_codel13EOF1415## 生效配置16[root@kesrwc01 ~]# sysctl -p限制用户可使用的资源数量对系统的稳定性非常重要，可以通过调整资源限制数量改进系统性能：
1## 以节点一为例2[root@kesrwc01 ~]# cat<<-EOF>>/etc/security/limits.conf3* soft nofile 6553604* hard nofile 6553605* soft nproc 6553606* hard nproc 6553607* soft core 6553608* hard core 6553609* soft memlock 5000000010* hard memlock 5000000011EOFsystemd-logind 服务中引入的一个特性 RemoveIPC，会造成程序信号丢失等问题，只有Redhat7 及以上和一些特殊的国产Linux的版本需要修改，改之前可先查看此项是否为默认yes)，需要设置 RemoveIPC=no：
1## 以节点一为例2[root@kesrwc01 ~]# sed -i 's/#RemoveIPC=no/RemoveIPC=no/' /etc/systemd/logind.conf3[root@kesrwc01 ~]# grep RemoveIPC /etc/systemd/logind.conf4RemoveIPC=no5[root@kesrwc01 ~]# sed -i 's/#DefaultTasksAccounting=yes/DefaultTasksAccounting=no/' /etc/systemd/system.conf6[root@kesrwc01 ~]# sed -i 's/#DefaultTasksMax=80%/DefaultTasksMax= 65536/' /etc/systemd/system.conf7[root@kesrwc01 ~]# cat /etc/systemd/system.conf | grep DefaultTask8DefaultTasksAccounting=no9DefaultTasksMax= 6553610# 重新加载 systemd 守护进程并重启 systemd-logind 服务生效11[root@kesrwc01 ~]# systemctl daemon-reload12[root@kesrwc01 ~]# systemctl restart systemd-logind.serviceKES 安装对于 /tmp 目录有一定要求，至少需要 10G 空间，否则安装时会报警并使用 kingbase 用户家目录作为替代：
1Now launch installer...2正在准备进行安装3警告:/tmp 磁盘空间不足！正在尝试将 /home/kingbase 用于安装基础和 tmp dir。手动挂载 /tmp 目录空间：
1## 以节点一为例2[root@kesrwc01 ~]# cat<<-EOF>>/etc/fstab3tmpfs /tmp tmpfs size=10G 0 04EOF5[root@kesrwc01 ~]# mount -o remount /tmp6[root@kesrwc01 ~]# df -h | grep /tmp7tmpfs 10G 16K 10G 1% /tmp访问数据库需要使用 kingbase 主机用户，（为了方便）配置 kingbase 用户环境变量：
1## 以节点一为例2[kingbase@kesrwc01 ~]$ cat<<-\EOF>>/home/kingbase/.bash_profile3export KINGBASE_HOME=/KingbaseES/V9/cluster/kingbase4export KINGBASE_DATA=/data5export LD_LIBRARY_PATH=$KINGBASE_HOME/lib:/lib:/usr/lib:/usr/lib646export PATH=$KINGBASE_HOME/bin:/usr/sbin:$PATH7export PS1="[`whoami`@`hostname`:"'$PWD]$ '8alias ksystem='ksql test system'9alias ksso='ksql test sso'10## 解决 clear 命令无法使用问题11export TERMINFO=/usr/share/terminfo12export TERM=vt10013EOF两个节点都需要配置，配置完成后生效环境变量：
1[kingbase@kesrwc01 ~]$ source ~/.bash_profile配置好环境变量之后就可以不需要到指定路径下执行命令了。
这个不是必须配置，但是建议配置，用于在系统中配置 NTP 实现时间同步。
⭐️ 配置 Linux 时间同步可以参考为之前写的文章：Linux 一键配置时钟同步全攻略
安装 KingbaseRWC 之前需要在主节点安装 KingbaseES 数据库软件。
在主节点都上传 KES 最新的安装包：
KingbaseES_V009R001C002B0014_Lin64_install.iso到 /install 目录下，直接官网下载即可！
以节点一为例：
1## 挂载 iso 文件需要使用 root 用户，安装包上传在 /install 目录，挂载到 /mnt 目录下2[root@kesrwc01 ~]# umount /mnt3[root@kesrwc01 ~]# mount -o loop /install/KingbaseES_V009R001C002B0014_Lin64_install.iso /mnt/4## 将挂载出来的安装文件拷贝到 /install 目录下5[root@kesrwc01 ~]# cp -r /mnt/* /install6## 复制完成后取消安装 iso 的挂载7[root@kesrwc01 ~]# umount /mnt8## 由于之前 root 复制的安装文件权限为 root，所以需要重新授权 kingbase 用户9[root@kesrwc01 ~]# chown -R kingbase:kingbase /install10[root@kesrwc01 ~]# chmod -R 775 /install11[root@kesrwc01 ~]# su - kingbase12[kingbase@kesrwc01 ~]$ cd /install/setup13## 修改 silent.cfg 模板文件14[kingbase@kesrwc01:/install/setup]$ cat<<-EOF>/install/setup/silent.cfg15KB_LICENSE_PATH=16CHOSEN_INSTALL_SET=Custom17CHOSEN_FEATURE_LIST=DEPLOY18USER_INSTALL_DIR=/KingbaseES/V919USER_SELECTED_DATA_FOLDER=/data20DB_PORT=5432121DB_USER=system22DB_PASS=23DB_PASS2=24ENCODING_PARAM=UTF825LOCALE_PARAM=zh_CN.UTF-826INITCUSTOM=27DATABASE_MODE_PARAM=ORACLE28CASE_SENSITIVE_PARAM=YES29BLOCK_SIZE_PARAM=8k30AUTHENTICATION_METHOD_PARAM=scram-sha-25631EOF32## 静默安装33[kingbase@kesrwc01:/install/setup]$ cd /install34[kingbase@kesrwc01:/install]$ ./setup.sh -i silent -f /install/setup/silent.cfg35Now launch installer...36 Verifying JVM...Complete.安装成功后，在 ${USER_INSTALL_DIR}/ClientTools/guitools/DeployTools/zip/ 目录下拷贝安装文件到 /install 目录下：
1[kingbase@kesrwc01 ~]$ cd /KingbaseES/V9/ClientTools/guitools/DeployTools/zip/2[kingbase@kesrwc01 zip]$ ls3cluster_install.sh db.zip install.conf securecmdd.zip trust_cluster.sh4[kingbase@kesrwc01 zip]$ cp * /install/接下来就可以部署 KingbaseRWC 集群了。
静默安装需要先配置 install.conf 文件：
1[kingbase@kesrwc01 install]$ vi install.conf2## 修改以下几个必要参数，具体参数请参考官方文档，根据实际情况进行修改34all_ip=(192.168.6.87 192.168.6.88)5install_dir="/KingbaseES/V9/cluster/"6zip_package="/install/db.zip"7data_directory="/data"8trusted_servers="192.168.6.254"静默安装需要配置免密，使用 root 用户执行 trust_cluster.sh 脚本即可：
1[root@kesrwc01 ~]# cd /install/2[root@kesrwc01 install]# sh trust_cluster.sh 3[INFO] set password-free between root and kingbase4Generating public/private rsa key pair.5...6...7...89## 过程中根据提示输入密码即可10root@192.168.6.88's password: 11id_rsa 100% 2602 167.8KB/s 00:00 12id_rsa.pub 100% 567 70.4KB/s 00:00 13authorized_keys 100% 567 74.1KB/s 00:00 14known_hosts 1516...17...18...19connect to "192.168.6.87" from current node by 'ssh' kingbase:0..... OK20connect to "192.168.6.87" from current node by 'ssh' root:0..... OK21connect to "192.168.6.88" from "192.168.6.87" by 'ssh' kingbase->kingbase:0 .... OK22connect to "192.168.6.88" from "192.168.6.87" by 'ssh' root->root:0 root->kingbase:0 kingbase->root:0.... OK23connect to "192.168.6.88" from current node by 'ssh' kingbase:0..... OK24connect to "192.168.6.88" from current node by 'ssh' root:0..... OK25connect to "192.168.6.87" from "192.168.6.88" by 'ssh' kingbase->kingbase:0 .... OK26connect to "192.168.6.87" from "192.168.6.88" by 'ssh' root->root:0 root->kingbase:0 kingbase->root:0.... OK27check ssh connection success!切换到 kingbase 用户，一键静默安装：
1[kingbase@kesrwc01 ~]$ cd /install/2[kingbase@kesrwc01 install]$ sh cluster_install.sh3[CONFIG_CHECK] will deploy the cluster of DG4[CONFIG_CHECK] file format is correct ... OK5[CONFIG_CHECK] encoding: UTF8 OK6[CONFIG_CHECK] locale: zh_CN.UTF-8 OK7[CONFIG_CHECK] the number of license_num matches the length of all_ip or the number of license_num is 1 ... OK8[RUNNING] check if the host can be reached from current node and between all nodes by ssh ...9[RUNNING] success connect to "192.168.6.87" from current node by 'ssh' ... OK10[RUNNING] success connect to "192.168.6.87" from "192.168.6.87" by 'ssh' ... OK11[RUNNING] success connect to "192.168.6.88" from "192.168.6.87" by 'ssh' ... OK12[RUNNING] success connect to "192.168.6.88" from current node by 'ssh' ... OK13[RUNNING] success connect to "192.168.6.87" from "192.168.6.88" by 'ssh' ... OK14[RUNNING] success connect to "192.168.6.88" from "192.168.6.88" by 'ssh' ... OK15[RUNNING] chmod /bin/ping ...16[RUNNING] chmod /bin/ping ... Done17[RUNNING] ping access rights OK18[RUNNING] check the db is running or not...19[RUNNING] the db is not running on "192.168.6.87:54321" ..... OK20[RUNNING] the db is not running on "192.168.6.88:54321" ..... OK21[RUNNING] check the sys_securecmdd is running or not...22[RUNNING] the sys_securecmdd is not running on "192.168.6.87:8890" ..... OK23[RUNNING] the sys_securecmdd is not running on "192.168.6.88:8890" ..... OK24[RUNNING] check if the install dir (create dir and check it's owner/permission) ...25[RUNNING] check if the install dir (create dir and check it's owner/permission) on "192.168.6.87" ... OK26[RUNNING] check if the install dir (create dir and check it's owner/permission) on "192.168.6.88" ... OK27[RUNNING] check if the dir "/KingbaseES/V9/cluster/kingbase" is already exist ...28[RUNNING] the dir "/KingbaseES/V9/cluster/kingbase" is not exist on "192.168.6.87" ..... OK29[RUNNING] the dir "/KingbaseES/V9/cluster/kingbase" is not exist on "192.168.6.88" ..... OK30[RUNNING] check the data directory (create it and check whether it is empty) ...31[RUNNING] when use_exist_data=0, create the empty data directory on "192.168.6.87" ..... OK32[RUNNING] when use_exist_data=0, create the empty data directory on "192.168.6.88" ..... OK332025-02-08 06:58:14 [INFO] start to check system parameters on 192.168.6.87 ...342025-02-08 06:58:14 [WARNING] [GSSAPIAuthentication] yes (should be: no) on 192.168.6.87352025-02-08 06:58:15 [INFO] [UseDNS] is null on 192.168.6.87362025-02-08 06:58:15 [INFO] [UsePAM] yes on 192.168.6.87372025-02-08 06:58:15 [INFO] [ulimit.open files] 655360 on 192.168.6.87382025-02-08 06:58:16 [INFO] [ulimit.open proc] 655360 on 192.168.6.87392025-02-08 06:58:16 [INFO] [ulimit.core size] 655360 (no unlimited) on 192.168.6.87402025-02-08 06:58:16 [INFO] the value of [ulimit.core size] is wrong, now will change it on 192.168.6.87 ...412025-02-08 06:58:16 [INFO] change ulimit.core size on 192.168.6.87 ...422025-02-08 06:58:17 [INFO] change ulimit.core size on 192.168.6.87 ... Done432025-02-08 06:58:17 [INFO] [ulimit.core size] unlimited on 192.168.6.87442025-02-08 06:58:17 [INFO] [ulimit.mem lock] 50000000 on 192.168.6.87452025-02-08 06:58:19 [INFO] [kernel.sem] 5010 641280 5010 256 on 192.168.6.87462025-02-08 06:58:19 [INFO] [RemoveIPC] no on 192.168.6.87472025-02-08 06:58:19 [INFO] [DefaultTasksAccounting] no on 192.168.6.87482025-02-08 06:58:20 [INFO] write file "/etc/udev/rules.d/kingbase.rules" on 192.168.6.87492025-02-08 06:58:21 [INFO] [crontab] chmod /usr/bin/crontab ...502025-02-08 06:58:22 [INFO] [crontab] chmod /usr/bin/crontab ... Done512025-02-08 06:58:22 [INFO] [crontab access] OK522025-02-08 06:58:23 [INFO] [cron.allow] add kingbase to cron.allow ...532025-02-08 06:58:23 [INFO] [cron.allow] add kingbase to cron.allow ... Done542025-02-08 06:58:24 [INFO] [crontab auth] crontab is accessible by kingbase now on 192.168.6.87552025-02-08 06:58:24 [INFO] [SELINUX] disabled on 192.168.6.87562025-02-08 06:58:25 [INFO] [firewall] down on 192.168.6.87572025-02-08 06:58:26 [INFO] [The memory] OK on 192.168.6.87582025-02-08 06:58:26 [INFO] [The hard disk] OK on 192.168.6.87592025-02-08 06:58:26 [INFO] [ping] chmod /bin/ping ...602025-02-08 06:58:27 [INFO] [ping] chmod /bin/ping ... Done612025-02-08 06:58:27 [INFO] [ping access] OK622025-02-08 06:58:27 [INFO] [/bin/cp --version] on 192.168.6.87 OK632025-02-08 06:58:27 [INFO] [Virtual IP] Not configured on 192.168.6.87642025-02-08 06:58:27 [INFO] start to check system parameters on 192.168.6.88 ...652025-02-08 06:58:28 [WARNING] [GSSAPIAuthentication] yes (should be: no) on 192.168.6.88662025-02-08 06:58:28 [INFO] [UseDNS] is null on 192.168.6.88672025-02-08 06:58:28 [INFO] [UsePAM] yes on 192.168.6.88682025-02-08 06:58:29 [INFO] [ulimit.open files] 655360 on 192.168.6.88692025-02-08 06:58:29 [INFO] [ulimit.open proc] 655360 on 192.168.6.88702025-02-08 06:58:29 [INFO] [ulimit.core size] 655360 (no unlimited) on 192.168.6.88712025-02-08 06:58:29 [INFO] the value of [ulimit.core size] is wrong, now will change it on 192.168.6.88 ...722025-02-08 06:58:29 [INFO] change ulimit.core size on 192.168.6.88 ...732025-02-08 06:58:30 [INFO] change ulimit.core size on 192.168.6.88 ... Done742025-02-08 06:58:30 [INFO] [ulimit.core size] unlimited on 192.168.6.88752025-02-08 06:58:31 [INFO] [ulimit.mem lock] 50000000 on 192.168.6.88762025-02-08 06:58:32 [INFO] [kernel.sem] 5010 641280 5010 256 on 192.168.6.88772025-02-08 06:58:32 [INFO] [RemoveIPC] no on 192.168.6.88782025-02-08 06:58:32 [INFO] [DefaultTasksAccounting] no on 192.168.6.88792025-02-08 06:58:33 [INFO] write file "/etc/udev/rules.d/kingbase.rules" on 192.168.6.88802025-02-08 06:58:34 [INFO] [crontab] chmod /usr/bin/crontab ...812025-02-08 06:58:34 [INFO] [crontab] chmod /usr/bin/crontab ... Done822025-02-08 06:58:35 [INFO] [crontab access] OK832025-02-08 06:58:36 [INFO] [cron.allow] add kingbase to cron.allow ...842025-02-08 06:58:36 [INFO] [cron.allow] add kingbase to cron.allow ... Done852025-02-08 06:58:36 [INFO] [crontab auth] crontab is accessible by kingbase now on 192.168.6.88862025-02-08 06:58:37 [INFO] [SELINUX] disabled on 192.168.6.88872025-02-08 06:58:37 [INFO] [firewall] down on 192.168.6.88882025-02-08 06:58:38 [INFO] [The memory] OK on 192.168.6.88892025-02-08 06:58:38 [INFO] [The hard disk] OK on 192.168.6.88902025-02-08 06:58:38 [INFO] [ping] chmod /bin/ping ...912025-02-08 06:58:39 [INFO] [ping] chmod /bin/ping ... Done922025-02-08 06:58:39 [INFO] [ping access] OK932025-02-08 06:58:39 [INFO] [/bin/cp --version] on 192.168.6.88 OK942025-02-08 06:58:39 [INFO] [Virtual IP] Not configured on 192.168.6.8895[INSTALL] create the install dir "/KingbaseES/V9/cluster/kingbase" on every host ...96[INSTALL] success to create the install dir "/KingbaseES/V9/cluster/kingbase" on "192.168.6.87" ..... OK97[INSTALL] success to create the install dir "/KingbaseES/V9/cluster/kingbase" on "192.168.6.88" ..... OK98[INSTALL] success to access the zip_package "/install/db.zip" on "192.168.6.87" ..... OK99[INSTALL] decompress the "/install/db.zip" to "/KingbaseES/V9/cluster/kingbase/__tmp_decompress__"100[INSTALL] success to recreate the tmp dir "/KingbaseES/V9/cluster/kingbase/__tmp_decompress__" on "192.168.6.87" ..... OK101[INSTALL] success to decompress the "/install/db.zip" to "/KingbaseES/V9/cluster/kingbase/__tmp_decompress__" on "192.168.6.87"..... OK102[INSTALL] scp the dir "/KingbaseES/V9/cluster/kingbase/__tmp_decompress__" to "/KingbaseES/V9/cluster/kingbase" on all host103[INSTALL] try to copy the install dir "/KingbaseES/V9/cluster/kingbase" to "192.168.6.87" .....104[INSTALL] success to scp the install dir "/KingbaseES/V9/cluster/kingbase" to "192.168.6.87" ..... OK105[INSTALL] try to copy the install dir "/KingbaseES/V9/cluster/kingbase" to "192.168.6.88" .....106[INSTALL] success to scp the install dir "/KingbaseES/V9/cluster/kingbase" to "192.168.6.88" ..... OK107[INSTALL] remove the dir "/KingbaseES/V9/cluster/kingbase/__tmp_decompress__"108[INSTALL] change the auth of bin directory on 192.168.6.87 ...109[INSTALL] change the auth of bin directory on 192.168.6.88 ...110[INSTALL] check license_file ...111[INSTALL] success to access license_file on 192.168.6.87: /KingbaseES/V9/cluster/kingbase/bin/license.dat112[INSTALL] check license_file ...113[INSTALL] success to access license_file on 192.168.6.88: /KingbaseES/V9/cluster/kingbase/bin/license.dat114[INSTALL] set the archive_command to "exit 0" and the archive dir is NULL115[INSTALL] the archive dir is NULL, not do archive ...116[INSTALL] create the dir "etc" "log" on all host117[RUNNING] config sys_securecmdd and start it ...118[RUNNING] config the sys_securecmdd port to 8890 ...119[RUNNING] success to config the sys_securecmdd port on 192.168.6.87 ... OK120successfully initialized the sys_securecmdd, please use "/KingbaseES/V9/cluster/kingbase/bin/sys_HAscmdd.sh start" to start the sys_securecmdd121[RUNNING] success to config sys_securecmdd on 192.168.6.87 ... OK122Created symlink /etc/systemd/system/multi-user.target.wants/securecmdd.service → /etc/systemd/system/securecmdd.service.123[RUNNING] success to start sys_securecmdd on 192.168.6.87 ... OK124[RUNNING] config sys_securecmdd and start it ...125[RUNNING] config the sys_securecmdd port to 8890 ...126[RUNNING] success to config the sys_securecmdd port on 192.168.6.88 ... OK127successfully initialized the sys_securecmdd, please use "/KingbaseES/V9/cluster/kingbase/bin/sys_HAscmdd.sh start" to start the sys_securecmdd128[RUNNING] success to config sys_securecmdd on 192.168.6.88 ... OK129Created symlink /etc/systemd/system/multi-user.target.wants/securecmdd.service → /etc/systemd/system/securecmdd.service.130[RUNNING] success to start sys_securecmdd on 192.168.6.88 ... OK131[RUNNING] check if the host can be reached between all nodes by scmd ...132[RUNNING] success connect to "192.168.6.87" from "192.168.6.87" by '/KingbaseES/V9/cluster/kingbase/bin/sys_securecmd' ... OK133[RUNNING] success connect to "192.168.6.88" from "192.168.6.87" by '/KingbaseES/V9/cluster/kingbase/bin/sys_securecmd' ... OK134[RUNNING] success connect to "192.168.6.87" from "192.168.6.88" by '/KingbaseES/V9/cluster/kingbase/bin/sys_securecmd' ... OK135[RUNNING] success connect to "192.168.6.88" from "192.168.6.88" by '/KingbaseES/V9/cluster/kingbase/bin/sys_securecmd' ... OK136[INSTALL] begin to init the database on "192.168.6.87" ...137The database cluster will be initialized with locales138 COLLATE: zh_CN.UTF-8139 CTYPE: zh_CN.UTF-8140 MESSAGES: C141 MONETARY: zh_CN.UTF-8142 NUMERIC: zh_CN.UTF-8143 TIME: zh_CN.UTF-8144The files belonging to this database system will be owned by user "kingbase".145This user must also own the server process.146147The default text search configuration will be set to "simple".148149The comparision of strings is case-sensitive.150Data page checksums are enabled.151152initdb: could not find suitable text search configuration for locale "zh_CN.UTF-8"153fixing permissions on existing directory /data ... ok154creating subdirectories ... ok155selecting dynamic shared memory implementation ... posix156selecting default max_connections ... 100157selecting default shared_buffers ... 128MB158selecting default time zone ... Asia/Shanghai159creating configuration files ... ok160Begin setup encrypt device161initializing the encrypt device ... ok162running bootstrap script ... ok163performing post-bootstrap initialization ... ok164create security database ... ok165load security database ... ok166syncing data to disk ... ok167168Success. You can now start the database server using:169170 /KingbaseES/V9/cluster/kingbase/bin/sys_ctl -D /data -l logfile start171172[INSTALL] end to init the database on "192.168.6.87" ... OK173[INSTALL] wirte the kingbase.conf on "192.168.6.87" ...174[INSTALL] wirte the kingbase.conf on "192.168.6.87" ... OK175[INSTALL] wirte the es_rep.conf on "192.168.6.87" ...176[INSTALL] wirte the es_rep.conf on "192.168.6.87" ... OK177[INSTALL] wirte the sys_hba.conf on "192.168.6.87" ...178[INSTALL] wirte the sys_hba.conf on "192.168.6.87" ... OK179[INSTALL] wirte the .encpwd on every host180[INSTALL] write the repmgr.conf on every host181[INSTALL] write the repmgr.conf on "192.168.6.87" ...182[INSTALL] write the repmgr.conf on "192.168.6.87" ... OK183[INSTALL] write the repmgr.conf on "192.168.6.88" ...184[INSTALL] write the repmgr.conf on "192.168.6.88" ... OK185[INSTALL] start up the database on "192.168.6.87" ...186[INSTALL] /KingbaseES/V9/cluster/kingbase/bin/sys_ctl -w -t 60 -l /KingbaseES/V9/cluster/kingbase/logfile -D /data start187waiting for server to start.... done188server started189[INSTALL] start up the database on "192.168.6.87" ... OK190[INSTALL] create the database "esrep" and user "esrep" for repmgr ...191CREATE DATABASE192CREATE ROLE193GRANT194GRANT ROLE195[INSTALL] create the database "esrep" and user "esrep" for repmgr ... OK196[INSTALL] register the primary on "192.168.6.87" ...197[INFO] connecting to primary database...198[NOTICE] attempting to install extension "repmgr"199[NOTICE] "repmgr" extension successfully installed200[NOTICE] primary node record (ID: 1) registered201[INSTALL] register the primary on "192.168.6.87" ... OK202[INSTALL] clone and start up the standby ...203clone the standby on "192.168.6.88" ...204/KingbaseES/V9/cluster/kingbase/bin/repmgr -h 192.168.6.87 -U esrep -d esrep -p 54321 --fast-checkpoint --upstream-node-id 1 standby clone205[NOTICE] destination directory "/data" provided206[INFO] connecting to source node207[DETAIL] connection string is: host=192.168.6.87 user=esrep port=54321 dbname=esrep208[DETAIL] current installation size is 87 MB209[NOTICE] checking for available walsenders on the source node (2 required)210[NOTICE] checking replication connections can be made to the source server (2 required)211[INFO] checking and correcting permissions on existing directory "/data"212[INFO] creating replication slot as user "esrep"213[NOTICE] starting backup (using sys_basebackup)...214[INFO] executing:215 /KingbaseES/V9/cluster/kingbase/bin/sys_basebackup -l "repmgr base backup" -D /data -h 192.168.6.87 -p 54321 -U esrep -c fast -X stream -S repmgr_slot_2 216[NOTICE] standby clone (using sys_basebackup) complete217[NOTICE] you can now start your Kingbase server218[HINT] for example: sys_ctl -D /data start219[HINT] after starting the server, you need to register this standby with "repmgr standby register"220clone the standby on "192.168.6.88" ... OK221start up the standby on "192.168.6.88" ...222/KingbaseES/V9/cluster/kingbase/bin/sys_ctl -w -t 60 -l /KingbaseES/V9/cluster/kingbase/logfile -D /data start223waiting for server to start.... done224server started225start up the standby on "192.168.6.88" ... OK226register the standby on "192.168.6.88" ...227[INFO] connecting to local node "node2" (ID: 2)228[INFO] connecting to primary database229[INFO] standby registration complete230[NOTICE] standby node "node2" (ID: 2) successfully registered231[INSTALL] register the standby on "192.168.6.88" ... OK232[INSTALL] start up the whole cluster ...2332025-02-08 07:00:44 Ready to start all DB ...2342025-02-08 07:00:44 begin to start DB on "[192.168.6.87]".2352025-02-08 07:00:45 DB on "[192.168.6.87]" already started, connect to check it.2362025-02-08 07:00:47 DB on "[192.168.6.87]" start success.2372025-02-08 07:00:47 Try to ping trusted_servers on host 192.168.6.87 ...2382025-02-08 07:00:50 Try to ping trusted_servers on host 192.168.6.88 ...2392025-02-08 07:00:53 begin to start DB on "[192.168.6.88]".2402025-02-08 07:00:53 DB on "[192.168.6.88]" already started, connect to check it.2412025-02-08 07:00:55 DB on "[192.168.6.88]" start success.242 ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string 243----+-------+---------+-----------+----------+----------+----------+----------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------244 1 | node1 | primary | * running | | default | 100 | 1 | | host=192.168.6.87 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=9000245 2 | node2 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.6.88 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=90002462025-02-08 07:00:55 The primary DB is started.2472025-02-08 07:00:55 begin to start repmgrd on "[192.168.6.87]".248[2025-02-08 07:00:56] [NOTICE] using provided configuration file "/KingbaseES/V9/cluster/kingbase/bin/../etc/repmgr.conf"249[2025-02-08 07:00:56] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/hamgr.log"2502512025-02-08 07:00:58 repmgrd on "[192.168.6.87]" start success.2522025-02-08 07:00:58 begin to start repmgrd on "[192.168.6.88]".253[2025-02-08 06:59:49] [NOTICE] using provided configuration file "/KingbaseES/V9/cluster/kingbase/bin/../etc/repmgr.conf"254[2025-02-08 06:59:49] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/hamgr.log"2552562025-02-08 07:01:01 repmgrd on "[192.168.6.88]" start success.257 ID | Name | Role | Status | Upstream | repmgrd | PID | Paused? | Upstream last seen258----+-------+---------+-----------+----------+---------+-------+---------+--------------------259 1 | node1 | primary | * running | | running | 60877 | no | n/a 260 2 | node2 | standby | running | node1 | running | 68396 | no | 1 second(s) ago 261[2025-02-08 07:01:04] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/kbha.log"262263[2025-02-08 06:59:59] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/kbha.log"2642652025-02-08 07:01:11 Done.266[INSTALL] start up the whole cluster ... OK等待几分钟就安装完成了。
检查集群状态：
1[kingbase@kesrwc01:/home/kingbase]$ repmgr cluster show2 ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string 3----+-------+---------+-----------+----------+----------+----------+----------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------4 1 | node1 | primary | * running | | default | 100 | 1 | | host=192.168.6.87 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=90005 2 | node2 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.6.88 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=900067[kingbase@kesrwc01:/home/kingbase]$ repmgr service status8 ID | Name | Role | Status | Upstream | repmgrd | PID | Paused? | Upstream last seen9----+-------+---------+-----------+----------+-------------+-----+---------+--------------------10 1 | node1 | primary | * running | | not running | n/a | n/a | n/a 11 2 | node2 | standby | running | node1 | not running | n/a | n/a | n/a在任意节点上执行一键关闭集群：
1[kingbase@kesrwc01:/home/kingbase]$ sys_monitor.sh stop22025-02-08 07:33:20 Ready to stop all DB ...32025-02-08 07:33:32 begin to stop repmgrd on "[192.168.6.87]".42025-02-08 07:33:33 repmgrd on "[192.168.6.87]" already stopped.52025-02-08 07:33:33 begin to stop repmgrd on "[192.168.6.88]".62025-02-08 07:33:34 repmgrd on "[192.168.6.88]" already stopped.72025-02-08 07:33:34 begin to stop DB on "[192.168.6.88]".8waiting for server to shut down.... done9server stopped102025-02-08 07:33:34 DB on "[192.168.6.88]" stop success.112025-02-08 07:33:34 begin to stop DB on "[192.168.6.87]".12waiting for server to shut down.... done13server stopped142025-02-08 07:33:35 DB on "[192.168.6.87]" stop success.152025-02-08 07:33:36 Done.检查各节点相关进程和端口是否已经释放：
1[root@kesrwc01 ~]# netstat -anlp|grep -E '54321'|column -t2[root@kesrwc02 ~]# netstat -anlp|grep -E '54321'|column -t如果需要关闭主机的话，启动主机需要先启动主节点，然后依次启动备节点，异步备库节点。然后在任意节点上执行一键启动集群：
1[kingbase@kesrwc02:/home/kingbase]$ sys_monitor.sh start22025-02-08 07:47:02 Ready to start all DB ...32025-02-08 07:47:02 begin to start DB on "[192.168.6.88]".4waiting for server to start..../bin/sh: /data/../logfile: Permission denied5 stopped waiting6sys_ctl: could not start server7Examine the log output.82025-02-08 07:47:03 execute to start DB on "[192.168.6.88]" failed.92025-02-08 07:47:04 Start DB on localhost(192.168.6.88) failed, will do nothing and exit.这里启动报错了，看报错说是无法访问 /data/../logfile 日志文件，看一下脚本里怎么调用的：
1[kingbase@kesrwc02:/data]$ cd /KingbaseES/V9/cluster/kingbase/bin/2[kingbase@kesrwc02:/KingbaseES/V9/cluster/kingbase/bin]$ grep sys_ctl sys_monitor.sh 3 execute_command ${execute_user} $host "${sys_bindir}/sys_ctl -D ${data_directory} -w -t 60 -l ${data_directory}/../logfile start"4 execute_command ${execute_user} $host "${sys_bindir}/sys_ctl -D ${data_directory} -m fast -w -t 60 -l ${data_directory}/../logfile stop"5 ${sys_bindir}/sys_ctl -D ${data_directory} status >/dev/null 2>&16 execute_command ${execute_user} $db_ip "${sys_bindir}/sys_ctl -D $data_directory reload" 2>&17 ${sys_bindir}/sys_ctl -D ${data_directory} status >/dev/null 2>&1可以看到脚本在启动数据库时，调用的命令是：
1${sys_bindir}/sys_ctl -D ${data_directory} -w -t 60 -l ${data_directory}/../logfile start这里指定的 -l 值是 ${data_directory}/../logfile，这个 data_directory 是获取的 repmgr.conf 配置文件值：
1[kingbase@kesrwc02:/KingbaseES/V9/cluster/kingbase/etc]$ cat repmgr.conf | grep data_directory2data_directory='/data'检查确实没有这个文件：
1[kingbase@kesrwc02:/data]$ ll /data/../logfile2ls: 无法访问 '/data/../logfile': 没有那个文件或目录但是之前集群安装的时候，看日志里面输出的启动数据库的命令是：
1/KingbaseES/V9/cluster/kingbase/bin/sys_ctl -w -t 60 -l /KingbaseES/V9/cluster/kingbase/logfile -D /data start那这看起来不就是 data_directory 的值取错了吗？应该是取 KINGBASE_HOME 环境变量才对吧！
1[kingbase@kesrwc02:/home/kingbase]$ ll $KINGBASE_HOME/logfile2-rw------- 1 kingbase kingbase 1644 2月 8 08:40 /KingbaseES/V9/cluster/kingbase//logfile好家伙，修改一下 sys_monitor.sh 脚本，将 sys_ctl 启停数据库命令中的 ${data_directory}/../logfile 替换为 ${KINGBASE_HOME}/logfile 即可：
1[kingbase@kesrwc02:/KingbaseES/V9/cluster/kingbase/bin]$ sed -i 's/${data_directory}\/..\/logfile/${KINGBASE_HOME}\/logfile/' /KingbaseES/V9/cluster/kingbase/bin/sys_monitor.sh确保已修改成功：
1[kingbase@kesrwc02:/KingbaseES/V9/cluster/kingbase/bin]$ grep '${KINGBASE_HOME}/logfile' sys_monitor.sh 2 execute_command ${execute_user} $host "${sys_bindir}/sys_ctl -D ${data_directory} -w -t 60 -l ${KINGBASE_HOME}/logfile start"3 execute_command ${execute_user} $host "${sys_bindir}/sys_ctl -D ${data_directory} -m fast -w -t 60 -l ${KINGBASE_HOME}/logfile stop"再次启动集群：
1[kingbase@kesrwc01:/home/kingbase]$ sys_monitor.sh start22025-02-08 08:41:06 Ready to start all DB ...32025-02-08 08:41:06 begin to start DB on "[192.168.6.87]".4waiting for server to start.... done5server started62025-02-08 08:41:07 execute to start DB on "[192.168.6.87]" success, connect to check it.72025-02-08 08:41:08 DB on "[192.168.6.87]" start success.82025-02-08 08:41:08 Try to ping trusted_servers on host 192.168.6.87 ...92025-02-08 08:41:11 Try to ping trusted_servers on host 192.168.6.88 ...102025-02-08 08:41:14 begin to start DB on "[192.168.6.88]".11waiting for server to start.... done12server started132025-02-08 08:41:16 execute to start DB on "[192.168.6.88]" success, connect to check it.142025-02-08 08:41:17 DB on "[192.168.6.88]" start success.15 ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string 16----+-------+---------+-----------+----------+----------+----------+----------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------17 1 | node1 | primary | * running | | default | 100 | 1 | | host=192.168.6.87 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=900018 2 | node2 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.6.88 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=9000192025-02-08 08:41:17 The primary DB is started.202025-02-08 08:41:17 begin to start repmgrd on "[192.168.6.87]".21[2025-02-08 08:41:19] [NOTICE] using provided configuration file "/KingbaseES/V9/cluster/kingbase/bin/../etc/repmgr.conf"22[2025-02-08 08:41:19] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/hamgr.log"23242025-02-08 08:41:21 repmgrd on "[192.168.6.87]" start success.252025-02-08 08:41:21 begin to start repmgrd on "[192.168.6.88]".26[2025-02-08 08:40:12] [NOTICE] using provided configuration file "/KingbaseES/V9/cluster/kingbase/bin/../etc/repmgr.conf"27[2025-02-08 08:40:12] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/hamgr.log"28292025-02-08 08:41:24 repmgrd on "[192.168.6.88]" start success.30 ID | Name | Role | Status | Upstream | repmgrd | PID | Paused? | Upstream last seen31----+-------+---------+-----------+----------+---------+-------+---------+--------------------32 1 | node1 | primary | * running | | running | 70859 | no | n/a 33 2 | node2 | standby | running | node1 | running | 78011 | no | 1 second(s) ago 34[2025-02-08 08:41:30] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/kbha.log"3536[2025-02-08 08:40:27] [NOTICE] redirecting logging output to "/KingbaseES/V9/cluster/kingbase/log/kbha.log"37382025-02-08 08:41:39 Done.集群启动成功，检查集群状态：
1[kingbase@kesrwc02:/home/kingbase]$ repmgr cluster show2 ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string 3----+-------+---------+-----------+----------+----------+----------+----------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------4 1 | node1 | primary | * running | | default | 100 | 1 | | host=192.168.6.87 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=90005 2 | node2 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.6.88 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=2 keepalives_interval=2 keepalives_count=3 tcp_user_timeout=9000修改 /data/sys_hba.conf 配置文件：
1## 以节点一为例2[kingbase@kesrwc01 ~]$ vi /data/sys_hba.conf34# TYPE DATABASE USER ADDRESS METHOD56# "local" is for Unix domain socket connections only7local all all scram-sha-25689## 将 local 认证方式 scram-sha-256 修改为 trust 即可1# TYPE DATABASE USER ADDRESS METHOD23# "local" is for Unix domain socket connections only4local all all trust重启数据库生效：
1[kingbase@kesrwc01:/home/kingbase]$ sys_ctl restart2等待服务器进程关闭 .... 完成3服务器进程已经关闭4等待服务器进程启动 ....2025-02-08 07:14:20.703 CST [64559] LOG: config the real archive_command string as soon as possible to archive WAL files52025-02-08 07:14:20.711 CST [64559] LOG: sepapower extension initialized62025-02-08 07:14:20.717 CST [64559] LOG: starting KingbaseES V009R001C002B0014 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28), 64-bit72025-02-08 07:14:20.717 CST [64559] LOG: listening on IPv4 address "0.0.0.0", port 5432182025-02-08 07:14:20.717 CST [64559] LOG: listening on IPv6 address "::", port 5432192025-02-08 07:14:20.719 CST [64559] LOG: listening on Unix socket "/tmp/.s.KINGBASE.54321"102025-02-08 07:14:20.921 CST [64559] LOG: redirecting log output to logging collector process112025-02-08 07:14:20.921 CST [64559] HINT: Future log output will appear in directory "sys_log".12 完成13服务器进程已经启动本地免密连接：
1[kingbase@kesrwc01:/home/kingbase]$ ksql test system2输入 "help" 来获取帮助信息.34test=#主节点连接数据库做一些数据测试：
1[kingbase@kesrwc01:/home/kingbase]$ ksql test system2输入 "help" 来获取帮助信息.34test=# show data_directory;5 data_directory 6----------------7 /data8(1 行记录)910test=# \conninfo11以用户 "system" 的身份，通过套接字"/tmp"在端口"54321"连接到数据库 "test"12test=# create database lucifer;13CREATE DATABASE14test=# \c lucifer15您现在以用户名"system"连接到数据库"lucifer"。16lucifer=# create table test (id int,name varchar2(20));17CREATE TABLE18lucifer=# insert into test values (1,'lucifer');19INSERT 0 120lucifer=# select * from test;21 id | name 22----+---------23 1 | lucifer24(1 行记录)2526lucifer=# \q备节点查询数据，查看是否同步：
1[kingbase@kesrwc02:/home/kingbase]$ ksql lucifer system2输入 "help" 来获取帮助信息.34## 数据正常同步5lucifer=# select * from test;6 id | name 7----+---------8 1 | lucifer9(1 行记录)1011## 测试备节点确实无法写入，只能查询12lucifer=# insert into test values (2,'lucifer');13ERROR: cannot execute INSERT in a read-only transaction数据同步正常。
其实我一开始测试过图形化部署，在银河麒麟 V10 SP3 图形化部署集群-添加节点失败，疑似 BUG。
期待官方可以重现解决这个问题。