在知识库中专注阅读，并随时返回相关工具与课程
大家好，这里是公众号 DBA学习之路，分享一些学习国产数据库路上的知识和经验。 为了给大家提供一些技术交流的平台，目前已成立的技术交流群： - Oracle 数据库交流群 - 国产数据库交流群 - Linux 技术交流群 - ChatGPT 4o 免费体验群 需要进群（均已超过 200 人，需要邀请）可以添加号主微信：Lucifer-0622，备注对应的群名即可。 @TOC目录 昨天看到一个 Op
系统学习 Oracle DBA
开通会员 ¥128/月起
查看完整课程大家好，这里是公众号 DBA学习之路，分享一些学习国产数据库路上的知识和经验。
本章目标：掌握本章核心知识点 前置要求：完成前序章节学习 预计时长：60 分钟
为了给大家提供一些技术交流的平台，目前已成立的技术交流群：
需要进群（均已超过 200 人，需要邀请）可以添加号主微信：Lucifer-0622，备注对应的群名即可。
昨天看到一个 OpenTenBase创新应用挑战赛 活动，非常感兴趣，打算学习了解一下 OpenTenBase 开源数据库。
本文简单介绍 OpenTenBase 以及如何安装部署。
OpenTenBase 是一个提供写可靠性，多主节点数据同步的关系数据库集群平台。你可以将 OpenTenBase 配置一台或者多台主机上， OpenTenBase 数据存储在多台物理主机上面。数据表的存储有两种方式， 分别是 distributed 或者 replicated ，当向 OpenTenBase 发送查询 SQL 时， OpenTenBase 会自动向数据节点发出查询语句并获取最终结果。
OpenTenBase 采用分布式集群架构（如下图）， 该架构分布式为无共享(share nothing)模式，节点之间相应独立，各自处理自己的数据，处理后的结果可能向上层汇总或在节点间流转，各处理单元之间通过网络协议进行通信，并行处理和扩展能力更好，这也意味着只需要简单的 x86 服务器就可以部署 OpenTenBase 数据库集群。
OpenTenBase 架构图：
简单解读一下 OpenTenBase 的三大模块：
接下来，让我们来看看如何从源码开始，完成到 OpenTenBase 集群环境的搭建。
本文使用两台服务器上搭建 1GTM 主，1GTM 备，2CN 主（CN 主之间对等，因此无需备 CN），2DN 主，2DN 备的集群，该集群为具备容灾能力的最小配置。
OpenTenBase 安装最低要求：
集群规划如下：
检查操作系统版本信息：
1## 以节点一为例2[root@oct01 ~]# cat /etc/os-release 3NAME="TencentOS Server"4VERSION="3.3 (Final)"5ID="tencentos"6ID_LIKE="rhel fedora centos"7VERSION_ID="3.3"8PLATFORM_ID="platform:el8"9PRETTY_NAME="TencentOS Server 3.3 (Final)"10ANSI_COLOR="0;31"11CPE_NAME="cpe:/o:tencentos:tencentos:3"12HOME_URL="https://cloud.tencent.com/product/ts"1314TENCENT_SUPPORT_PRODUCT="tencentos"15TENCENT_SUPPORT_PRODUCT_VERSION="3"16NAME_ORIG="TencentOS Server"数据库安装均建议关闭防火墙：
1## 以节点一为例2[root@otc01 ~]# systemctl stop firewalld3[root@otc01 ~]# systemctl disable firewalld4Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.5Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.所有节点建议关闭 SELNUX，防止造成一些不必要的问题：
1## 以节点一为例2## 这里使用 setenforce 0 临时生效3[root@oct01 ~]# /usr/sbin/setenforce 04[root@oct01 ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config5[root@oct01 ~]# sestatus6SELinux status: enabled7SELinuxfs mount: /sys/fs/selinux8SELinux root directory: /etc/selinux9Loaded policy name: targeted10Current mode: permissive11Mode from config file: disabled12Policy MLS status: enabled13Policy deny_unknown status: allowed14Memory protection checking: actual (secure)15Max kernel policy version: 31将 SELINUX 参数设置为 disabled，即 SELINUX=disabled 保存退出后，需要重新启动才能生效。
挂载 iso 镜像：
1[root@oct01 ~]# mount /dev/sr0 /mnt2mount: /mnt: WARNING: device write-protected, mounted read-only.一键配置本地软件源：
1## 备份系统初始配置文件2mkdir -p /etc/yum.repos.d/bak3mv /etc/yum.repos.d/* /etc/yum.repos.d/bak4## 一键配置软件源，默认 ISO 安装镜像挂载在 /mnt 目录下5cat <<-EOF > /etc/yum.repos.d/local.repo6[BaseOS]7name=BaseOS8baseurl=file:///mnt/BaseOS9enabled=110gpgcheck=011[AppStream]12name=AppStream13baseurl=file:///mnt/AppStream14enabled=115gpgcheck=016EOF17## 查看配置好的软件源18cat /etc/yum.repos.d/local.repoOpenTenBase 对部署的环境有一些依赖包要求：
1## TencentOS 2、TencentOS 3、OpenCloudOS、CentOS 7、CentOS 82[root@oct01 ~]# dnf install -y gcc make readline-devel zlib-devel openssl-devel uuid uuid-devel bison flex --skip-broken安装完成后检查是否安装成功：
1[root@oct01 ~]# rpm -q gcc make readline-devel zlib-devel openssl-devel uuid uuid-devel bison flex2gcc-8.5.0-22.tl3.1.x86_643make-4.2.1-11.tl3.x86_644readline-devel-7.0-10.tl3.x86_645zlib-devel-1.2.11-25.tl3.x86_646openssl-devel-1.1.1k-12.tl3.1.x86_647未安装软件包 uuid-devel 8bison-3.0.4-10.tl3.x86_649flex-2.6.1-9.tl3.x86_64📢 注意：在 TencentOS 3 的本地软件源中默认没有 uuid-devel 包，而是在 PowerTools 中，所以需要自行下载安装，否则 OpenTenBase 源码编译的时候会报错：
1checking for uuid_export in -luuid... no2configure: error: library 'ossp-uuid' or 'uuid' is required for OSSP UUIDuuid-devel-1.6.2-43.tl3.x86_64.rpm 下载链接 🔗： https://mirrors.tencent.com/tlinux/3.3/PowerTools/x86_64/os/Packages/uuid-devel-1.6.2-43.tl3.x86_64.rpm
下载好之后上传安装即可：
1## 以节点一为例2[root@oct01 ~]# rpm -ivh uuid-devel-1.6.2-43.tl3.x86_64.rpm 3警告：uuid-devel-1.6.2-43.tl3.x86_64.rpm: 头V4 RSA/SHA256 Signature, 密钥 ID 6ca3b8cc: NOKEY4Verifying... ################################# [100%]5准备中... ################################# [100%]6正在升级/安装...7 1:uuid-devel-1.6.2-43.tl3 ################################# [100%]安装后再次检查依赖包是否安装：
1[root@oct01 ~]# rpm -q gcc make readline-devel zlib-devel openssl-devel uuid uuid-devel bison flex2gcc-8.5.0-22.tl3.1.x86_643make-4.2.1-11.tl3.x86_644readline-devel-7.0-10.tl3.x86_645zlib-devel-1.2.11-25.tl3.x86_646openssl-devel-1.1.1k-12.tl3.1.x86_647uuid-1.6.2-43.tl3.x86_648uuid-devel-1.6.2-43.tl3.x86_649bison-3.0.4-10.tl3.x86_6410flex-2.6.1-9.tl3.x86_64依赖包全部安装完成。
1## 以节点一为例2[root@oct01 ~]# useradd -d /home/opentenbase -m opentenbase3## 设置用户密码为 opentenbase4[root@oct01 ~]# echo "opentenbase:opentenbase" | chpasswd创建安装所需目录并且授权：
1## 以节点一为例2[root@oct01 ~]# mkdir -p /soft3[root@oct01 ~]# mkdir -p /opentenbase/{pgxc_ctl,install,global}4[root@oct01 ~]# mkdir -p /opentenbase/data/{gtm,coord,coord_archlog,dn001,dn002,datanode_archlog}5[root@oct01 ~]# chown -R opentenbase:opentenbase {/soft,/opentenbase}6[root@oct01 ~]# chmod -R 775 {/soft,/opentenbase}建议将所有节点之间的 SSH 配置免密登录，然后 deployment 和 init 就会 SSH 到各个节点的机器上，不需要输入密码了：
1## 以节点一为例2## 切换到 opentenbase 用户3[root@oct01 ~]# su - opentenbase45## 生成 ssh 密钥，一路回车即可6[opentenbase@oct01 ~]$ ssh-keygen -t rsa7Generating public/private rsa key pair.8Enter file in which to save the key (/home/opentenbase/.ssh/id_rsa): 9Created directory '/home/opentenbase/.ssh'.10Enter passphrase (empty for no passphrase): 11Enter same passphrase again: 12Your identification has been saved in /home/opentenbase/.ssh/id_rsa.13Your public key has been saved in /home/opentenbase/.ssh/id_rsa.pub.14The key fingerprint is:15SHA256:kCmuNwPhMiIue6ruEDC+ubrJvLL56WA7IA0Vb+QWeNc opentenbase@oct0116The key's randomart image is:17+---[RSA 3072]----+18| .oo . |19| o+...oE |20|o...*.+ |21|+o = . . |22|*+o . S |23|*++o |24|*=. + |25|B*+o o |26|//* |27+----[SHA256]-----+2829## 将公钥拷贝到所有节点（包括自己），中间需要输入一次密码30[opentenbase@oct01 ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub opentenbase@192.168.6.8731[opentenbase@oct01 ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub opentenbase@192.168.6.883233## 测试免密互信是否成功34[opentenbase@oct01 ~]$ ssh 192.168.6.8735[opentenbase@oct01 ~]$ ssh 192.168.6.88其他节点同样操作。
建议在自己电脑获取 OpenTenBase 源码：
1╭─lucifer@Lucifer-7 /Volumes/DBA/Github2╰─$ git clone https://github.com/OpenTenBase/OpenTenBase3正克隆到 'OpenTenBase'...4remote: Enumerating objects: 18129, done.5remote: Counting objects: 100% (4633/4633), done.6remote: Compressing objects: 100% (810/810), done.7remote: Total 18129 (delta 4102), reused 3863 (delta 3822), pack-reused 13496 (from 1)8接收对象中: 100% (18129/18129), 31.03 MiB | 20.64 MiB/s, 完成.9处理 delta 中: 100% (11304/11304), 完成.当然也可以直接去下载源码包：https://github.com/OpenTenBase/OpenTenBase/tags
下载后上传到数据库主机并解压（主节点）：
1[root@oct01 ~]# cd /soft/2[root@oct01 soft]# chown opentenbase:opentenbase OpenTenBase-2.6.0.tar.gz 3[root@oct01 soft]# ll4总用量 248005-rwxr-xr-x. 1 opentenbase opentenbase 25394783 11月 28 14:18 OpenTenBase-2.6.0.tar.gz67## 解压源码包8[root@oct01 soft]# su - opentenbase9[opentenbase@oct01 ~]$ cd /soft/10[opentenbase@oct01 soft]$ tar -xf OpenTenBase-2.6.0.tar.gz 11[opentenbase@oct01 soft]$ ls12OpenTenBase-2.6.0 OpenTenBase-2.6.0.tar.gz使用 opentenbase 用户一键配置环境变量：
1## 以节点一为例2[opentenbase@oct01 ~]$ cat<<-\EOF>>/home/opentenbase/.bash_profile3export OPENTENBASE_HOME=/opentenbase/install/opentenbase_bin_v2.64export PATH=$OPENTENBASE_HOME/bin:$PATH5export LD_LIBRARY_PATH=$OPENTENBASE_HOME/lib:${LD_LIBRARY_PATH}6export LC_ALL=C7EOF至此，准备工作就完成了。
主节点需要先编译安装源码包，用于后续集群发布部署：
1## 进入源码目录2[opentenbase@oct01 ~]$ cd /soft/OpenTenBase-2.6.034## 授予脚本执行权限5[opentenbase@oct01 OpenTenBase-2.6.0]$ chmod +x configure*67## 配置编译选项8[opentenbase@oct01 OpenTenBase-2.6.0]$ ./configure --prefix=/opentenbase/install/opentenbase_bin_v2.6 --enable-user-switch --with-openssl --with-ossp-uuid CFLAGS=-g910## 编译安装软件11[opentenbase@oct01 OpenTenBase-2.6.0]$ make clean && make -sj && make install1213## 编译 contruib 工具14[opentenbase@oct01 OpenTenBase-2.6.0]$ chmod +x contrib/pgxc_ctl/make_signature15[opentenbase@oct01 OpenTenBase-2.6.0]$ cd contrib16[opentenbase@oct01 OpenTenBase-2.6.0]$ make -sj && make install至此，主节点源码编译安装完成。
接下来可以进入到集群初始化阶段，为了方便用户，OpenTenBase 提供了专用的配置和操作工具：pgxc_ctl 来协助用户快速搭建并管理集群：
1## 查看 pgxc_ctl 命令帮助文档2[opentenbase@oct01 ~]$ pgxc_ctl --help3/usr/bin/bash4pgxc_ctl [option ...] [command]5option:6 -c or --configuration conf_file: Specify configruration file.7 -v or --verbose: Specify verbose output.8 -V or --version: Print version and exit.9 -l or --logdir log_directory: specifies what directory to write logs.10 -L or --logfile log_file: Specifies log file.11 --home home_direcotry: Specifies pgxc_ctl work director.12 -i or --infile input_file: Specifies inptut file.13 -o or --outfile output_file: Specifies output file.14 -h or --help: Prints this message and exits.15For more deatils, refer to pgxc_ctl reference manual included in16postgres-xc reference manual.首先需要将前文所述的节点的 ip，端口，目录写入到配置文件 pgxc_ctl.conf 中：
1[opentenbase@oct01 ~]$ cd /opentenbase/pgxc_ctl/2[opentenbase@oct01 pgxc_ctl]$ vi pgxc_ctl.conf配置文件内容如下：
1#!/bin/bash2# Double Node Config34IP_1=192.168.6.875IP_2=192.168.6.8867pgxcInstallDir=/opentenbase/install/opentenbase_bin_v2.68pgxcOwner=opentenbase9defaultDatabase=postgres10pgxcUser=$pgxcOwner11tmpDir=/tmp12localTmpDir=$tmpDir13configBackup=n14configBackupHost=pgxc-linker15configBackupDir=$HOME/pgxc16configBackupFile=pgxc_ctl.bak171819#---- GTM ----------20gtmName=gtm21gtmMasterServer=$IP_122gtmMasterPort=5000123gtmMasterDir=/opentenbase/data/gtm24gtmExtraConfig=none25gtmMasterSpecificExtraConfig=none26gtmSlave=y27gtmSlaveServer=$IP_228gtmSlavePort=5000129gtmSlaveDir=/opentenbase/data/gtm30gtmSlaveSpecificExtraConfig=none3132#---- Coordinators -------33coordMasterDir=/opentenbase/data/coord34coordArchLogDir=/opentenbase/data/coord_archlog3536coordNames=(cn001 cn002 )37coordPorts=(30004 30004 )38poolerPorts=(31110 31110 )39coordPgHbaEntries=(0.0.0.0/0)40coordMasterServers=($IP_1 $IP_2)41coordMasterDirs=($coordMasterDir $coordMasterDir)42coordMaxWALsernder=243coordMaxWALSenders=($coordMaxWALsernder $coordMaxWALsernder )44coordSlave=n45coordSlaveSync=n46coordArchLogDirs=($coordArchLogDir $coordArchLogDir)4748coordExtraConfig=coordExtraConfig49cat > $coordExtraConfig <<EOF50#================================================51# Added to all the coordinator postgresql.conf52# Original: $coordExtraConfig5354include_if_exists = '/opentenbase/global/global_opentenbase.conf'5556wal_level = replica57wal_keep_segments = 256 58max_wal_senders = 459archive_mode = on 60archive_timeout = 1800 61archive_command = 'echo 0' 62log_truncate_on_rotation = on 63log_filename = 'postgresql-%M.log' 64log_rotation_age = 4h 65log_rotation_size = 100MB66hot_standby = on 67wal_sender_timeout = 30min 68wal_receiver_timeout = 30min 69shared_buffers = 1024MB 70max_pool_size = 200071log_statement = 'ddl'72log_destination = 'csvlog'73logging_collector = on74log_directory = 'pg_log'75listen_addresses = '*'76max_connections = 20007778EOF7980coordSpecificExtraConfig=(none none)81coordExtraPgHba=coordExtraPgHba82cat > $coordExtraPgHba <<EOF8384local all all trust85host all all 0.0.0.0/0 trust86host replication all 0.0.0.0/0 trust87host all all ::1/128 trust88host replication all ::1/128 trust899091EOF929394coordSpecificExtraPgHba=(none none)95coordAdditionalSlaves=n 96cad1_Sync=n9798#---- Datanodes ---------------------99dn1MstrDir=/opentenbase/data/dn001100dn2MstrDir=/opentenbase/data/dn002101dn1SlvDir=/opentenbase/data/dn001102dn2SlvDir=/opentenbase/data/dn002103dn1ALDir=/opentenbase/data/datanode_archlog104dn2ALDir=/opentenbase/data/datanode_archlog105106primaryDatanode=dn001107datanodeNames=(dn001 dn002)108datanodePorts=(40004 40004)109datanodePoolerPorts=(41110 41110)110datanodePgHbaEntries=(0.0.0.0/0)111datanodeMasterServers=($IP_1 $IP_2)112datanodeMasterDirs=($dn1MstrDir $dn2MstrDir)113dnWALSndr=4114datanodeMaxWALSenders=($dnWALSndr $dnWALSndr)115116datanodeSlave=y117datanodeSlaveServers=($IP_2 $IP_1)118datanodeSlavePorts=(50004 54004)119datanodeSlavePoolerPorts=(51110 51110)120datanodeSlaveSync=n121datanodeSlaveDirs=($dn1SlvDir $dn2SlvDir)122datanodeArchLogDirs=($dn1ALDir/dn001 $dn2ALDir/dn002)123124datanodeExtraConfig=datanodeExtraConfig125cat > $datanodeExtraConfig <<EOF126#================================================127# Added to all the coordinator postgresql.conf128# Original: $datanodeExtraConfig129130include_if_exists = '/opentenbase/global/global_opentenbase.conf'131listen_addresses = '*' 132wal_level = replica 133wal_keep_segments = 256 134max_wal_senders = 4135archive_mode = on 136archive_timeout = 1800 137archive_command = 'echo 0' 138log_directory = 'pg_log' 139logging_collector = on 140log_truncate_on_rotation = on 141log_filename = 'postgresql-%M.log' 142log_rotation_age = 4h 143log_rotation_size = 100MB144hot_standby = on 145wal_sender_timeout = 30min 146wal_receiver_timeout = 30min 147shared_buffers = 1024MB 148max_connections = 4000 149max_pool_size = 4000150log_statement = 'ddl'151log_destination = 'csvlog'152wal_buffers = 1GB153154EOF155156datanodeSpecificExtraConfig=(none none)157datanodeExtraPgHba=datanodeExtraPgHba158cat > $datanodeExtraPgHba <<EOF159160local all all trust161host all all 0.0.0.0/0 trust162host replication all 0.0.0.0/0 trust163host all all ::1/128 trust164host replication all ::1/128 trust165166167EOF168169170datanodeSpecificExtraPgHba=(none none)171172datanodeAdditionalSlaves=n173walArchive=n如上是结合本文演示环境的 IP，端口，数据库目录，二进制目录等规划来写的 pgxc_ctl.conf 文件。具体实践中只需按照自己的实际情况配置好即可。
在主节点配置好 pgxc_ctl.conf 文件后，使用 pgxc_ctl 工具将二进制包部署到所有节点：
1[opentenbase@oct01 pgxc_ctl]$ pgxc_ctl -c /opentenbase/pgxc_ctl/pgxc_ctl.conf2/usr/bin/bash3Installing pgxc_ctl_bash script as /home/opentenbase/pgxc_ctl/pgxc_ctl_bash.4Installing pgxc_ctl_bash script as /home/opentenbase/pgxc_ctl/pgxc_ctl_bash.5Reading configuration using /home/opentenbase/pgxc_ctl/pgxc_ctl_bash --home /home/opentenbase/pgxc_ctl --configuration /opentenbase/pgxc_ctl/pgxc_ctl.conf6Finished reading configuration.7 ******** PGXC_CTL START ***************89Current directory: /home/opentenbase/pgxc_ctl10PGXC这里输入 deploy all 命令来完成分发：
1## PGXC 后面需要输入指令 deploy all 回车继续2PGXC deploy all3Deploying Postgres-XL components to all the target servers.4Prepare tarball to deploy ... 5Deploying to the server 192.168.6.87.6Deploying to the server 192.168.6.88.7Deployment done.8## 输入 exit 退出 PGXC 工具9PGXC exit登录到所有节点，检查二进制包是否分发完成：
1[root@otc02 ~]# su - opentenbase2[opentenbase@otc02 ~]$ cd /opentenbase/install/opentenbase_bin_v2.6/3[opentenbase@otc02 opentenbase_bin_v2.6]$ ll4total 125drwxrwxr-x. 2 opentenbase opentenbase 4096 Nov 28 16:52 bin6drwxrwxr-x. 4 opentenbase opentenbase 4096 Nov 28 16:51 include7drwxrwxr-x. 4 opentenbase opentenbase 4096 Nov 28 16:51 lib8drwxrwxr-x. 4 opentenbase opentenbase 35 Nov 28 16:52 share至此，二进制包分发完成。
这里需要先配置一下 /etc/environment 文件，否则初始化会报错很多命令找不到：
1## 报错如下，很多命令找不到2PGXC init all3Initialize GTM master4bash: initgtm: command not found5bash: gtm: command not found6bash: gtm_ctl: command not found7Done.8Start GTM master9bash: gtm_ctl: command not found配置 /etc/environment 文件（所有节点均需配置）：
1## 首先查看 opentenbase 用户的 PATH 变量值2[opentenbase@oct01 ~]$ echo $PATH3/opentenbase/install/opentenbase_bin_v2.6/bin:/home/opentenbase/.local/bin:/home/opentenbase/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin45## 切换到 root 用户，写入 PATH 变量值6[root@oct01 ~]# cat<<-EOF>>/etc/environment7PATH=/opentenbase/install/opentenbase_bin_v2.6/bin:/home/opentenbase/.local/bin:/home/opentenbase/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin8EOF910## 尝试切换到 opentenbase 用户，未报错则代表配置正常11[root@oct01 ~]# su - opentenbase 12[opentenbase@oct01 ~]$所有节点均配置完成后，参照上一步分发二进制包，继续使用 pgxc_ctl 工具初始化集群，
1[opentenbase@oct01 ~]$ pgxc_ctl -c /opentenbase/pgxc_ctl/pgxc_ctl.conf 2/usr/bin/bash3Installing pgxc_ctl_bash script as /home/opentenbase/pgxc_ctl/pgxc_ctl_bash.4Installing pgxc_ctl_bash script as /home/opentenbase/pgxc_ctl/pgxc_ctl_bash.5Reading configuration using /home/opentenbase/pgxc_ctl/pgxc_ctl_bash --home /home/opentenbase/pgxc_ctl --configuration /opentenbase/pgxc_ctl/pgxc_ctl.conf6Finished reading configuration.7 ******** PGXC_CTL START ***************89Current directory: /home/opentenbase/pgxc_ctl10PGXC执行 init all 命令：
1## PGXC 后面需要输入指令 init all 回车继续2PGXC init all3Initialize GTM master4The files belonging to this GTM system will be owned by user "opentenbase".5This user must also own the server process.678fixing permissions on existing directory /opentenbase/data/gtm ... ok9creating configuration files ... ok10creating xlog dir ... ok1112Success.133076450112:2024-11-29 09:36:17.425 CST -LOG: listening on IPv4 address "0.0.0.0", port 5000114LOCATION: StreamServerPort, pqcomm.c:397153076450112:2024-11-29 09:36:17.426 CST -LOG: listening on IPv6 address "::", port 5000116LOCATION: StreamServerPort, pqcomm.c:397173076450112:2024-11-29 09:36:17.427 CST -LOG: listening on Unix socket "/tmp/.s.GTM.50001"18LOCATION: StreamServerPort, pqcomm.c:391193076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData context:20LOCATION: GTM_PrintControlData, gtm_xlog.c:2013213076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->gtm_control_version = 2018071622LOCATION: GTM_PrintControlData, gtm_xlog.c:2014233076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->xlog_seg_size = 209715224LOCATION: GTM_PrintControlData, gtm_xlog.c:2015253076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->xlog_blcksz = 409626LOCATION: GTM_PrintControlData, gtm_xlog.c:2016273076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->state = 128LOCATION: GTM_PrintControlData, gtm_xlog.c:2017293076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->CurrBytePos = 408030LOCATION: GTM_PrintControlData, gtm_xlog.c:2018313076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->PrevBytePos = 408032LOCATION: GTM_PrintControlData, gtm_xlog.c:2019333076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->thisTimeLineID = 134LOCATION: GTM_PrintControlData, gtm_xlog.c:2020353076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->prevCheckPoint = 0/036LOCATION: GTM_PrintControlData, gtm_xlog.c:2021373076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->checkPoint = 0/100038LOCATION: GTM_PrintControlData, gtm_xlog.c:2022393076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->gts = 100000000040LOCATION: GTM_PrintControlData, gtm_xlog.c:2023413076450112:2024-11-29 09:36:17.429 CST -LOG: ControlData->time = 173284417742LOCATION: GTM_PrintControlData, gtm_xlog.c:202443waiting for server to shut down....Received signal 1544 done45server stopped46Done.47## ....48## 中间部分省略49## ....502024-11-29 09:36:56.519 CST [120196,coord(0.0)] HINT: Future log output will appear in directory "pg_log".51Done.52ALTER NODE cn001 WITH (HOST='192.168.6.87', PORT=30004);53ALTER NODE54CREATE NODE cn002 WITH (TYPE='coordinator', HOST='192.168.6.88', PORT=30004);55CREATE NODE56CREATE NODE dn001 WITH (TYPE='datanode', HOST='192.168.6.87', PORT=40004, PRIMARY, PREFERRED);57CREATE NODE58CREATE NODE dn002 WITH (TYPE='datanode', HOST='192.168.6.88', PORT=40004);59CREATE NODE60SELECT pgxc_pool_reload();61 pgxc_pool_reload 62------------------63 t64(1 row)6566CREATE NODE cn001 WITH (TYPE='coordinator', HOST='192.168.6.87', PORT=30004);67CREATE NODE68ALTER NODE cn002 WITH (HOST='192.168.6.88', PORT=30004);69ALTER NODE70CREATE NODE dn001 WITH (TYPE='datanode', HOST='192.168.6.87', PORT=40004, PRIMARY);71CREATE NODE72CREATE NODE dn002 WITH (TYPE='datanode', HOST='192.168.6.88', PORT=40004, PREFERRED);73CREATE NODE74SELECT pgxc_pool_reload();75 pgxc_pool_reload 76------------------77 t78(1 row)7980Done.81EXECUTE DIRECT ON (dn001) 'CREATE NODE cn001 WITH (TYPE=''coordinator'', HOST=''192.168.6.87'', PORT=30004)';82EXECUTE DIRECT83EXECUTE DIRECT ON (dn001) 'CREATE NODE cn002 WITH (TYPE=''coordinator'', HOST=''192.168.6.88'', PORT=30004)';84EXECUTE DIRECT85EXECUTE DIRECT ON (dn001) 'ALTER NODE dn001 WITH (TYPE=''datanode'', HOST=''192.168.6.87'', PORT=40004, PRIMARY, PREFERRED)';86EXECUTE DIRECT87EXECUTE DIRECT ON (dn001) 'CREATE NODE dn002 WITH (TYPE=''datanode'', HOST=''192.168.6.88'', PORT=40004, PREFERRED)';88EXECUTE DIRECT89EXECUTE DIRECT ON (dn001) 'SELECT pgxc_pool_reload()';90 pgxc_pool_reload 91------------------92 t93(1 row)9495EXECUTE DIRECT ON (dn002) 'CREATE NODE cn001 WITH (TYPE=''coordinator'', HOST=''192.168.6.87'', PORT=30004)';96EXECUTE DIRECT97EXECUTE DIRECT ON (dn002) 'CREATE NODE cn002 WITH (TYPE=''coordinator'', HOST=''192.168.6.88'', PORT=30004)';98EXECUTE DIRECT99EXECUTE DIRECT ON (dn002) 'CREATE NODE dn001 WITH (TYPE=''datanode'', HOST=''192.168.6.87'', PORT=40004, PRIMARY, PREFERRED)';100EXECUTE DIRECT101EXECUTE DIRECT ON (dn002) 'ALTER NODE dn002 WITH (TYPE=''datanode'', HOST=''192.168.6.88'', PORT=40004, PREFERRED)';102EXECUTE DIRECT103EXECUTE DIRECT ON (dn002) 'SELECT pgxc_pool_reload()';104 pgxc_pool_reload 105------------------106 t107(1 row)108109Done.这里初始化日志太长，篇幅原因，只展示部分日志。
我在 init 集群这一步遇到了很多坑，很容易报错，终端会打印出错误日志，通过查看错误原因，更改配置即可。也可以通过查看错误日志来排查：
1## 日志路径是 $HOME/pgxc_ctl/pgxc_log 2[opentenbase@oct01 pgxc_log]$ pwd3/home/opentenbase/pgxc_ctl/pgxc_log4## 所有的日志都记录在这里5[opentenbase@oct01 pgxc_log]$ ls 6115286_pgxc_ctl.log 123051_pgxc_ctl.log 123061_pgxc_ctl.log 123071_pgxc_ctl.log 123083_pgxc_ctl.log 125018_pgxc_ctl.log 125060_pgxc_ctl.log重新初始化集群之前需要先执行 clean all 命令删除已经初始化的文件：
1PGXC clean all清理之后可以重新执行 init all 命令重新发起初始化。
通过 pgxc_ctl 工具的 monitor all 命令可以查看集群状态：
1## 输入命令 monitor all 查看集群状态2PGXC monitor all3Running: gtm master4Running: gtm slave5Running: coordinator master cn0016Running: coordinator master cn0027Running: datanode master dn0018Running: datanode slave dn0019Running: datanode master dn00210Running: datanode slave dn002通过 pgxc_ctl 工具的 stop all 命令来停止集群，stop all 后面可以加上参数 -m fast 或者是 -m immediate 来决定如何停止各个节点：
1PGXC help stop23stop [ -m smart | fast | immediate ] all4stop gtm [ master | slave | all ]5stop gtm_proxy [ all | nodename ... ]6stop [ -m smart | fast | immediate ] coordinator nodename ... 7stop [ -m smart | fast | immediate ] coordinator [ master | slave ] [ all | nodename ... ] 8stop [ -m smart | fast | immediate ] datanode nodename ... 9stop [ -m smart | fast | immediate ] datanode [ master | slave ] [ all | nodename ... ] 1011Stops specified node12For more details, please see the pgxc_ctl documentation使用 stop all -m fast 命令关闭集群：
1PGXC stop all -m fast2Stopping all the coordinator masters.3Stopping coordinator master cn001.4Stopping coordinator master cn002.5Done.6Stopping all the datanode slaves.7Stopping datanode slave dn001.8Stopping datanode slave dn002.9Stopping all the datanode masters.10Stopping datanode master dn001.11Stopping datanode master dn002.12Done.13Stop GTM slave14waiting for server to shut down..... done15server stopped16Stop GTM master17waiting for server to shut down.... done18server stopped集群关闭成功。
通过 pgxc_ctl 工具的 start all 命令来启动集群：
1PGXC help start23start all4start nodename ...5start gtm [ master | slave | all ]6start gtm_proxy [ all | nodename ... ]7start coordinator nodename ...8start coordinator [ master | slave ] [ all | nodename ... ]9start datanode nodename ...10start datanode [ master | slave ] [ all | nodename ... ]1112Starts specified node13For more details, please see the pgxc_ctl documentation使用 start all 命令启动集群：
1PGXC start all2Start GTM master3waiting for server to shut down.... done4server stopped5server starting6228882240:2024-11-29 09:58:17.525 CST -LOG: listening on IPv4 address "0.0.0.0", port 500017LOCATION: StreamServerPort, pqcomm.c:3978228882240:2024-11-29 09:58:17.525 CST -LOG: listening on IPv6 address "::", port 500019LOCATION: StreamServerPort, pqcomm.c:39710228882240:2024-11-29 09:58:17.525 CST -LOG: listening on Unix socket "/tmp/.s.GTM.50001"11LOCATION: StreamServerPort, pqcomm.c:39112228882240:2024-11-29 09:58:17.525 CST -LOG: ControlData context:13LOCATION: GTM_PrintControlData, gtm_xlog.c:201314228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->gtm_control_version = 2018071615LOCATION: GTM_PrintControlData, gtm_xlog.c:201416228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->xlog_seg_size = 209715217LOCATION: GTM_PrintControlData, gtm_xlog.c:201518228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->xlog_blcksz = 409619LOCATION: GTM_PrintControlData, gtm_xlog.c:201620228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->state = 121LOCATION: GTM_PrintControlData, gtm_xlog.c:201722228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->CurrBytePos = 835592023LOCATION: GTM_PrintControlData, gtm_xlog.c:201824228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->PrevBytePos = 835584025LOCATION: GTM_PrintControlData, gtm_xlog.c:201926228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->thisTimeLineID = 127LOCATION: GTM_PrintControlData, gtm_xlog.c:202028228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->prevCheckPoint = 0/60000029LOCATION: GTM_PrintControlData, gtm_xlog.c:202130228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->checkPoint = 0/80000031LOCATION: GTM_PrintControlData, gtm_xlog.c:202232228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->gts = 313827945433LOCATION: GTM_PrintControlData, gtm_xlog.c:202334228882240:2024-11-29 09:58:17.526 CST -LOG: ControlData->time = 173284549635LOCATION: GTM_PrintControlData, gtm_xlog.c:202436Start GTM slavewaiting for server to shut down..... done37server stopped38server starting393723429696:2024-11-29 09:58:21.856 CST -LOG: listening on IPv4 address "0.0.0.0", port 5000140LOCATION: StreamServerPort, pqcomm.c:397413723429696:2024-11-29 09:58:21.856 CST -LOG: listening on IPv6 address "::", port 5000142LOCATION: StreamServerPort, pqcomm.c:397433723429696:2024-11-29 09:58:21.856 CST -LOG: listening on Unix socket "/tmp/.s.GTM.50001"44LOCATION: StreamServerPort, pqcomm.c:391453723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData context:46LOCATION: GTM_PrintControlData, gtm_xlog.c:2013473723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->gtm_control_version = 2018071648LOCATION: GTM_PrintControlData, gtm_xlog.c:2014493723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->xlog_seg_size = 209715250LOCATION: GTM_PrintControlData, gtm_xlog.c:2015513723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->xlog_blcksz = 409652LOCATION: GTM_PrintControlData, gtm_xlog.c:2016533723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->state = 154LOCATION: GTM_PrintControlData, gtm_xlog.c:2017553723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->CurrBytePos = 835605256LOCATION: GTM_PrintControlData, gtm_xlog.c:2018573723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->PrevBytePos = 835592058LOCATION: GTM_PrintControlData, gtm_xlog.c:2019593723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->thisTimeLineID = 160LOCATION: GTM_PrintControlData, gtm_xlog.c:2020613723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->prevCheckPoint = 0/6000E462LOCATION: GTM_PrintControlData, gtm_xlog.c:2021633723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->checkPoint = 0/80000064LOCATION: GTM_PrintControlData, gtm_xlog.c:2022653723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->gts = 373843779366LOCATION: GTM_PrintControlData, gtm_xlog.c:2023673723429696:2024-11-29 09:58:21.857 CST -LOG: ControlData->time = 173284550068LOCATION: GTM_PrintControlData, gtm_xlog.c:202469Done.70Starting coordinator master.71Starting coordinator master cn00172ERROR: target coordinator master cn001 is already running now. Skip initialization.73Starting coordinator master cn00274ERROR: target coordinator master cn002 is already running now. Skip initialization.75Done.76Starting all the datanode masters.77Starting datanode master dn001.78WARNING: datanode master dn001 is running now. Skipping.79Starting datanode master dn002.80WARNING: datanode master dn002 is running now. Skipping.81Done.82Starting all the datanode slaves.83Starting datanode slave dn001.84Starting datanode slave dn002.85pg_ctl: another server might be running; trying to start server anyway862024-11-29 01:58:23.665 GMT [103473,coord(0.0)] LOG: skipping missing configuration file "/opentenbase/global/global_opentenbase.conf"872024-11-29 01:58:23.665 GMT [103473,coord(0.0)] LOG: skipping missing configuration file "/opentenbase/global/global_opentenbase.conf"882024-11-29 09:58:23.703 CST [103473,coord(0.0)] LOG: other_pid is datanode, result:openten+ 102251 1 0 09:50 ? 00:00:02 /opentenbase/install/opentenbase_bin_v2.6/bin/postgres --datanode -D /opentenbase/data/dn00189902024-11-29 09:58:23.704 CST [103473,coord(0.0)] FATAL: lock file "postmaster.pid" already exists912024-11-29 09:58:23.704 CST [103473,coord(0.0)] HINT: Is another postmaster (PID 102251) running in data directory "/opentenbase/data/dn001"?92pg_ctl: could not start server93Examine the log output.94pg_ctl: another server might be running; trying to start server anyway952024-11-29 01:58:22.954 GMT [125626,coord(0.0)] LOG: skipping missing configuration file "/opentenbase/global/global_opentenbase.conf"962024-11-29 01:58:22.955 GMT [125626,coord(0.0)] LOG: skipping missing configuration file "/opentenbase/global/global_opentenbase.conf"972024-11-29 09:58:22.979 CST [125626,coord(0.0)] LOG: other_pid is datanode, result:openten+ 124233 1 0 09:50 ? 00:00:02 /opentenbase/install/opentenbase_bin_v2.6/bin/postgres --datanode -D /opentenbase/data/dn00298992024-11-29 09:58:22.980 CST [125626,coord(0.0)] FATAL: lock file "postmaster.pid" already exists1002024-11-29 09:58:22.980 CST [125626,coord(0.0)] HINT: Is another postmaster (PID 124233) running in data directory "/opentenbase/data/dn002"?101pg_ctl: could not start server102Examine the log output.103Done.集群启动成功。
访问 OpenTenBase 集群和访问单机的 PostgreSQL 基本上无差别，我们可以通过任意一个 CN 访问数据库集群：例如通过连接 CN 节点 select pgxc_node 表即可查看集群的拓扑结构（当前的配置下备机不会展示在 pgxc_node 中），在 Linux 命令行下通过 psql 访问的具体示例如下：
1[opentenbase@oct01 ~]$ psql -h 192.168.6.87 -p 30004 -d postgres -U opentenbase2psql (PostgreSQL 10.0 OpenTenBase V2)3Type "help" for help.45postgres=# \d6Did not find any relations.7postgres=# select * from pgxc_node;8 node_name | node_type | node_port | node_host | nodeis_primary | nodeis_preferred | node_id | node_cluster_name 9-----------+-----------+-----------+--------------+----------------+------------------+------------+---------------------10 gtm | G | 50001 | 192.168.6.87 | t | f | 428125959 | opentenbase_cluster11 cn001 | C | 30004 | 192.168.6.87 | f | f | -264077367 | opentenbase_cluster12 cn002 | C | 30004 | 192.168.6.88 | f | f | -674870440 | opentenbase_cluster13 dn001 | D | 40004 | 192.168.6.87 | t | t | 2142761564 | opentenbase_cluster14 dn002 | D | 40004 | 192.168.6.88 | f | f | -17499968 | opentenbase_cluster15(5 rows)OpenTenBase 使用 datanode group 来增加节点的管理灵活度，要求有一个 default group 才能使用，因此需要预先创建。一般情况下，会将节点的所有 datanode 节点加入到 default group 里：
1postgres=# create default node group default_group with (dn001,dn002);2CREATE NODE GROUP另外一方面，OpenTenBase 的数据分布为了增加灵活度，加了中间逻辑层来维护数据记录到物理节点的映射，我们叫 sharding，所以需要预先创建 sharding 表：
1postgres=# create sharding group to group default_group;2CREATE SHARDING GROUP创建数据库，用户，创建表，增删查改：
1postgres=# create database lucifer;2CREATE DATABASE3postgres=# create user lucifer with password 'lucifer';4CREATE ROLE5postgres=# alter database lucifer owner to lucifer;6ALTER DATABASE7postgres=# \c lucifer lucifer8You are now connected to database "lucifer" as user "lucifer".9lucifer=> create table test(id bigint, str text) distribute by shard(id);10CREATE TABLE11lucifer=> insert into test values(1, 'tencent'), (2, 'shenzhen');12COPY 213lucifer=> select * from test;14 id | str 15----+----------16 1 | tencent17 2 | shenzhen18(2 rows)至此，就可以跟使用单机数据库一样来访问 OpenTenBase 数据库集群了。
TencentOS 3 在 PowerTools 中支持了 OpenTenBase 的 RPM 安装，可以直接安装单机版。
如果是联网情况，只需要开启 PowerTools 源即可实现一键安装 OpenTenBase：
1[PowerTools]2name=TencentOS Server $releasever - PowerTools3baseurl=http://mirrors.tencent.com/tlinux/$releasever/PowerTools/$basearch/os/4 http://mirrors.tencentyun.com/tlinux/$releasever/PowerTools/$basearch/os/5gpgcheck=16enabled=17gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-TencentOS-3一键安装单机 OpenTenBase：
1yum install -y OpenTenBase-2.6.0-1.tl3.x86_64.rpm如果是离线环境，则可以下载 OpenTenBase-2.6.0-1.tl3.x86_64.rpm：
上传主机后手动进行安装即可。
本文主要介绍如何从源码开始，一步一步搭建一个完整的 OpenTenBase 集群，网上相关的文档比较少，摸索过程中遇到了不少坑，好在最终都解决！
如果有遗漏或者不足的地方，欢迎评论区补充或者投稿，感谢阅读！
达梦数据库安装最详细教程
一招教你学会达梦数据库的免密登录
一文讲透达梦数据库的大小写敏感
效率翻倍！达梦数据库 disql 使用技巧全攻略
达梦数据库参数配置与一键优化指南
达梦 AWR 报告快速上手指南
金仓数据库 KingbaseES V9 单机安装指南
KingbaseES KSQL 免密登录的几种方式
KingbaseES 控制文件冗余与恢复秘籍
金仓 KingbaseES RAC 入门指南
GBase 8a MPP 集群部署最佳实践
GBase 8a GDCA 认证课后练习题大全（题库） 南大通用 GBASE 8s V8.8 数据库最全安装指南 GBase 8s GDCA 认证课后练习题大全（题库）
GBase 8s 数据库巡检报告及一键巡检脚本
YashanDB 一键生成 AWR 报告
YashanDB 数据库安装部署
YashanDB 共享集群（YAC）入门指南与技术详解