docker搭建redis集群
环境
- 系统 麒麟Kylin （V10 SP3）
- Redis 7.2.4
前期准备
- 调整系统参数
less -N /etc//sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
kernel.sysrq=0
net.ipv4.ip_forward=0
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.default.send_redirects=0
net.ipv4.conf.all.accept_source_route=0
net.ipv4.conf.default.accept_source_route=0
net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.default.accept_redirects=0
net.ipv4.conf.all.secure_redirects=0
net.ipv4.conf.default.secure_redirects=0
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_responses=1
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.default.rp_filter=1
net.ipv4.tcp_syncookies=1
kernel.dmesg_restrict=1
net.ipv6.conf.all.accept_redirects=0
net.ipv6.conf.default.accept_redirects=0
###新增以下配置
vm.overcommit_memory = 1
net.core.somaxconn=65535
#配置生效
sysctl -p
- 创建配置文件
注：
如果节点配置文件设置了密码：requirepass xxxxx
那么节点配置文件需要添加 masterauth xxxxx 参数
cat create_conf.sh
#!/bin/bash
for port in $(seq 6379 6384);
do
mkdir -p common/node-${port}/conf
touch common/node-${port}/conf/redis.conf
cat << EOF > common/node-${port}/conf/redis.conf
port ${port}
requirepass xxxxx （redis 密码，可以不设）
masterauth xxxxx
bind 0.0.0.0
protected-mode no
daemonize no
appendonly yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 10.0.xx.xx (替换为宿主机IP)
cluster-announce-port ${port}
cluster-announce-bus-port 1${port}
EOF
done
- 创建数据目录
mkdir data/node-{6379..6384}
查看目录
[root@Kylin redis]# ls
common create_conf.sh data docker-compose.yml
查看数据目录
[root@Kylin redis]# tree data
data
├── node-6379
├── node-6380
├── node-6381
├── node-6382
├── node-6383
└── node-6384
服务启动
- 查看docker-cmopose文件
version: "3"
services:
redis1:
image: redis:7.2.4
container_name: redis-1
restart: on-failure:3
privileged: true
network_mode: "host"
environment:
- TZ=Asia/Shanghai
ports:
- "6379:6379"
- "16379:16379"
volumes:
- ./common/node-6379/conf/redis.conf:/usr/local/etc/redis/redis.conf
- ./data/node-6379:/data
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
redis2:
image: redis:7.2.4
container_name: redis-2
restart: on-failure:3
privileged: true
network_mode: "host"
environment:
- TZ=Asia/Shanghai
ports:
- "6380:6380"
- "16380:16380"
volumes:
- ./common/node-6380/conf/redis.conf:/usr/local/etc/redis/redis.conf
- ./data/node-6380:/data
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
redis3:
image: redis:7.2.4
container_name: redis-3
restart: on-failure:3
privileged: true
network_mode: "host"
environment:
- TZ=Asia/Shanghai
ports:
- "6381:6381"
- "16381:16381"
volumes:
- ./common/node-6381/conf/redis.conf:/usr/local/etc/redis/redis.conf
- ./data/node-6381:/data
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
redis4:
image: redis:7.2.4
container_name: redis-4
restart: on-failure:3
privileged: true
network_mode: "host"
environment:
- TZ=Asia/Shanghai
ports:
- "6382:6382"
- "16382:16382"
volumes:
- ./common/node-6382/conf/redis.conf:/usr/local/etc/redis/redis.conf
- ./data/node-6382:/data
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
redis5:
image: redis:7.2.4
container_name: redis-5
restart: on-failure:3
privileged: true
network_mode: "host"
environment:
- TZ=Asia/Shanghai
ports:
- "6383:6383"
- "16383:16383"
volumes:
- ./common/node-6383/conf/redis.conf:/usr/local/etc/redis/redis.conf
- ./data/node-6383:/data
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
redis6:
image: redis:7.2.4
container_name: redis-6
restart: on-failure:3
privileged: true
network_mode: "host"
environment:
- TZ=Asia/Shanghai
ports:
- "6384:6384"
- "16384:16384"
volumes:
- ./common/node-6384/conf/redis.conf:/usr/local/etc/redis/redis.conf
- ./data/node-6384:/data
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
- 服务查看
[root@Kylin redis]# docker-compose up -d
[+] Running 12/12
✔ Container redis-1 Started 2.6s
✔ Container redis-3 Started 2.6s
✔ Container redis-4 Started 2.4s
✔ Container redis-5 Started 2.6s
✔ Container redis-2 Started 2.4s
✔ Container redis-6 Started 1.8s
! redis2 Published ports are discarded when using host network mode 0.0s
! redis4 Published ports are discarded when using host network mode 0.0s
! redis1 Published ports are discarded when using host network mode 0.0s
! redis3 Published ports are discarded when using host network mode 0.0s
! redis5 Published ports are discarded when using host network mode 0.0s
! redis6 Published ports are discarded when using host network mode 0.0s
[root@Kylin redis]# docker-compose ps -a
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
redis-1 redis:7.2.4 "docker-entrypoint.s…" redis1 About an hour ago Up About an hour
redis-2 redis:7.2.4 "docker-entrypoint.s…" redis2 About an hour ago Up About an hour
redis-3 redis:7.2.4 "docker-entrypoint.s…" redis3 About an hour ago Up About an hour
redis-4 redis:7.2.4 "docker-entrypoint.s…" redis4 About an hour ago Up About an hour
redis-5 redis:7.2.4 "docker-entrypoint.s…" redis5 About an hour ago Up About an hour
redis-6 redis:7.2.4 "docker-entrypoint.s…" redis6 About an hour ago Up About an hour
[root@Kylin redis]# ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 511 0.0.0.0:16379 0.0.0.0:*
LISTEN 0 511 0.0.0.0:16380 0.0.0.0:*
LISTEN 0 511 0.0.0.0:16381 0.0.0.0:*
LISTEN 0 511 0.0.0.0:16382 0.0.0.0:*
LISTEN 0 511 0.0.0.0:16383 0.0.0.0:*
LISTEN 0 511 0.0.0.0:16384 0.0.0.0:*
LISTEN 0 511 0.0.0.0:6379 0.0.0.0:*
LISTEN 0 511 0.0.0.0:6380 0.0.0.0:*
LISTEN 0 511 0.0.0.0:6381 0.0.0.0:*
LISTEN 0 511 0.0.0.0:6382 0.0.0.0:*
LISTEN 0 511 0.0.0.0:6383 0.0.0.0:*
LISTEN 0 511 0.0.0.0:6384 0.0.0.0:*
配置集群
#随机登录一个redis节点
docker exec -it redis-1 sh
redis-cli -a "Redis密码" --cluster create 10.0.xx.xx:6379 10.0.xx.xx:6380 10.0.xx.xx:6381 10.0.xx.xx:6382 10.0.xx.xx:6383 10.0.xx.xx:6384 --cluster-replicas 1
提示输入yes即可，等待出现以下内容，这表示成功
[0K] All nodes agree about slots configuration.
Check for open slots...
>>> Check slots coverage...
[0K] All 16384 slots covered.
测试
#再次登录redis-1 容器
docker exec -it redis-1 sh
#查看集群情况
redis-cli -a “redis密码”
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:7303
cluster_stats_messages_pong_sent:7326
cluster_stats_messages_sent:14629
cluster_stats_messages_ping_received:7321
cluster_stats_messages_pong_received:7303
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:14629
total_cluster_links_buffer_limit_exceeded:0
#查看集群节点
127.0.0.1:6379> cluster nodes
5646099b2a3859c412af566459510a93e27d2bad 10.0.xx.xx:6379@16379 myself,master - 0 1708599347000 1 connected 0-5460
df87c420ee15dc5909e406b947608571b33dc5df 10.0.xx.xx:6380@16380 master - 0 1708599349000 2 connected 5461-10922
14a93c49f6e12ef97f1292a9535286a04481d78b 10.0.xx.xx:6382@16382 slave 2f6e68e92e958fba56a0bddf555475ba9ab4878b 0 1708599350000 3 connected
6b62228ecdd9df253638d92a43c5c8eb59c10760 10.0.xx.xx:6383@16383 slave 5646099b2a3859c412af566459510a93e27d2bad 0 1708599348000 1 connected
2f6e68e92e958fba56a0bddf555475ba9ab4878b 10.0.xx.xx:6381@16381 master - 0 1708599350299 3 connected 10923-16383
217d7296d3d17c062546d9e58292f96cd15d13c3 10.0.xx.xx:6384@16384 slave df87c420ee15dc5909e406b947608571b33dc5df 0 1708599349000 2 connected
#从其他服务器登录链接redis写入数据进行验证
登录10.0.11.20
ssh [email protected]
docker exec -it redis sh （登录11.20的redis服务）
redis-cli -c -a "Redis密码" -h 10.0.xx.xx -p 6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.0.xx.xx:6379> set test 'hello world'
-> Redirected to slot [6918] located at 10.0.xx.xx:6380
OK
10.0.xx.xx:6380> get test
"hello world"
##通slave 容器日志验证结果
docker logs -f redis-4
1:S 23 Feb 2024 03:28:23.106 * Connecting to MASTER 10.0.xx.xx:6381
1:S 23 Feb 2024 03:28:23.106 * MASTER <-> REPLICA sync started
1:S 23 Feb 2024 03:28:23.106 * Cluster state changed: ok
1:S 23 Feb 2024 03:28:23.106 * Non blocking connect for SYNC fired the event.
1:S 23 Feb 2024 03:28:23.211 * Master replied to PING, replication can continue...
1:S 23 Feb 2024 03:28:23.279 * Trying a partial resynchronization (request 135dc76f87b7b3d07f78542e878deb65f1a733ad:1).
1:S 23 Feb 2024 03:28:28.681 * Full resync from master: 841dce9a89d9c248965abf04bd1a93d8ce1c8bf3:0
1:S 23 Feb 2024 03:28:28.684 * MASTER <-> REPLICA sync: receiving streamed RDB from master with EOF to disk
1:S 23 Feb 2024 03:28:28.684 * Discarding previously cached master state.
1:S 23 Feb 2024 03:28:28.684 * MASTER <-> REPLICA sync: Flushing old data
1:S 23 Feb 2024 03:28:28.685 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 23 Feb 2024 03:28:28.723 * Loading RDB produced by version 7.2.4
1:S 23 Feb 2024 03:28:28.723 * RDB age 0 seconds
1:S 23 Feb 2024 03:28:28.723 * RDB memory usage when created 1.61 Mb
1:S 23 Feb 2024 03:28:28.723 * Done loading RDB, keys loaded: 0, keys expired: 0.
1:S 23 Feb 2024 03:28:28.723 * MASTER <-> REPLICA sync: Finished with success
1:S 23 Feb 2024 03:28:28.723 * Creating AOF incr file temp-appendonly.aof.incr on background rewrite
1:S 23 Feb 2024 03:28:28.724 * Background append only file rewriting started by pid 21
21:C 23 Feb 2024 03:28:28.747 * Successfully created the temporary AOF base file temp-rewriteaof-bg-21.aof
21:C 23 Feb 2024 03:28:28.748 * Fork CoW for AOF rewrite: current 4 MB, peak 4 MB, average 4 MB
1:S 23 Feb 2024 03:28:28.830 * Background AOF rewrite terminated with success
1:S 23 Feb 2024 03:28:28.830 * Successfully renamed the temporary AOF base file temp-rewriteaof-bg-21.aof into appendonly.aof.3.base.rdb
1:S 23 Feb 2024 03:28:28.830 * Successfully renamed the temporary AOF incr file temp-appendonly.aof.incr into appendonly.aof.3.incr.aof
1:S 23 Feb 2024 03:28:28.878 * Removing the history file appendonly.aof.2.incr.aof in the background
1:S 23 Feb 2024 03:28:28.878 * Removing the history file appendonly.aof.2.base.rdb in the background
1:S 23 Feb 2024 03:28:28.921 * Background AOF rewrite finished successfully
官方集群搭建教程
创建redis集群
集群搭建脚本