如果安装redis 5.0以下版本,则需要redis、ruby、redis-trib等各种组件安装,比较繁琐,在redis 5.0及以后,不需要ruby,仅在安装redis后执行一行命令即可。

前置条件:

  • Docker
  • redis:5.0.0镜像
1
2
[root@lemonlzy redis]# docker images | grep redis
redis 5.0.0 1babb1dde7e1 24 months ago 94.9MB

集群搭建


创建Redis 容器

1. 创建redis自定义network
1
$ docker network create redis-net

备注:创建redis-net虚拟网卡 目的是让docker容器能与宿主(centos7)桥接网络 并间接与外界连接

2. 查看redis-net虚拟网卡网关ip

这里的ip用于后续配置文件中的集群ip配置:

1
2
[root@lemonlzy redis]# docker network inspect redis-net | grep "Gateway" | grep --color=auto -P '(\d{1,3}.){3}\d{1,3}' -o
172.20.0.1
3. 创建redis配置文件

在自定义路径下下创建文件夹redis用于存放自定义的集群配置文件,并在该文件夹中新建redis6379.conf文件(以下演示中目录为/data/redis):

1
2
3
4
5
6
7
8
9
10
[root@lemonlzy redis]# cat redis6379.conf
port 6379 # 节点端口
cluster-enabled yes # 启用cluster集群模式
cluster-config-file nodes.conf # 集群配置自定义文件名
cluster-node-timeout 5000 # 超时时间
cluster-announce-ip 172.20.0.2 # 为各节点网卡分配ip,根据redis-net的ip自行调整
cluster-announce-port 6379 # 节点映射端口
cluster-announce-bus-port 16379 # 节点总线端
appendonly yes # 开启持久化模式
protected-mode no # 关闭protected-mode模式,此时外部网络可以直接访问

如果在开启防火墙的情况下,上述配置文件的自定义端口均需加入防火墙规则

4. 创建集群配置文件

创建redis配置文件的目录结构:

1
$ mkdir -p /data/redis/{6379/data/,6380/data/,6381/data/,6382/data/,6383/data/,6384/data/,6385/data/,6386/data/,6387/data/}

并将每一个配置文件移动至对应目录下(我这次开放6379-6387端口,所以配置文件调整为redis6379.conf-redis6387.conf),目录结构如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
[root@lemonlzy redis]# tree -L 2
.
|-- 6379
| |-- data
| `-- redis6379.conf
|-- 6380
| |-- data
| `-- redis6380.conf
|-- 6381
| |-- data
| `-- redis6381.conf
|-- 6382
| |-- data
| `-- redis6382.conf
|-- 6383
| |-- data
| `-- redis6383.conf
|-- 6384
| |-- data
| `-- redis6384.conf
|-- 6385
| |-- data
| `-- redis6385.conf
|-- 6386
| |-- data
| `-- redis6386.conf
|-- 6387
| |-- data
| `-- redis6387.conf

里面每一个配置文件都需调整为对应的端口,且cluster-announce-ip字段需逐步自增1,即redis-net的ip为172.20.0.1redis6379.confcluster-announce-ip172.20.0.2redis6380.confcluster-announce-ip172.20.0.3redis6381.confcluster-announce-ip172.20.0.4

5. 创建redis容器
1
2
3
4
5
6
7
8
9
docker run -d -p6379:6379 -p16379:16379 --restart always --name redis-6379 --net redis-net --privileged=true -v /data/redis/6379/redis6379.conf:/etc/redis/redis.conf -v /data/redis/6379/data:/data  redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf
docker run -d -p6380:6380 -p16380:16380 --restart always --name redis-6380 --net redis-net --privileged=true -v /data/redis/6380/redis6380.conf:/etc/redis/redis.conf -v /data/redis/6380/data:/data redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf
docker run -d -p6381:6381 -p16381:16381 --restart always --name redis-6381 --net redis-net --privileged=true -v /data/redis/6381/redis6381.conf:/etc/redis/redis.conf -v /data/redis/6381/data:/data redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf
docker run -d -p6382:6382 -p16382:16382 --restart always --name redis-6382 --net redis-net --privileged=true -v /data/redis/6382/redis6382.conf:/etc/redis/redis.conf -v /data/redis/6382/data:/data redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf
docker run -d -p6383:6383 -p16383:16383 --restart always --name redis-6383 --net redis-net --privileged=true -v /data/redis/6383/redis6383.conf:/etc/redis/redis.conf -v /data/redis/6383/data:/data redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf
docker run -d -p6384:6384 -p16384:16384 --restart always --name redis-6384 --net redis-net --privileged=true -v /data/redis/6384/redis6384.conf:/etc/redis/redis.conf -v /data/redis/6384/data:/data redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf
docker run -d -p6385:6385 -p16385:16385 --restart always --name redis-6385 --net redis-net --privileged=true -v /data/redis/6385/redis6385.conf:/etc/redis/redis.conf -v /data/redis/6385/data:/data redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf
docker run -d -p6386:6386 -p16386:16386 --restart always --name redis-6386 --net redis-net --privileged=true -v /data/redis/6386/redis6386.conf:/etc/redis/redis.conf -v /data/redis/6386/data:/data redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf
docker run -d -p6387:6387 -p16387:16387 --restart always --name redis-6387 --net redis-net --privileged=true -v /data/redis/6387/redis6387.conf:/etc/redis/redis.conf -v /data/redis/6387/data:/data redis:5.0.0 /usr/local/bin/redis-server /etc/redis/redis.conf

如果路径、配置文件名均为自定义,请自行调整docker命令挂载目录的地址以及暴露的端口号。

6. 查看容器分配ip

确认ip配置正确,每个容器的ip应为自定义配置中定义的cluster-announce-ip,用于后续进行集群生成:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
[root@lemonlzy redis]# docker network inspect redis-net
[
{
"Name": "redis-net",
"Id": "1937a531c5e70d8c4ed63bfabfb7e1aad0d7e7ee822a066b0014e024c2d6b0bf",
"Created": "2020-10-16T10:14:39.376661537+08:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.20.0.0/16",
"Gateway": "172.20.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"27619ee4a487834c87becf22faa8969fb07c5e60d338ef7a3251de7a5d890339": {
"Name": "redis-6386",
"EndpointID": "8bbea88a19ac96cdede3d693d2822040ae7bab41cd3f998be161ea852ba9fe5b",
"MacAddress": "02:42:ac:14:00:09",
"IPv4Address": "172.20.0.9/16",
"IPv6Address": ""
},
"4446a895ef9d0e9e01a0f04b766702ab6baccd2dc7e694f29daef44bafabe0b8": {
"Name": "redis-6383",
"EndpointID": "4eed1b3c8fb85ef55cfd8ca0ea6602098574eb2592f1725719e012540716d1ca",
"MacAddress": "02:42:ac:14:00:06",
"IPv4Address": "172.20.0.6/16",
"IPv6Address": ""
},
"4a10842ba1fb6c9fea9a854c9df368b6862a8a58a6735f2a17225cde4f1e33db": {
"Name": "redis-6382",
"EndpointID": "1db34d8233dc368774a9834c1ed82a11155639fc8eec9092fd0c2440d94e86f9",
"MacAddress": "02:42:ac:14:00:05",
"IPv4Address": "172.20.0.5/16",
"IPv6Address": ""
},
"4f8f2f65bb94c28ce441710bd90156872fb1f3ed02d41262630d09538df1470b": {
"Name": "redis-6384",
"EndpointID": "645d701717b703ff63824201f3f5df87eaf29979a5c4ce3c20831a000aec89a2",
"MacAddress": "02:42:ac:14:00:07",
"IPv4Address": "172.20.0.7/16",
"IPv6Address": ""
},
"8f52949e3b3c79f3f768aebd5e5d4e8a9e25ca124cd053ded2d89e163fb1d6a4": {
"Name": "redis-6381",
"EndpointID": "620e6546cd432010c77864b634c6729955a3884b4d80f47f546cbe24ac877ade",
"MacAddress": "02:42:ac:14:00:04",
"IPv4Address": "172.20.0.4/16",
"IPv6Address": ""
},
"92a627152f8f9a43785cb19fcbd11e376764f407a24ce06d447c20eff6f216c2": {
"Name": "redis-6379",
"EndpointID": "fb56afd5727b26576fa75649c76f123b2abfa9e1cbe36f0f7f91799600833402",
"MacAddress": "02:42:ac:14:00:02",
"IPv4Address": "172.20.0.2/16",
"IPv6Address": ""
},
"969c705823c24aa71b99d17bc366180dacc9dde0e53342f2f8943e88c544aa28": {
"Name": "redis-6380",
"EndpointID": "6094ce50275df3bca5a1577abe617bf124a0c7de85d73749f4ae889ab3ddc825",
"MacAddress": "02:42:ac:14:00:03",
"IPv4Address": "172.20.0.3/16",
"IPv6Address": ""
},
"cb67e53e99a89a3eedf5468abcb504e46225b110b75a6da20c67800e06d97836": {
"Name": "redis-6385",
"EndpointID": "e958b91b00157d015cdb30722706cc285b6019dce4235477e5bf15435ab4d2b1",
"MacAddress": "02:42:ac:14:00:08",
"IPv4Address": "172.20.0.8/16",
"IPv6Address": ""
},
"e2015e6591c1be45ec09375f238dd1e798076d251e0a291618988d2f273c3ccd": {
"Name": "redis-6387",
"EndpointID": "33312557d761552a0f56f7a62cd092a835316209e5042464200e4607fe56f2b8",
"MacAddress": "02:42:ac:14:00:0a",
"IPv4Address": "172.20.0.10/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}
]

启动集群

1. 进入任意redis容器
1
2
[root@lemonlzy redis]# docker exec -it redis-6379 bash
root@92a627152f8f:/data#
2. 创建redis集群

一行命令创建集群,此处的ip与端口号调整为上述自定义的端口号,若无自行改动直接复制使用即可:

1
redis-cli --cluster create 172.20.0.2:6379 172.20.0.3:6380 172.20.0.4:6381 172.20.0.5:6382 172.20.0.6:6383 172.20.0.7:6384 172.20.0.8:6385  172.20.0.9:6386  172.20.0.10:6387 --cluster-replicas 1

具体的命令执行结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
[root@lemonlzy redis]# docker exec -it redis-6379 bash
root@92a627152f8f:/data# redis-cli --cluster create 172.20.0.2:6379 172.20.0.3:6380 172.20.0.4:6381 172.20.0.5:6382 172.20.0.6:6383 172.20.0.7:6384 172.20.0.8:6385 172.20.0.9:6386 172.20.0.10:6387 --cluster-replicas 1
>>> Performing hash slots allocation on 9 nodes...
Master[0] -> Slots 0 - 4095
Master[1] -> Slots 4096 - 8191
Master[2] -> Slots 8192 - 12287
Master[3] -> Slots 12288 - 16383
Adding replica 172.20.0.6:6383 to 172.20.0.2:6379
Adding replica 172.20.0.7:6384 to 172.20.0.3:6380
Adding replica 172.20.0.8:6385 to 172.20.0.4:6381
Adding replica 172.20.0.9:6386 to 172.20.0.5:6382
Adding extra replicas...
Adding replica 172.20.0.10:6387 to 172.20.0.2:6379
M: 0b8fce7d53267e09b591bd89884f7bdc9f302c1f 172.20.0.2:6379
slots:[0-4095] (4096 slots) master
M: c6d8bd75d9f46f471cd64f7f61f2d23841a293bc 172.20.0.3:6380
slots:[4096-8191] (4096 slots) master
M: fd223524965320fba7b30b4acff26a331370a32f 172.20.0.4:6381
slots:[8192-12287] (4096 slots) master
M: 933bcb5a0124a3c83c3f161b867016827b216020 172.20.0.5:6382
slots:[12288-16383] (4096 slots) master
S: 9c3cd6804fa682c08b6bacc0fc91b3bed4b83470 172.20.0.6:6383
replicates 0b8fce7d53267e09b591bd89884f7bdc9f302c1f
S: 9bffee0bedb8b1e92f2a092047f2989c11ceeac6 172.20.0.7:6384
replicates c6d8bd75d9f46f471cd64f7f61f2d23841a293bc
S: ec1fc924a2bccb42522b8bba3ffa207770b2efc3 172.20.0.8:6385
replicates fd223524965320fba7b30b4acff26a331370a32f
S: e36c059618291399117f23d4bb924a00ec5fba89 172.20.0.9:6386
replicates 933bcb5a0124a3c83c3f161b867016827b216020
S: e6e8a2461293628dcc0995fcef66f8bd89c802c6 172.20.0.10:6387
replicates 0b8fce7d53267e09b591bd89884f7bdc9f302c1f
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 172.20.0.2:6379)
M: 0b8fce7d53267e09b591bd89884f7bdc9f302c1f 172.20.0.2:6379
slots:[0-4095] (4096 slots) master
2 additional replica(s)
S: e36c059618291399117f23d4bb924a00ec5fba89 172.20.0.1:6386
slots: (0 slots) slave
replicates 933bcb5a0124a3c83c3f161b867016827b216020
S: ec1fc924a2bccb42522b8bba3ffa207770b2efc3 172.20.0.1:6385
slots: (0 slots) slave
replicates fd223524965320fba7b30b4acff26a331370a32f
S: 9c3cd6804fa682c08b6bacc0fc91b3bed4b83470 172.20.0.1:6383
slots: (0 slots) slave
replicates 0b8fce7d53267e09b591bd89884f7bdc9f302c1f
M: fd223524965320fba7b30b4acff26a331370a32f 172.20.0.1:6381
slots:[8192-12287] (4096 slots) master
1 additional replica(s)
M: c6d8bd75d9f46f471cd64f7f61f2d23841a293bc 172.20.0.1:6380
slots:[4096-8191] (4096 slots) master
1 additional replica(s)
M: 933bcb5a0124a3c83c3f161b867016827b216020 172.20.0.1:6382
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
S: 9bffee0bedb8b1e92f2a092047f2989c11ceeac6 172.20.0.1:6384
slots: (0 slots) slave
replicates c6d8bd75d9f46f471cd64f7f61f2d23841a293bc
S: e6e8a2461293628dcc0995fcef66f8bd89c802c6 172.20.0.1:6387
slots: (0 slots) slave
replicates 0b8fce7d53267e09b591bd89884f7bdc9f302c1f
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

登录redis,查看主从信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
root@92a627152f8f:/data# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.20.0.1,port=6387,state=online,offset=84,lag=1
slave1:ip=172.20.0.1,port=6383,state=online,offset=84,lag=1
master_replid:6f556f924ab7026a6dd1029aa5b6cd19eb32c607
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:84
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:84

查看集群信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:9
cluster_size:4
cluster_current_epoch:9
cluster_my_epoch:1
cluster_stats_messages_ping_sent:99
cluster_stats_messages_pong_sent:106
cluster_stats_messages_sent:205
cluster_stats_messages_ping_received:98
cluster_stats_messages_pong_received:99
cluster_stats_messages_meet_received:8
cluster_stats_messages_received:205
3. 测试
1
2
3
4
5
6
root@86dc7ab9335a:/data# redis-cli -c -h 172.20.0.2 -p 6379
172.20.0.2:6379> set name zhangsan
-> Redirected to slot [5798] located at 172.20.0.3:6380
OK
172.20.0.3:6380> get name
"zhangsan"

在6379设置了key为name,value为zhangsan的键值对,在6380端口使用get仍可获取到对应的value值,集群搭建成功。

如果出现以下报错,原因是没有用集群模式连接(连接节点命令,没有加 -c 参数),请参考上述步骤连接。

1
2
127.0.0.1:6379> set name zhangshan
(error) MOVED 5798 172.20.0.3:6380

外部访问,访问本机ip+6379-6387任一端口号即可。