redis线上问题处理

1 ERR max number of clients reached

2023-06-14 15:24:34 ERROR redis.clients.jedis.JedisSentinelPool          Lost connection to Sentinel at 10.101.10.113:26379. Sleeping 5000ms and retrying.
redis.clients.jedis.exceptions.JedisDataException: ERR max number of clients reached
        at redis.clients.jedis.Protocol.processError(Protocol.java:132)
        at redis.clients.jedis.Protocol.process(Protocol.java:166)
        at redis.clients.jedis.Protocol.read(Protocol.java:220)
        at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:278)
        at redis.clients.jedis.Connection.getUnflushedObjectMultiBulkReply(Connection.java:240)
        at redis.clients.jedis.Connection.getObjectMultiBulkReply(Connection.java:245)
        at redis.clients.jedis.Jedis.sentinelGetMasterAddrByName(Jedis.java:3058)
        at redis.clients.jedis.JedisSentinelPool$MasterListener.run(JedisSentinelPool.java:364)

进入到哨兵模式，无法获取到连接。看来应该更改的是sentinel.conf中的maxclients,修改redis.conf中的没用

./redis-cli -h 10.101.10.103 -p 26379
10.101.10.103:26379> info
ERR max number of clients reached

跟进查看到redis.conf中配置已经设置为1万了，那么为什么还会报错呢？

除了这个配置，还需要修改tcp-keepalive 60，加快回收连接速度
修改配置后，重新进入

[bwuser@centos79 bin]$ ./redis-cli -h 10.101.10.103 -p 26379
10.101.10.103:26379> info
# Server
redis_version:3.0.7
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:5b3481d3a8abe975
redis_mode:sentinel
os:Linux 3.10.0-1160.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.8.5
process_id:29862
run_id:a97d15c42de81eca5f628a6c57fc364d49ff4101
tcp_port:26379
uptime_in_seconds:128
uptime_in_days:0
hz:13
lru_clock:9011610
config_file:/home/bwuser/appdata/redis-3.0.7/sentinel.conf

# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=mymaster,status=ok,address=10.101.10.103:6379,slaves=2,sentinels=3

上面的配置没有看到connected_clients,但可以查看到有那些客户端连接进去。

10.101.10.103:26379> client list
id=78 addr=10.101.10.127:14191 fd=88 name= age=191 idle=2 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=sentinel
id=16 addr=10.101.10.127:64743 fd=28 name= age=204 idle=23 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
id=32 addr=10.101.10.90:18883 fd=44 name= age=203 idle=203 flags=N db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=subscribe
id=73 addr=10.101.10.127:57158 fd=83 name= age=199 idle=11 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=sentinel
id=64 addr=10.101.10.128:2470 fd=74 name= age=202 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=sentinel

2 redis哨兵模式进去删不正确的数据