Nacos源码解析:Nacos配置管理-客户端篇

​本文源码来源于Nacos2.1.0版本https://github.com/alibaba/nacos/releases/tag/2.1.0

一、一个例子

Nacos获取配置以及监听配置变更的基本使用方式如下:

public class NacosConfigExample {

    public static void main(String[] args) throws NacosException, InterruptedException {
        String dataId = "test";
        String group = "DEFAULT_GROUP";
        Properties properties = new Properties();
        properties.put("namespace","test");
        properties.put("serverAddr", "192.168.153.101");

        //创建NacosConfigService的实例
        ConfigService configService = NacosFactory.createConfigService(properties);

        //获取配置内容
        String content = configService.getConfig(dataId, group, 5000);
        System.out.println("content:" + content);

        //添加监听器,监听配置的变更
        configService.addListener(dataId, group, new Listener() {
            @Override
            public void receiveConfigInfo(String configInfo) {
                //收到新的配置
                System.out.println("receive:" + configInfo);
            }

            @Override
            public Executor getExecutor() {
                return null;
            }
        });
    }
}

二、ConfigService的实现

由上面的示例知道,Nacos是通过ConfigService类对配置进行管理的,通过NacosFactory创建NacosConfigService的实例,其基本属性和构造函数如下(删减了部分非关键代码):

public class NacosConfigService implements ConfigService {
	 //封装了通过grpc方式管理Nacos配置的接口,包括发布/获取/删除配置以及监听配置变更等功能
    private final ClientWorker worker;

    //命名空间
    private String namespace;

    //配置过滤器管理器
    private final ConfigFilterChainManager configFilterChainManager;
    
	public NacosConfigService(Properties properties) throws NacosException {
	        ValidatorUtils.checkInitParam(properties);
	        initNamespace(properties);
	
	        //初始化configFilterChainManager,用于管理配置过滤器链,ConfigFilterChainManager#addFilter()可对配置进行过滤处理
	        //过滤器的加载以SPI的方式实现,用户可自定义实现过滤器
	        this.configFilterChainManager = new ConfigFilterChainManager(properties);
	
	        //serverListManager用于管理Nacos服务端的服务器列表信息
	        //如果properties包含serverAddr,则使用固定服务器;否则,则会在启动时通过http接口方式动态获取服务器信息
	        ServerListManager serverListManager = new ServerListManager(properties);
	        //启动serverListManager,如果不是使用固定服务器,则会通过http接口方式获取服务器信息,并且创建一个定时任务,每隔30秒获取一次服务器信息
	        serverListManager.start();
	
	        //ClientWorker封装了通过grpc方式管理Nacos配置的接口,包括发布/获取/删除配置以及监听配置变更等功能
	        this.worker = new ClientWorker(this.configFilterChainManager, serverListManager, properties);
	
	        // will be deleted in 2.0 later versions
	        //目前采用的通信方式是grpc,所以该agent已经废弃
	        agent = new ServerHttpAgent(serverListManager);
	}
}

主要关注两个地方:

  • ServerListManager: 负责管理服务器列表信息,可使用固定服务器信息,也可采用动态获取的方式,线上环境推荐使用动态获取方式,方便上线/下线Nacos服务器
  • ClientWorker: Nacos2.0之后采用grpc作为通信的方式,ClientWorker封装了通过grpc方式管理Nacos配置的接口

ClientWorker的基本属性和构造函数如下(删减了部分非关键代码):

public class ClientWorker implements Closeable {
	//groupKey -> cacheData, groupKey = dataId+group+tenant
    private final AtomicReference<Map<String, CacheData>> cacheMap = new AtomicReference<>(new HashMap<>());

    //配置过滤链管理器
    private final ConfigFilterChainManager configFilterChainManager;
    
    //rpc通信客户端
    private ConfigTransportClient agent;
  
    //新建CacheData时,是否从远程服务器同步配置
    private boolean enableRemoteSyncConfig = false;

	public ClientWorker(final ConfigFilterChainManager configFilterChainManager, ServerListManager serverListManager,
            final Properties properties) throws NacosException {
        this.configFilterChainManager = configFilterChainManager;
        init(properties);

        //实例化ConfigRpcTransportClient
        agent = new ConfigRpcTransportClient(properties, serverListManager);
        int count = ThreadUtils.getSuitableThreadCount(THREAD_MULTIPLE);
        ScheduledExecutorService executorService = Executors
                .newScheduledThreadPool(Math.max(count, MIN_THREAD_NUM), r -> {
                    Thread t = new Thread(r);
                    t.setName("com.alibaba.nacos.client.Worker");
                    t.setDaemon(true);
                    return t;
                });
        agent.setExecutor(executorService);
        //启动agent,这里实际上调用ConfigRpcTransportClient#startInternal(),会创建一个线程,用于不断轮训配置是否发生变更
        agent.start();
    }
}

既然上面提到了ConfigRpcTransportClient,那就先简单说一下部分代码的实现(删减了部分非关键代码):

public class ConfigRpcTransportClient extends ConfigTransportClient {
	 //中文直译过来就是监听执行铃声。实际就是长度为1的阻塞队列
     private final BlockingQueue<Object> listenExecutebell = new ArrayBlockingQueue<Object>(1);

     //铃声项,仅作为listenExecutebell的元素
     private Object bellItem = new Object();

	 //最后一次同步所有监听的配置的时间
     private long lastAllSyncTime = System.currentTimeMillis();
     
     public ConfigRpcTransportClient(Properties properties, ServerListManager serverListManager) {
         super(properties, serverListManager);
     }
     
     @Override
     public void notifyListenConfig() {
     	 //以非阻塞方式添加元素,队列长度为1
         listenExecutebell.offer(bellItem);
     } 
	
	 @Override
     public void startInternal() {
         executor.schedule(() -> {
             while (!executor.isShutdown() && !executor.isTerminated()) {
                 try {
                     //这里是一个阻塞队列,如果有数据,直接返回;没有数据则等待5s
                     //注:当服务端配置变更的时候,会通知客户端(ConfigChangeNotifyRequest),客户端则会调用ConfigRpcTransportClient#notifyListenConfig(),往listenExecutebell添加数据
                     //具体见ConfigRpcTransportClient#initRpcClientHandler()
                     listenExecutebell.poll(5L, TimeUnit.SECONDS);
                     if (executor.isShutdown() || executor.isTerminated()) {
                         continue;
                     }

                     //执行配置变更监听,这里会以grpc的方式将本地缓存的配置的md5发送给服务端进行比较
                     //若服务端返回的数据不为空,说明有配置发生变更,则再以grpc的方式请求最新的配置
                     executeConfigListen();
                 } catch (Exception e) {
                     LOGGER.error("[ rpc listen execute ] [rpc listen] exception", e);
                 }
             }
         }, 0L, TimeUnit.MILLISECONDS);
     }
}

从上面ConfigRpcTransportClient的代码可以看到,其使用一个长度为1的阻塞队列listenExecutebell实现了实时或定时监听配置变更并更新。(这个设计太巧妙了.jpg)

三、配置的获取(NacosConfigService#getConfig)

 public String getConfig(String dataId, String group, long timeoutMs) throws NacosException {
     return getConfigInner(namespace, dataId, group, timeoutMs);
}

getConfig()实际上是调用内部一个私有方法getConfigInner():

private String getConfigInner(String tenant, String dataId, String group, long timeoutMs) throws NacosException {
 	group = blank2defaultGroup(group);
    ParamUtils.checkKeyParam(dataId, group);
    ConfigResponse cr = new ConfigResponse();

    cr.setDataId(dataId);
    cr.setTenant(tenant);
    cr.setGroup(group);

    //1.优先使用本地配置文件
    String content = LocalConfigInfoProcessor.getFailover(worker.getAgentName(), dataId, group, tenant);
    if (content != null) {
        LOGGER.warn("[{}] [get-config] get failover ok, dataId={}, group={}, tenant={}, config={}",
                worker.getAgentName(), dataId, group, tenant, ContentUtils.truncateContent(content));
        cr.setContent(content);
        String encryptedDataKey = LocalEncryptedDataKeyProcessor
                .getEncryptDataKeyFailover(agent.getName(), dataId, group, tenant);
        cr.setEncryptedDataKey(encryptedDataKey);
        configFilterChainManager.doFilter(null, cr);
        content = cr.getContent();
        return content;
    }

    try {
        //2.本地配置文件不存在,从远程服务器获取配置。当获取配置成功时,会写入本地快照
        ConfigResponse response = worker.getServerConfig(dataId, group, tenant, timeoutMs, false);
        cr.setContent(response.getContent());
        cr.setEncryptedDataKey(response.getEncryptedDataKey());
        configFilterChainManager.doFilter(null, cr);
        content = cr.getContent();

        return content;
    } catch (NacosException ioe) {
        if (NacosException.NO_RIGHT == ioe.getErrCode()) {
            throw ioe;
        }
        LOGGER.warn("[{}] [get-config] get from server error, dataId={}, group={}, tenant={}, msg={}",
                worker.getAgentName(), dataId, group, tenant, ioe.toString());
    }

    LOGGER.warn("[{}] [get-config] get snapshot ok, dataId={}, group={}, tenant={}, config={}",
            worker.getAgentName(), dataId, group, tenant, ContentUtils.truncateContent(content));

    //3.远程服务器不可用时,使用本地配置快照做容灾处理。快照会在适当的时机更新,但没有过期机制
    content = LocalConfigInfoProcessor.getSnapshot(worker.getAgentName(), dataId, group, tenant);
    cr.setContent(content);
    String encryptedDataKey = LocalEncryptedDataKeyProcessor
            .getEncryptDataKeyFailover(agent.getName(), dataId, group, tenant);
    cr.setEncryptedDataKey(encryptedDataKey);
    configFilterChainManager.doFilter(null, cr);
    content = cr.getContent();
    return content;
}

由上面的代码可知,配置的获取包含三种方式:

  1. 优先获取本地配置,默认路径为:/{user.home}/nacos/config/fixed-{namespace}-{server.ip}_{server.port}_nacos/data/config-data-tenant/{namespace}/{group}/{dataId}
  2. 本地配置不存在,从远程Nacos服务器获取配置,获取成功则写入本地快照
  3. 从服务器获取配置失败,使用本地快照做容灾处理,快照会在适当的时机更新,但没有过期机制,默认路径为:/{user.home}/nacos/config/fixed-{namespace}-{server.ip}_{server.port}_nacos/data/snapshot-tenant/{namespace}/{group}/{dataId}

四、配置的监听(NacosConfigService#addListener)

public void addListener(String dataId, String group, Listener listener) throws NacosException {
   worker.addTenantListeners(dataId, group, Arrays.asList(listener));
}

由上面代码可知,addLIstener()实际上是调用了ClientWorker的addTenantListeners()方法。下面看下其实现:

public void addTenantListeners(String dataId, String group, List<? extends Listener> listeners)
         throws NacosException {
     group = blank2defaultGroup(group);
     String tenant = agent.getTenant();
     //如果不存在对应的CacheData则新建一个。CacheData放在一个map中,key为dataId+group+tenant组成
     CacheData cache = addCacheDataIfAbsent(dataId, group, tenant);
     synchronized (cache) {
         for (Listener listener : listeners) {
             cache.addListener(listener);
         }
         //添加监听器后,需与服务器端的配置做比较同步
         cache.setSyncWithServer(false);

         //通知监听配置变更,可理解为发布一个监听变更的事件。从上面agent(ConfigRpcTransportClient)的实现中看到,它会去轮询发布的事件并做配置变更监听和更新(调用ConfigRpcTransportClient#executeConfigListen)
         agent.notifyListenConfig();
     }
}

public CacheData addCacheDataIfAbsent(String dataId, String group, String tenant) throws NacosException {
    CacheData cache = getCache(dataId, group, tenant);
    if (null != cache) {
        return cache;
    }
    //key = dataId + group + tenant
    String key = GroupKey.getKeyTenant(dataId, group, tenant);
    synchronized (cacheMap) {
        CacheData cacheFromMap = getCache(dataId, group, tenant);
        // multiple listeners on the same dataid+group and race condition,so
        // double check again
        // other listener thread beat me to set to cacheMap
        //多个线程并发为同个配置设置监听器,需做二次检查
        if (null != cacheFromMap) {
            cache = cacheFromMap;
            // reset so that server not hang this check
            cache.setInitializing(true);
        } else {
            //初始化本地缓存配置数据
            cache = new CacheData(configFilterChainManager, agent.getName(), dataId, group, tenant);
            //计算当前配置对应的任务id,一个任务负责处理多个配置的变更监听,perTaskConfigSize =3000
            int taskId = cacheMap.get().size() / (int) ParamUtil.getPerTaskConfigSize();
            cache.setTaskId(taskId);
            // fix issue # 1317
            if (enableRemoteSyncConfig) {
                //从远程服务器获取配置内容
                ConfigResponse response = getServerConfig(dataId, group, tenant, 3000L, false);
                cache.setContent(response.getContent());
            }
        }
        
        Map<String, CacheData> copy = new HashMap<>(this.cacheMap.get());
        copy.put(key, cache);
        cacheMap.set(copy);
    }
    LOGGER.info("[{}] [subscribe] {}", agent.getName(), key);
    
    MetricsMonitor.getListenConfigCountMonitor().set(cacheMap.get().size());
    
    return cache;
}

从上面的代码可以看到,客户端使用CacheData类来缓存或管理配置信息,其基本实现如下(删减了部分非关键代码):

public class CacheData {
    //配置过滤链
    private final ConfigFilterChainManager configFilterChainManager;

    //dataId
    public final String dataId;

    //分组
    public final String group;

    //租户(命名空间)
    public final String tenant;

    //监听器列表,这里使用的是ManagerListenerWrap,是对原有Listener的包装
    private final CopyOnWriteArrayList<ManagerListenerWrap> listeners;

    //配置(content)的md5
    private volatile String md5;

    //是否使用本地配置
    private volatile boolean isUseLocalConfig = false;
    
    //本地配置最后修改时间
    private volatile long localConfigLastModified;

    //配置内容
    private volatile String content;

    //加密key
    private volatile String encryptedDataKey;
  
    //最后修改时间戳
    private volatile AtomicLong lastModifiedTs = new AtomicLong(0);

    //任务id
    private int taskId;

    //是否初始化中
    private volatile boolean isInitializing = true;
    
    //是否从服务端同步了最新数据(content)
    private volatile boolean isSyncWithServer = false;
	
	public CacheData(ConfigFilterChainManager configFilterChainManager, String name, String dataId, String group,
            String tenant) {
        if (null == dataId || null == group) {
            throw new IllegalArgumentException("dataId=" + dataId + ", group=" + group);
        }
        this.name = name;
        this.configFilterChainManager = configFilterChainManager;
        this.dataId = dataId;
        this.group = group;
        this.tenant = tenant;
        this.listeners = new CopyOnWriteArrayList<>();
        this.isInitializing = true;
        if (initSnapshot) {
            this.content = loadCacheContentFromDiskLocal(name, dataId, group, tenant);
            this.md5 = getMd5String(content);
        }
    }
   
   private static class ManagerListenerWrap {
        //是否在执行监听器中
        boolean inNotifying = false;
        
        //监听器
        final Listener listener;
        
        //最新配置的md5
        String lastCallMd5 = CacheData.getMd5String(null);
        
        //最新的配置内容
        String lastContent = null;
        
        ManagerListenerWrap(Listener listener) {
            this.listener = listener;
        }
        
        ManagerListenerWrap(Listener listener, String md5) {
            this.listener = listener;
            this.lastCallMd5 = md5;
        }
        
        ManagerListenerWrap(Listener listener, String md5, String lastContent) {
            this.listener = listener;
            this.lastCallMd5 = md5;
            this.lastContent = lastContent;
        }
        
        @Override
        public boolean equals(Object obj) {
            if (null == obj || obj.getClass() != getClass()) {
                return false;
            }
            if (obj == this) {
                return true;
            }
            ManagerListenerWrap other = (ManagerListenerWrap) obj;
            return listener.equals(other.listener);
        }
        
        @Override
        public int hashCode() {
            return super.hashCode();
        }
        
    }
}

接着看下配置变更监听(ConfigRpcTransportClient#executeConfigListen)的实现:

@Override
public void executeConfigListen() {
    //taskId -> List
    Map<String, List<CacheData>> listenCachesMap = new HashMap<String, List<CacheData>>(16);
    //taskId -> List
    Map<String, List<CacheData>> removeListenCachesMap = new HashMap<String, List<CacheData>>(16);

    long now = System.currentTimeMillis();
    //ALL_SYNC_INTERNAL=5分钟,每隔5分钟需要检查所有监听的配置
    boolean needAllSync = now - lastAllSyncTime >= ALL_SYNC_INTERNAL;

    for (CacheData cache : cacheMap.get().values()) {
        synchronized (cache) {
            if (cache.isSyncWithServer()) {
                //从服务器同步了配置(md5),检查监听器的md5是否与当前CacheData的md5一致,不一致则通知监听器并更新其md5
                cache.checkListenerMd5();
                if (!needAllSync) {
                    continue;
                }
            }
            
            if (!CollectionUtils.isEmpty(cache.getListeners())) {
                //有监听器的CacheData放到listenCachesMap中
                if (!cache.isUseLocalConfigInfo()) {
                    List<CacheData> cacheDatas = listenCachesMap.get(String.valueOf(cache.getTaskId()));
                    if (cacheDatas == null) {
                        cacheDatas = new LinkedList<>();
                        listenCachesMap.put(String.valueOf(cache.getTaskId()), cacheDatas);
                    }
                    cacheDatas.add(cache);
                }
            } else if (CollectionUtils.isEmpty(cache.getListeners())) {
                //没有监听器的CacheData放到removeListenCachesMap中
                if (!cache.isUseLocalConfigInfo()) {
                    List<CacheData> cacheDatas = removeListenCachesMap.get(String.valueOf(cache.getTaskId()));
                    if (cacheDatas == null) {
                        cacheDatas = new LinkedList<>();
                        removeListenCachesMap.put(String.valueOf(cache.getTaskId()), cacheDatas);
                    }
                    cacheDatas.add(cache);
                }
            }
        }
    }
    
    boolean hasChangedKeys = false;
    
    if (!listenCachesMap.isEmpty()) {
        for (Map.Entry<String, List<CacheData>> entry : listenCachesMap.entrySet()) {
            String taskId = entry.getKey();
            Map<String, Long> timestampMap = new HashMap<>(listenCachesMap.size() * 2);
            
            List<CacheData> listenCaches = entry.getValue();
            for (CacheData cacheData : listenCaches) {
                timestampMap.put(GroupKey.getKeyTenant(cacheData.dataId, cacheData.group, cacheData.tenant),
                        cacheData.getLastModifiedTs().longValue());
            }
            
            ConfigBatchListenRequest configChangeListenRequest = buildConfigRequest(listenCaches);
            configChangeListenRequest.setListen(true);
            try {
                //根据taskId获取对应的RpcClient,没有则创建一个
                RpcClient rpcClient = ensureRpcClient(taskId);
                //向服务端发送请求,检查配置是否变更
                ConfigChangeBatchListenResponse configChangeBatchListenResponse = (ConfigChangeBatchListenResponse) requestProxy(
                        rpcClient, configChangeListenRequest);
                //请求成功
                if (configChangeBatchListenResponse != null && configChangeBatchListenResponse.isSuccess()) {
                    //配置变更的key集合
                    Set<String> changeKeys = new HashSet<String>();
                    if (!CollectionUtils.isEmpty(configChangeBatchListenResponse.getChangedConfigs())) {
                        //服务端返回的响应中存在配置变更的数据
                        hasChangedKeys = true;
                        for (ConfigChangeBatchListenResponse.ConfigContext changeConfig : configChangeBatchListenResponse.getChangedConfigs()) {
                            String changeKey = GroupKey.getKeyTenant(changeConfig.getDataId(), changeConfig.getGroup(),
                                            changeConfig.getTenant());
                            changeKeys.add(changeKey);
                            boolean isInitializing = cacheMap.get().get(changeKey).isInitializing();
                            //从服务器获取最新的配置内容,检查监听器的md5是否与当前CacheData的md5一致,不一致则通知监听器并更新其md5
                            //!isInitializing=true表示服务端在配置变更时需要通知客户端
                            refreshContentAndCheck(changeKey, !isInitializing);
                        }
                        
                    }
                    
                    //handler content configs
                    for (CacheData cacheData : listenCaches) {
                        String groupKey = GroupKey
                                .getKeyTenant(cacheData.dataId, cacheData.group, cacheData.getTenant());
                        if (!changeKeys.contains(groupKey)) {
                            //sync:cache data md5 = server md5 && cache data md5 = all listeners md5.
                            synchronized (cacheData) {
                                if (!cacheData.getListeners().isEmpty()) {
                                    Long previousTimesStamp = timestampMap.get(groupKey);
                                    ///修改CacheData的lastModifiedTs,注意这里是采用cas方式修改的
                                    //这里存在一个并发问题,在服务端配置变更的时候会通知客户端(ConfigChangeNotifyRequest),客户端修改了lastModifiedTs,
                                    //并将syncWithServer修改为false,同时通知监听配置(调用notifyListenConfig())
                                    //即当前lastModifiedTs != previousTimesStamp,不能将syncWithServer设置为true,需在下次执行executeConfigListen()拉取最新的配置
                                    if (previousTimesStamp != null && !cacheData.getLastModifiedTs().compareAndSet(previousTimesStamp,
                                            System.currentTimeMillis())) {
                                        continue;
                                    }
                                    //设置为true,表示已经从服务器同步了最新配置(md5)
                                    cacheData.setSyncWithServer(true);
                                }
                            }
                        }
                        //设置为非初始化中的,服务端在配置变更时需要通知客户端
                        cacheData.setInitializing(false);
                    }
                    
                }
            } catch (Exception e) {
                
                LOGGER.error("Async listen config change error ", e);
                try {
                    Thread.sleep(50L);
                } catch (InterruptedException interruptedException) {
                    //ignore
                }
            }
        }
    }
    
    if (!removeListenCachesMap.isEmpty()) {
        //没有监听器的CacheData,向服务端发送取消监听请求
        for (Map.Entry<String, List<CacheData>> entry : removeListenCachesMap.entrySet()) {
            String taskId = entry.getKey();
            List<CacheData> removeListenCaches = entry.getValue();
            ConfigBatchListenRequest configChangeListenRequest = buildConfigRequest(removeListenCaches);
            configChangeListenRequest.setListen(false);
            try {
                RpcClient rpcClient = ensureRpcClient(taskId);
                boolean removeSuccess = unListenConfigChange(rpcClient, configChangeListenRequest);
                if (removeSuccess) {
                    for (CacheData cacheData : removeListenCaches) {
                        synchronized (cacheData) {
                            if (cacheData.getListeners().isEmpty()) {
                                //移除对应的CacheData
                                ClientWorker.this.removeCache(cacheData.dataId, cacheData.group, cacheData.tenant);
                            }
                        }
                    }
                }
                
            } catch (Exception e) {
                LOGGER.error("async remove listen config change error ", e);
            }
            try {
                Thread.sleep(50L);
            } catch (InterruptedException interruptedException) {
                //ignore
            }
        }
    }
    
    if (needAllSync) {
        //更新最后一次同步所有监听配置的时间为当前时间
        lastAllSyncTime = now;
    }

    //If has changed keys,notify re sync md5.
    if (hasChangedKeys) {
        notifyListenConfig();
    }
}

上述代码中使用到ConfigRpcTransportClient#ensureRpcClient创建一个RpcClient:

private RpcClient ensureRpcClient(String taskId) throws NacosException {
    synchronized (ClientWorker.this) {
        Map<String, String> labels = getLabels();
        Map<String, String> newLabels = new HashMap<>(labels);
        newLabels.put("taskId", taskId);

        //相同的taskId使用同一个RpcClient
        RpcClient rpcClient = RpcClientFactory
                .createClient(uuid + "_config-" + taskId, getConnectionType(), newLabels);
        if (rpcClient.isWaitInitiated()) {
            //RpcClient未初始化,进行初始化
            //初始化RpcClient处理器,如:配置变更通知请求处理器、连接监听器(连接成功/断开连接)
            initRpcClientHandler(rpcClient);

            rpcClient.setTenant(getTenant());
            rpcClient.clientAbilities(initAbilities());

            //启动RpcClient,这里会去连接服务器
            rpcClient.start();
        }
        
        return rpcClient;
    }
}

下面先看下RpcClient的基本属性(删减部分非关键代码):

public abstract class RpcClient implements Closeable {
 	//服务器列表工厂
    private ServerListFactory serverListFactory;
    
    //连接/断开事件队列
    protected BlockingQueue<ConnectionEvent> eventLinkedBlockingQueue = new LinkedBlockingQueue<>();
    
    //客户端状态
    protected volatile AtomicReference<RpcClientStatus> rpcClientStatus = new AtomicReference<>(
            RpcClientStatus.WAIT_INIT);
    
    //事件调度线程池
    protected ScheduledExecutorService clientEventExecutor;
    
    //重连信号,一个长度为1的阻塞队列
    private final BlockingQueue<ReconnectContext> reconnectionSignal = new ArrayBlockingQueue<>(1);
    
    //连接信息
    protected volatile Connection currentConnection;
    
    //保持活跃时间
    private long keepAliveTime = 5000L;
    
    //最后活跃时间
    private long lastActiveTimeStamp = System.currentTimeMillis();
    
    //连接事件监听器列表
    protected List<ConnectionEventListener> connectionEventListeners = new ArrayList<>();

    //服务端请求处理器列表
    protected List<ServerRequestHandler> serverRequestHandlers = new ArrayList<>();
}

初始化RpcClient处理器ConfigRpcTransportClient#initRpcClientHandler:

private void initRpcClientHandler(final RpcClient rpcClientInner) {
    //1.注册配置变更通知请求处理器(请求来自于服务端)
    rpcClientInner.registerServerRequestHandler((request) -> {
        if (request instanceof ConfigChangeNotifyRequest) {
            ConfigChangeNotifyRequest configChangeNotifyRequest = (ConfigChangeNotifyRequest) request;
            LOGGER.info("[{}] [server-push] config changed. dataId={}, group={},tenant={}",
                    rpcClientInner.getName(), configChangeNotifyRequest.getDataId(),
                    configChangeNotifyRequest.getGroup(), configChangeNotifyRequest.getTenant());
            String groupKey = GroupKey
                    .getKeyTenant(configChangeNotifyRequest.getDataId(), configChangeNotifyRequest.getGroup(),
                            configChangeNotifyRequest.getTenant());
            
            CacheData cacheData = cacheMap.get().get(groupKey);
            if (cacheData != null) {
                synchronized (cacheData) {
               		//修改lastModifiedTs为当前时间
                    cacheData.getLastModifiedTs().set(System.currentTimeMillis());
                    //设置为false,则需要从服务端同步最新的配置
                    cacheData.setSyncWithServer(false);
                    //通知配置变更(可理解为发布一个事件,通过异步的方式拉取最新的配置)
                    notifyListenConfig();
                }
                
            }
            return new ConfigChangeNotifyResponse();
        }
        return null;
    });

    //2.注册客户端配置度量处理器(请求来自服务端)
    rpcClientInner.registerServerRequestHandler((request) -> {
        if (request instanceof ClientConfigMetricRequest) {
            ClientConfigMetricResponse response = new ClientConfigMetricResponse();
            response.setMetrics(getMetrics(((ClientConfigMetricRequest) request).getMetricsKeys()));
            return response;
        }
        return null;
    });

    //3.注册连接监听器
    rpcClientInner.registerConnectionListener(new ConnectionEventListener() {
        
        @Override
        public void onConnected() {
            LOGGER.info("[{}] Connected,notify listen context...", rpcClientInner.getName());
            //连接成功,通知监听配置
            notifyListenConfig();
        }
        
        @Override
        public void onDisConnect() {
            String taskId = rpcClientInner.getLabels().get("taskId");
            LOGGER.info("[{}] DisConnected,clear listen context...", rpcClientInner.getName());
            Collection<CacheData> values = cacheMap.get().values();
            
            for (CacheData cacheData : values) {
                if (StringUtils.isNotBlank(taskId)) {
                    if (Integer.valueOf(taskId).equals(cacheData.getTaskId())) {
                        //断开连接后,需要从新的服务器同步配置
                        cacheData.setSyncWithServer(false);
                    }
                } else {
                    //断开连接后,需要从新的服务器同步配置
                    cacheData.setSyncWithServer(false);
                }
            }
        }
        
    });

    //4.设置服务器列表工厂
    rpcClientInner.serverListFactory(new ServerListFactory() {
        @Override
        public String genNextServer() {
            return ConfigRpcTransportClient.super.serverListManager.getNextServerAddr();
        }
        
        @Override
        public String getCurrentServer() {
            return ConfigRpcTransportClient.super.serverListManager.getCurrentServerAddr();
        }
        
        @Override
        public List<String> getServerList() {
            return ConfigRpcTransportClient.super.serverListManager.serverUrls;
        }
    });

    //5.注册服务器列表变更事件订阅者
    NotifyCenter.registerSubscriber(new Subscriber<ServerlistChangeEvent>() {
        @Override
        public void onEvent(ServerlistChangeEvent event) {
            rpcClientInner.onServerListChange();
        }
        
        @Override
        public Class<? extends Event> subscribeType() {
            return ServerlistChangeEvent.class;
        }
    });
}

initRpcClientHandler主要干了5件事:

  1. 注册配置变更通知请求处理器: 当服务端配置发生变更时,如果客户端监听了该配置,则会通知客户端,通过异步的方式拉取最新的配置
  2. 注册客户端配置度量请求处理器: 处理服务端请求度量信息的请求
  3. 注册连接监听器: 连接成功时,通知监听配置;断开连接时,将相关的CacheData的syncWithServer设置为false,这样在下次执行监听配置变更时会建立新的连接去同步配置
  4. 设置服务器列表工厂: 服务器信息工厂,实际是通过ServerListManager获取
  5. 注册服务器列表变更事件订阅者: 在服务器列表发生变更时,做切换重连

启动RpcClient:

public final void start() throws NacosException {
    boolean success = rpcClientStatus.compareAndSet(RpcClientStatus.INITIALIZED, RpcClientStatus.STARTING);
    if (!success) {
        return;
    }
    
    clientEventExecutor = new ScheduledThreadPoolExecutor(2, r -> {
        Thread t = new Thread(r);
        t.setName("com.alibaba.nacos.client.remote.worker");
        t.setDaemon(true);
        return t;
    });
    
    //1.提交连接成功/断开连接事件处理任务
    clientEventExecutor.submit(() -> {
        //轮训获取连接建立/断开事件
        while (!clientEventExecutor.isTerminated() && !clientEventExecutor.isShutdown()) {
            ConnectionEvent take;
            try {
                //以阻塞的方式获取
                take = eventLinkedBlockingQueue.take();
                if (take.isConnected()) {
                    //执行ConnectionEventListener#onConnected
                    notifyConnected();
                } else if (take.isDisConnected()) {
                    //执行ConnectionEventListener#onDisConnect
                    notifyDisConnected();
                }
            } catch (Throwable e) {
                // Do nothing
            }
        }
    });

    //2.提交重连事件处理任务
    clientEventExecutor.submit(() -> {
        while (true) {
            try {
                if (isShutdown()) {
                    break;
                }
                //获取重连事件信息,keepAliveTime = 5秒,相当于每隔5秒做一次健康检查
                ReconnectContext reconnectContext = reconnectionSignal.poll(keepAliveTime, TimeUnit.MILLISECONDS);
                if (reconnectContext == null) {
                    // check alive time.
                    //不存在重连事件,判断客户端心跳是否超时
                    if (System.currentTimeMillis() - lastActiveTimeStamp >= keepAliveTime) {
                        //心跳超时,做心跳检测
                        boolean isHealthy = healthCheck();
                        if (!isHealthy) {
                            //非健康状态
                            if (currentConnection == null) {
                                continue;
                            }
                            LoggerUtils.printIfInfoEnabled(LOGGER,
                                    "[{}] Server healthy check fail, currentConnection = {}", name, currentConnection.getConnectionId());
                            
                            RpcClientStatus rpcClientStatus = RpcClient.this.rpcClientStatus.get();
                            if (RpcClientStatus.SHUTDOWN.equals(rpcClientStatus)) {
                                //客户端已关闭
                                break;
                            }

                            //修改RpcClient为UNHEALTHY状态
                            boolean statusFLowSuccess = RpcClient.this.rpcClientStatus                                   .compareAndSet(rpcClientStatus, RpcClientStatus.UNHEALTHY);
                            if (statusFLowSuccess) {
                                //初始化重连事件上下文,服务器信息为null
                                reconnectContext = new ReconnectContext(null, false);
                            } else {
                                continue;
                            }
                        } else {
                            //健康状态,修改最近活跃时间
                            lastActiveTimeStamp = System.currentTimeMillis();
                            continue;
                        }
                    } else {
                        continue;
                    }    
                }
                
                if (reconnectContext.serverInfo != null) {
                    boolean serverExist = false;
                    //判断服务器信息是否包含在当前服务器列表中,是则重置其端口
                    for (String server : getServerListFactory().getServerList()) {
                        ServerInfo serverInfo = resolveServerInfo(server);
                        if (serverInfo.getServerIp().equals(reconnectContext.serverInfo.getServerIp())) {
                            serverExist = true;
                            reconnectContext.serverInfo.serverPort = serverInfo.serverPort;
                            break;
                        }
                    }
                    
                    if (!serverExist) {
                        //服务器信息不包含在当前服务器列表中,reconnectContext.serverInfo置为null
                        LoggerUtils.printIfInfoEnabled(LOGGER,
                                "[{}] Recommend server is not in server list, ignore recommend server {}", name,
                                reconnectContext.serverInfo.getAddress());
                        
                        reconnectContext.serverInfo = null;
                        
                    }
                }

                //重连服务端
                reconnect(reconnectContext.serverInfo, reconnectContext.onRequestFail);
            } catch (Throwable throwable) {
                // Do nothing
            }
        }
    });
    
    // connect to server, try to connect to server sync RETRY_TIMES times, async starting if failed.
    Connection connectToServer = null;
    rpcClientStatus.set(RpcClientStatus.STARTING);
    
    int startUpRetryTimes = RETRY_TIMES;
    //3.连接到服务器
    while (startUpRetryTimes > 0 && connectToServer == null) {
        try {
            startUpRetryTimes--;
            ServerInfo serverInfo = nextRpcServer();       
            LoggerUtils.printIfInfoEnabled(LOGGER, "[{}] Try to connect to server on start up, server: {}", name,
                    serverInfo);
            
            connectToServer = connectToServer(serverInfo);
        } catch (Throwable e) {
            LoggerUtils.printIfWarnEnabled(LOGGER,
                    "[{}] Fail to connect to server on start up, error message = {}, start up retry times left: {}",
                    name, e.getMessage(), startUpRetryTimes);
        }
        
    }
    
    if (connectToServer != null) {
        //连接成功
        LoggerUtils.printIfInfoEnabled(LOGGER, "[{}] Success to connect to server [{}] on start up, connectionId = {}",
                name, connectToServer.serverInfo.getAddress(), connectToServer.getConnectionId());
        this.currentConnection = connectToServer;
        rpcClientStatus.set(RpcClientStatus.RUNNING);

        //添加连接事件
        eventLinkedBlockingQueue.offer(new ConnectionEvent(ConnectionEvent.CONNECTED));
    } else {
        //连接失败,异步重连,实际上就是添加一个重连事件到reconnectionSignal中
        switchServerAsync();
    }

    //4.注册连接重置请求处理器
    registerServerRequestHandler(new ConnectResetRequestHandler());
    
    //注册客户端检测请求处理器
    registerServerRequestHandler(request -> {
        if (request instanceof ClientDetectionRequest) {
            return new ClientDetectionResponse();
        }
        return null;
    });
}

RpcClinet启动主要做4件事:

  1. 提交连接成功/断开连接事件处理任务: 通过轮询的方式判断是否有连接成功/断开连接的事件,然后调用连接事件监听器执行ConnectionEventListener处理
  2. 提交重连事件处理任务: 通过轮询的方式判断是否有重连信号,有的话则进行重连;没有的话则判断当前连接是否心跳超时,超时则判断其是否健康可用,若不可用且客户端处理非关闭状态,则进行重连
  3. 连接到服务器: 建立连接,成功则发布连接成功事件,失败则进行异步重试
  4. 注册连接重置请求处理器: 处理服务端发送的连接重置请求

注:上面提到,如果CacheData的taskId相同,使用同个RpcClient,但这是在监听配置的时候才会创建CacheData,在获取配置的时候是没有创建CacheData的,那获取配置的时候是使用哪个RpcClient的?实际上,获取配置的时候指定了一个taskId=0的RpcClient

五、总结

本文主要讲述了Nacos客户端获取配置以及监听配置的实现原理和源代码。

你可能感兴趣的:(Nacos,java,开源软件)