springboot 多数据源实现读写分离
springboot 多数据源实现读写分离
你好！在大多数项目中，数据库的读写走的都是一个库，这在并发量小的时候，完全没问题，但是一旦并发上来，数据库的读写压力会变的很大，在实际业务中，读写分离是有必要的，特别是在读多写少的业务场景中；
多数据源配置
- 数据库的主从搭建 ，数据库的主从主从搭建不是本文的重点，这里略过；
- springboot配置文件 ，对配置文件进行修改，修改结果如下：
spring:
datasource:
dynamic-datasource:
enable: true
slave:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/my_db2
username: root
password: 123456
# Hikari 连接池配置
hikari:
# 最小空闲连接数量
minimum-idle: 1
# 空闲连接存活最大时间，默认600000（10分钟）
idle-timeout: 180000
# 连接池最大连接数，默认是10
maximum-pool-size: 4
# 此属性控制从池返回的连接的默认自动提交行为,默认值：true
auto-commit: true
# 连接池名称
pool-name: ${spring.application.name}
# 此属性控制池中连接的最长生命周期，值0表示无限生命周期，默认1800000即30分钟
max-lifetime: 1800000
# 数据库连接超时时间,默认30秒，即30000
connection-timeout: 30000
master:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/my_db
username: root
password: 123456
# Hikari 连接池配置
hikari:
# 最小空闲连接数量
minimum-idle: 1
# 空闲连接存活最大时间，默认600000（10分钟）
idle-timeout: 180000
# 连接池最大连接数，默认是10
maximum-pool-size: 4
# 此属性控制从池返回的连接的默认自动提交行为,默认值：true
auto-commit: true
# 连接池名称
pool-name: ${spring.application.name}
# 此属性控制池中连接的最长生命周期，值0表示无限生命周期，默认1800000即30分钟
max-lifetime: 1800000
# 数据库连接超时时间,默认30秒，即30000
connection-timeout: 30000
组件实现
- 多数据切换注解
用于方法或者类名上
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface DataSourceSelector {
DataSourceType type() default DataSourceType.MASTER;
}
- 数据源枚举
public enum DataSourceType {
MASTER, SLAVE
}
- 配置类配置多数据源
@Configuration(proxyBeanMethods = false)
@AutoConfigureBefore({DataSourceAutoConfiguration.class})
@EnableConfigurationProperties({DataSourceProperties.class,DynamicDataSourceProperties.class})
@RequiredArgsConstructor
public class DynamicDataSourceAutoConfiguration {
@Bean(name = "masterDataSource")
@ConfigurationProperties(prefix = "spring.datasource.master")
public DataSource masterDataSource() {
return DataSourceBuilder.create().build();
}
@Bean(name = "salveDataSource")
@ConfigurationProperties(prefix = "spring.datasource.slave")
public DataSource salveDataSource() {
return DataSourceBuilder.create().build();
}
@Primary
@Bean(name = "dynamicDataSource")
public DataSource dynamicDataSource(@Qualifier("masterDataSource") DataSource master, @Qualifier("salveDataSource") DataSource salve) {
DynamicDataSource dynamicDataSource = new DynamicDataSource();
Map<Object, Object> targetDataSources = new HashMap<>();
targetDataSources.put(DataSourceType.MASTER, master);
targetDataSources.put(DataSourceType.SLAVE, salve);
dynamicDataSource.setTargetDataSources(targetDataSources);
dynamicDataSource.setDefaultTargetDataSource(master);
return dynamicDataSource;
}
}
- 数据源路由
从TheadLocal拿到数据源类型，路由到指定的数据源
public class DynamicDataSource extends AbstractRoutingDataSource {
private static final Logger log = LoggerFactory.getLogger(DynamicDataSource.class);
@Override
protected Object determineCurrentLookupKey() {
DataSourceType dataSourceType = DataSourceContextHolder.getDataSourceType();
log.info("use datasource: {}", dataSourceType);
return dataSourceType;
}
}
- 切换上下文（ThreadLocal）
注：这里IS_DATA_SOURCE_SELECTED_HOLDER用于异步场景
public class DataSourceContextHolder {
private DataSourceContextHolder() {
}
/**
* 线程本地环境
*/
private static final ThreadLocal<DataSourceType> CONTEXT_TYPE_HOLDER = ThreadLocal.withInitial(() -> DataSourceType.MASTER);
private static final ThreadLocal<Boolean> IS_DATA_SOURCE_SELECTED_HOLDER = ThreadLocal.withInitial(() -> Boolean.FALSE);
/**
* 设置数据源类型：枚举式
*/
public static void setDataSourceType(DataSourceType dbType) {
Assert.notNull(dbType, "DataSourceType cannot be null");
CONTEXT_TYPE_HOLDER.set(dbType);
}
/**
* 当前线程已选择数据源
*/
public static void inDataSourceSelected() {
IS_DATA_SOURCE_SELECTED_HOLDER.set(Boolean.TRUE);
}
/**
* 获取数据源类型：枚举式
*/
public static DataSourceType getDataSourceType() {
return CONTEXT_TYPE_HOLDER.get();
}
public static boolean isDataSourceSelected() {
return IS_DATA_SOURCE_SELECTED_HOLDER.get();
}
/**
* 清除数据源类型
*/
public static void clearDataSourceType() {
CONTEXT_TYPE_HOLDER.remove();
}
public static void clearIsDataSourceSelected() {
IS_DATA_SOURCE_SELECTED_HOLDER.remove();
}
}
- 对注解进行切面
这里主要的逻辑就是对容器所有的代理类进行扫描，看是否在方法或者类上有DataSourceSelector注解，如果有对类生成代理对象；
注：这里使AbstractAutoProxyCreator进行注解的拦截处理，和传统的AOP写法不太一样，这种写法更适合于组件。
public class DynamicDataSourceCreator extends AbstractAutoProxyCreator {
private static final Set<String> PROXY_SET = new HashSet<>();
@Override
protected Object[] getAdvicesAndAdvisorsForBean(Class<?> aClass, String s, TargetSource targetSource) throws BeansException {
return new Object[]{new DynamicDataSourceMethodInterceptor()};
}
@SneakyThrows
@Override
protected Object wrapIfNecessary(Object bean, String beanName, Object cacheKey) {
synchronized (PROXY_SET) {
if (PROXY_SET.contains(beanName)) {
return bean;
}
Class<?> serviceInterface = SpringProxyUtils.findTargetClass(bean);
Class<?>[] interfacesIfJdk = SpringProxyUtils.findInterfaces(bean);
if (!CommUtils.existsAnnotation(new Class[]{serviceInterface}) && !CommUtils.existsAnnotation(interfacesIfJdk)) {
return bean;
}
if (!AopUtils.isAopProxy(bean)) {
bean = super.wrapIfNecessary(bean, beanName, cacheKey);
} else {
AdvisedSupport advised = SpringProxyUtils.getAdvisedSupport(bean);
Advisor[] advisor = buildAdvisors(beanName, getAdvicesAndAdvisorsForBean(null, null, null));
for (Advisor avr : advisor) {
advised.addAdvisor(0, avr);
}
}
PROXY_SET.add(beanName);
return bean;
}
}
}
- 切面逻辑
这里主要是拿到注解中的数据源类型，然后put到ThreadLocal中，用于后续路由
public class DynamicDataSourceMethodInterceptor implements MethodInterceptor {
@Override
@SneakyThrows
public Object invoke(MethodInvocation methodInvocation) {
if (DataSourceContextHolder.isDataSourceSelected()) {
return methodInvocation.proceed();
}
try {
Class<?> targetClass = methodInvocation.getThis() != null ? AopUtils.getTargetClass(methodInvocation.getThis()) : null;
Method specificMethod = ClassUtils.getMostSpecificMethod(methodInvocation.getMethod(), targetClass);
DataSourceSelector methodAnnotation = getAnnotation(specificMethod, targetClass, DataSourceSelector.class);
if (Objects.nonNull(methodAnnotation)) {
CommUtils.setDataSourceType(methodAnnotation.type());
} else {
CommUtils.setDataSourceType(DataSourceType.MASTER);
}
return methodInvocation.proceed();
} finally {
CommUtils.clearContext();
}
}
private <T extends Annotation> T getAnnotation(Method method, Class<?> targetClass, Class<T> annotationClass) {
return Optional.ofNullable(method).map(m -> m.getAnnotation(annotationClass))
.orElse(Optional.ofNullable(targetClass).map(t -> t.getAnnotation(annotationClass)).orElse(null));
}
}
注意事项
- 在读写分离中写入到从库是一个非常严重的事件，因此为了避免误用注解，导致写 操作路由到从库，可以做一个sql语句的拦截，对写操作，只能路由到主库，这里是一个mybatis拦截器的逻辑;
@Intercepts({
@Signature(type = Executor.class, method = "update", args = {MappedStatement.class,
Object.class}),
@Signature(type = Executor.class, method = "query", args = {MappedStatement.class,
Object.class, RowBounds.class, ResultHandler.class})})
@RequiredArgsConstructor
public class DynamicDataSourceInterceptor implements Interceptor {
private static final String REGEX = ".*insert\\u0020.*|.*delete\\u0020.*|.*update\\u0020.*";
private static final Set<String> cacheSet = new ConcurrentSkipListSet<>();
@Override
public Object intercept(Invocation invocation) throws Throwable {
if (Objects.equals(DataSourceType.SLAVE, DataSourceContextHolder.getDataSourceType())) {
Object[] objects = invocation.getArgs();
MappedStatement ms = (MappedStatement) objects[0];
BoundSql boundSql = ms.getSqlSource().getBoundSql(objects[1]);
boolean actualTransactionActive = TransactionSynchronizationManager
.isActualTransactionActive();
String sql = boundSql.getSql().toLowerCase(Locale.CHINA)
.replaceAll("[\\t\\n\\r]", " ");
if (cacheSet.contains(ms.getId())) {
throw new IllegalStateException(String.format("The current sql:[%s] should use the master database", boundSql.getSql().toLowerCase(Locale.CHINA)));
}
boolean isMatch = sql.matches(REGEX);
if (isMatch || actualTransactionActive) {
if (isMatch) {
cacheSet.add(ms.getId());
}
throw new IllegalStateException(String.format("The current sql:[%s] should use the master database", boundSql.getSql().toLowerCase(Locale.CHINA)));
}
}
return invocation.proceed();
}
@Override
public Object plugin(Object target) {
if (target instanceof Executor) {
return Plugin.wrap(target, this);
} else {
return target;
}
}
}
- 在未使用注解时，默认路由到主库，仅当通过注解指定为从库，才会路由到从库；
- 在使用注解时，目标类必须被spring代理，且不能在类里方法调用，否则会失效，参考@Transactional失效场景；
- 考虑到现实情况，目前使用jdk 自带的TheadLocal，后续若有异步查询场景可以考虑替换为阿里TTL；