1. Dubbo--动态编译和SPI机制
2020-07-03 本文已影响0人
FrodeWY
Dubbo分层架构
image.png
Dubbo采用分层架构,除了Service和Config层为Api层,其他各层都是SPI层,意味着下面各层都是组件化的,可替换的。Dubbo增强了JDK提供的标准SPI功能,SPI层通过实现拓展点接口来提供服务;Dubbo增强的SPI增加了对拓展点的AOP和IOC支持,一个拓展点可以直接使用setter()方法注入其它拓展点,,并不会一次性实例化拓展点的所有实现类,避免了当拓展点实现类初始化很耗时,但有没有用到时带来的资源浪费问题,增强的SPI是用到某个实现类的时候才会对具体实现类进行实例化。
Dubbo的适配器原理
前面说到Dubbo为每个功能点提供了一个SPI拓展接口,但是每个拓展接口可能对应一系列拓展实现类,Dubbo是如何选择使用哪个拓展实现类的呢?Dubbo是通过适配器模式实现的。
- Dubbo使用动态编译技术为拓展接口生成适配器类,拿Protocol举例,Dubbo会为Protocol生成一个适配器类Protocol
$Adaptive,Dubbo需要使用Protocol
实例时其实就是使用Protocol$Adaptive的对象实例来获取具体的SPI实现类的,代码如下:
package org.apache.dubbo.rpc;
import org.apache.dubbo.common.extension.ExtensionLoader;
public class Protocol$Adaptive implements org.apache.dubbo.rpc.Protocol {
public void destroy() {
throw new UnsupportedOperationException("The method public abstract void org.apache.dubbo.rpc.Protocol.destroy() of interface org.apache.dubbo.rpc.Protocol is not adaptive method!");
}
public int getDefaultPort() {
throw new UnsupportedOperationException("The method public abstract int org.apache.dubbo.rpc.Protocol.getDefaultPort() of interface org.apache.dubbo.rpc.Protocol is not adaptive method!");
}
public org.apache.dubbo.rpc.Exporter export(org.apache.dubbo.rpc.Invoker arg0) throws org.apache.dubbo.rpc.RpcException {
if (arg0 == null) throw new IllegalArgumentException("org.apache.dubbo.rpc.Invoker argument == null");
if (arg0.getUrl() == null)
throw new IllegalArgumentException("org.apache.dubbo.rpc.Invoker argument getUrl() == null");
org.apache.dubbo.common.URL url = arg0.getUrl();
String extName = (url.getProtocol() == null ? "dubbo" : url.getProtocol());
if (extName == null)
throw new IllegalStateException("Failed to get extension (org.apache.dubbo.rpc.Protocol) name from url (" + url.toString() + ") use keys([protocol])");
org.apache.dubbo.rpc.Protocol extension = (org.apache.dubbo.rpc.Protocol) ExtensionLoader.getExtensionLoader(org.apache.dubbo.rpc.Protocol.class).getExtension(extName);
return extension.export(arg0);
}
public org.apache.dubbo.rpc.Invoker refer(java.lang.Class arg0, org.apache.dubbo.common.URL arg1) throws org.apache.dubbo.rpc.RpcException {
if (arg1 == null) throw new IllegalArgumentException("url == null");
org.apache.dubbo.common.URL url = arg1;
String extName = (url.getProtocol() == null ? "dubbo" : url.getProtocol());
if (extName == null)
throw new IllegalStateException("Failed to get extension (org.apache.dubbo.rpc.Protocol) name from url (" + url.toString() + ") use keys([protocol])");
org.apache.dubbo.rpc.Protocol extension = (org.apache.dubbo.rpc.Protocol) ExtensionLoader.getExtensionLoader(org.apache.dubbo.rpc.Protocol.class).getExtension(extName);
return extension.refer(arg0, arg1);
}
}
可以看到无论#export,还是#refer函数都是通过
org.apache.dubbo.rpc.Protocol extension = (org.apache.dubbo.rpc.Protocol) ExtensionLoader.getExtensionLoader(org.apache.dubbo.rpc.Protocol.class).getExtension(extName);
来获取具体的拓展点实例。适配器类是通过url中传递的参数,去加载Protocol的SPI实现。
- 正常情况下,我们是把所有java源文件静态编译成字节码文件,再由JVM统一加载,而动态编译是运行时将源文件编译成字节码文件,然后使用字节码文件创建对象实例。
刚才我们看到了,Dubbo为Protocol生成对应的适配器类,是如何生成的呢?
Protocol生成适配器类时序图
image.png
- 在ServiceConfig中Protocal的定义是:
private static final Protocol protocol = ExtensionLoader.getExtensionLoader(Protocol.class).getAdaptiveExtension();
2.org.apache.dubbo.common.extension.ExtensionLoader#getExtensionLoader
首先获取Protocol 对应的ExtensionLoader。
public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
if (type == null) {
throw new IllegalArgumentException("Extension type == null");
}
if (!type.isInterface()) {
throw new IllegalArgumentException("Extension type (" + type + ") is not an interface!");
}
if (!withExtensionAnnotation(type)) {
throw new IllegalArgumentException("Extension type (" + type +
") is not an extension, because it is NOT annotated with @" + SPI.class.getSimpleName() + "!");
}
//尝试根据type从缓存中获取
ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
if (loader == null) {
//如果缓存中没有,创建一个type类型的ExtensionLoader,并放入缓存
EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader<T>(type));
loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
}
return loader;
}
3.org.apache.dubbo.common.extension.ExtensionLoader#getAdaptiveExtension
获取Protocol 的适配器类实例
public T getAdaptiveExtension() {
//从缓存中尝试获取适配器类实例, Holder<Object> cachedAdaptiveInstance缓存了创建好的适配器类实例
Object instance = cachedAdaptiveInstance.get();
//double check方式创建适配器类实例
if (instance == null) {
if (createAdaptiveInstanceError == null) {
synchronized (cachedAdaptiveInstance) {
instance = cachedAdaptiveInstance.get();
if (instance == null) {
try {
//创建适配器类实例
instance = createAdaptiveExtension();
//放入缓存中
cachedAdaptiveInstance.set(instance);
} catch (Throwable t) {
createAdaptiveInstanceError = t;
throw new IllegalStateException("Failed to create adaptive instance: " + t.toString(), t);
}
}
}
} else {
throw new IllegalStateException("Failed to create adaptive instance: " + createAdaptiveInstanceError.toString(), createAdaptiveInstanceError);
}
}
return (T) instance;
}
4.org.apache.dubbo.common.extension.ExtensionLoader#createAdaptiveExtension
如果缓存中没有,需要创建适配器类
private T createAdaptiveExtension() {
try {
return injectExtension((T) getAdaptiveExtensionClass().newInstance());
} catch (Exception e) {
throw new IllegalStateException("Can't create adaptive extension " + type + ", cause: " + e.getMessage(), e);
}
}
5.org.apache.dubbo.common.extension.ExtensionLoader#getAdaptiveExtensionClass
获取适配器类的class对象
private Class<?> getAdaptiveExtensionClass() {
//首先加载META-INF/services/,META-INF/dubbo/ ,META-INF/dubbo/internal/路径下的type对应的文件,加载文件中所有的SPI实现类
getExtensionClasses();
if (cachedAdaptiveClass != null) {
//如果拓展点存在使用@Adaptive修饰的实现类时,直接使用该拓展实现类作为作为适配器类的class对象
return cachedAdaptiveClass;
}
return cachedAdaptiveClass = createAdaptiveExtensionClass();
}
6.org.apache.dubbo.common.extension.ExtensionLoader#createAdaptiveExtensionClass
如果没有使用@Adaptive修饰的拓展实现类,则需要通过动态编译生成拓展点的适配器类
private Class<?> createAdaptiveExtensionClass() {
//生成type对应的拓展点的适配器类内容
String code = new AdaptiveClassCodeGenerator(type, cachedDefaultName).generate();
ClassLoader classLoader = findClassLoader();
//获取Compiler 拓展点的适配器类,Compiler 的拓展实现有javassistCompiler,JdkCompiler两种。
org.apache.dubbo.common.compiler.Compiler compiler = ExtensionLoader.getExtensionLoader(org.apache.dubbo.common.compiler.Compiler.class).getAdaptiveExtension();
//通过Compiler 的实现类动态编译code,生成type类对应的适配器类Class对象
return compiler.compile(code, classLoader);
}
7.org.apache.dubbo.common.extension.ExtensionLoader#injectExtension
通过setter方法注入实现类对象依赖的其他拓展点实现类
private T injectExtension(T instance) {
try {
if (objectFactory != null) {
for (Method method : instance.getClass().getMethods()) {
if (isSetter(method)) {
/**
* Check {@link DisableInject} to see if we need auto injection for this property
*/
if (method.getAnnotation(DisableInject.class) != null) {
continue;
}
Class<?> pt = method.getParameterTypes()[0];
if (ReflectUtils.isPrimitives(pt)) {
continue;
}
try {
String property = getSetterProperty(method);
Object object = objectFactory.getExtension(pt, property);
if (object != null) {
method.invoke(instance, object);
}
} catch (Exception e) {
logger.error("Failed to inject via method " + method.getName()
+ " of interface " + type.getName() + ": " + e.getMessage(), e);
}
}
}
}
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
return instance;
}
通过上面的过程一系列操作,最终获取Protocol对应的适配器类
加载SPI实现类
再讲如何通过URL上的参数获取具体的SPI实现类实例前,先说一下Duboo是如何加载SPI的实现类的
- org.apache.dubbo.common.extension.ExtensionLoader#getExtensionClasses
加载org.apache.dubbo.common.extension.ExtensionLoader#type对应的所以拓展点实现
private Map<String, Class<?>> getExtensionClasses() {
//先从缓存中尝试获取
Map<String, Class<?>> classes = cachedClasses.get();
if (classes == null) {
synchronized (cachedClasses) {
classes = cachedClasses.get();
if (classes == null) {
//如果缓存为空,说明还没有加载type对应的所以拓展点实现,需要加载
classes = loadExtensionClasses();
//放入缓存
cachedClasses.set(classes);
}
}
}
return classes;
}
- org.apache.dubbo.common.extension.ExtensionLoader#loadExtensionClasses
加载META-INF/services/,META-INF/dubbo/ ,META-INF/dubbo/internal/路径下的type对应的文件,加载文件中所有的SPI实现类,并放入响应缓存中
private Map<String, Class<?>> loadExtensionClasses() {
//如果@SPI注解中存在默认名称,将默认名称放入cachedDefaultName中
cacheDefaultExtensionName();
Map<String, Class<?>> extensionClasses = new HashMap<>();
//加载指定路径下的SPI文件中的实现类
loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName());
loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName().replace("org.apache", "com.alibaba"));
loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName());
loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName().replace("org.apache", "com.alibaba"));
loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName());
loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName().replace("org.apache", "com.alibaba"));
return extensionClasses;
}
- org.apache.dubbo.common.extension.ExtensionLoader#loadDirectory
加载指定dir下,type对应的SPI实现类
private void loadDirectory(Map<String, Class<?>> extensionClasses, String dir, String type) {
//加载的文件名称
String fileName = dir + type;
try {
Enumeration<java.net.URL> urls;
ClassLoader classLoader = findClassLoader();
//获取fileName对应的资源URL
if (classLoader != null) {
urls = classLoader.getResources(fileName);
} else {
urls = ClassLoader.getSystemResources(fileName);
}
if (urls != null) {
while (urls.hasMoreElements()) {
java.net.URL resourceURL = urls.nextElement();
//加载resourceURL对应文件中的SPI实现类
loadResource(extensionClasses, classLoader, resourceURL);
}
}
} catch (Throwable t) {
logger.error("Exception occurred when loading extension class (interface: " +
type + ", description file: " + fileName + ").", t);
}
}
- org.apache.dubbo.common.extension.ExtensionLoader#loadResource
加载resourceURL对应的文件,将文件中的SPI实现类加载到缓存中
private void loadResource(Map<String, Class<?>> extensionClasses, ClassLoader classLoader, java.net.URL resourceURL) {
try {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(resourceURL.openStream(), StandardCharsets.UTF_8))) {
String line;
while ((line = reader.readLine()) != null) {
final int ci = line.indexOf('#');
if (ci >= 0) {
line = line.substring(0, ci);
}
line = line.trim();
if (line.length() > 0) {
try {
String name = null;
int i = line.indexOf('=');
if (i > 0) {
//SPI实现类对应的名称
name = line.substring(0, i).trim();
//SPI实现类类路径
line = line.substring(i + 1).trim();
}
if (line.length() > 0) {
//加载line 对应的实现类
loadClass(extensionClasses, resourceURL, Class.forName(line, true, classLoader), name);
}
} catch (Throwable t) {
IllegalStateException e = new IllegalStateException("Failed to load extension class (interface: " + type + ", class line: " + line + ") in " + resourceURL + ", cause: " + t.getMessage(), t);
exceptions.put(line, e);
}
}
}
}
} catch (Throwable t) {
logger.error("Exception occurred when loading extension class (interface: " +
type + ", class file: " + resourceURL + ") in " + resourceURL, t);
}
}
- org.apache.dubbo.common.extension.ExtensionLoader#loadClass
private void loadClass(Map<String, Class<?>> extensionClasses, java.net.URL resourceURL, Class<?> clazz, String name) throws NoSuchMethodException {
if (!type.isAssignableFrom(clazz)) {
throw new IllegalStateException("Error occurred when loading extension class (interface: " +
type + ", class line: " + clazz.getName() + "), class "
+ clazz.getName() + " is not subtype of interface.");
}
//如果类上有@Adaptive,表示这个类就是type SPI对应的适配器类,不需要动态编译产生适配器类
if (clazz.isAnnotationPresent(Adaptive.class)) {
//放入缓存
cacheAdaptiveClass(clazz);
} else if (isWrapperClass(clazz)) {
//缓存包装类,以便于之后对SPI实现类实例包装,增强
cacheWrapperClass(clazz);
} else {
clazz.getConstructor();
if (StringUtils.isEmpty(name)) {
/**如果name为空,尝试从@Extension的Value中获取name,如果类上也没有@Extension,就尝试将clazz.getSimpleName()解析成键值对,key就是name,
例如com.foo.XxxProtocol==>xxx=com.foo.XxxProtocol*/
name = findAnnotationName(clazz);
if (name.length() == 0) {
throw new IllegalStateException("No such extension name for the class " + clazz.getName() + " in the config " + resourceURL);
}
}
String[] names = NAME_SEPARATOR.split(name);
if (ArrayUtils.isNotEmpty(names)) {
//将第一个别名,还有clazz存入cachedActivates中,cachedActivates缓存了key:name,value:clazz上的@Activete注解
cacheActivateClass(clazz, names[0]);
for (String n : names) {
//将clazz,以及对应的name 放入cachedNames中缓存,其实只有第一个别名会被放入ConcurrentMap<Class<?>, String> cachedNames中缓存
cacheName(clazz, n);
//将name,clazz放入extensionClasses中
saveInExtensionClass(extensionClasses, clazz, name);
}
}
}
}
通过getExtensionClasses()加载指定路径下的SPI文件中的SPI实现类,对cachedClasses,cachedDefaultName,cachedAdaptiveClass,cachedWrapperClasses,cachedActivates,cachedNames
进行了填充,便于后期获取拓展点的实现类
通过适配器类获取SPI拓展点具体实现类
之前我们说了,Dubbo会为每个SPI拓展点生成适配器类,也了解了SPI实现类的加载过程,那如何通过适配器类获取具体的SPI实现类实例呢?
public org.apache.dubbo.rpc.Invoker refer(java.lang.Class arg0, org.apache.dubbo.common.URL arg1) throws org.apache.dubbo.rpc.RpcException {
if (arg1 == null) throw new IllegalArgumentException("url == null");
org.apache.dubbo.common.URL url = arg1;
String extName = (url.getProtocol() == null ? "dubbo" : url.getProtocol());
if (extName == null)
throw new IllegalStateException("Failed to get extension (org.apache.dubbo.rpc.Protocol) name from url (" + url.toString() + ") use keys([protocol])");
org.apache.dubbo.rpc.Protocol extension = (org.apache.dubbo.rpc.Protocol) ExtensionLoader.getExtensionLoader(org.apache.dubbo.rpc.Protocol.class).getExtension(extName);
return extension.refer(arg0, arg1);
}
以Protocol$Adaptive#refer为例,首先从URL中获取protocol参数值extName,在通过ExtensionLoader加载extName对应的拓展点实现。
- org.apache.dubbo.common.extension.ExtensionLoader#getExtension
获取实例名称对应的SPI实现实例
public T getExtension(String name) {
if (StringUtils.isEmpty(name)) {
throw new IllegalArgumentException("Extension name == null");
}
if ("true".equals(name)) {
return getDefaultExtension();
}
//还是先尝试从缓存中获取
Holder<Object> holder = getOrCreateHolder(name);
Object instance = holder.get();
//dubbo check
if (instance == null) {
synchronized (holder) {
instance = holder.get();
if (instance == null) {
//如果缓存中没有则创建name对应的SPI实现
instance = createExtension(name);
//放入缓存
holder.set(instance);
}
}
}
return (T) instance;
}
- org.apache.dubbo.common.extension.ExtensionLoader#createExtension
创建name对应的SPI实现类实例
private T createExtension(String name) {
//获取name对应的SPI实现类Class对象
Class<?> clazz = getExtensionClasses().get(name);
if (clazz == null) {
throw findException(name);
}
try {
//尝试从缓存中获取
T instance = (T) EXTENSION_INSTANCES.get(clazz);
if (instance == null) {
//如果缓存中没有则通过clazz.newInstance()创建实例,并放入缓存EXTENSION_INSTANCES中
EXTENSION_INSTANCES.putIfAbsent(clazz, clazz.newInstance());
instance = (T) EXTENSION_INSTANCES.get(clazz);
}
//注入实例依赖的其他SPI实现类实例
injectExtension(instance);
Set<Class<?>> wrapperClasses = cachedWrapperClasses;
//如果存在Wrapper类,利用Wrapper类对实现类进行增强,这里利用了装饰器设计模式
if (CollectionUtils.isNotEmpty(wrapperClasses)) {
for (Class<?> wrapperClass : wrapperClasses) {
instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance));
}
}
return instance;
} catch (Throwable t) {
throw new IllegalStateException("Extension instance (name: " + name + ", class: " +
type + ") couldn't be instantiated: " + t.getMessage(), t);
}
}
总结
到这里为止,Dubbo如何通过适配器类获取特定SPI实现类实例的原理就讲完了。总结一下,Dubbo会为每一个SPI拓展接口通过动态编译生成一个适配器类,并且加载指定路径下的SPI文件,解析文件,加载文件中的SPI实现类,将加载的SPI实现类的class对象放入cachedClasses中缓存,当需要用到某个SPI实现类时候才会去初始化,当需要使用拓展接口的适配器类时,会根据url中的参数加载特定的SPI实现类。
参考
1.<<深度剖析Apache Dubbo 核心技术内幕>>
