一: Hugegraph 源代码 : HugeGraph对象
2019-06-18 本文已影响0人
NazgulSun
hugraph对象为百度的开源图数据库,实现了Tinkerpop框架,本系列解读 hugegraph-core 的开源代码,加深图数据的理解。
hugeraph项目 提供了 api 功能,用户可以调用 api 添加节点,本文就以 这个 增加节点的api为起点,一步一步讲解 hugegraph是如何实现的。
addVertex 作为 api 的入口
@POST
@Timed
@Status(Status.CREATED)
@Consumes(APPLICATION_JSON)
@Produces(APPLICATION_JSON_WITH_CHARSET)
public String create(@Context GraphManager manager,
@PathParam("graph") String graph,
JsonVertex jsonVertex) {
LOG.debug("Graph [{}] create vertex: {}", graph, jsonVertex);
checkCreatingBody(jsonVertex);
HugeGraph g = graph(manager, graph);
Vertex vertex = commit(g, () -> g.addVertex(jsonVertex.properties()));
return manager.serializer(g).writeVertex(vertex);
}
可以看到 所有的起点就是需要 创建一个 HugeGraph对象,Hugegraph 对象里面 包含了 addVertex 方法。
本文就重点讲解 HugeGraph是如何创建的
graph(manager, graph)
public static HugeGraph graph(GraphManager manager, String graph) {
HugeGraph g = manager.graph(graph);
if (g == null) {
throw new NotFoundException(String.format(
"Graph '%s' does not exist", graph));
}
return g;
}
实际是使用graphManager的map,根据 graph名字获取对象。
那么graphManager 是关键对象
构建GraphManager
hugegraph 使用GraphManager直接获取HugeGraph对象
对于GraphManger对象
private final Map<String, Graph> graphs;
private final HugeAuthenticator authenticator;
public GraphManager(HugeConfig conf) {
this.graphs = new ConcurrentHashMap<>();
if (conf.get(ServerOptions.AUTHENTICATOR).isEmpty()) {
this.authenticator = null;
} else {
this.authenticator = HugeAuthenticator.loadAuthenticator(conf);
}
this.loadGraphs(conf.getMap(ServerOptions.GRAPHS));
this.checkBackendVersionOrExit();
this.restoreUncompletedTasks();
this.addMetrics(conf);
}
由于hugegraph是支持多图的,所有有GrappManager保存一个 多图的引用map。
在构建GraphManager对象的时候
最重要的一步是
this.loadGraphs(conf.getMap(ServerOptions.GRAPHS));
private void loadGraph(String name, String path) {
final Graph graph = GraphFactory.open(path);
this.graphs.put(name, graph);
LOG.info("Graph '{}' was successfully configured via '{}'", name, path);
if (this.requireAuthentication() &&
!(graph instanceof HugeGraphAuthProxy)) {
LOG.warn("You may need to support access control for '{}' with {}",
path, HugeFactoryAuthProxy.GRAPH_FACTORY);
}
}
可以看到 关键的一句 final Graph graph = GraphFactory.open(path);
这个 Graph 和GraphFactory 都是 tinkerpop的标准接口。
path参数就是 我们的图的配置文件,默认是 conf/hugegraph.properties.
在配置文件中 (conf/hugegraph.propreties)我们有一行:
gremlin.graph=com.baidu.hugegraph.HugeFactory
这一行指定了具体创建图的 工厂类,查看 HugeFactory 的open方法:
public static synchronized HugeGraph open(Configuration config) {
HugeConfig conf = new HugeConfig(config);
String name = conf.get(CoreOptions.STORE);
HugeGraph graph = graphs.get(name);
if (graph == null || graph.closed()) {
graph = new HugeGraph(conf);
graphs.put(name, graph);
} else {
String backend = conf.get(CoreOptions.BACKEND);
E.checkState(backend.equals(graph.backend()),
"Graph name '%s' has been used by backend '%s'",
name, graph.backend());
}
return graph;
}
这里面 调用了 HugeGraph的 构造方法
graph = new HugeGraph(conf);
下面看看 HugeGraph 到底是如何构造的?
构建 HugeGraph 对象
public HugeGraph(HugeConfig configuration) {
this.configuration = configuration;
this.schemaEventHub = new EventHub("schema");
this.indexEventHub = new EventHub("index");
final int limit = configuration.get(CoreOptions.RATE_LIMIT);
this.rateLimiter = limit > 0 ? RateLimiter.create(limit) : null;
this.taskManager = TaskManager.instance();
this.features = new HugeFeatures(this, true);
this.name = configuration.get(CoreOptions.STORE);
this.closed = false;
this.mode = GraphMode.NONE;
LockUtil.init(this.name);
try {
this.storeProvider = this.loadStoreProvider();
} catch (BackendException e) {
String message = "Failed to init backend store";
LOG.error("{}: {}", message, e.getMessage());
throw new HugeException(message);
}
this.tx = new TinkerpopTransaction(this);
this.taskManager.addScheduler(this);
this.variables = null;
}
- rateLimiter 设置hugegraph 增删改查的节点速度
- features 设置hugegraph 对于tinkerpop的特性支持,目前hugegraph是对tinkerpop的一个标准子集
- LockUtil.init(this.name) 初始化图上的锁,对于图的操作有很多需要异步加锁的地方,锁机制后面专门讲解
- this.storeProvider = this.loadStoreProvider() 加载 存储模块。hugegraph支持多种存储终端,在这里根据配置信息初始化存储终端
- 设置对应的事务对象 this.tx = new TinkerpopTransaction(this);
初始化 storeProvider
private BackendStoreProvider loadStoreProvider() {
String backend = this.configuration.get(CoreOptions.BACKEND);
LOG.info("Opening backend store '{}' for graph '{}'",
backend, this.name);
return BackendProviderFactory.open(backend, this.name);
}
调用 BackendProviderFactory.open(backend, this.name) 方法
public static BackendStoreProvider open(String backend, String graph) {
if (InMemoryDBStoreProvider.matchType(backend)) {
return InMemoryDBStoreProvider.instance(graph);
}
Class<? extends BackendStoreProvider> clazz = providers.get(backend);
BackendException.check(clazz != null,
"Not exists BackendStoreProvider: %s", backend);
assert BackendStoreProvider.class.isAssignableFrom(clazz);
BackendStoreProvider instance = null;
try {
instance = clazz.newInstance();
} catch (Exception e) {
throw new BackendException(e);
}
BackendException.check(backend.equalsIgnoreCase(instance.type()),
"BackendStoreProvider with type '%s' " +
"can't be opened by key '%s'",
instance.type(), backend);
instance.open(graph);
return instance;
}
使用反射的方法 获取 BackendStoreProvider 对象, 对于 hugegraph 提供了多个 provider,
根据配置信息获取具体的Provider。
看下在哪里配置了不同的provider?
hugegraph 有一个子module, hugegraph-dist, 这里面有相应的配置信息
下面代码来自 RegisterUtil
public static void registerCassandra() {
// Register config
OptionSpace.register("cassandra",
"com.baidu.hugegraph.backend.store.cassandra.CassandraOptions");
// Register serializer
SerializerFactory.register("cassandra",
"com.baidu.hugegraph.backend.store.cassandra.CassandraSerializer");
// Register backend
BackendProviderFactory.register("cassandra",
"com.baidu.hugegraph.backend.store.cassandra.CassandraStoreProvider");
}
初始化 transcation
this.tx = new TinkerpopTransaction(this);
TinkerpopTransaction 继承自 tinkerpop 的抽象类 AbstractThreadLocalTransaction ,
该抽象类实现了 tinkerpop的 Transaction 接口,定了以tinkerpop的事务标准。
从TinkerpopTransaction 来看, hugegraph 定义了 GraphTransaction 和schemaTransaction
private class TinkerpopTransaction extends AbstractThreadLocalTransaction {
private AtomicInteger refs;
private ThreadLocal<Boolean> opened;
// Backend transactions
private ThreadLocal<GraphTransaction> graphTransaction;
private ThreadLocal<SchemaTransaction> schemaTransaction;
private SchemaTransaction schemaTransaction() {
/*
* NOTE: this method may be called even tx is not opened,
* the reason is for reusing backend tx.
* so we don't call this.verifyOpened() here.
*/
SchemaTransaction schemaTx = this.schemaTransaction.get();
if (schemaTx == null) {
schemaTx = openSchemaTransaction();
this.schemaTransaction.set(schemaTx);
}
return schemaTx;
}
private GraphTransaction graphTransaction() {
/*
* NOTE: this method may be called even tx is not opened,
* the reason is for reusing backend tx.
* so we don't call this.verifyOpened() here.
*/
GraphTransaction graphTx = this.graphTransaction.get();
if (graphTx == null) {
graphTx = openGraphTransaction();
this.graphTransaction.set(graphTx);
}
return graphTx;
}
实际通过 openGraphTransaction() 和 openSchemaTransaction(); 来创建事务对象。
hugegraph 的 addvertex 以及 对schema 的修改都是通过 这两个对象的接口实现。
对于 hugegraph 的事务管理,将在下一章重点接收。
到目前为止, hugegraph对象就创建好了,接口以使用相应的方法。
请看下回分解。