hive启用sentry
2020-07-22 本文已影响0人
david9
基于CDH6.2.0环境
安装sentry服务
在cloudera manager中添加sentry服务：
1595380703.jpg
1595380751(1).jpg
1595380789(1).jpg
并在hive中开启sentry服务：
1595381082(1).jpg
1595381142(1).jpg
如果集群中没有开启kerbos，需要在sentry-site.xml中添加如下配置：
<property>
<name>sentry.hive.testing.mode</name>
<value>true</value>
</property>
企业微信截图_1595389436803.png
在beeline中登录hive：
beeline -u jdbc:hive2://YOUR_HIVE_HOST:10000/default -n hive
执行：
create role admin_role;
grant all on server server1 to role admin_role;
grant role admin_role to group hive;
这样就可以通过hive用户管理sentry权限
问题
在cdh文档中，sentry创建函数需要CREATE ON SERVER或CREATE ON DATABASE权限，但是实测不需要（使用Direct JAR Reference Configuration，未使用推荐的Reloadable Aux JAR Configuration），只需要给对应的hdfs uri权限以及/tmp目录权限即可，暂时没深究
0: jdbc:hive2://cdh1:10000/default> show current roles;
...
+-------------------+
| role |
+-------------------+
| david_2020_test1 |
+-------------------+
...
0: jdbc:hive2://cdh1:10000/default> show grant role david_2020_test1;
...
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+----------------+----------+
| database | table | partition | column | principal_name | principal_type | privilege | grant_option | grant_time | grantor |
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+----------------+----------+
| default | | | | david_2020_test1 | ROLE | SELECT | false | 1595414036000 | -- |
| file:///tmp/ | | | | david_2020_test1 | ROLE | * | false | 1595413722000 | -- |
| hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar | | | | david_2020_test1 | ROLE | * | false | 1595390462000 | -- |
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+----------------+----------+
...
0: jdbc:hive2://cdh1:10000/default> create function LowerUDF as 'com.test.LowerUDF' using jar 'hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar';
INFO : Compiling command(queryId=hive_20200723154145_61a10681-8dca-44ae-b62a-8f19e1824ba6): create function LowerUDF as 'com.test.LowerUDF' using jar 'hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar'
INFO : Semantic Analysis Completed
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling command(queryId=hive_20200723154145_61a10681-8dca-44ae-b62a-8f19e1824ba6); Time taken: 0.125 seconds
INFO : Executing command(queryId=hive_20200723154145_61a10681-8dca-44ae-b62a-8f19e1824ba6): create function LowerUDF as 'com.test.LowerUDF' using jar 'hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar'
INFO : Starting task [Stage-0:FUNC] in serial mode
INFO : Added [/tmp/a114e074-ee0e-4342-bb12-04e6e83f42ae_resources/test-udf-1.0-SNAPSHOT.jar] to class path
INFO : Added resources: [hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar]
INFO : Completed executing command(queryId=hive_20200723154145_61a10681-8dca-44ae-b62a-8f19e1824ba6); Time taken: 0.017 seconds
INFO : OK
No rows affected (0.158 seconds)
需要 file:///tmp/权限的原因在HiveAuthzBindingHook中
case HiveParser.TOK_CREATEFUNCTION:
String udfClassName = BaseSemanticAnalyzer.unescapeSQLString(ast.getChild(1).getText());
try {
CodeSource udfSrc =
Class.forName(udfClassName, true, Utilities.getSessionSpecifiedClassLoader())
.getProtectionDomain().getCodeSource();
if (udfSrc == null) {
throw new SemanticException("Could not resolve the jar for UDF class " + udfClassName);
}
String udfJar = udfSrc.getLocation().getPath();
if (udfJar == null || udfJar.isEmpty()) {
throw new SemanticException("Could not find the jar for UDF class " + udfClassName +
"to validate privileges");
}
udfURIs.add(parseURI(udfSrc.getLocation().toString(), true));
} catch (ClassNotFoundException e) {
List<String> functionJars = getFunctionJars(ast);
if (functionJars.isEmpty()) {
throw new SemanticException("Error retrieving udf class:" + e.getMessage(), e);
} else {
// Add the jars from the command "Create function using jar" to the access list
// Defer to hive to check if the class is in the jars
for(String jar : functionJars) {
udfURIs.add(parseURI(jar, false));
}
}
}
// create/drop function is allowed with any database
currDB = Database.ALL;
break;
- udfSrc会从classpath中找udf对应的jar包，如果找不到，抛出ClassNotFoundException，使用create function语句中的uri。
- 函数一旦创建，当前session会创建类似/tmp/a114e074-ee0e-4342-bb12-04e6e83f42ae_resources/xxx.jar之类的缓存，并加入classpath。
- 后续语句的校验都会走classpath流程，且获取的uri路径是file:///tmp/a114e074-ee0e-4342-bb12-04e6e83f42ae_resources/xxx.jar，如果没有file:///tmp/权限，则权限校验会报错。
- 这可能是sentry的一个bug，获取jar资源的uri应该永远返回using jar语句中的uri才对。
参考：
Direct JAR Reference Configuration
Hive Aux JARs Directory Configuration
Reloadable Aux JAR Configuration
Managing the Sentry Service
Managing Apache Hive User-Defined Functions
