elasticsearch的plugin中实现jni调用

环境

• JDK1.8
• elasticsearch5.3.3

示例

生成JNI动态链接库

1. 创建工作目录并进入
   mkdir es_jni_test; cd es_jni_test

2. 编写HelloWorld.java类(调用sayHello native方法，用于生成jni头文件)

package com.clark010.playgroud.es.jni;

public class HelloWorld {
    // native方法
    public static native String sayHello(String name); 

    public static void main(String[] args) {
      String text = sayHello("clark010"); 
      System.out.println(text);
    }

    static {
      System.loadLibrary("HelloWorld"); 
    }
}

2. 编译Java类
   mkdir classes; $JAVA_HOME/bin/javac HelloWorld.java -d ./classes
   类文件生成在classes目录下

3. 生成jni头文件
   $JAVA_HOME/bin/javah -jni -classpath ./classes -d ./jni com.clark010.playgroud.es.jni.HelloWorld
   会在在jni目录下自动生成com_clark010_playgroud_es_jni_HelloWorld.h头文件，
   文件内容如下：

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_clark010_playgroud_es_jni_HelloWorld */

#ifndef _Included_com_clark010_playgroud_es_jni_HelloWorld
#define _Included_com_clark010_playgroud_es_jni_HelloWorld
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_clark010_playgroud_es_jni_HelloWorld
 * Method:    sayHello
 * Signature: (Ljava/lang/String;)Ljava/lang/String;
 */
JNIEXPORT jstring JNICALL Java_com_clark010_playgroud_es_jni_HelloWorld_sayHello
  (JNIEnv *, jclass, jstring);

#ifdef __cplusplus
}
#endif
#endif

4. 实现jni接口函数

// com_clark010_playgroud_es_jni_HelloWorld.c
 
#include "com_clark010_playgroud_es_jni_HelloWorld.h"
 
#ifdef __cplusplus
extern "C"
{
#endif
 
/*
 * Class:     com_clark010_playgroud_es_jni_HelloWorld
 * Method:    sayHello
 * Signature: (Ljava/lang/String;)Ljava/lang/String;
 */
JNIEXPORT jstring JNICALL Java_com_clark010_playgroud_es_jni_HelloWorld_sayHello(
            JNIEnv *env, jclass cls, jstring j_str)
{
    const char *c_str = NULL;
    char buff[128] = { 0 };
    c_str = (*env)->GetStringUTFChars(env, j_str, NULL);
    if (c_str == NULL)
    {
        printf("out of memory.\n");
        return NULL;
    }
    sprintf(buff, "hello %s", c_str);
        (*env)->ReleaseStringUTFChars(env, j_str, c_str);
    return (*env)->NewStringUTF(env, buff);
}
#ifdef __cplusplus
}
#endif

5. 编译JNI动态链接库
   gcc -I$JAVA_HOME/include -I$JAVA_HOME/include/linux -fPIC -shared jni/com_clark010_playgroud_es_jni_HelloWorld.c -o libHelloWorld.so
   在当前路径libHelloWorld.so动态链接库

6. 测试
   执行第一步生成的HelloWorld测试程序
   $JAVA_HOME/bin/java -classpath ./classes -Djava.library.path=./ com.clark010.playgroud.es.jni.HelloWorld

输出 :hello clark010

7. 编写es插件
   7.1 实现插件

// TestPlugin class
package com.clark010.playgroud.es.plugin.test;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.elasticsearch.plugins.ActionPlugin;
import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.plugins.ScriptPlugin;
import org.elasticsearch.script.NativeScriptFactory;

import java.util.Collections;
import java.util.List;

public class TestPlugin extends Plugin implements ActionPlugin, ScriptPlugin {
  private final static Logger LOGGER = LogManager.getLogger(TestPlugin.class);

  public TestPlugin() {
    super();
    LOGGER.warn("Create the Test Plugin and installed it into elasticsearch");
  }

  public List<NativeScriptFactory> getNativeScripts() {
    return Collections.<NativeScriptFactory>singletonList(new TestPluginFactory());
  }
}

// TestPluginFactory class
package com.clark010.playgroud.es.plugin.test;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.elasticsearch.common.Nullable;
import org.elasticsearch.index.fielddata.ScriptDocValues;
import org.elasticsearch.script.AbstractDoubleSearchScript;
import org.elasticsearch.script.ExecutableScript;
import org.elasticsearch.script.NativeScriptFactory;
import org.elasticsearch.SpecialPermission;

import com.clark010.playgroud.es.jni.HelloWorld;

import java.security.AccessController;
import java.security.PrivilegedAction;

import java.util.List;
import java.util.Map;

public class TestPluginFactory implements NativeScriptFactory {
  private final static Logger LOGGER = LogManager.getLogger(TestPlugin.class);

 static {
        final SecurityManager sm = System.getSecurityManager();
        if (sm != null) {
            sm.checkPermission(new SpecialPermission());
        }
        AccessController.doPrivileged(new PrivilegedAction() {
             public Object run() {
                    LOGGER.info("load library: HelloWorld");
                   // privileged code goes here, for example:
                    System.loadLibrary("HelloWorld");
                    LOGGER.info("load library success");
                    return null; // nothing to return
            }
      });
 }

  public ExecutableScript newScript(@Nullable Map<String, Object> params) {
    return new JyhScript(params);
  }

  public boolean needsScores() {
    return false;
  }
  public String getName() {
    return "testplugin";
  }

  protected class JyhScript extends AbstractDoubleSearchScript {
    private double price;

    //params就是搜索请求中传入的参数
    public JyhScript(@Nullable Map<String,Object> params){
        LOGGER.info("init JyhScript");
    }

    @Override
    public double runAsDouble() {
      LOGGER.info("JNI output info:" + HelloWorld.sayHello("clark010"));
      return 0.0;
}

创建项目通过maven编译生成jar包，具体参考es_jni
mvn package -DskipTests
将target/releases下生成zip包拷贝到测试es的plugins目录下解压（test-plugin目录），目录下有如下文件
7.1.1 plugin jar包

7.1.2 plugin-descriptor.properties

description=testplugin
version=1.0
name=TestPlugin
site=${elasticsearch.plugin.site}
jvm=true
classname=com.clark010.playgroud.es.plugin.test.TestPlugin
java.version=1.8
elasticsearch.version=5.3.3
isolated=${elasticsearch.plugin.isolated}

7.1.3 plugin-security.policy

  grant {
    // needed to generate runtime classes
    permission java.lang.RuntimePermission "loadLibrary.HelloWorld";
  };

插件权限问题详细说明请参考：

• modules-scripting-security
• plugin-authors

7.2 重新启动es
7.2.1 指定java.library.path到libHelloWorld.so的生成目录，即当前目录
export ES_JAVA_OPTS="$ES_JAVA_OPTS -Djava.library.path=./"
7.2.2 启动elasticsearch（先停掉老的服务）
$ES_HOME/bin/elasticsearch

7.3 测试

# 插入
curl -XPUT localhost:9200/m2c/item/1 -d '{
    "id":"11111",
    "sproduct":"nike",
    "iismerchant":[1]
}'

# 查询
curl -XGET localhost:9200/m2c/item/_search -d '{
  "query": {
    "function_score": {
      "query": {
        "match": {"sproduct":"nike"}
      },
      "script_score": {
        "script": {
          "inline":"testplugin",
          "lang": "native",
          "params": {
          }
        }
      }
    }
  }
}'

查看es日志，输出：hello clark010

备忘

• 如果没有通过配置java的安全策略，则会报如下异常(并且es节点会异常退出)

Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "loadLibrary.HelloWorld")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_131]
at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_131]
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_131]
at java.lang.SecurityManager.checkLink(SecurityManager.java:835) ~[?:1.8.0_131]
at java.lang.Runtime.loadLibrary0(Runtime.java:864) ~[?:1.8.0_131]
at java.lang.System.loadLibrary(System.java:1122) ~[?:1.8.0_131]
...........

• 如果JNI有依赖其他的动态链接库，则需要在ES启动前执行export LD_LIBRARY_PATH=$libs_dir