【Ovirt 笔记】engine-manage-domains

分析整理的版本为 Ovirt 3.4.5 版本。

命令使用方式：engine-manage-domains <action> [<args>]

Available actions:
    add         add a domain using specified provider and user
    edit        edit an existing domain
    delete      delete an existing domain
    validate    validate the current configuration
    list        list the current configuration

Add domain:
    engine-manage-domains add --domain=DOMAIN --provider=PROVIDER --user=USER  [--add-permissions] [--config-file=CFG_FILE] [--ldap-servers=SERVERS] [--resolve-kdc] [--password-file=PASS_FILE] [--change-password-msg]

Edit domain:
    engine-manage-domains edit --domain=DOMAIN [--provider=PROVIDER] [--user=USER]  [--add-permissions] [--config-file=CFG_FILE] [--ldap-servers=SERVERS] [--resolve-kdc]  [--password-file=PASS_FILE] [--change-password-msg]

Delete domain:
    engine-manage-domains delete --domain=DOMAIN [--force] [--config-file=CFG_FILE] [--password-file=PASS_FILE]

Validate configuration:
    engine-manage-domains validate [--report] [--config-file=CFG_FILE]

List configuration:
    engine-manage-domains list [--config-file=CFG_FILE]

Options:
--add-permissions
    Add engine superuser permissions to the user.

--change-password-msg
    Reads interactively a URL or a message to be returned to the user in case the password has expired.

--config-file=CFG_FILE
    Use the given alternate configuration file.

--domain=DOMAIN
    The domain you wish to perform the action on.

--force
    Skip confirmation of a delete operation.

--help
    Show this help message and exit.

--ldap-servers=SERVERS
    A comma delimited list of LDAP servers to be set to the domain.

--log-file=LOG_FILE
    Sets file to write logging into (if not set nothing is logged).

--log-level=LOG_LEVEL
    Sets log level, one of DEBUG (default), INFO, WARN, ERROR (case insensitive).

--log4j-config=XML_FILE
    Sets log4j.xml file which logging configuration is loaded from.

--provider=PROVIDER
    The LDAP provider type of server used for the domain, can be one of (case insensitive):
        ad        Microsoft Active Directory
        ipa       freeIPA
        rhds      Red Hat Directory Server
        itds      IBM Tivoli Directory Server
        oldap     OpenLDAP

--report
    Report all validation error, if occured (default behaviour is to exit when a validation error occurs).

--resolve-kdc
    Resolve KDC servers using DNS (don't assume they are the same as LDAP servers).

--user=USER
    The domain user.

--password-file=PASS_FILE
    A  file containing the password (if it's not set, the password will be read interactively).

命令采用了 shell 调用 java 的方式进行实现。

engine-manage-domains.sh 利用 jboss-modules.jar 查询指定的模块 org.ovirt.engine.core.tools 执行启动类 org.ovirt.engine.core.domains.ManageDomainsExecutor

exec "${JAVA_HOME}/bin/java" \
    -Djboss.modules.write-indexes=false \
    -jar "${JBOSS_HOME}/jboss-modules.jar" \
    -dependencies org.ovirt.engine.core.tools \
    -class org.ovirt.engine.core.domains.ManageDomainsExecutor \
    "$@"

1. ManageDomainsExecutor 执行 main 方法，通过 ManageDomainsArguments 处理验证参数。

mdArgs = new ManageDomainsArguments();
mdArgs.parse(args);

2. 事务逻辑通过 ManageDomains 类进行处理。

ManageDomains util = new ManageDomains(mdArgs);
// it's existence is checked during the parser validation
util.init();
util.createConfigurationProvider();
util.runCommand();

3. ManageDomains 执行的相关事务。

public void runCommand() throws ManageDomainsResult {
    String action = args.get(ARG_ACTION);

    if (ACTION_ADD.equals(action)) {
        addDomain();
    } else if (ACTION_EDIT.equals(action)) {
        editDomain();
    } else if (ACTION_DELETE.equals(action)) {
        deleteDomain();
    } else if (ACTION_VALIDATE.equals(action)) {
        validate();
    } else if (ACTION_LIST.equals(action)) {
        getConfiguration();
    }
}

4. 有关 AD 的操作使用了 ldap 服务，加载配置信息。

util.createConfigurationProvider();

public void createConfigurationProvider() throws ManageDomainsResult {
    String engineConfigProperties = createTempPropFile();
    try {
        String engineConfigExecutable = utilityConfiguration.getEngineConfigExecutablePath();
        String adUserName = getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.AdUserName);
        String domainName = getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.DomainName);
        String ldapSecurityAuthentication =
                getConfigValue(engineConfigExecutable,
                        engineConfigProperties,
                        ConfigValues.LDAPSecurityAuthentication);
        String adUserPassword =
                getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.AdUserPassword);
        String adUserId = getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.AdUserId);
        String ldapServers =
                getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.LdapServers);
        String ldapProviderTypes =
                getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.LDAPProviderTypes);
        String ldapPort =
                getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.LDAPServerPort);
        if (ldapPort == null) {
            ldapPort = DEFAULT_LDAP_SERVER_PORT;
        }
        String changePasswordUrl =
                getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.ChangePasswordMsg);
        if (changePasswordUrl == null) {
            changePasswordUrl = "";
        }
        String saslQOP =
                getConfigValue(engineConfigExecutable, engineConfigProperties, ConfigValues.SASL_QOP);

        configurationProvider =
                new ConfigurationProvider(adUserName,
                        adUserPassword,
                        domainName,
                        ldapSecurityAuthentication,
                        ldapServers,
                        adUserId,
                        ldapProviderTypes,
                        utilityConfiguration.getEngineConfigExecutablePath(),
                        engineConfigProperties, ldapPort, changePasswordUrl, saslQOP);

    } catch (Throwable e) {
        throw new ManageDomainsResult(ManageDomainsResultEnum.FAILED_READING_CURRENT_CONFIGURATION, e.getMessage());
    }
}

• 实际通过调用 engine-config.sh 获取。

public String getConfigValue(String engineConfigExecutable, String engineConfigProperties, ConfigValues enumValue)
        throws IOException,
        InterruptedException {
    Process engineConfigProcess =
            Runtime.getRuntime().exec(engineConfigExecutable + " -g "
                    + enumValue.name() + " --cver=general" + " -p " + engineConfigProperties);
    int retVal = engineConfigProcess.waitFor();
    if (retVal == 0) {
        InputStream processOutput = engineConfigProcess.getInputStream();
        return convertStreamToString(processOutput);
    } else {
        InputStream errorOutput = engineConfigProcess.getErrorStream();
        throw new FailedReadingConfigValueException(enumValue.name(), convertStreamToString(errorOutput));
    }
}

例如：

engine-config.sh -g AdUserName --cver=general -p /tmp/engine-config25691...2950properties

从数据库中表 vdc_options 读取信息。

validate

• 先将数据库中已添加的域和域用户名、密码读出，然后去 ldap 服务器验证用户名密码是否正确，验证与添加域时候的验证一样。

ldap://30cloud-4137917.sdcad.com:389/DC=sdcad,DC=com

注意：数据库中域用户的密码存放在 AdUserPassword 内置参数中（多个域用户之间由逗号分隔），其加密方式和门户管理员密码 AdminPassword 一样。

add

• 查询验证 ldap 服务。

List<String> ldapServers = getLdapServers(domainName);

dns:///_ldap._tcp.sdcad.com

• 查询验证 Kdc 服务。

validateKdcServers(authMode, domainName);

dns:///_kerberos._tcp.sdcad.com

• 授权模式 GSSAPI，根据模板文件 krb5.conf.template

krbConfCreator = new KrbConfCreator(gssapiDomainsString, useDnsLookup, ldapServersPerGSSAPIDomains, domainRealmMappingFile);

/backend/manager/modules/utils/src/main/resources/krb5.conf.template

• 创建 krb5 配置文件，路径和名称是 krb5confFile 变量值，如果存在会先备份再重新创建。

krb5 配置文件路径的读取从 engine-manage-domains.conf 配置文件中获取

jaasFile=/usr/share/ovirt-engine/conf/jaas.conf
krb5confFile=/etc/ovirt-engine/krb5.conf
engineConfigExecutable=/usr/share/ovirt-engine/bin/engine-config.sh
localHostEntry=localhost
useDnsLookup=true

• krb5 配置文件内容如下，该文件的核心内容是前两行。

[libdefaults]

default_realm = SDCAD.COM
dns_lookup_realm = true
dns_lookup_kdc = true
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = no
default_tkt_enctypes = arcfour-hmac-md5
udp_preference_limit = 1

#realms

#domain_realm

• 读取 JAAS 配置文件，进行验证。

AuthenticationResult authResult = authenticate(realm, username, password, pathToJAASFile, pathToKrb5ConfFile);

private AuthenticationResult checkAuthentication(String username, String password) {

    AuthenticationResult result = AuthenticationResult.OK;
    try {

        lc = new LoginContext("KerberosUtil", new KerberosUtilCallbackHandler(username, password));
        lc.login();
        log.debug("Check authentication finished successfully ");
    } catch (LoginException ex) {
        String resultMessage = ex.getMessage();
        String formattedMessage = ERROR_PREFIX + " exception message: " + resultMessage;
        log.error(formattedMessage);
        log.debug("", ex);

        KerberosReturnCodeParser parser = new KerberosReturnCodeParser();
        result = parser.parse(resultMessage);
        if (result != AuthenticationResult.OTHER) {
            return result;
        } else {
            System.out.println(formattedMessage);
        }
    }
    return result;
}

• JAAS 配置文件内容，路径所在 jaasFile=/usr/share/ovirt-engine/conf/jaas.conf。

/**
 * Login Configuration for JAAS.
 *
 * Specify that Kerberos v5 is a required login module for the
 * example classes: GssExample and Mutual.
 */
KerberosUtil {
  com.sun.security.auth.module.Krb5LoginModule required client=TRUE;
};

• 通过 LoginContext 调用 com.sun.security.auth.module. Krb5LoginModule 进行 JAAS 验证。

lc = new LoginContext("KerberosUtil", new KerberosUtilCallbackHandler(username, password));
lc.login();

实现 JAAS（Java Authentication Authorization Service） 一个简单验证实例。

String pathToKrb5File = "/etc/ovirt-engine/krb5.conf";
String pathToJAASFile = "/usr/share/ovirt-engine/conf/jaas.conf";
String username = "sdcadmin";
String password = "Kvm_123456";
System.setProperty("java.security.krb5.conf", pathToKrb5File);
System.setProperty("java.security.auth.login.config", pathToJAASFile);
try {
    lc = new LoginContext("KerberosUtil", new KerberosUtilCallbackHandler(username, password));
    lc.login();
    System.out.println("Check authentication finished successfully");
} catch (LoginException ex) {
    System.out.println("auth failed.");
    ex.printStackTrace();
}

delete

• 先从表 vdc_options 中将域名、域用户、密码（多个域在数据中以逗号分隔）读出存入 HashMap 中，然后 remove 掉传入的 --domain 参数，再将 HashMap 中的内容存入数据库相应的字段。

• 方式与增加一致，通过 engine-config.sh 实现。

• HashMap 是 DomainsConfigurationEntry 类的一个属性。

protected Map<String, String> valuePerDomain = new HashMap<String, String>();

public void removeValueForDomain(String domain) {
    valuePerDomain.remove(domain);
}

• 修改也是类似的过程。

public void setValueForDomain(String domain, String value) {
    valuePerDomain.put(domain, value);
}

• 最终通过以下方法生效。

public void setConfigValue(ConfigValues enumValue, DomainsConfigurationEntry entry, boolean passedAsValue)
        throws ManageDomainsResult {

    log.info("Setting value for " + enumValue.toString() + " to " + entry.getDomainsLoggingEntry());
    File passFile = null;

    try {
        StringBuilder executeCmd = new StringBuilder(engineConfigExecutable);
        executeCmd.append(" -s ").append(enumValue.name());
        if (passedAsValue) {
            executeCmd.append("=" + entry.getDomainsConfigurationEntry());
        } else {
            passFile = createPassFile(entry.getDomainsConfigurationEntry());
            executeCmd.append(" --admin-pass-file " + passFile.getAbsolutePath());
        }
        executeCmd.append(" -p " + engineConfigProperties);
        Process engineConfigProcess = Runtime.getRuntime().exec(executeCmd.toString());

        int retVal = engineConfigProcess.waitFor();
        if (retVal != 0) {
            throw new ManageDomainsResult(ManageDomainsResultEnum.FAILED_SETTING_CONFIGURATION_VALUE_FOR_OPTION,
                    enumValue.name() + " - execute command: " + executeCmd.toString());
        }
    } catch (Throwable e) {
        throw new ManageDomainsResult(ManageDomainsResultEnum.FAILED_SETTING_CONFIGURATION_VALUE_FOR_OPTION_WITH_DETAILS,
                enumValue.name(), e.getMessage());
    } finally {
        disposePassFile(passFile);
    }
}

engine-config.sh -s AdUserName=xx.com -p /tmp/engine-config25691...2950properties

• 查询域用户时报 Kerberos 错误原因调查。

错误信息

2015-10-29 17:11:36,249 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp-/127.0.0.1:8702-5) Failed ldap search server ldap://sdcad.com:389 using user sdcadmin@SDCAD.COM due to General connection problem due to javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))]. We should try the next server

错误原因

DNS 解析有错误，配置正确的 DNS 解析即可。

/etc/resolv.conf

nameserver 192.168.xx.xx