Tomcat启动分析(五) - Connector

在分析Lifecycle接口之后，本文分析Connector组件的初始化和启动过程。

Connector

与其他组件一样，Connector类也继承了LifecycleMBeanBase类，其构造函数和成员变量如下所示：

public class Connector extends LifecycleMBeanBase  {
    private static final Log log = LogFactory.getLog(Connector.class);
    // ------------------------------------------------------------ Constructor
    public Connector() {
        this(null);
    }

    public Connector(String protocol) {
        setProtocol(protocol);
        // Instantiate protocol handler
        ProtocolHandler p = null;
        try {
            Class<?> clazz = Class.forName(protocolHandlerClassName);
            p = (ProtocolHandler) clazz.getConstructor().newInstance();
        } catch (Exception e) {
            log.error(sm.getString(
                    "coyoteConnector.protocolHandlerInstantiationFailed"), e);
        } finally {
            this.protocolHandler = p;
        }

        if (Globals.STRICT_SERVLET_COMPLIANCE) {
            uriCharset = StandardCharsets.ISO_8859_1;
        } else {
            uriCharset = StandardCharsets.UTF_8;
        }
    }
    // ----------------------------------------------------- Instance Variables
    /**
     * The <code>Service</code> we are associated with (if any).
     */
    protected Service service = null;
    protected boolean allowTrace = false;
    protected long asyncTimeout = 30000;
    protected boolean enableLookups = false;
    protected boolean xpoweredBy = false;
    protected int port = -1;
    protected String proxyName = null;
    protected int proxyPort = 0;
    protected int redirectPort = 443;
    protected String scheme = "http";
    protected boolean secure = false;
    protected static final StringManager sm = StringManager.getManager(Connector.class);
    private int maxCookieCount = 200;
    protected int maxParameterCount = 10000;
    protected int maxPostSize = 2 * 1024 * 1024;
    protected int maxSavePostSize = 4 * 1024;
    protected String parseBodyMethods = "POST";
    protected HashSet<String> parseBodyMethodsSet;
    protected boolean useIPVHosts = false;
    protected String protocolHandlerClassName ="org.apache.coyote.http11.Http11NioProtocol";
    protected final ProtocolHandler protocolHandler;
    protected Adapter adapter = null;
    @Deprecated
    protected String URIEncoding = null;
    protected String URIEncodingLower = null;
    private Charset uriCharset = StandardCharsets.UTF_8;
    protected boolean useBodyEncodingForURI = false;
}

成员变量的含义可以参考Connector配置文档，以下的属性值得特别注意：

• URIEncoding属性是用来对URI百分号编码解码时用的编码，从Tomcat 8开始，URIEncoding属性的默认值是UTF-8；
• useBodyEncodingForURI属性默认为false，若设置为true那么Tomcat会使用Content-Type头或ServletRequest接口的setCharacterEncoding方法指定的编码解析查询字符串，若编码不被支持，那么使用默认的ISO-8859-1。请注意该属性只适用于查询字符串，不适用于URI的路径部分。
• 以上两个属性的解释也可参考Tomcat Wiki。

初始化Connector

initInternal方法主要做了以下几件事：

• 创建一个CoyoteAdapter并关联到此Connector上；
• 初始化此Connector的protocolHandler。

@Override
protected void initInternal() throws LifecycleException {
    super.initInternal();
    // Initialize adapter
    adapter = new CoyoteAdapter(this);
    protocolHandler.setAdapter(adapter);
    // Make sure parseBodyMethodsSet has a default
    if (null == parseBodyMethodsSet) {
        setParseBodyMethods(getParseBodyMethods());
    }
    if (protocolHandler.isAprRequired() && !AprLifecycleListener.isAprAvailable()) {
        throw new LifecycleException(sm.getString("coyoteConnector.protocolHandlerNoApr",
                getProtocolHandlerClassName()));
    }
    if (AprLifecycleListener.isAprAvailable() && AprLifecycleListener.getUseOpenSSL() &&
            protocolHandler instanceof AbstractHttp11JsseProtocol) {
        AbstractHttp11JsseProtocol<?> jsseProtocolHandler =
                (AbstractHttp11JsseProtocol<?>) protocolHandler;
        if (jsseProtocolHandler.isSSLEnabled() &&
                jsseProtocolHandler.getSslImplementationName() == null) {
            // OpenSSL is compatible with the JSSE configuration, so use it if APR is available
            jsseProtocolHandler.setSslImplementationName(OpenSSLImplementation.class.getName());
        }
    }

    try {
        protocolHandler.init();
    } catch (Exception e) {
        throw new LifecycleException(sm.getString("coyoteConnector.protocolHandlerInitializationFailed"), e);
    }
}

启动Connector

Connector类的startInternal方法启动了关联的protocolHandler：

@Override
protected void startInternal() throws LifecycleException {
    // Validate settings before starting
    if (getPort() < 0) {
        throw new LifecycleException(sm.getString(
                "coyoteConnector.invalidPort", Integer.valueOf(getPort())));
    }

    setState(LifecycleState.STARTING);

    try {
        protocolHandler.start();
    } catch (Exception e) {
        throw new LifecycleException(
                sm.getString("coyoteConnector.protocolHandlerStartFailed"), e);
    }
}

Http11NioProtocol

本节以常用的Http11NioProtocol分析ProtocolHandler的初始化和启动过程，Http11NioProtocol的类层次结构如下图所示。

Http11NioProtocol类层次结构.png

Http11NioProtocol对象在被构造时，为其自己关联了一个NioEndpoint，类层次结构上的构造函数代码如下：

public Http11NioProtocol() {
    super(new NioEndpoint());
}

public AbstractHttp11JsseProtocol(AbstractJsseEndpoint<S> endpoint) {
    super(endpoint);
}

public AbstractHttp11Protocol(AbstractEndpoint<S> endpoint) {
    super(endpoint);
    setConnectionTimeout(Constants.DEFAULT_CONNECTION_TIMEOUT);
    ConnectionHandler<S> cHandler = new ConnectionHandler<>(this);
    setHandler(cHandler);
    getEndpoint().setHandler(cHandler);
}

public AbstractProtocol(AbstractEndpoint<S> endpoint) {
    this.endpoint = endpoint;
    setSoLinger(Constants.DEFAULT_CONNECTION_LINGER);
    setTcpNoDelay(Constants.DEFAULT_TCP_NO_DELAY);
}

Http11NioProtocol的init和start方法都在其父类AbstractProtocol中定义，部分代码如下：

public abstract class AbstractProtocol<S> implements ProtocolHandler, MBeanRegistration {
    // 省略一些代码
    private final AbstractEndpoint<S> endpoint;

    @Override
    public void init() throws Exception {
        if (getLog().isInfoEnabled()) {
            getLog().info(sm.getString("abstractProtocolHandler.init", getName()));
        }
        // 省略一些JMX相关代码
        String endpointName = getName();
        endpoint.setName(endpointName.substring(1, endpointName.length()-1));
        endpoint.setDomain(domain);
        endpoint.init();
    }

    @Override
    public void start() throws Exception {
        if (getLog().isInfoEnabled()) {
            getLog().info(sm.getString("abstractProtocolHandler.start", getName()));
        }
        endpoint.start();
        // Start async timeout thread
        asyncTimeout = new AsyncTimeout();
        Thread timeoutThread = new Thread(asyncTimeout, getNameInternal() + "-AsyncTimeout");
        int priority = endpoint.getThreadPriority();
        if (priority < Thread.MIN_PRIORITY || priority > Thread.MAX_PRIORITY) {
            priority = Thread.NORM_PRIORITY;
        }
        timeoutThread.setPriority(priority);
        timeoutThread.setDaemon(true);
        timeoutThread.start();
    }
    // 省略一些代码
}

• 初始化过程除了JMX相关的代码就是初始化关联的端点。首先调用端点的setName方法设置名称，然后执行初始化工作；
• 启动过程启动了关联的端点。

端点名称与ProtocolHandler实现有关，AbstractProtocol类中的getName、getNameInternal和getNamePrefix是用于获取ProtocolHandler名称的函数，代码如下：

public String getName() {
    return ObjectName.quote(getNameInternal());
}

private String getNameInternal() {
    StringBuilder name = new StringBuilder(getNamePrefix());
    name.append('-');
    if (getAddress() != null) {
        name.append(getAddress().getHostAddress());
        name.append('-');
    }
    int port = getPort();
    if (port == 0) {
        // Auto binding is in use. Check if port is known
        name.append("auto-");
        name.append(getNameIndex());
        port = getLocalPort();
        if (port != -1) {
            name.append('-');
            name.append(port);
        }
    } else {
        name.append(port);
    }
    return name.toString();
}

protected abstract String getNamePrefix();

Http11NioProtocol类实现的getNamePrefix方法如下，所以端点名有类似“http-nio-端口号”这种形式，这在日志输出时有体现，其他ProtocolHandler同理。

@Override
protected String getNamePrefix() {
    if (isSSLEnabled()) {
        return ("https-" + getSslImplementationShortName()+ "-nio");
    } else {
        return ("http-nio");
    }
}

下面分析端点AbstractEndpoint和实现类NioEndpoint。

AbstractEndpoint

AbstractEndpoint类的层次结构如下图所示：

AbstractEndpoint类层次结构.png

AbstractEndpoint类的部分代码如下：

public abstract class AbstractEndpoint<S> {
    // 省略一些代码
    protected volatile boolean running = false;
    protected volatile boolean paused = false;
    protected volatile boolean internalExecutor = true;
    private volatile LimitLatch connectionLimitLatch = null;
    protected SocketProperties socketProperties = new SocketProperties();
    public SocketProperties getSocketProperties() {
        return socketProperties;
    }
    protected Acceptor[] acceptors;
    protected SynchronizedStack<SocketProcessorBase<S>> processorCache;
    private int acceptCount = 100;
    public void setAcceptCount(int acceptCount) { if (acceptCount > 0) this.acceptCount = acceptCount; }
    public int getAcceptCount() { return acceptCount; }
    /**
     * Acceptor thread count.
     */
    protected int acceptorThreadCount = 1;
    public void setAcceptorThreadCount(int acceptorThreadCount) {
        this.acceptorThreadCount = acceptorThreadCount;
    }
    public int getAcceptorThreadCount() { return acceptorThreadCount; }
    // 省略一些代码
    private boolean bindOnInit = true;
    public boolean getBindOnInit() { return bindOnInit; }
    public void setBindOnInit(boolean b) { this.bindOnInit = b; }
    private volatile BindState bindState = BindState.UNBOUND;

    private Executor executor = null;
    public void setExecutor(Executor executor) {
        this.executor = executor;
        this.internalExecutor = (executor == null);
    }
    public Executor getExecutor() { return executor; }

    public abstract void bind() throws Exception;
    public abstract void startInternal() throws Exception;

    public void init() throws Exception {
        if (bindOnInit) {
            bind();
            bindState = BindState.BOUND_ON_INIT;
        }
        if (this.domain != null) {
            // Register endpoint (as ThreadPool - historical name)
            oname = new ObjectName(domain + ":type=ThreadPool,name=\"" + getName() + "\"");
            Registry.getRegistry(null, null).registerComponent(this, oname, null);

            for (SSLHostConfig sslHostConfig : findSslHostConfigs()) {
                registerJmx(sslHostConfig);
            }
        }
    }

    public final void start() throws Exception {
        if (bindState == BindState.UNBOUND) {
            bind();
            bindState = BindState.BOUND_ON_START;
        }
        startInternal();
    }
    // 省略一些代码
}

• acceptCount、acceptorThreadCount等很多都是端点的属性，上述代码没有全部包括；
• executor表示处理请求用的工作线程池，internalExecutor变量表示该线程池是否由内部创建，外部线程池是指server.xml中Connector元素executor属性引用的线程池；
• Acceptor是静态内部类，表示Acceptor线程；
• 在init函数中，bindOnInit是一个布尔值，true表示在init时绑定地址，false表示在start时绑定地址，默认是true。初始化时会调用bind抽象方法并做与JMX相关的工作；
• start函数可以看到bindOnInit的作用，若还未绑定则先绑定再调用startInternal抽象方法。

属性赋值

端点的属性是在何时被赋值的呢？这还要回到前文所述的解析server.xml的过程中。在解析server.xml时为Server/Service/Connector创建了一个ConnectorCreateRule和一个SetAllPropertiesRule。
ConnectorCreateRule创建了Connector实例，并调用ProtocolHandler如AbstractProtocol的setExecutor方法将executor属性值引用的外部工作线程池设置到与AbstractProtocol关联的AbstractEndpoint上，sslImplementationName同理：

@Override
public void begin(String namespace, String name, Attributes attributes)
        throws Exception {
    Service svc = (Service)digester.peek();
    Executor ex = null;
    if ( attributes.getValue("executor")!=null ) {
        ex = svc.getExecutor(attributes.getValue("executor"));
    }
    Connector con = new Connector(attributes.getValue("protocol"));
    if (ex != null) {
        setExecutor(con, ex);
    }
    String sslImplementationName = attributes.getValue("sslImplementationName");
    if (sslImplementationName != null) {
        setSSLImplementationName(con, sslImplementationName);
    }
    digester.push(con);
}

private static void setExecutor(Connector con, Executor ex) throws Exception {
    Method m = IntrospectionUtils.findMethod(con.getProtocolHandler().getClass(),"setExecutor",new Class[] {java.util.concurrent.Executor.class});
    if (m!=null) {
        m.invoke(con.getProtocolHandler(), new Object[] {ex});
    }else {
        log.warn(sm.getString("connector.noSetExecutor", con));
    }
}

SetAllPropertiesRule这个规则只排除了executor和sslImplementationName两个属性的赋值，并使用IntrospectionUtils.setProperty为属性赋值。Connector元素上可配置的属性列表可以参见官方文档，可以分成三种类型：

• 只属于Connector，如scheme等;
• 只属于EndPoint，如bindOnInit、acceptCount等；
• 共存于Connector和EndPoint，如port、redirectPort等。

因此，属性赋值也分为三种：

• 对于第一种属性，IntrospectionUtils.setProperty会调用恰当的setter方法；
• 对于第二种属性，IntrospectionUtils.setProperty会调用Connector的setProperty方法

  public boolean setProperty(String name, String value) {
      String repl = name;
      if (replacements.get(name) != null) {
          repl = replacements.get(name);
      }
      return IntrospectionUtils.setProperty(protocolHandler, repl, value);
  }
  

  该方法会接着在ProtocolHandler上赋值，AbstractProtocol的setProperty方法如下：

  public boolean setProperty(String name, String value) {
      return endpoint.setProperty(name, value);
  }
  

  接着调用AbstractEndPoint的setProperty方法，如果属性名以socket.开头那么将值设置socketProperties的对应属性上，否则设置到AbstractEndPoint的自身成员变量上：

  public boolean setProperty(String name, String value) {
      setAttribute(name, value);
      final String socketName = "socket.";
      try {
          if (name.startsWith(socketName)) {
              return IntrospectionUtils.setProperty(socketProperties, name.substring(socketName.length()), value);
          } else {
              return IntrospectionUtils.setProperty(this,name,value,false);
          }
      }catch ( Exception x ) {
          getLog().error("Unable to set attribute \""+name+"\" to \""+value+"\"",x);
          return false;
      }
  }
  

• 对于第三种属性，上述两个赋值过程都会执行。

NioEndpoint

NioEndpoint继承了AbstractEndpoint抽象类，部分代码如下：

public class NioEndpoint extends AbstractJsseEndpoint<NioChannel> {
    public static final int OP_REGISTER = 0x100; //register interest op
    // ----------------------------------------------------------------- Fields
    private NioSelectorPool selectorPool = new NioSelectorPool();
    private ServerSocketChannel serverSock = null;
    private volatile CountDownLatch stopLatch = null;
    private SynchronizedStack<PollerEvent> eventCache;
    private SynchronizedStack<NioChannel> nioChannels;
    /**
     * Priority of the poller threads.
     */
    private int pollerThreadPriority = Thread.NORM_PRIORITY;
    public void setPollerThreadPriority(int pollerThreadPriority) { this.pollerThreadPriority = pollerThreadPriority; }
    public int getPollerThreadPriority() { return pollerThreadPriority; }

    /**
     * Poller thread count.
     */
    private int pollerThreadCount = Math.min(2,Runtime.getRuntime().availableProcessors());
    public void setPollerThreadCount(int pollerThreadCount) { this.pollerThreadCount = pollerThreadCount; }
    public int getPollerThreadCount() { return pollerThreadCount; }

    private long selectorTimeout = 1000;
    public void setSelectorTimeout(long timeout){ this.selectorTimeout = timeout;}
    public long getSelectorTimeout(){ return this.selectorTimeout; }

    /**
     * The socket poller.
     */
    private Poller[] pollers = null;
    private AtomicInteger pollerRotater = new AtomicInteger(0);

    /**
     * Return an available poller in true round robin fashion.
     *
     * @return The next poller in sequence
     */
    public Poller getPoller0() {
        int idx = Math.abs(pollerRotater.incrementAndGet()) % pollers.length;
        return pollers[idx];
    }

    // 省略一些代码
}

• NIO特定XML属性的赋值过程同上；
• selectorPool是一个NioSelectorPool类型的选择器池；
• serverSock是端点监听的监听套接字通道；
• SynchronizedStack是Tomcat自己实现的一个栈，入栈和出栈操作都是synchronized的。eventCache和nioChannels是两个栈，分别存放轮询事件和Nio通道。

1. 初始化

NioEndpoint实现了AbstractEndpoint类的bind抽象方法，这里看到了熟悉的ServerSocketChannel等Java NIO的内容，打开通道和绑定地址：

@Override
public void bind() throws Exception {
    serverSock = ServerSocketChannel.open();
    socketProperties.setProperties(serverSock.socket());
    InetSocketAddress addr = (getAddress()!=null?new InetSocketAddress(getAddress(),getPort()):new InetSocketAddress(getPort()));
    serverSock.socket().bind(addr,getAcceptCount());
    serverSock.configureBlocking(true); //mimic APR behavior

    // Initialize thread count defaults for acceptor, poller
    if (acceptorThreadCount == 0) {
        // FIXME: Doesn't seem to work that well with multiple accept threads
        acceptorThreadCount = 1;
    }
    if (pollerThreadCount <= 0) {
        //minimum one poller thread
        pollerThreadCount = 1;
    }
    setStopLatch(new CountDownLatch(pollerThreadCount));

    // Initialize SSL if needed
    initialiseSsl();
    selectorPool.open();
}

2. 启动

NioEndpoint实现了AbstractEndpoint类的startInternal抽象方法，代码如下：

/**
 * Start the NIO endpoint, creating acceptor, poller threads.
 */
@Override
public void startInternal() throws Exception {
    if (!running) {
        running = true;
        paused = false;
        processorCache = new SynchronizedStack<>(SynchronizedStack.DEFAULT_SIZE,
                socketProperties.getProcessorCache());
        eventCache = new SynchronizedStack<>(SynchronizedStack.DEFAULT_SIZE,
                        socketProperties.getEventCache());
        nioChannels = new SynchronizedStack<>(SynchronizedStack.DEFAULT_SIZE,
                socketProperties.getBufferPool());
        // Create worker collection
        if ( getExecutor() == null ) {
            createExecutor();
        }
        initializeConnectionLatch();
        // Start poller threads
        pollers = new Poller[getPollerThreadCount()];
        for (int i=0; i<pollers.length; i++) {
            pollers[i] = new Poller();
            Thread pollerThread = new Thread(pollers[i], getName() + "-ClientPoller-"+i);
            pollerThread.setPriority(threadPriority);
            pollerThread.setDaemon(true);
            pollerThread.start();
        }
        startAcceptorThreads();
    }
}

启动过程做了以下几件事：

• 先用getExecutor函数判断端点是否已关联外部的线程池（见上文属性赋值的分析），若没有则先调用createExecutor创建内部工作线程池；
• initializeConnectionLatch函数利用maxConnections属性创建了LimitLatch对象并赋值给connectionLimitLatch成员变量；
• 创建轮询Poller线程（Poller是NioEndPoint的内部类）；
• 创建Acceptor线程（Acceptor是AbstractEndPoint和NioEndPoint的内部类）。

createExecutor和startAcceptorThreads都定义在父类AbstractEndpoint中，代码如下，其中的getName函数返回端点的名称用以设置线程名称（ProtocolHandler初始化时会给端点设置名称，可以参阅上文Http11NioProtocol的初始化分析）。

public void createExecutor() {
    internalExecutor = true;
    TaskQueue taskqueue = new TaskQueue();
    TaskThreadFactory tf = new TaskThreadFactory(getName() + "-exec-", daemon, getThreadPriority());
    executor = new ThreadPoolExecutor(getMinSpareThreads(), getMaxThreads(), 60, TimeUnit.SECONDS,taskqueue, tf);
    taskqueue.setParent( (ThreadPoolExecutor) executor);
}

protected final void startAcceptorThreads() {
    int count = getAcceptorThreadCount();
    acceptors = new Acceptor[count];

    for (int i = 0; i < count; i++) {
        acceptors[i] = createAcceptor();
        String threadName = getName() + "-Acceptor-" + i;
        acceptors[i].setThreadName(threadName);
        Thread t = new Thread(acceptors[i], threadName);
        t.setPriority(getAcceptorThreadPriority());
        t.setDaemon(getDaemon());
        t.start();
    }
}

protected abstract Acceptor createAcceptor();

所以三种线程的名称分别是：

• 工作线程的名称需要看是外部线程池还是内部线程池：若是外部线程池则是Executor元素的namePrefix属性值加计数，内部线程池则是端点名称加exec加计数；
• 轮询线程的名称是端点名称加ClientPoller加计数；
• Acceptor线程的名称是端点名称加Acceptor加计数。

NioEndpoint实现的createAcceptor方法如下，Acceptor是NioEndpoint的成员内部类：

@Override
protected AbstractEndpoint.Acceptor createAcceptor() {
    return new Acceptor();
}

三种线程的作用请看下一篇文章。