ThreadPoolExecutor 源码浅析（一）
2018-03-03 本文已影响11人
WJL3333
这篇主要以代码分析为主,文中的代码有注释进行解析.
1.Executor接口
public interface Executor {
/**
* Executes the given command at some time in the future. The command
* may execute in a new thread, in a pooled thread, or in the calling
* thread, at the discretion of the {@code Executor} implementation.
*
* @param command the runnable task
* @throws RejectedExecutionException if this task cannot be
* accepted for execution
* @throws NullPointerException if command is null
*/
void execute(Runnable command);
}
只提供了一个execute方法作为任务执行的抽象
- ExecutorService接口
ExecutorService接口继承了Executor接口增加了一些
- 生命周期相关方法
- void shutdown();
- List<Runnable> shutdownNow();
- ....
- 任务提交方法
- <T> Future<T> submit(Callable<T> task);
- <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks, long timeout, TimeUnit unit) throws InterruptedException;
- ...
3.AbstractExecutorService抽象类
该抽象类只是实现了一些任务提交方法。生命周期相关方法和execute方法留给实现类自己实现
1.newTaskFor 提供了一个便捷方法来方便调用
protected <T> RunnableFuture<T> newTaskFor(Callable<T> callable) {
return new FutureTask<T>(callable);
}
2.submit 实际操作还是调用了execute方法
public <T> Future<T> submit(Callable<T> task) {
if (task == null) throw new NullPointerException();
RunnableFuture<T> ftask = newTaskFor(task);
execute(ftask);
return ftask;
}
3.invokeAll
public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks)
throws InterruptedException {
if (tasks == null)
throw new NullPointerException();
ArrayList<Future<T>> futures = new ArrayList<Future<T>>(tasks.size());
boolean done = false;
try {
//将任务全部提交到线程池
for (Callable<T> t : tasks) {
RunnableFuture<T> f = newTaskFor(t);
futures.add(f);
execute(f);
}
//阻塞直到所有任务完成
for (int i = 0, size = futures.size(); i < size; i++) {
Future<T> f = futures.get(i);
if (!f.isDone()) {
try {
f.get();
} catch (CancellationException ignore) {
} catch (ExecutionException ignore) {
}
}
}
done = true;
//返回的Future都是可以马上得到结果的
return futures;
} finally {
//execute方法会抛出异常，如果没有结束则所有任务取消。
if (!done)
for (int i = 0, size = futures.size(); i < size; i++)
futures.get(i).cancel(true);
}
}
- ThreadPoolExecutor
该类继承了抽象类AbstractExecutorService
4.1 状态控制
内部使用了一个AtomicInteger来保存生命周期相关的信息和线程数信息。
使用一个Integer前3位来表示Executor的生命周期状态。
后29位来表示工作线程数。
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
private static final int COUNT_BITS = Integer.SIZE - 3;
private static final int CAPACITY = (1 << COUNT_BITS) - 1;
// runState is stored in the high-order bits
private static final int RUNNING = -1 << COUNT_BITS;
private static final int SHUTDOWN = 0 << COUNT_BITS;
private static final int STOP = 1 << COUNT_BITS;
private static final int TIDYING = 2 << COUNT_BITS;
private static final int TERMINATED = 3 << COUNT_BITS;
// Packing and unpacking ctl
private static int runStateOf(int c) { return c & ~CAPACITY; }
private static int workerCountOf(int c) { return c & CAPACITY; }
private static int ctlOf(int rs, int wc) { return rs | wc; }
生命周期
RUNNING
接收任务并处理队列中的任务
SHUTDOWN
不接受新任务但是处理工作队列中的任务
STOP
不接受新任务不处理工作队列中的任务，而且中断正在运行的任务线程
TIDY
所有任务都已经终止，工作线程为0，将要运行terminated方法
TERMINATED
terminated()方法返回
4.2 一些volatile变量
volatile 变量使用条件
1.对变量的写入不依赖变量的当前值，或者能保证只有一个变量能更新该变量的值
2.变量的改变不会影响其他变量的状态
3.（那么）该变量的使用就不需要加锁
/*
* All user control parameters are declared as volatiles so that
* ongoing actions are based on freshest values, but without need
* for locking, since no internal invariants depend on them
* changing synchronously with respect to other actions.
*/
private volatile ThreadFactory threadFactory;
private volatile RejectedExecutionHandler handler;
private volatile long keepAliveTime;
private volatile boolean allowCoreThreadTimeOut;
private volatile int corePoolSize;
private volatile int maximumPoolSize;
4.3 Worker
封装了线程池的线程。
继承了AbstractQueuedSynchronizer实现了一个不可重入的派他锁。
并拥有一个管理线程终端状态的方法，该方法避免了为了唤醒等待任务的线程而中断一个正在执行的任务。
private final class Worker
extends AbstractQueuedSynchronizer
implements Runnable
{
/**
* This class will never be serialized, but we provide a
* serialVersionUID to suppress a javac warning.
*/
private static final long serialVersionUID = 6138294804551838833L;
/** Thread this worker is running in. Null if factory fails. */
final Thread thread;
/** Initial task to run. Possibly null. */
Runnable firstTask;
/** Per-thread task counter */
volatile long completedTasks;
/**
* Creates with given first task and thread from ThreadFactory.
* @param firstTask the first task (null if none)
*/
Worker(Runnable firstTask) {
setState(-1); // inhibit interrupts until runWorker
this.firstTask = firstTask;
this.thread = getThreadFactory().newThread(this);
}
/** Delegates main run loop to outer runWorker */
public void run() {
runWorker(this); //下面会说到这个方法。4.8
}
// Lock methods
//
// The value 0 represents the unlocked state.
// The value 1 represents the locked state.
protected boolean isHeldExclusively() {
return getState() != 0;
}
protected boolean tryAcquire(int unused) {
if (compareAndSetState(0, 1)) {
setExclusiveOwnerThread(Thread.currentThread());
return true;
}
return false;
}
protected boolean tryRelease(int unused) {
setExclusiveOwnerThread(null);
setState(0);
return true;
}
public void lock() { acquire(1); }
public boolean tryLock() { return tryAcquire(1); }
public void unlock() { release(1); }
public boolean isLocked() { return isHeldExclusively(); }
void interruptIfStarted() {
Thread t;
if (getState() >= 0 && (t = thread) != null && !t.isInterrupted()) {
try {
t.interrupt();
} catch (SecurityException ignore) {
}
}
}
}
4.4 设置线程池状态的方法
/**
* Transitions to TERMINATED state if either (SHUTDOWN and pool
* and queue empty) or (STOP and pool empty). If otherwise
* eligible to terminate but workerCount is nonzero, interrupts an
* idle worker to ensure that shutdown signals propagate. This
* method must be called following any action that might make
* termination possible -- reducing worker count or removing tasks
* from the queue during shutdown. The method is non-private to
* allow access from ScheduledThreadPoolExecutor.
*/
final void tryTerminate() {
for (;;) {
int c = ctl.get();
//条件一
//判断线程池状态，如果正在运行，现在是TIDY或者TERMINATED，
//又或者现在是SHUTDOWN状态，但是工作队列非空则无需继续处理
if (isRunning(c) ||
runStateAtLeast(c, TIDYING) ||
(runStateOf(c) == SHUTDOWN && ! workQueue.isEmpty()))
return;
//条件二
//工作线程为0,可以关闭线程池,则中断空闲线程
if (workerCountOf(c) != 0) { // Eligible to terminate
interruptIdleWorkers(ONLY_ONE);
return;
}
//为什么使用CAS操作还上锁了呢
//无视条件一和条件二的代码则相当于在循环中获取CAS变量的状态
//之后比较并测试，上锁为了保证terminated方法只调用一次。
//下面的CAS操作将线程池状态设置为TIDY，如果当前线程设置成功了(返回true)
//那么其他线程同时进入到这个方法的时候就会被条件一中的runStateAtLeast条件过滤而返回
//而且不上锁的话。可能会出现当前线程刚刚将状态设置为TIDY，
//而另外一个线程因为CAS操作失败之后，重新获取状态为TIDY，而且调用CAS操作将TIDY设置为TIDY
//这样terminated方法又会调佣一遍。就会出现问题。
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
try {
terminated();
} finally {
ctl.set(ctlOf(TERMINATED, 0));
termination.signalAll();
}
return;
}
} finally {
mainLock.unlock();
}
// else retry on failed CAS
}
}
4.5 控制工作线程中断的方法
interruptWorkers中断所有Worker，调用Worker.interruptIfStarted()
线程池中的所有Worker是保存在一个HashSet(workers)中的（为保证线程安全，对这个集合的操作需要得到mainLock）
//中断空闲进程
//空闲进程的判断：
//Worker本身是一个互斥锁，如果当前Worker正在工作这个锁是被lock的。
//那么其他线程想要得到Worker的锁（tryLock）会返回false那么这个Worker就不是空闲进程。
/**
*@param onlyOne If true, interrupt at most one worker. This is
* called only from tryTerminate when termination is otherwise
* enabled but there are still other workers. In this case, at
* most one waiting worker is interrupted to propagate shutdown
* signals in case all threads are currently waiting.
* Interrupting any arbitrary thread ensures that newly arriving
* workers since shutdown began will also eventually exit.
* To guarantee eventual termination, it suffices to always
* interrupt only one idle worker, but shutdown() interrupts all
* idle workers so that redundant workers exit promptly, not
* waiting for a straggler task to finish.
*/
private void interruptIdleWorkers(boolean onlyOne) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
for (Worker w : workers) {
Thread t = w.thread;
if (!t.isInterrupted() && w.tryLock()) {
try {
t.interrupt();
} catch (SecurityException ignore) {
} finally {
w.unlock();
}
}
if (onlyOne)
break;
}
} finally {
mainLock.unlock();
}
}
4.6 addWorker
用来增加线程
private boolean addWorker(Runnable firstTask, boolean core) {
//该代码块逻辑，如果成功增加了线程数目而且线程池运行状态不变则继续运行。略过
retry:
for (;;) {
int c = ctl.get();
int rs = runStateOf(c);
// Check if queue empty only if necessary.
if (rs >= SHUTDOWN &&
! (rs == SHUTDOWN &&
firstTask == null &&
! workQueue.isEmpty()))
return false;
for (;;) {
int wc = workerCountOf(c);
if (wc >= CAPACITY ||
wc >= (core ? corePoolSize : maximumPoolSize))
return false;
if (compareAndIncrementWorkerCount(c))
break retry;
c = ctl.get(); // Re-read ctl
if (runStateOf(c) != rs)
continue retry;
// else CAS failed due to workerCount change; retry inner loop
}
}
//增加Worker主要逻辑
//只要一个Worker构造出来就已经包含一个Thread了（见上构造方法）
//但是如果ThreadFactory无法创建线程则会返回null
boolean workerStarted = false;
boolean workerAdded = false;
Worker w = null;
try {
w = new Worker(firstTask);
final Thread t = w.thread;
if (t != null) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
// Recheck while holding lock.
// Back out on ThreadFactory failure or if
// shut down before lock acquired.
int rs = runStateOf(ctl.get());
if (rs < SHUTDOWN ||
(rs == SHUTDOWN && firstTask == null)) {
if (t.isAlive()) // precheck that t is startable
throw new IllegalThreadStateException();
// Thread.isAlive()它表示线程当前是否为可用状态，
// 如果线程已经启动，并且当前没有任何异常的话，则返回true，否则为false
// 一个Worker中的线程应该是未启动的。返回true则抛出异常。
//添加到HashSet中
workers.add(w);
int s = workers.size();
if (s > largestPoolSize)
largestPoolSize = s;
workerAdded = true;
}
} finally {
mainLock.unlock();
}
if (workerAdded) {
t.start();//线程执行
workerStarted = true;
}
}
} finally {
//如果上面抛出异常了或者ThreadFactory无法创建异常，会执行addWorkerFailed方法
if (! workerStarted)
addWorkerFailed(w);
}
return workerStarted;
}
4.7 addWorkerFailed
/**
* Rolls back the worker thread creation.
* - removes worker from workers, if present
* - decrements worker count
* - rechecks for termination, in case the existence of this
* worker was holding up termination
*/
//从HashSet中移除Worker，减少Worker数量
private void addWorkerFailed(Worker w) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
if (w != null)
workers.remove(w);
decrementWorkerCount();
//[为什么调用]tryTerminate()?
tryTerminate();
} finally {
mainLock.unlock();
}
}
4.8 启动Worker
final void runWorker(Worker w) {
Thread wt = Thread.currentThread();
Runnable task = w.firstTask;
w.firstTask = null;
// allow interrupts
//见上4.5
w.unlock();
boolean completedAbruptly = true;
try {
//从队列中获取任务getTask()
while (task != null || (task = getTask()) != null) {
w.lock();
// If pool is stopping, ensure thread is interrupted;
// if not, ensure thread is not interrupted. This
// requires a recheck in second case to deal with
// shutdownNow race while clearing interrupt
if ((runStateAtLeast(ctl.get(), STOP) ||
(Thread.interrupted() &&
runStateAtLeast(ctl.get(), STOP))) &&
!wt.isInterrupted())
wt.interrupt();
//如果调用线程被中断了而Worker的线程没被中断，则中断worker中的线程。
try {
beforeExecute(wt, task);
Throwable thrown = null;
try {
task.run();
} catch (RuntimeException x) {
thrown = x; throw x;
} catch (Error x) {
thrown = x; throw x;
} catch (Throwable x) {
thrown = x; throw new Error(x);
} finally {
afterExecute(task, thrown);
}
} finally {
task = null;
w.completedTasks++;
w.unlock();
}
}
completedAbruptly = false;
} finally {
processWorkerExit(w, completedAbruptly);
}
}
4.9 processWorkerExit
private void processWorkerExit(Worker w, boolean completedAbruptly) {
if (completedAbruptly) // If abrupt, then workerCount wasn't adjusted
decrementWorkerCount();
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
completedTaskCount += w.completedTasks;
workers.remove(w);
} finally {
mainLock.unlock();
}
tryTerminate();
int c = ctl.get();
if (runStateLessThan(c, STOP)) {
if (!completedAbruptly) {
//正常退出则判断是否应该增加Worker
int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
if (min == 0 && ! workQueue.isEmpty())
min = 1;
if (workerCountOf(c) >= min)
return; // replacement not needed
}
//增加Worker 4.6
addWorker(null, false);
}
}
4.10 execute方法
/**
* Executes the given task sometime in the future. The task
* may execute in a new thread or in an existing pooled thread.
*
* If the task cannot be submitted for execution, either because this
* executor has been shutdown or because its capacity has been reached,
* the task is handled by the current {@code RejectedExecutionHandler}.
*
* @param command the task to execute
* @throws RejectedExecutionException at discretion of
* {@code RejectedExecutionHandler}, if the task
* cannot be accepted for execution
* @throws NullPointerException if {@code command} is null
*/
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
*
* 3. If we cannot queue task, then we try to add a new
* thread. If it fails, we know we are shut down or saturated
* and so reject the task.
*/
int c = ctl.get();
//少于核心数目，增加worker并直接运行且execute返回
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
c = ctl.get();
}
//offer如果能添加到任务队列中则返回true（不能添加则队列满了）
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
//重新获取线程池状态，如果不是RUNNING状态则移除任务，
//调用RejectedExecutionHandler方法来处理拒绝任务的逻辑。
if (! isRunning(recheck) && remove(command))
reject(command);
//如果工作线程数目为0则新建线程
//这里的逻辑
//上面offer返回true，则说明入队。
//addWorker方法 4.6
//会直接启动Worker线程，运行runWorker() 该方法中如果Worker中的任务为null
//则会从队列中取任务4.8
//所以这里会在线程数为0时创建线程
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
else if (!addWorker(command, false))
//无法执行任务条件则拒绝
reject(command);
}
4.11 getTask
private Runnable getTask() {
boolean timedOut = false; // Did the last poll() time out?
for (;;) {
int c = ctl.get();
int rs = runStateOf(c);
// Check if queue empty only if necessary.
// 线程池关闭
if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
//降低工作线程数
decrementWorkerCount();
return null;
}
int wc = workerCountOf(c);
// Are workers subject to culling?
boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
//如果超过最大线程数 或者 获取任务超时
//而且在工作队列为空的时候还存在空闲的工作线程
//则降低工作线程数目
if ((wc > maximumPoolSize || (timed && timedOut))
&& (wc > 1 || workQueue.isEmpty())) {
if (compareAndDecrementWorkerCount(c))
return null;
continue;
}
try {
Runnable r = timed ?
workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
workQueue.take();
if (r != null)
return r;
timedOut = true;
} catch (InterruptedException retry) {
timedOut = false;
}
}
}
