【算法笔记】队列相关基础
2019-12-26 本文已影响0人
程序员Anthony
一些知识
以下内容部分来自极客时间 -王争-数据结构与算法之美 的学习笔记
一、什么是队列？
1.先进者先出，这就是典型的“队列”结构。
2.支持两个操作：入队enqueue()，放一个数据到队尾；出队dequeue()，从队头取一个元素。
3.所以，和栈一样，队列也是一种操作受限的线性表。
二、如何实现队列？
1.队列API
public interface Queue<T> {
public void enqueue(T item); //入队
public T dequeue(); //出队
public int size(); //统计元素数量
public boolean isNull(); //是否为空
}
2.数组实现（顺序队列）：
3.链表实现（链式队列）：
4.循环队列（基于数组）：
三、队列有哪些常见的应用？
队列的应用非常广泛，特别是一些具有某些额外特性的队列，比如循环队列、阻塞队列、并发队列。它们在很多偏底层的系统、框架、中间件的开发中，起着关键性的作用。比如高性能队列 Disruptor、Linux 环形缓存，都用到了循环并发队列；Java concurrent 并发包利用 ArrayBlockingQueue 来实现公平锁等。
关于如何实现无锁并发队列
可以使用 cas + 数组的方式实现。
队列的其他应用
分布式消息队列，如 kafka 也是一种队列。
1.阻塞队列
1）在队列的基础上增加阻塞操作，就成了阻塞队列。
2）阻塞队列就是在队列为空的时候，从队头取数据会被阻塞，因为此时还没有数据可取，直到队列中有了数据才能返回；如果队列已经满了，那么插入数据的操作就会被阻塞，直到队列中有空闲位置后再插入数据，然后在返回。
3）从上面的定义可以看出这就是一个“生产者-消费者模型”。这种基于阻塞队列实现的“生产者-消费者模型”可以有效地协调生产和消费的速度。当“生产者”生产数据的速度过快，“消费者”来不及消费时，存储数据的队列很快就会满了，这时生产者就阻塞等待，直到“消费者”消费了数据，“生产者”才会被唤醒继续生产。不仅如此，基于阻塞队列，我们还可以通过协调“生产者”和“消费者”的个数，来提高数据处理效率，比如配置几个消费者，来应对一个生产者。
2.并发队列
1）在多线程的情况下，会有多个线程同时操作队列，这时就会存在线程安全问题。能够有效解决线程安全问题的队列就称为并发队列。
2）并发队列简单的实现就是在enqueue()、dequeue()方法上加锁，但是锁粒度大并发度会比较低，同一时刻仅允许一个存或取操作。
3）实际上，基于数组的循环队列利用CAS原子操作，可以实现非常高效的并发队列。这也是循环队列比链式队列应用更加广泛的原因。
3.线程池资源枯竭是的处理
在资源有限的场景，当没有空闲资源时，基本上都可以通过“队列”这种数据结构来实现请求排队。
四、思考
1.除了线程池这种池结构会用到队列排队请求，还有哪些类似线程池结构或者场景中会用到队列的排队请求呢？
2.今天讲到并发队列，关于如何实现无锁的并发队列，网上有很多讨论。对这个问题，你怎么看？
队列的java语言数组实现
// 用数组实现的队列
public class ArrayQueue {
// 数组：items，数组大小：n
private String[] items;
private int n = 0;
// head表示队头下标，tail表示队尾下标
private int head = 0;
private int tail = 0;
// 申请一个大小为capacity的数组
public ArrayQueue(int capacity) {
items = new String[capacity];
n = capacity;
}
// 入队
public boolean enqueue(String item) {
// 如果tail == n 表示队列已经满了
if (tail == n) return false;
items[tail] = item;
++tail;
return true;
}
// 出队
public String dequeue() {
// 如果head == tail 表示队列为空
if (head == tail) return null;
String ret = items[head];
++head;
return ret;
}
}
一个是先进先出，一个是先进后出
数据搬移解决队列满的问题
// 入队操作，将item放入队尾
public boolean enqueue(String item) {
// tail == n表示队列末尾没有空间了
if (tail == n) {
// tail ==n && head==0，表示整个队列都占满了
if (head == 0) return false;
// 数据搬移
for (int i = head; i < tail; ++i) {
items[i-head] = items[i];
}
// 搬移完之后重新更新head和tail
tail -= head;
head = 0;
}
items[tail] = item;
++tail;
return true;
}
基于链表的队列实现
入队时，tail->next= new_node, tail = tail->next；出队时，head = head->next
public class LinkedQueue {
//定义一个节点类
private class Node{
String value;
Node next;
}
//记录队列元素个数
private int size = 0;
//head指向队头结点，tail指向队尾节点
private Node head;
private Node tail;
//申请一个队列
public LinkedQueue(){}
//入队
public boolean enqueue(String item){
Node newNode = new Node();
newNode.value = item;
if (size == 0) head = newNode;
else tail.next = newNode;
tail = newNode;
size++;
return true;
}
//出队
public String dequeue(){
String res = null;
if(size == 0) return res;
if(size == 1) tail = null;
res = head.value;
head = head.next;
size--;
return res;
}
}
循环队列
public class CircularQueue {
// 数组：items，数组大小：n
private String[] items;
private int n = 0;
// head表示队头下标，tail表示队尾下标
private int head = 0;
private int tail = 0;
// 申请一个大小为capacity的数组
public CircularQueue(int capacity) {
items = new String[capacity];
n = capacity;
}
// 入队
public boolean enqueue(String item) {
// 队列满了
if ((tail + 1) % n == head) return false;
items[tail] = item;
tail = (tail + 1) % n;
return true;
}
// 出队
public String dequeue() {
// 如果head == tail 表示队列为空
if (head == tail) return null;
String ret = items[head];
head = (head + 1) % n;
return ret;
}
}
队列的C语言实现
来自https://github.com/TheAlgorithms/C
////////////////////////////////////////////////////////////////////////////////
//INCLUDES
#include <stdio.h>
#include <stdlib.h>
////////////////////////////////////////////////////////////////////////////////
//MACROS: CONSTANTS
////////////////////////////////////////////////////////////////////////////////
//DATA STRUCTURES
struct node {
int data;
struct node* next;
struct node* pre;
} *head, *tail, *tmp;
////////////////////////////////////////////////////////////////////////////////
//GLOBAL VARIABLES
int count;
////////////////////////////////////////////////////////////////////////////////
//FORWARD DECLARATIONS
void create();
void enque(int x);
int deque();
int peek();
int size();
int isEmpty();
////////////////////////////////////////////////////////////////////////////////
//MAIN ENTRY POINT
int main(int argc, char const *argv[]) {
create();
enque(5);
return 0;
}
void create() {
head = NULL;
tail = NULL;
}
/**
* Puts an item into the Queue.
*/
void enque(int x) {
if(head == NULL) {
head = (struct node *)malloc(1 * sizeof(struct node));
head->data = x;
head->pre = NULL;
tail = head;
} else {
tmp = (struct node *)malloc(1 * sizeof(struct node));
tmp->data = x;
tmp->next = tail;
tail = tmp;
}
}
/**
* Takes the next item from the Queue.
*/
int deque() {
int returnData;
if(head == NULL) {
printf("ERROR: Deque from empty queue.\n");
exit(1);
} else {
returnData = head->data;
if(head->pre == NULL)
head = NULL;
else
head = head->pre;
head->next = NULL;
}
}
/**
* Returns the size of the Queue.
*/
int size() {
return count;
}
Queue 在《算法 4》中的实现
/******************************************************************************
* Compilation: javac Queue.java
* Execution: java Queue < input.txt
* Dependencies: StdIn.java StdOut.java
* Data files: https://algs4.cs.princeton.edu/13stacks/tobe.txt
*
* A generic queue, implemented using a linked list.
*
* % java Queue < tobe.txt
* to be or not to be (2 left on queue)
*
******************************************************************************/
package edu.princeton.cs.algs4;
import java.util.Iterator;
import java.util.NoSuchElementException;
/**
* The {@code Queue} class represents a first-in-first-out (FIFO)
* queue of generic items.
* It supports the usual <em>enqueue</em> and <em>dequeue</em>
* operations, along with methods for peeking at the first item,
* testing if the queue is empty, and iterating through
* the items in FIFO order.
* <p>
* This implementation uses a singly linked list with a static nested class for
* linked-list nodes. See {@link LinkedQueue} for the version from the
* textbook that uses a non-static nested class.
* See {@link ResizingArrayQueue} for a version that uses a resizing array.
* The <em>enqueue</em>, <em>dequeue</em>, <em>peek</em>, <em>size</em>, and <em>is-empty</em>
* operations all take constant time in the worst case.
* <p>
* For additional documentation, see <a href="https://algs4.cs.princeton.edu/13stacks">Section 1.3</a> of
* <i>Algorithms, 4th Edition</i> by Robert Sedgewick and Kevin Wayne.
*
* @author Robert Sedgewick
* @author Kevin Wayne
*
* @param <Item> the generic type of an item in this queue
*/
public class Queue<Item> implements Iterable<Item> {
private Node<Item> first; // beginning of queue
private Node<Item> last; // end of queue
private int n; // number of elements on queue
// helper linked list class
private static class Node<Item> {
private Item item;
private Node<Item> next;
}
/**
* Initializes an empty queue.
*/
public Queue() {
first = null;
last = null;
n = 0;
}
/**
* Returns true if this queue is empty.
*
* @return {@code true} if this queue is empty; {@code false} otherwise
*/
public boolean isEmpty() {
return first == null;
}
/**
* Returns the number of items in this queue.
*
* @return the number of items in this queue
*/
public int size() {
return n;
}
/**
* Returns the item least recently added to this queue.
*
* @return the item least recently added to this queue
* @throws NoSuchElementException if this queue is empty
*/
public Item peek() {
if (isEmpty()) throw new NoSuchElementException("Queue underflow");
return first.item;
}
/**
* Adds the item to this queue.
*
* @param item the item to add
*/
public void enqueue(Item item) {
Node<Item> oldlast = last;
last = new Node<Item>();
last.item = item;
last.next = null;
if (isEmpty()) first = last;
else oldlast.next = last;
n++;
}
/**
* Removes and returns the item on this queue that was least recently added.
*
* @return the item on this queue that was least recently added
* @throws NoSuchElementException if this queue is empty
*/
public Item dequeue() {
if (isEmpty()) throw new NoSuchElementException("Queue underflow");
Item item = first.item;
first = first.next;
n--;
if (isEmpty()) last = null; // to avoid loitering
return item;
}
/**
* Returns a string representation of this queue.
*
* @return the sequence of items in FIFO order, separated by spaces
*/
public String toString() {
StringBuilder s = new StringBuilder();
for (Item item : this) {
s.append(item);
s.append(' ');
}
return s.toString();
}
/**
* Returns an iterator that iterates over the items in this queue in FIFO order.
*
* @return an iterator that iterates over the items in this queue in FIFO order
*/
public Iterator<Item> iterator() {
return new LinkedIterator(first);
}
// an iterator, doesn't implement remove() since it's optional
private class LinkedIterator implements Iterator<Item> {
private Node<Item> current;
public LinkedIterator(Node<Item> first) {
current = first;
}
public boolean hasNext() { return current != null; }
public void remove() { throw new UnsupportedOperationException(); }
public Item next() {
if (!hasNext()) throw new NoSuchElementException();
Item item = current.item;
current = current.next;
return item;
}
}
/**
* Unit tests the {@code Queue} data type.
*
* @param args the command-line arguments
*/
public static void main(String[] args) {
Queue<String> queue = new Queue<String>();
while (!StdIn.isEmpty()) {
String item = StdIn.readString();
if (!item.equals("-"))
queue.enqueue(item);
else if (!queue.isEmpty())
StdOut.print(queue.dequeue() + " ");
}
StdOut.println("(" + queue.size() + " left on queue)");
}
}
/******************************************************************************
* Copyright 2002-2020, Robert Sedgewick and Kevin Wayne.
*
* This file is part of algs4.jar, which accompanies the textbook
*
* Algorithms, 4th edition by Robert Sedgewick and Kevin Wayne,
* Addison-Wesley Professional, 2011, ISBN 0-321-57351-X.
* http://algs4.cs.princeton.edu
*
*
* algs4.jar is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* algs4.jar is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with algs4.jar. If not, see http://www.gnu.org/licenses.
******************************************************************************/
LinkedQueue在《算法 4》中的实现：
/******************************************************************************
* Compilation: javac LinkedQueue.java
* Execution: java LinkedQueue < input.txt
* Dependencies: StdIn.java StdOut.java
* Data files: https://algs4.cs.princeton.edu/13stacks/tobe.txt
*
* A generic queue, implemented using a singly linked list.
*
* % java Queue < tobe.txt
* to be or not to be (2 left on queue)
*
******************************************************************************/
package edu.princeton.cs.algs4;
import java.util.Iterator;
import java.util.NoSuchElementException;
/**
* The {@code LinkedQueue} class represents a first-in-first-out (FIFO)
* queue of generic items.
* It supports the usual <em>enqueue</em> and <em>dequeue</em>
* operations, along with methods for peeking at the first item,
* testing if the queue is empty, and iterating through
* the items in FIFO order.
* <p>
* This implementation uses a singly linked list with a non-static nested class
* for linked-list nodes. See {@link Queue} for a version that uses a static nested class.
* The <em>enqueue</em>, <em>dequeue</em>, <em>peek</em>, <em>size</em>, and <em>is-empty</em>
* operations all take constant time in the worst case.
* <p>
* For additional documentation, see <a href="https://algs4.cs.princeton.edu/13stacks">Section 1.3</a> of
* <i>Algorithms, 4th Edition</i> by Robert Sedgewick and Kevin Wayne.
*
* @author Robert Sedgewick
* @author Kevin Wayne
*/
public class LinkedQueue<Item> implements Iterable<Item> {
private int n; // number of elements on queue
private Node first; // beginning of queue
private Node last; // end of queue
// helper linked list class
private class Node {
private Item item;
private Node next;
}
/**
* Initializes an empty queue.
*/
public LinkedQueue() {
first = null;
last = null;
n = 0;
assert check();
}
/**
* Is this queue empty?
* @return true if this queue is empty; false otherwise
*/
public boolean isEmpty() {
return first == null;
}
/**
* Returns the number of items in this queue.
* @return the number of items in this queue
*/
public int size() {
return n;
}
/**
* Returns the item least recently added to this queue.
* @return the item least recently added to this queue
* @throws java.util.NoSuchElementException if this queue is empty
*/
public Item peek() {
if (isEmpty()) throw new NoSuchElementException("Queue underflow");
return first.item;
}
/**
* Adds the item to this queue.
* @param item the item to add
*/
public void enqueue(Item item) {
Node oldlast = last;
last = new Node();
last.item = item;
last.next = null;
if (isEmpty()) first = last;
else oldlast.next = last;
n++;
assert check();
}
/**
* Removes and returns the item on this queue that was least recently added.
* @return the item on this queue that was least recently added
* @throws java.util.NoSuchElementException if this queue is empty
*/
public Item dequeue() {
if (isEmpty()) throw new NoSuchElementException("Queue underflow");
Item item = first.item;
first = first.next;
n--;
if (isEmpty()) last = null; // to avoid loitering
assert check();
return item;
}
/**
* Returns a string representation of this queue.
* @return the sequence of items in FIFO order, separated by spaces
*/
public String toString() {
StringBuilder s = new StringBuilder();
for (Item item : this)
s.append(item + " ");
return s.toString();
}
// check internal invariants
private boolean check() {
if (n < 0) {
return false;
}
else if (n == 0) {
if (first != null) return false;
if (last != null) return false;
}
else if (n == 1) {
if (first == null || last == null) return false;
if (first != last) return false;
if (first.next != null) return false;
}
else {
if (first == null || last == null) return false;
if (first == last) return false;
if (first.next == null) return false;
if (last.next != null) return false;
// check internal consistency of instance variable n
int numberOfNodes = 0;
for (Node x = first; x != null && numberOfNodes <= n; x = x.next) {
numberOfNodes++;
}
if (numberOfNodes != n) return false;
// check internal consistency of instance variable last
Node lastNode = first;
while (lastNode.next != null) {
lastNode = lastNode.next;
}
if (last != lastNode) return false;
}
return true;
}
/**
* Returns an iterator that iterates over the items in this queue in FIFO order.
* @return an iterator that iterates over the items in this queue in FIFO order
*/
public Iterator<Item> iterator() {
return new LinkedIterator();
}
// an iterator, doesn't implement remove() since it's optional
private class LinkedIterator implements Iterator<Item> {
private Node current = first;
public boolean hasNext() { return current != null; }
public void remove() { throw new UnsupportedOperationException(); }
public Item next() {
if (!hasNext()) throw new NoSuchElementException();
Item item = current.item;
current = current.next;
return item;
}
}
/**
* Unit tests the {@code LinkedQueue} data type.
*
* @param args the command-line arguments
*/
public static void main(String[] args) {
LinkedQueue<String> queue = new LinkedQueue<String>();
while (!StdIn.isEmpty()) {
String item = StdIn.readString();
if (!item.equals("-"))
queue.enqueue(item);
else if (!queue.isEmpty())
StdOut.print(queue.dequeue() + " ");
}
StdOut.println("(" + queue.size() + " left on queue)");
}
}
/******************************************************************************
* Copyright 2002-2020, Robert Sedgewick and Kevin Wayne.
*
* This file is part of algs4.jar, which accompanies the textbook
*
* Algorithms, 4th edition by Robert Sedgewick and Kevin Wayne,
* Addison-Wesley Professional, 2011, ISBN 0-321-57351-X.
* http://algs4.cs.princeton.edu
*
*
* algs4.jar is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* algs4.jar is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with algs4.jar. If not, see http://www.gnu.org/licenses.
******************************************************************************/
ResizingArrayQueue在《算法 4》中的实现：
/******************************************************************************
* Compilation: javac ResizingArrayQueue.java
* Execution: java ResizingArrayQueue < input.txt
* Dependencies: StdIn.java StdOut.java
* Data files: https://algs4.cs.princeton.edu/13stacks/tobe.txt
*
* Queue implementation with a resizing array.
*
* % java ResizingArrayQueue < tobe.txt
* to be or not to be (2 left on queue)
*
******************************************************************************/
package edu.princeton.cs.algs4;
import java.util.Iterator;
import java.util.NoSuchElementException;
/**
* The {@code ResizingArrayQueue} class represents a first-in-first-out (FIFO)
* queue of generic items.
* It supports the usual <em>enqueue</em> and <em>dequeue</em>
* operations, along with methods for peeking at the first item,
* testing if the queue is empty, and iterating through
* the items in FIFO order.
* <p>
* This implementation uses a resizing array, which double the underlying array
* when it is full and halves the underlying array when it is one-quarter full.
* The <em>enqueue</em> and <em>dequeue</em> operations take constant amortized time.
* The <em>size</em>, <em>peek</em>, and <em>is-empty</em> operations takes
* constant time in the worst case.
* <p>
* For additional documentation, see <a href="https://algs4.cs.princeton.edu/13stacks">Section 1.3</a> of
* <i>Algorithms, 4th Edition</i> by Robert Sedgewick and Kevin Wayne.
*
* @author Robert Sedgewick
* @author Kevin Wayne
*/
public class ResizingArrayQueue<Item> implements Iterable<Item> {
private Item[] q; // queue elements
private int n; // number of elements on queue
private int first; // index of first element of queue
private int last; // index of next available slot
/**
* Initializes an empty queue.
*/
public ResizingArrayQueue() {
q = (Item[]) new Object[2];
n = 0;
first = 0;
last = 0;
}
/**
* Is this queue empty?
* @return true if this queue is empty; false otherwise
*/
public boolean isEmpty() {
return n == 0;
}
/**
* Returns the number of items in this queue.
* @return the number of items in this queue
*/
public int size() {
return n;
}
// resize the underlying array
private void resize(int capacity) {
assert capacity >= n;
Item[] copy = (Item[]) new Object[capacity];
for (int i = 0; i < n; i++) {
copy[i] = q[(first + i) % q.length];
}
q = copy;
first = 0;
last = n;
}
/**
* Adds the item to this queue.
* @param item the item to add
*/
public void enqueue(Item item) {
// double size of array if necessary and recopy to front of array
if (n == q.length) resize(2*q.length); // double size of array if necessary
q[last++] = item; // add item
if (last == q.length) last = 0; // wrap-around
n++;
}
/**
* Removes and returns the item on this queue that was least recently added.
* @return the item on this queue that was least recently added
* @throws java.util.NoSuchElementException if this queue is empty
*/
public Item dequeue() {
if (isEmpty()) throw new NoSuchElementException("Queue underflow");
Item item = q[first];
q[first] = null; // to avoid loitering
n--;
first++;
if (first == q.length) first = 0; // wrap-around
// shrink size of array if necessary
if (n > 0 && n == q.length/4) resize(q.length/2);
return item;
}
/**
* Returns the item least recently added to this queue.
* @return the item least recently added to this queue
* @throws java.util.NoSuchElementException if this queue is empty
*/
public Item peek() {
if (isEmpty()) throw new NoSuchElementException("Queue underflow");
return q[first];
}
/**
* Returns an iterator that iterates over the items in this queue in FIFO order.
* @return an iterator that iterates over the items in this queue in FIFO order
*/
public Iterator<Item> iterator() {
return new ArrayIterator();
}
// an iterator, doesn't implement remove() since it's optional
private class ArrayIterator implements Iterator<Item> {
private int i = 0;
public boolean hasNext() { return i < n; }
public void remove() { throw new UnsupportedOperationException(); }
public Item next() {
if (!hasNext()) throw new NoSuchElementException();
Item item = q[(i + first) % q.length];
i++;
return item;
}
}
/**
* Unit tests the {@code ResizingArrayQueue} data type.
*
* @param args the command-line arguments
*/
public static void main(String[] args) {
ResizingArrayQueue<String> queue = new ResizingArrayQueue<String>();
while (!StdIn.isEmpty()) {
String item = StdIn.readString();
if (!item.equals("-")) queue.enqueue(item);
else if (!queue.isEmpty()) StdOut.print(queue.dequeue() + " ");
}
StdOut.println("(" + queue.size() + " left on queue)");
}
}
/******************************************************************************
* Copyright 2002-2020, Robert Sedgewick and Kevin Wayne.
*
* This file is part of algs4.jar, which accompanies the textbook
*
* Algorithms, 4th edition by Robert Sedgewick and Kevin Wayne,
* Addison-Wesley Professional, 2011, ISBN 0-321-57351-X.
* http://algs4.cs.princeton.edu
*
*
* algs4.jar is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* algs4.jar is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with algs4.jar. If not, see http://www.gnu.org/licenses.
******************************************************************************/
这里总结下算法4中三种queue实现的区别：
- 都支持了FIFO的队列，Queue和LinkedQueue是支持的泛类的items
- LinkedQueue中增加了check方法，有许多对错误的判断，值得借鉴
- LinkedList:a singly linked list with a non-static nested class for linked-list nodes. 用Linked List实现，实现非静态的内部类节点Node
Queue:This implementation uses a singly linked list with a static nested class for linked-list nodes. 用Linked List实现， 实现静态的内部类节点Node
ResizingArrayQueue:This implementation uses a resizing array 用Array实现
