Selective Search 选择性搜索算法

参考：https://www.learnopencv.com/selective-search-for-object-detection-cpp-python/

图像产生候选子区域的方法

The most straightforward way to generate smaller sub-regions (patches) is called the Sliding Window approach. Sliding window approach is good for fixed aspect ratio objects such as faces or pedestrians.
However, the sliding window approach has several limitations. These limitations are overcome by a class of algorithms called the “Region Proposal” algorithms. Selective Search is one of the most popular Region Proposal algorithms.

• 滑窗法
• 候选区域法，代表：Selective Search，fast, high Recall 保证选出的框都是可靠的，减少框分类耗时

候选区域法产生的 box，分为 FP / TP；TP 越多 Recall 越高

选择性搜索算法

参考：物体检测之选择性搜索(Selective Search)

1. 使用 Efficient Graph-Based Image Segmentation, MIT, 2004 论文里的方法产生初始的分割区域
2. 使用相似度计算方法合并一些小的区域

Efficient Graph-Based Image Segmentation 分割结果

不能使用原始分割图的区域作为候选区域，因为：

• 大部分物体在原始分割图里都被分为多个区域
• 原始分割图无法体现物体之间的 遮挡和包含，所以说 难度 instance seg > scene seg

我们不要需要完美的的分割区域，我们只想要 和实际物体高度重合 的区域就行了。

上图中，最下面的图属于 Oversegmented Image

---

问题：选择性搜索算法如何计算两个区域的相似度的呢？

主要是通过以下四个方面：

• Color 颜色
• Texture 纹理
• Size 大小
• Shape Compatibility 形状兼容性

最终的相似度是这四个值取不同的权重相加。

• 颜色：RGB 形成 75-D 直方图

• 纹理：8 邻域图像求导，导数范围 [-255, 255] 划分 10 个区间
  总共 240-D = 3 channels × 8 directions × 10 bins

• 大小
  越小的 region， 值越大，越优先合并

• 形状兼容性：包含两个 region 的边界框 相对两个 region 的补集（巧妙😀）
  补集越小，两个区域越能完好契合，但是对于边界犬牙差互的有点不行

• 两个 region 最终相似度

实验

结果显示 产生的 box 效果一般

top 100 top 200

selective_search.py

"""
https://www.learnopencv.com/selective-search-for-object-detection-cpp-python/
- f|q: f=fast, q=quality
- l|m: less box, more box
结果可看 产生的 box 效果一般
"""
import cv2

im = cv2.imread('../breakfast.jpg')

# resize image
newHeight = 400
newWidth = int(im.shape[1] * newHeight / im.shape[0])
im = cv2.resize(im, (newWidth, newHeight))

# create Selective Search Segmentation Object using default parameters
ss = cv2.ximgproc.segmentation.createSelectiveSearchSegmentation()

# set input image on which we will run segmentation
ss.setBaseImage(im)

method = 'f'  # f=fast, q=quality

if method == 'f':  # fast but low recall
    ss.switchToSelectiveSearchFast()
elif method == 'q':  # high recall but slow
    ss.switchToSelectiveSearchQuality()
else:
    exit(1)

# run selective search segmentation on input image
rects = ss.process()  # f:453, q:1354
print('Total Number of Region Proposals: {}'.format(len(rects)))

# number of region proposals to show
numShowRects = 100
# increment to increase/decrease total number of reason proposals to be shown
increment = 50

while True:
    # create a copy of original image
    imOut = im.copy()

    # itereate over all the region proposals
    for i, rect in enumerate(rects):
        # draw rectangle for region proposal till numShowRects
        if i < numShowRects:
            x, y, w, h = rect  # 这种格式
            cv2.rectangle(imOut, (x, y), (x + w, y + h), (0, 0, 255), 1, cv2.LINE_AA)
        else:
            break

    # show output
    cv2.imshow("Output", imOut)

    # record key press
    k = cv2.waitKey(0) & 0xFF

    # more
    if k == ord('m'):
        numShowRects += increment  # increase total number of rectangles to show by increment
    # less
    elif k == ord('l') and numShowRects > increment:
        numShowRects -= increment  # decrease total number of rectangles to show by increment
    # quit
    elif k == ord('q'):
        break

cv2.destroyAllWindows()