MySQL-Innodb-批量刷脏的场景

2020-10-05  本文已影响0人  多血

buf_flush_page_cleaner_coordinator协调线程的主循环主线程以最多1s的间隔或者收到buf_flush_event事件就会触发进行一轮的刷脏。
批量刷脏主要有3个场景。

  1. 同步刷脏
    如果 buf_flush_sync_lsn > 0, 则因为redo log free space 不够了, 那么我们需要进入同步刷脏阶段了。同步刷脏场景下,所有需要写脏数据库的用户线程都会堵塞,这是很严重的情况。
  2. 正常刷脏
    最常见逻辑 srv_check_activity(last_activity), 也就是系统有正常活动,有DML/DDL, 这个时候会通过 page_cleaner_flush_pages_recommendation() 函数去合理的判断应该刷多少个page, 既不抖动, 也能够满足刷脏需求
  3. 空闲刷脏
    如果系统没有DML\DDL活动,且ret_sleep == OS_SYNC_TIME_EXCEEDED,说明比较空闲。空闲的情况下因为服务器IO比较空闲,所以Innodb使用buf_flush_page_cleaner_coordinator线程本身进行刷新,刷新的块数计算比较简单就是innodb_io_capacity设置的值。
/******************************************************************//**
page_cleaner thread tasked with flushing dirty pages from the buffer
pools. As of now we'll have only one coordinator.
@return a dummy parameter */
extern "C"
os_thread_ret_t
DECLARE_THREAD(buf_flush_page_cleaner_coordinator)(
/*===============================================*/
    void*   arg MY_ATTRIBUTE((unused)))
            /*!< in: a dummy parameter required by
            os_thread_create */
{
    /* 忽略一些逻辑 */
    while (srv_shutdown_state == SRV_SHUTDOWN_NONE) {
        if (ret_sleep != OS_SYNC_TIME_EXCEEDED
            && srv_flush_sync
            && buf_flush_sync_lsn > 0) {
            /* 场景1 */
        } else if (srv_check_activity(last_activity)) {
            ulint   n_to_flush;
            lsn_t   lsn_limit = 0;

            /* Estimate pages from flush_list to be flushed */
            if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
                last_activity = srv_get_activity_count();
                n_to_flush =
                    page_cleaner_flush_pages_recommendation(
                        &lsn_limit, last_pages);
            } else {
                n_to_flush = 0;
            }
            /* 场景2 */
        } else if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
            /* 场景3 */
            /* no activity, slept enough */
            }
        } else {
            /* no activity, but woken up by event */
            n_flushed = 0;
        }
    }
」

三种场景下的具体工作

同步刷脏
pc_request(ULINT_MAX, lsn_limit),会把lsn小于lsn_limit的都flush到硬盘,同时coordinator线程本身也会参与刷脏。

            /* woke up for flush_sync */
            mutex_enter(&page_cleaner->mutex);
            lsn_t   lsn_limit = buf_flush_sync_lsn;
            buf_flush_sync_lsn = 0;
            mutex_exit(&page_cleaner->mutex);

            /* Request flushing for threads */
            pc_request(ULINT_MAX, lsn_limit);

            ib_time_monotonic_ms_t tm = ut_time_monotonic_ms();

            /* Coordinator also treats requests */
            while (pc_flush_slot() > 0) {}

            /* only coordinator is using these counters,
            so no need to protect by lock. */
            page_cleaner->flush_time += ut_time_monotonic_ms() - tm;
            page_cleaner->flush_pass++;

            /* Wait for all slots to be finished */
            ulint   n_flushed_lru = 0;
            ulint   n_flushed_list = 0;
            pc_wait_finished(&n_flushed_lru, &n_flushed_list);

            if (n_flushed_list > 0 || n_flushed_lru > 0) {
                buf_flush_stats(n_flushed_list, n_flushed_lru);

                MONITOR_INC_VALUE_CUMULATIVE(
                    MONITOR_FLUSH_SYNC_TOTAL_PAGE,
                    MONITOR_FLUSH_SYNC_COUNT,
                    MONITOR_FLUSH_SYNC_PAGES,
                    n_flushed_lru + n_flushed_list);
            }

            n_flushed = n_flushed_lru + n_flushed_list;

正常刷脏
通过page_cleaner_flush_pages_recommendation计算需要刷新的页。

            ulint   n_to_flush;
            lsn_t   lsn_limit = 0;

            /* Estimate pages from flush_list to be flushed */
            if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
                last_activity = srv_get_activity_count();
                n_to_flush =
                    page_cleaner_flush_pages_recommendation(
                        &lsn_limit, last_pages);
            } else {
                n_to_flush = 0;
            }

            /* Request flushing for threads */
            pc_request(n_to_flush, lsn_limit);

            ib_time_monotonic_ms_t tm = ut_time_monotonic_ms();

            /* Coordinator also treats requests */
            while (pc_flush_slot() > 0) {
                /* No op */
            }

            /* only coordinator is using these counters,
            so no need to protect by lock. */
            page_cleaner->flush_time += ut_time_monotonic_ms() - tm;
            page_cleaner->flush_pass++ ;

            /* Wait for all slots to be finished */
            ulint   n_flushed_lru = 0;
            ulint   n_flushed_list = 0;

            pc_wait_finished(&n_flushed_lru, &n_flushed_list);

            if (n_flushed_list > 0 || n_flushed_lru > 0) {
                buf_flush_stats(n_flushed_list, n_flushed_lru);
            }

            if (ret_sleep == OS_SYNC_TIME_EXCEEDED) {
                last_pages = n_flushed_list;
            }

            n_evicted += n_flushed_lru;
            n_flushed_last += n_flushed_list;

            n_flushed = n_flushed_lru + n_flushed_list;

            if (n_flushed_lru) {
                MONITOR_INC_VALUE_CUMULATIVE(
                    MONITOR_LRU_BATCH_FLUSH_TOTAL_PAGE,
                    MONITOR_LRU_BATCH_FLUSH_COUNT,
                    MONITOR_LRU_BATCH_FLUSH_PAGES,
                    n_flushed_lru);
            }

            if (n_flushed_list) {
                MONITOR_INC_VALUE_CUMULATIVE(
                    MONITOR_FLUSH_ADAPTIVE_TOTAL_PAGE,
                    MONITOR_FLUSH_ADAPTIVE_COUNT,
                    MONITOR_FLUSH_ADAPTIVE_PAGES,
                    n_flushed_list);
            }

空闲刷脏
空闲刷脏是coordinator自己进行,直接按照PCT_IO(100)来生成刷新数量。
#define PCT_IO(p) ((ulong) (srv_io_capacity * ((double) (p) / 100.0)))

            buf_flush_lists(PCT_IO(100), LSN_MAX, &n_flushed);

            n_flushed_last += n_flushed;

            if (n_flushed) {
                MONITOR_INC_VALUE_CUMULATIVE(
                    MONITOR_FLUSH_BACKGROUND_TOTAL_PAGE,
                    MONITOR_FLUSH_BACKGROUND_COUNT,
                    MONITOR_FLUSH_BACKGROUND_PAGES,
                    n_flushed);

            }

http://mysql.taobao.org/monthly/2018/09/02/
https://www.jianshu.com/p/6991304a8e26
https://mp.weixin.qq.com/s/o2OlvRiybIsqi7WU_Kvhiw

上一篇下一篇

猜你喜欢

热点阅读