cinder配额 quota 使用及代码分析
2017-07-17 本文已影响144人
笨手笨脚越
封装抽象
quota相关功能实现在cinder\quota.py实现，包含了引擎、资源、驱动三个大类抽象封装。看起来quota.py用的是设计模式“抽象工厂模式”，引擎选择驱动、定义资源，调用驱动管理资源。
一、资源
资源其实就是对配额的封装，封装了资源名、默认值、数据库统计函数dbapi等。
资源分类列表：
| 类名 | 描述 | 属性 |
|---|---|---|
| cinder.quota.BaseResource | 定义单个配额资源 | name（资源名）、flag（控制默认值）、parent_project_id（当前租户的附租户）、quota方法用于获取资源的使用量、default方法用户获取默认值 |
| cinder.quota.AbsoluteResource | 无预留的资源 | pass无定义 |
| cinder.quota.ReservableResource | 可预留的资源 | sync （dbapi 方法名，统计配额使用信息。如范例） |
| cinder.quota.CountableResource | 可统计的资源，cinder代码里没看到使用 | count（统计函数） |
| cinder.quota.VolumeTypeResource | 为卷类型定义的资源，继承ReservableResource | volume_type_name（卷类型名）、volume_type_id（卷类型id） |
注意：
-
ReservableResource：相比BaseResource，多了sync方法，sync会被驱动调用，用于在计算配额之前，先同步配额信息（到本地和数据库）。ReservableResource只能用于project绑定的资源。
-
CountableResource：相比BaseResource，多了count方法，count方法必须给出一个函数，自己计算配额，其返回值里会包含配额实际使用值。
Resource类关系图.png
范例：ReservableResource资源'volume'的sync：cinder.db.sqlalchemy.api._sync_volumes
def _sync_volumes(context, project_id, session, volume_type_id=None,
volume_type_name=None):
# 根据volume_type_id和project_id统计卷数量和卷空间使用量
(volumes, _gigs) = _volume_data_get_for_project(
context, project_id, volume_type_id=volume_type_id, session=session)
key = 'volumes'
if volume_type_name:
key += '_' + volume_type_name
return {key: volumes}
二、引擎
定义了资源集。调用驱动来实现查询统计功能。
引擎列表：
| 类名 | 描述 | resources |
|---|---|---|
| cinder.quota.QuotaEngine | 配额引擎，基类 | |
| cinder.quota.VolumeTypeQuotaEngine | 卷类型配额引擎 | 'volumes', 'per_volume_gigabytes','snapshots','gigabytes','backups','backup_gigabytes' |
| cinder.quota.CGQuotaEngine | 一致性组的配额的引擎 | consistencygroups |
| cinder.quota.GroupQuotaEngine | 组配额的引擎 | groups |
三、驱动
驱动是可配置的，对应配置项quota_driver，默认值是cinder.quota.DbQuotaDriver
cfg.StrOpt('quota_driver',
default="cinder.quota.DbQuotaDriver",
help='Default driver to use for quota checks')
驱动列表：
| 名称 | 描述 |
|---|---|
| cinder.quota.DbQuotaDriver | Number of volume gigabytes allowed per tenant |
| cinder.quota.NestedDbQuotaDriver | Number of Block Storage snapshots allowed per tenant. |
quota主要操作四张数据表：
- reservations表，定义每个项目配额的增量。
图片.png
- quota_usage表，定义每个项目配额的已使用量和预留量。
quota_usage表.png
- quota_classes表，定义了配额类的配额。操作界面上的默认配额就是保存在这个表里。
quota_classes表.png
- quotas表，定义了项目的配额。是如果仅仅是调用API接口或者client指令
openstack project create pro3创建项目，是不会同时创建项目对应的专用配额的。但是如果在管理界面上创建项目，horizon会同时调用cinder的quota接口创建三个“gigabytes”、“volumes”、“snapshots” cinder专用配额，另外还会调neutron、nova的配额接口创建它们专用的配额。
quotas表.png
创建project操作界面.png
指令功能介绍
quotas相关的指令：
quota-class-show Lists quotas for a quota class.
quota-class-update Updates quotas for a quota class.
quota-defaults Lists default quotas for a tenant.
quota-delete Delete the quotas for a tenant.
quota-show Lists quotas for a tenant.
quota-update Updates quotas for a tenant.
quota-usage Lists quota usage for a tenant.
quota-class-show、quota-class-update、quota-defaults是对quota_classes表操作，quota-delete、quota-show、quota-update 主要对quota表操作，quota-usage是对quota_classes、quota_usage操作。
作为管理员用户，我们可以修改块存储租户的配额，也可以对新租户定义默认配额。
1. 列出默认List all default quotas for all tenants, as follows:
$ cinder quota-defaults tenantID
[root@node1 ~]# cinder quota-defaults admin
+------------------------------+-------+
| Property | Value |
+------------------------------+-------+
| backup_gigabytes | 1000 |
| backups | 10 |
| gigabytes | 1000 |
| gigabytes_netapp_volume_type | -1 |
| gigabytes_nfs_common | -1 |
| gigabytes_vmware | -1 |
| gigabytes_vmware-type | -1 |
| per_volume_gigabytes | -1 |
| snapshots | 10 |
| snapshots_netapp_volume_type | -1 |
| snapshots_nfs_common | -1 |
| snapshots_vmware | -1 |
| snapshots_vmware-type | -1 |
| volumes | 10 |
| volumes_netapp_volume_type | -1 |
| volumes_nfs_common | -1 |
| volumes_vmware | -1 |
| volumes_vmware-type | -1 |
+------------------------------+-------+
quota-defaults 代码分析：
cinder.api.contrib.quotas.QuotaSetsController#defaults:
return self._format_quota_set(id, QUOTAS.get_defaults(context, project_id=id))
# QUOTAS 是类对象 cinder.quota.VolumeTypeQuotaEngine()
cinder.quota.QuotaEngine#get_defaults:
def get_defaults(self, context, project_id=None):
return self._driver.get_defaults(context, self.resources,
project_id)
_driver 是 cinder.quota.QuotaEngine#_driver，代码可得cinder.quota.DbQuotaDriver
@property
def _driver(self):
# _driver_class是__init__构造函数里传入设置的，没传为None
if self._driver_class:
return self._driver_class
if not self._quota_driver_class:
# 读配置项cfg.StrOpt('quota_driver',default="cinder.quota.DbQuotaDriver",
self._quota_driver_class = CONF.quota_driver
if isinstance(self._quota_driver_class, six.string_types):
# 动态导入类对象
self._quota_driver_class = importutils.import_object(
self._quota_driver_class)
self._driver_class = self._quota_driver_class
return self._driver_class
cinder.quota.DbQuotaDriver#get_defaults:
def get_defaults(self, context, resources, project_id=None):
quotas = {}
default_quotas = {}
# cfg.BoolOpt('use_default_quota_class',default=True,
if CONF.use_default_quota_class:
# 查询'quota_classes'表，过滤出class_name = 'defualt'的记录，
default_quotas = db.quota_class_get_defaults(context)
# resources 来自cinder.quota.VolumeTypeQuotaEngine的@property 方法
for resource in resources.values():
if default_quotas:
if resource.name not in default_quotas:
versionutils.report_deprecated_feature(LOG, _(
"Default quota for resource: %(res)s is set "
"by the default quota flag: quota_%(res)s, "
"it is now deprecated. Please use the "
"default quota class for default "
"quota.") % {'res': resource.name})
# default_quotas的值 复写 resources ，如果default_quotas里不包含resource，则使用resource的default属性。default属性说明见下文！
quotas[resource.name] = default_quotas.get(resource.name,
resource.default)
return quotas
代码可知cinder.quota.DbQuotaDriver的get_defaults方法不区分租户project_id，所以如果配置项'quota_driver'使用"cinder.quota.DbQuotaDriver"，调用 cinder quota-defaults ${project_id} 的结果都是一样的。
cinder.quota.VolumeTypeQuotaEngine#resources:
@property
def resources(self):
"""Fetches all possible quota resources."""
result = {}
# Global quotas.
argses = [('volumes', '_sync_volumes', 'quota_volumes'),
('per_volume_gigabytes', None, 'per_volume_size_limit'),
('snapshots', '_sync_snapshots', 'quota_snapshots'),
('gigabytes', '_sync_gigabytes', 'quota_gigabytes'),
('backups', '_sync_backups', 'quota_backups'),
('backup_gigabytes', '_sync_backup_gigabytes',
'quota_backup_gigabytes')]
# 根据上面定义的argses获得ReservableResource列表，
for args in argses:
resource = ReservableResource(*args)
result[resource.name] = resource
# 查询得volume_type列表
volume_types = db.volume_type_get_all(context.get_admin_context(),
False)
for volume_type in volume_types.values():
for part_name in ('volumes', 'gigabytes', 'snapshots'):
# 对每一个volume_type，按照规则 name = "%s_%s" % (part_name, self.volume_type_name)设置resource.name
resource = VolumeTypeResource(part_name, volume_type)
result[resource.name] = resource
# 返回ReservableResource和VolumeTypeResource组合的resoure列表
return result
#### (Pdb) p result
{'per_volume_gigabytes': <cinder.quota.ReservableResource object at 0x98e7090>, 'gigabytes': <cinder.quota.ReservableResource object at 0x98e70d0>, 'backup_gigabytes': <cinder.quota.ReservableResource object at 0x98e7150>, 'snapshots': <cinder.quota.ReservableResource object at 0x98e7050>, 'volumes': <cinder.quota.ReservableResource object at 0x9b8ffd0>, 'backups': <cinder.quota.ReservableResource object at 0x98e7110>}
看下ReservableResource和VolumeTypeResource 类的继承关系：
ReservableResource和VolumeTypeResource 对象的default属性，都继承自cinder.quota.BaseResource#default：
@property
def default(self):
"""Return the default value of the quota."""
if self.parent_project_id:
return 0
# 如果self.flag不是空，则返回CONF[self.flag]，否则返回-1
return CONF[self.flag] if self.flag else -1
总结：
- 如果配置项'quota_driver'使用"cinder.quota.DbQuotaDriver"或者不配置，cinder quota-defaults 指令打印结果和租户id无关。
- cinder quota-defaults 指令打印列表，包含了'volumes', 'per_volume_gigabytes','snapshots','gigabytes','backups','backup_gigabytes' 六个元素，另外还包含'volumes_${volume_type}', 'gigabytes_${volume_type}', 'snapshots_${volume_type}' 多个元素，元素的值由表 quota_classes 里 class_name='default' 的记录设置，如果表里没有，则由cinder.quota.BaseResource#default得到。
2. 根据quota-class列出quota列表:
cinder quota-class-show <class>
quota-defaults 代码分析：
如果配置项'quota_driver'使用"cinder.quota.DbQuotaDriver"或者不配置，走的是cinder.quota.DbQuotaDriver#get_class_quotas：
def get_class_quotas(self, context, resources, quota_class,
defaults=True):
"""Given list of resources, retrieve the quotas for given quota class.
:param context: The request context, for access checks.
:param resources: A dictionary of the registered resources.
:param quota_class: The name of the quota class to return
quotas for.
:param defaults: If True, the default value will be reported
if there is no specific value for the
resource.
"""
quotas = {}
default_quotas = {}
# 根据class_name在数据库表quota_classes查询出归属某类的配额属性
class_quotas = db.quota_class_get_all_by_name(context, quota_class)
if defaults:
# 数据库表quota_classes查出默认类的配额属性
default_quotas = db.quota_class_get_defaults(context)
for resource in resources.values():
# 检查cinder.quota.VolumeTypeQuotaEngine#resources的元素是否有跟class_quotas交集，
# 如果有，则用class_quotas的元素值覆盖
if resource.name in class_quotas:
quotas[resource.name] = class_quotas[resource.name]
continue
# 检查cinder.quota.VolumeTypeQuotaEngine#resources的元素是否有跟default_quotas交集，
# 如果有，则用default_quotas的元素值覆盖
if defaults:
quotas[resource.name] = default_quotas.get(resource.name,
resource.default)
return quotas
总结：
- 根据class_name在数据库表quota_classes表查出记录集class_quotas，再从class_name='default'在数据库表quota_classes表查出记录集default_quotas。
- 用class_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值。
- 用default_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值。
3. 根据quota-class列出quota列表:
cinder quota-show <tenant_id>
cinder.quota.DbQuotaDriver#get_project_quotas:
def get_project_quotas(self, context, resources, project_id,
quota_class=None, defaults=True,
usages=True):
"""Retrieve quotas for a project.
Given a list of resources, retrieve the quotas for the given
project.
:param context: The request context, for access checks.
:param resources: A dictionary of the registered resources.
:param project_id: The ID of the project to return quotas for.
:param quota_class: If project_id != context.project_id, the
quota class cannot be determined. This
parameter allows it to be specified. It
will be ignored if project_id ==
context.project_id.
:param defaults: If True, the quota class value (or the
default value, if there is no value from the
quota class) will be reported if there is no
specific value for the resource.
:param usages: If True, the current in_use, reserved and allocated
counts will also be returned.
"""
quotas = {}
# 在数据库表 quotas 根据 project_id 查询字段 hard_limit
project_quotas = db.quota_get_all_by_project(context, project_id)
allocated_quotas = None
default_quotas = None
# API调用的时候传入的usages=False
if usages:
# 根据 project_id 查询数据库表 quota_usages ，获得各quota resource的 in_use和 reserved
project_usages = db.quota_usage_get_all_by_project(context,
project_id)
# 在数据库表 quotas 根据 project_id 查询字段 allocated
allocated_quotas = db.quota_allocated_get_all_by_project(
context, project_id)
allocated_quotas.pop('project_id')
# Get the quotas for the appropriate class. If the project ID
# matches the one in the context, we use the quota_class from
# the context, otherwise, we use the provided quota_class (if
# any)
# 如果context有quota_class，用context.quota_class查询class_quotas
if project_id == context.project_id:
quota_class = context.quota_class
if quota_class:
class_quotas = db.quota_class_get_all_by_name(context, quota_class)
else:
class_quotas = {}
for resource in resources.values():
# Omit default/quota class values
# defaults = True
if not defaults and resource.name not in project_quotas:
continue
quota_val = project_quotas.get(resource.name)
# 如果 project_quota 是空，用 class_quota 的值赋值
if quota_val is None:
quota_val = class_quotas.get(resource.name)
# 如果 class_quota 赋的值也是空，用 default_quotas 的值赋值
if quota_val is None:
# Lazy load the default quotas
if default_quotas is None:
default_quotas = self.get_defaults(
context, resources, project_id)
quota_val = default_quotas[resource.name]
quotas[resource.name] = {'limit': quota_val}
# Include usages if desired. This is optional because one
# internal consumer of this interface wants to access the
# usages directly from inside a transaction.
if usages: # False
usage = project_usages.get(resource.name, {})
quotas[resource.name].update(
in_use=usage.get('in_use', 0),
reserved=usage.get('reserved', 0), )
if allocated_quotas: # None
quotas[resource.name].update(
allocated=allocated_quotas.get(resource.name, 0), )
return quotas
总结：
- 用project_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值
- 如果project_quotas的元素值为空，用class_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值。
- 如果class_quotas的元素值为空，用default_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值。
3. quota-usage 根据租户列出配额的使用量
cinder quota-usage <tenant_id>
核心代码跟 cinder quota-show 一样，走的是cinder.quota.DbQuotaDriver#get_project_quotas
cinder.quota.DbQuotaDriver#get_project_quotas:
def get_project_quotas(self, context, resources, project_id,
quota_class=None, defaults=True,
usages=True):
quotas = {}
# 在数据库表 quotas 根据 project_id 查询字段 hard_limit
project_quotas = db.quota_get_all_by_project(context, project_id)
allocated_quotas = None
default_quotas = None
# API调用的时候传入的usages=True
if usages:
# 根据 project_id 查询数据库表 quota_usages ，获得各quota resource的 in_use和 reserved
project_usages = db.quota_usage_get_all_by_project(context,
project_id)
# 在数据库表 quotas 根据 project_id 查询字段 allocated
allocated_quotas = db.quota_allocated_get_all_by_project(
context, project_id)
allocated_quotas.pop('project_id')
# Get the quotas for the appropriate class. If the project ID
# matches the one in the context, we use the quota_class from
# the context, otherwise, we use the provided quota_class (if
# any)
# 如果context有quota_class，用context.quota_class查询class_quotas
if project_id == context.project_id:
quota_class = context.quota_class
if quota_class:
class_quotas = db.quota_class_get_all_by_name(context, quota_class)
else:
class_quotas = {}
for resource in resources.values():
# Omit default/quota class values
# defaults = True
if not defaults and resource.name not in project_quotas:
continue
quota_val = project_quotas.get(resource.name)
# 如果 project_quota 是空，用 class_quota 的值赋值
if quota_val is None:
quota_val = class_quotas.get(resource.name)
# 如果 class_quota 赋的值也是空，用 default_quotas 的值赋值
if quota_val is None:
# Lazy load the default quotas
if default_quotas is None:
default_quotas = self.get_defaults(
context, resources, project_id)
quota_val = default_quotas[resource.name]
quotas[resource.name] = {'limit': quota_val}
# Include usages if desired. This is optional because one
# internal consumer of this interface wants to access the
# usages directly from inside a transaction.
if usages: # True
# 设置usage
usage = project_usages.get(resource.name, {})
quotas[resource.name].update(
in_use=usage.get('in_use', 0),
reserved=usage.get('reserved', 0), )
if allocated_quotas: # None
quotas[resource.name].update(
allocated=allocated_quotas.get(resource.name, 0), )
return quotas
