接口自动化测试---通用JSON解析算法
2017-08-28 本文已影响83人
Qi仔
JSON格式的数据在http接口的自动化测试中是一种常见的数据结构，在接口自动化脚本里解析JSON返回结果,验证数据的正确性是非常关键的一步，但JSON结构有简单有复杂，不同的接口返回不同的结果，自动化脚本中解析响应JSON数据占了一大部分的工作量，而且随着接口的变动，维护脚本也特别麻烦，今天在这里介绍一种通用的解析所有JSON 的通用算法，来解决这一问题。
算法解析：
首先来看一段较为复杂的JSON:
图-1
从图-1可以看出，这段JSON第一层包含了resultCode，resultDesc，data，dataList四个key值，其中resultCode，resultDesc，data 都是单个的字符串，而dataList 的值是一个列表，有俩元素，其中每个元素又是一个字典，包括id,name,caption,subjectId,notes,items,其中items中又是一个list,每个元素为一个字典，key值为 id,name,caption等....不难发现这段JSON中第二层和第三层中都包含相同的key: id，name, capiton,这也是选择这段JSON的原因。大多数的JSON中可能都会有相同的key,解析的时候如何精确的获取想要的key对应的值这也是需要考虑的场景之一。
其实JSON的整个结构就是 “树”，可以将上述JSON中的 第一个节点[JSON]看成树的根节点， resultCode，resultDesc 等节点看成叶子节点，而dataList 则可以看成父节点，dataList里的元素可以看成是子节点。
来一张更容易理解的图：A 为根节点，黄色的节点称是父节点，绿色节点则是叶子节点，图中的D 类比为图-1中的dataList节点，B,C为图一种的resultCode，resultDesc
图-2
解析JSON主要分为以下几步
第一步：
- 将JSON 转化为树，从根节点（A）开始遍历每一路径，即从根节点到叶子节点每一条路径作为一条数据，并且获取每一深度（JSON中的每一层）叶子节点的{key:value}，其中父节点的值不取（通常父节点可能是一个list,或者一个字典）,当前深度的叶子节点的值全取，与下一深度的叶子节点的值加到同一字典中。
- 如图-2，B,C,D,E是深度为2的节点，第一次遍历，第一条数据为{B,C},以此递归，直至遍历所有路径，可到图-2的遍历结果为(其中B节点理解为 {'B':'B'},图-3 为图-2的JSON格式)
图-3
按第一步解析结果如下：
1. [{'B': 'B'}, {'C': 'C'}]
2. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}]
3. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]
4. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}]
5. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}]
6. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}, {'T': 'T'}]
7. [{'B': 'B'}, {'C': 'C'}, {'U': 'U'}]
分析上述结果可以看出 第3组结果包含了第1,2组值，6包含了5,4的结果，因此可以将重复的数据进行一次过滤和筛选
第二步：
过滤重复数据，步骤一的结果可以过滤为：
3. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]
6.[{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}, {'T': 'T'}]
7. [{'B': 'B'}, {'C': 'C'}, {'U': 'U'}]
在这一步骤中，如果不同深度的叶子节点名称重复，则按照深度，依次在key后面加入序号1,2...
即如果有3个B,则解析结果为：
[{'B': 'B'}, {'B1': 'B1'}, {'B2: 'B2'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]
第三步：
根据想要获取的KEY,在第二步获取到的list中取出对应的value，KEY以列表的方式给出
keys = [key1,key2,key3....]，如要获取[B,C,F,G]
则最终结果为：
[{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}]
这样就轻而易举的解析处理我们想要的结果!
代码 -python算法实现
def responseComplexJson2LD(responseJson,assertKeys):
'''
:param responseJson: 类型：JSONObject 接口返回:JSON 格式
:param assertKeys: 类型：list 需要校验的key ['resultCode','resultDesc','id','name','caption','id1','notes','name1','caption1','cubeName','functions']
说明：如果key里有重复的解析为 key1,key2...
:return: list[dict]
'''
try:
if not isinstance(responseJson,dict):
raise Exception('responseJson 类型错误，必须为JSON格式')
result_list = []
def isInclude(dict1, dict2):
'''
:function: 判断 dict1 是否包含于dict2
:param dict1:
:param dict2:
:return: True False
'''
for key1 in dict1:
if key1 in dict2:
pass
else:
return False
return True
def parse_dict(responseJson, parent):
'''
:递归解析json,将json树解析成list[dict,dict....]
:param responseJson:
:param parent:
:return:
'''
if isinstance(responseJson, dict):
dp = parent[:]
not_l_d = [responseKey for responseKey in responseJson if not isinstance(responseJson[responseKey], (dict, list))]
for i in not_l_d:
dp.append({i: responseJson[i]})
dp_1 = dp[:]
result_list.append(dp)
for responseKey in responseJson:
if isinstance(responseJson[responseKey], (dict, list)):
parse_dict(responseJson[responseKey], dp_1)
elif isinstance(responseJson, list):
for i in responseJson:
dd = parent[:]
if not isinstance(i, (dict, list)):
dd.append(i)
result_list.append(dd)
else:
parse_dict(i, dd)
else:
dx = parent[:]
dx.append(responseJson)
result_list.append(dx)
parse_dict(responseJson, parent=[])
if not result_list:
return []
'''获取给定的assertKeys'''
last_result =[]
templist ={}
if assertKeys:
if isinstance(assertKeys,list):
for iter in result_list:
for key in iter:
thiskey = list(key.keys())[0]
'''如果有重复的key,解析为key1,key2...'''
i = 1
temp = thiskey
while True:
if thiskey in list(templist.keys()):
thiskey = thiskey + str(i)
i = i + 1
else:
templist.setdefault(thiskey,key.get(temp))
break
last_result.append(templist)
templist={}
else:
raise Exception("assertKeys 类型错误，必须为list")
'''将assertKeys里的定义的key加入dict,不在assertKeys 定义的除去'''
assert_result = []
templist = {}
for meta in last_result:
keys = list(meta.keys())
if isInclude(assertKeys,keys):
for key in assertKeys:
templist.setdefault(key,meta.get(key))
assert_result.append(templist)
templist={}
else:
pass
'''对返回的last_result中的元素去重'''
result = []
for meta in assert_result:
if meta not in result:
result.append(meta)
else:
pass
return result
except Exception as err:
Log.error(err)
return []
结果验证
以图-1中的json 为例测试
测试1：
keylist = ['resultCode']
responseComplexJson2LD(test,keylist)
返回：
{'resultCode': 100}
测试2：
keylist = ['resultCode','resultDesc']
responseComplexJson2LD(test,keylist)
返回：
{'resultCode': 100, 'resultDesc': '成功'}
测试3：
keylist = ['resultCode','resultDesc','id']
responseComplexJson2LD(test,keylist)
返回：
{'resultCode': 100, 'resultDesc': '成功', 'id': 1}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2}
测试4：--重复id,第二用id1标识
keylist = ['resultCode', 'resultDesc', 'id','id1']
responseComplexJson2LD(test,keylist)
返回：
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 1}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 2}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 20}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 3}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 4}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 5}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 6}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 21}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 22}
{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 23}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 7}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 8}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 9}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 10}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 24}
{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 25}
测试5：取第二个id
keylist = ['resultCode', 'resultDesc','id1']
responseComplexJson2LD(test,keylist)
返回：
{'resultCode': 100, 'resultDesc': '成功', 'id1': 1}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 2}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 20}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 3}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 4}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 5}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 6}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 21}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 22}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 23}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 7}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 8}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 9}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 10}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 24}
{'resultCode': 100, 'resultDesc': '成功', 'id1': 25}
测试6：只取caption 字段，俩caption 时第二个写caption1
keylist = ['caption','caption1]
responseComplexJson2LD(test,keylist)
返回：
{'caption': '交易', 'caption1': '交易金额'}
{'caption': '交易', 'caption1': '交易用户数'}
{'caption': '交易', 'caption1': '人均交易金额'}
{'caption': '交易', 'caption1': '订单数'}
{'caption': '交易', 'caption1': '订单金额'}
{'caption': '交易', 'caption1': '大订单数'}
{'caption': '交易', 'caption1': '大订单金额'}
{'caption': '交易', 'caption1': '平均订单金额'}
{'caption': '交易', 'caption1': '大订单数占比'}
{'caption': '交易', 'caption1': '平均大订单金额'}
{'caption': '盈亏', 'caption1': '盈利用户数'}
{'caption': '盈亏', 'caption1': '亏损用户数'}
{'caption': '盈亏', 'caption1': '净盈利值'}
{'caption': '盈亏', 'caption1': '净亏损值'}
{'caption': '盈亏', 'caption1': '盈利用户数占比'}
{'caption': '盈亏', 'caption1': '亏损用户数占比'}
附上图-1完整的Json报文：
data = {
"resultCode": 100,
"resultDesc": "成功",
"data": "",
"dataList": [
{
"id": 1,
"subjectId": "",
"name": "trading",
"caption": "交易",
"notes": "1.人均交易金额=用户累积交易金额/交易用户数<br>2.平均订单金额=用户累计订单金额/订单数<br>3.大订单：单笔手续费大于50元的订单（手续费＝每笔订单金额＊万分之八）<br>4.大订单数占比=大订单数/订单数<br>5.平均大订单金额=累计大订单金额/大订单数<br>",
"items": [
{
"id": 1,
"name": "pay_money",
"caption": "交易金额",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY",
"itemType": "SINGLE",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 2,
"name": "pay_user_num",
"caption": "交易用户数",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "ACCOUNT_ID",
"itemType": "SINGLE",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT_DISTINCT"
},
{
"id": 20,
"name": "pay_money,pay_user_num",
"caption": "人均交易金额",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY,ACCOUNT_ID",
"itemType": "COMBINE",
"calType": "/",
"valCalType": "MONEY",
"functions": "SUM,COUNT_DISTINCT"
},
{
"id": 3,
"name": "order_num",
"caption": "订单数",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "*",
"itemType": "SINGLE",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT"
},
{
"id": 4,
"name": "order_money",
"caption": "订单金额",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY",
"itemType": "SINGLE",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 5,
"name": "big_order_num",
"caption": "大订单数",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "*",
"itemType": "CONDITION",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT"
},
{
"id": 6,
"name": "big_order_money",
"caption": "大订单金额",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY",
"itemType": "CONDITION",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 21,
"name": "order_money,order_num",
"caption": "平均订单金额",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY,*",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "MONEY",
"functions": "SUM,COUNT"
},
{
"id": 22,
"name": "big_order_num,order_num",
"caption": "大订单数占比",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "*,*",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "PERCENT",
"functions": "COUNT,COUNT"
},
{
"id": 23,
"name": "big_order_money,big_order_num",
"caption": "平均大订单金额",
"cubeName": "TRANSACTION_ANALYSIS_V3",
"columns": "CONTQTY,*",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "MONEY",
"functions": "SUM,COUNT"
}
]
},
{
"id": 2,
"subjectId": "",
"name": "profit&loss",
"caption": "盈亏",
"notes": "<br>1.盈利用户数：当日用户中，净利润>0的用户数<br>2.亏损用户数：当日用户中，净利润<0的用户数<br>3.盈利用户占比=当日盈利用户数/当日总用户数<br>4.亏损用户占比=当日亏损用户数/当日总用户数<br>5.净盈利值：当日用户中，净利润>0的值求和<br>6.净亏损值：当日用户中，净利润<0的值求和<br>7.日净利润＝当日净资产-前一日净资产-当日入金+当日出金<br>8.月净利润=月末净资产-月初净资产-月内总入金+月内总出金<br>其他时间粒度计算方法类似月粒度<br>",
"items": [
{
"id": 7,
"name": "profit_user_count",
"caption": "盈利用户数",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "ACCOUNT_ID",
"itemType": "CONDITION",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT_DISTINCT"
},
{
"id": 8,
"name": "loss_user_count",
"caption": "亏损用户数",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "ACCOUNT_ID",
"itemType": "CONDITION",
"calType": "",
"valCalType": "COMMON",
"functions": "COUNT_DISTINCT"
},
{
"id": 9,
"name": "net_profit_num",
"caption": "净盈利值",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "NETPROFIT",
"itemType": "CONDITION",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 10,
"name": "net_loss_num",
"caption": "净亏损值",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "NETPROFIT",
"itemType": "CONDITION",
"calType": "",
"valCalType": "MONEY",
"functions": "SUM"
},
{
"id": 24,
"name": "profit_user_count,profit_total_user_count",
"caption": "盈利用户数占比",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "ACCOUNT_ID,ACCOUNT_ID",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "PERCENT",
"functions": "COUNT_DISTINCT,COUNT_DISTINCT"
},
{
"id": 25,
"name": "loss_user_count,profit_total_user_count",
"caption": "亏损用户数占比",
"cubeName": "NET_PROFIT_ANALYSIS_V3",
"columns": "ACCOUNT_ID,ACCOUNT_ID",
"itemType": "COMBINE_CONDITION",
"calType": "/",
"valCalType": "PERCENT",
"functions": "COUNT_DISTINCT,COUNT_DISTINCT"
}
]
}
]
}
