通过yarn timeline server api查询tez任
2018-08-09 本文已影响0人
ron_yang
这里面最坑爹的就是tez记录在timelineserver中的entity_type值:TEZ_DAG_ID
timelineserver并没有接口查询所有的entity_type
这个原因主要是因为timelineserver使用了levelDB作为序列化存储:https://zh.wikipedia.org/wiki/LevelDB
另外参考:
timelineserver的api文档:http://han-zw.iteye.com/blog/2332924
import json
import requests
parameter = {'limit': '11',
'_': '1533712088094'
}
ambari_user = 'admin'
ambari_pass = 'admin'
response = requests.get('http://{yarn-timelineserver-host}:8188/ws/v1/timeline/TEZ_DAG_ID', parameter)
json_data = response.json()
entities_ = json_data['entities']
entity = entities_[0]
otherinfo_ = entity['otherinfo']
res = {
'domain_':entity['domain'],
'dag_id':entity['entity'],
'app_id':otherinfo_['applicationId'],
'description_': otherinfo_['dagPlan']['dagContext']['description'],
'user_':otherinfo_['user'],
'time_taken_': otherinfo_['timeTaken'],
'start_time_': otherinfo_['startTime'],
'end_time_': otherinfo_['endTime'],
'queue_name_': otherinfo_['queueName'],
}
for k, v in res.iteritems():
print(k +'|' + str(v))