通过yarn timeline server api查询tez任

2018-08-09  本文已影响0人  ron_yang

这里面最坑爹的就是tez记录在timelineserver中的entity_type值:TEZ_DAG_ID
timelineserver并没有接口查询所有的entity_type
这个原因主要是因为timelineserver使用了levelDB作为序列化存储:https://zh.wikipedia.org/wiki/LevelDB

另外参考:
timelineserver的api文档:http://han-zw.iteye.com/blog/2332924

import json
import requests

parameter = {'limit': '11',
             '_': '1533712088094'
             }
ambari_user = 'admin'
ambari_pass = 'admin'
response = requests.get('http://{yarn-timelineserver-host}:8188/ws/v1/timeline/TEZ_DAG_ID', parameter)

json_data = response.json()
entities_ = json_data['entities']

entity = entities_[0]
otherinfo_ = entity['otherinfo']
res = {

    'domain_':entity['domain'],
    'dag_id':entity['entity'],

    'app_id':otherinfo_['applicationId'],
    'description_': otherinfo_['dagPlan']['dagContext']['description'],
    'user_':otherinfo_['user'],
    'time_taken_': otherinfo_['timeTaken'],
    'start_time_': otherinfo_['startTime'],
    'end_time_': otherinfo_['endTime'],
    'queue_name_': otherinfo_['queueName'],

}
for k, v in res.iteritems():
    print(k +'|' + str(v))
上一篇下一篇

猜你喜欢

热点阅读