Jmeter压测之jtl文件解析
2018-11-24 本文已影响0人
丫头可乐
一、背景
最近在做性能压测方面的工作，用的压测工具是Jmeter。Jmeter中有一个插件叫
jp@gc Throughput Shaping Timer。翻译过来就是吞吐量整形定时器。如下图：
Throughput Shaping Timer
它可以设计梯度的并发请求，并且每种并发请求量(RPS)都可以设定执行的时间。
压测完成后，可以通过Jmeter的聚合报告查看压测结果。如下图：
聚合报告
但是这里存在一个问题。就是如果我想要知道每一种RPS下的平均响应时间，TP90,TP95,TP99,错误率，吞吐量等信息，就没有办法从聚合报告中获取。可能Jmter提供了类似功能的插件，但是我没有找到。
我们都知道Jmeter的压测结果是jtl文件，不知道大家有没有看过jtl文件里面的内容。就是就是一行行的请求记录。如下图：
1542884219661,5,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,5
1542884219690,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,4
1542884219718,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219772,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219772,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219702,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219728,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219695,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219731,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219698,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219734,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219702,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219737,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,4
1542884219705,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219741,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219709,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219745,2,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,2
1542884219712,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219748,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,2
1542884219715,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219719,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219751,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219722,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219754,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,4
1542884219725,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219758,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219728,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219761,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,4
1542884219731,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219705,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219765,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219734,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,4
1542884219708,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,4
1542884219768,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219738,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219712,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
1542884219771,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
上面是我截取的一部分jtl文件的内容。下面做一下解释：
1542884219715,4,http请求,200,OK,Stepping Thread Group 1-1,text,true,318,1,1,3
timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,failureMessage,bytes,Latency
请求发出的绝对时间，响应时间，请求的标签，返回码，返回消息，请求所属的线程，数据类型，是否成功，失败信息，字节，响应时间
倒数第二位和第三位没搞懂是什么意思，不过不影响。
二、实现
有了上面的jtl知识后，相信有点程序基础的同学都可以设计程序来实现获得每种RPS的压测结果信息。
贴一下我自己的代码
首先是JtlResolver类
package com.lin.performance.testing;
import com.alibaba.fastjson.JSONObject;
import com.lin.performance.testing.po.SceneResult;
import org.apache.commons.lang3.time.DateFormatUtils;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
public class JtlResolver {
public String resloveJtl(File file,List<Integer> durationList) throws IOException {
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
String oneLine = null;
//所有的行
List<String> totalLines = new ArrayList<String>();
List<Long> timestampList = new ArrayList<Long>();
while ((oneLine = br.readLine()) != null){
totalLines.add(oneLine);
String[] split = oneLine.split(",");
String timestampStr = split[0];
Long timestamp = Long.valueOf(timestampStr);
timestampList.add(timestamp);
}
System.out.println("totalLines:"+totalLines.size());
System.out.println("第一行:"+totalLines.get(0));
System.out.println("最后一行："+totalLines.get(totalLines.size()-1));
System.out.println();
//压测开始时间
String allStartTimestampStr = totalLines.get(0).split(",")[0];
Long allStartTime = Long.valueOf(allStartTimestampStr);
//压测结束时间
String allEndTimestampStr = totalLines.get(totalLines.size() - 1).split(",")[0];
Long allEndTime = Long.valueOf(allEndTimestampStr);
Long startTime = allStartTime;
Long endTime = allEndTime;
Integer startIndex = 0;
for (int i=0;i<durationList.size();i++){
Integer duration = durationList.get(i);
if (durationList.size() != 1){
endTime = startTime + duration*1000;
}
if (i == (durationList.size() - 1)){
endTime = allEndTime;
}
//计算开始和结束
Integer endIndex = getEndIndex(endTime, timestampList);
getOneRpsResult(startIndex,endIndex,startTime,endTime,timestampList,totalLines);
startTime = endTime;
startIndex = endIndex;
}
return null;
}
private void getOneRpsResult(Integer startIndex,Integer endIndex,Long startTime,Long endTime,List<Long> timestampList,List<String> totalLines){
int index=0;
//总的响应时间
int sumElapsed=0;
Integer failSize = 0;
Integer totalBytes = 0;
List<Integer> elapsedList = new ArrayList<Integer>();
for (Integer i = startIndex;i<=endIndex;i++){
try {
String row = totalLines.get(i);
String[] split = row.split(",");
//响应时间
String elapsed = split[1];
sumElapsed += Integer.valueOf(elapsed);
elapsedList.add(Integer.valueOf(elapsed));
//成功与否
String success = split[7];
if (!"true".equals(success)){
failSize++;
}
//字节
String bytes = split[8];
totalBytes += Integer.valueOf(bytes);
index++;
}catch (Exception e){
System.out.println("exception i:"+i);
}
}
Collections.sort(elapsedList, new Comparator<Integer>() {
public int compare(Integer o1, Integer o2) {
return o1-o2;
}
});
Integer tp90 = elapsedList.size()*9/10;
Integer tp95 = elapsedList.size()*95/100;
Integer tp99 = elapsedList.size()*99/100;
Long l = timestampList.get(index-1) - timestampList.get(0);
SceneResult sceneResult = new SceneResult();
sceneResult.setStartTimeStr(DateFormatUtils.format(startTime,"yyyy-MM-dd HH:mm:ss"));
sceneResult.setEndTimeStr(DateFormatUtils.format(endTime,"yyyy-MM-dd HH:mm:ss"));
sceneResult.setSamples(index+"");
sceneResult.setAverage(sumElapsed/index+"");
sceneResult.setTp90(elapsedList.get(tp90)+"");
sceneResult.setTp95(elapsedList.get(tp95)+"");
sceneResult.setTp99(elapsedList.get(tp99)+"");
sceneResult.setMin(elapsedList.get(0)+"");
sceneResult.setMax(elapsedList.get(index-1)+"");
sceneResult.setMedian(elapsedList.get(index/2)+"");
sceneResult.setErrors(String.format("%.2f",failSize*100.0/index)+"%");
sceneResult.setThroughput(String.format("%.2f",index*1.0/(l*1.0/1000)));
sceneResult.setKbPerSec(String.format("%.2f",totalBytes*1.0/1024/(l*1.0/1000)));
System.out.println(JSONObject.toJSONString(sceneResult));
System.out.println();
}
private Integer getStartIndex(Long startTime,List<Long> timestampList){
int index = 0;
for (int i=0;i<timestampList.size()-1;i++){
if (startTime.longValue() >= timestampList.get(i).longValue() && startTime.longValue() <= timestampList.get(i+1).longValue()){
return index;
}
index++;
}
return index;
}
private Integer getEndIndex(Long endTime,List<Long> timestampList){
for (int i=timestampList.size()-1;i>=0;i--){
if (endTime.longValue() >= timestampList.get(i).longValue()){
return i;
}
}
return 0;
}
public static void main(String[] args) throws IOException {
List<Integer> durationList = new ArrayList<Integer>();
durationList.add(10);
durationList.add(10);
durationList.add(10);
durationList.add(10);
durationList.add(10);
durationList.add(10);
File file = new File("/Users/xmly/Documents/stresstest/jtl/welfare-banners-1-5.jtl");
new JtlResolver().resloveJtl(file,durationList);
}
}
然后是SceneResult类
package com.lin.performance.testing.po;
public class SceneResult {
/**开始压测时间*/
private String startTimeStr;
/**结束压测时间*/
private String endTimeStr;
/**压测请求数*/
private String samples;
/**平均响应时间*/
private String average;
/**90% Line*/
private String tp90;
/**95% Line*/
private String tp95;
/**99% Line*/
private String tp99;
/**最小请求时间*/
private String min;
/**最大请求时间*/
private String max;
/**错误率*/
private String errors;
/** throughput */
private String throughput;
/**吞吐量 KB/sec*/
private String kbPerSec;
/**中间时间*/
private String median;
public String getMedian() {
return median;
}
public void setMedian(String median) {
this.median = median;
}
public String getSamples() {
return samples;
}
public void setSamples(String samples) {
this.samples = samples;
}
public String getAverage() {
return average;
}
public void setAverage(String average) {
this.average = average;
}
public String getTp90() {
return tp90;
}
public void setTp90(String tp90) {
this.tp90 = tp90;
}
public String getTp95() {
return tp95;
}
public void setTp95(String tp95) {
this.tp95 = tp95;
}
public String getTp99() {
return tp99;
}
public void setTp99(String tp99) {
this.tp99 = tp99;
}
public String getMin() {
return min;
}
public void setMin(String min) {
this.min = min;
}
public String getMax() {
return max;
}
public void setMax(String max) {
this.max = max;
}
public String getErrors() {
return errors;
}
public void setErrors(String errors) {
this.errors = errors;
}
public String getThroughput() {
return throughput;
}
public void setThroughput(String throughput) {
this.throughput = throughput;
}
public String getKbPerSec() {
return kbPerSec;
}
public void setKbPerSec(String kbPerSec) {
this.kbPerSec = kbPerSec;
}
public String getStartTimeStr() {
return startTimeStr;
}
public void setStartTimeStr(String startTimeStr) {
this.startTimeStr = startTimeStr;
}
public String getEndTimeStr() {
return endTimeStr;
}
public void setEndTimeStr(String endTimeStr) {
this.endTimeStr = endTimeStr;
}
}
运行JtlResolver类中的main方法，获得的结果如下：
/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/tools.jar:/Users/xmly/workspace/java/performance-testing/target/classes:/Users/xmly/workspace/repository/org/apache/logging/log4j/log4j-core/2.9.1/log4j-core-2.9.1.jar:/Users/xmly/workspace/repository/org/apache/logging/log4j/log4j-api/2.10.0/log4j-api-2.10.0.jar:/Users/xmly/workspace/repository/org/apache/commons/commons-lang3/3.8.1/commons-lang3-3.8.1.jar:/Users/xmly/workspace/repository/com/alibaba/fastjson/1.2.53/fastjson-1.2.53.jar:/Users/xmly/workspace/repository/org/springframework/boot/spring-boot-starter-web/2.0.5.RELEASE/spring-boot-starter-web-2.0.5.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/boot/spring-boot-starter/2.0.5.RELEASE/spring-boot-starter-2.0.5.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/boot/spring-boot/2.0.5.RELEASE/spring-boot-2.0.5.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/boot/spring-boot-autoconfigure/2.0.5.RELEASE/spring-boot-autoconfigure-2.0.5.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/boot/spring-boot-starter-logging/2.0.5.RELEASE/spring-boot-starter-logging-2.0.5.RELEASE.jar:/Users/xmly/workspace/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/Users/xmly/workspace/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/Users/xmly/workspace/repository/org/apache/logging/log4j/log4j-to-slf4j/2.10.0/log4j-to-slf4j-2.10.0.jar:/Users/xmly/workspace/repository/org/slf4j/jul-to-slf4j/1.7.25/jul-to-slf4j-1.7.25.jar:/Users/xmly/workspace/repository/javax/annotation/javax.annotation-api/1.3.2/javax.annotation-api-1.3.2.jar:/Users/xmly/workspace/repository/org/yaml/snakeyaml/1.19/snakeyaml-1.19.jar:/Users/xmly/workspace/repository/org/springframework/boot/spring-boot-starter-json/2.0.5.RELEASE/spring-boot-starter-json-2.0.5.RELEASE.jar:/Users/xmly/workspace/repository/com/fasterxml/jackson/core/jackson-databind/2.9.6/jackson-databind-2.9.6.jar:/Users/xmly/workspace/repository/com/fasterxml/jackson/core/jackson-annotations/2.9.0/jackson-annotations-2.9.0.jar:/Users/xmly/workspace/repository/com/fasterxml/jackson/core/jackson-core/2.9.6/jackson-core-2.9.6.jar:/Users/xmly/workspace/repository/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.9.6/jackson-datatype-jdk8-2.9.6.jar:/Users/xmly/workspace/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.9.6/jackson-datatype-jsr310-2.9.6.jar:/Users/xmly/workspace/repository/com/fasterxml/jackson/module/jackson-module-parameter-names/2.9.6/jackson-module-parameter-names-2.9.6.jar:/Users/xmly/workspace/repository/org/springframework/boot/spring-boot-starter-tomcat/2.0.5.RELEASE/spring-boot-starter-tomcat-2.0.5.RELEASE.jar:/Users/xmly/workspace/repository/org/apache/tomcat/embed/tomcat-embed-core/8.5.34/tomcat-embed-core-8.5.34.jar:/Users/xmly/workspace/repository/org/apache/tomcat/embed/tomcat-embed-el/8.5.34/tomcat-embed-el-8.5.34.jar:/Users/xmly/workspace/repository/org/apache/tomcat/embed/tomcat-embed-websocket/8.5.34/tomcat-embed-websocket-8.5.34.jar:/Users/xmly/workspace/repository/org/hibernate/validator/hibernate-validator/6.0.12.Final/hibernate-validator-6.0.12.Final.jar:/Users/xmly/workspace/repository/javax/validation/validation-api/2.0.1.Final/validation-api-2.0.1.Final.jar:/Users/xmly/workspace/repository/org/jboss/logging/jboss-logging/3.3.2.Final/jboss-logging-3.3.2.Final.jar:/Users/xmly/workspace/repository/com/fasterxml/classmate/1.3.4/classmate-1.3.4.jar:/Users/xmly/workspace/repository/org/springframework/spring-web/5.0.9.RELEASE/spring-web-5.0.9.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/spring-beans/5.0.9.RELEASE/spring-beans-5.0.9.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/spring-webmvc/5.0.9.RELEASE/spring-webmvc-5.0.9.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/spring-aop/5.0.9.RELEASE/spring-aop-5.0.9.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/spring-context/5.0.9.RELEASE/spring-context-5.0.9.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/spring-expression/5.0.9.RELEASE/spring-expression-5.0.9.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/spring-core/5.0.9.RELEASE/spring-core-5.0.9.RELEASE.jar:/Users/xmly/workspace/repository/org/springframework/spring-jcl/5.0.9.RELEASE/spring-jcl-5.0.9.RELEASE.jar:/Users/xmly/workspace/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar com.ximalaya.performance.testing.JtlResolver
totalLines:133118
第一行:1542970924059,3,http请求,200,OK,Stepping Thread Group 1-1,text,true,1398,1,1,3
最后一行：1542970985058,1,http请求,200,OK,Stepping Thread Group 1-22,text,true,1398,1,1,1
{"average":"1","endTimeStr":"2018-11-23 19:02:14","errors":"0.00%","kbPerSec":"684.40","max":"21","median":"1","min":"0","samples":"5010","startTimeStr":"2018-11-23 19:02:04","throughput":"501.30","tp90":"2","tp95":"2","tp99":"2"}
{"average":"1","endTimeStr":"2018-11-23 19:02:24","errors":"0.00%","kbPerSec":"736.17","max":"4","median":"1","min":"0","samples":"7602","startTimeStr":"2018-11-23 19:02:14","throughput":"539.23","tp90":"2","tp95":"2","tp99":"2"}
{"average":"1","endTimeStr":"2018-11-23 19:02:34","errors":"0.00%","kbPerSec":"824.10","max":"11","median":"1","min":"0","samples":"9809","startTimeStr":"2018-11-23 19:02:24","throughput":"603.59","tp90":"1","tp95":"2","tp99":"2"}
{"average":"1","endTimeStr":"2018-11-23 19:02:44","errors":"0.00%","kbPerSec":"968.51","max":"11","median":"1","min":"0","samples":"18990","startTimeStr":"2018-11-23 19:02:34","throughput":"709.30","tp90":"1","tp95":"2","tp99":"2"}
{"average":"1","endTimeStr":"2018-11-23 19:02:54","errors":"0.00%","kbPerSec":"1351.61","max":"37","median":"1","min":"0","samples":"38010","startTimeStr":"2018-11-23 19:02:44","throughput":"989.87","tp90":"2","tp95":"2","tp99":"3"}
{"average":"0","endTimeStr":"2018-11-23 19:03:05","errors":"0.00%","kbPerSec":"1677.58","max":"22","median":"1","min":"0","samples":"53702","startTimeStr":"2018-11-23 19:02:54","throughput":"1228.63","tp90":"1","tp95":"1","tp99":"2"}
Process finished with exit code 0
这样我们就获得了每种RPS的压测结果信息。获得系统在不同的吞吐量的情况下性能表现。
