Repository files navigation

Jmeter JTL ans statiscts file parsing with Logstash and elasticsearch, you can find imageonDocker Hub, statistics.json file is generated with jmeter htmlreport.

• Where to get help:
  • Issues
  • Discussions
  • Documentation

• latest,7.17,7.17.14
• 8.4,8.4.2
• 8.3,8.3.3
• 8.2,8.2.3
• 8.1,8.1.3

1. Parse Standard JTL (CSV Format).
2. Possibility to filter requests based on regex filter (include and exclude filter) .
3. Flag samplers generated by TransactionController based on regex (by default '.+').
4. For TransactionController, calculate number of failing sampler and total.
5. Add relative time to compare different Executions, variable TESTSTART.MS should be logged withpropertysample_variables
6. Add Project name, test name, environment and executionId to organize results and compare different execution.
7. Split Label name to have multi tags by request (by default split by '/').
8. Flag subresult, when there is a redirection 302,.. Subrequest has a have a suffix like "-xx" when xx is the ordernumber
9. Supporting ElasticSearch and can be adapted for other tools.
10. can also index custom field logged in file withproperty :sample_variables

• 1. jmeter-logstash
• 2. Quick reference
  • 2.1. Image version
  • 2.2. Features
  • 2.3. Content
  • 2.4. Image Variants
• 3. Getting Started
  • 3.1. Create ElasticSearch stack (Only if using ElasticSearch & Kibana)
  • 3.2. Run Logstash
    • 3.2.1. Run With image from docker hub for Elasticsearch (way 1, preferred)
  • 3.3. Dashboards
    • 3.3.1. Kibana
  • 3.4. HOW-TO
    • 3.4.1. Example
  • 3.5. Parameters
    • 3.5.1. ElasticSearch configuration
    • 3.5.2. Logstash
  • 3.6. Fields
    • 3.6.1. Common
    • 3.6.2. JTL
    • 3.6.3. Statistics (Only ElasticSearch)
• 4. Troubleshooting & Limitation

Thejmeter-logstash images come in many flavors, each designed for a specific use case.The images version are based on component used to build image, default use elasticsearch output:

1. Logstash Version: 7.17.9 -> default for 7.17.

1. Create Optional docker network (Called jmeter). If not used remove "--net jmeter " from all following docker commandand adapt Elasticsearch url.

docker network create jmeter

2. Start elastic search container , Or use any Elasticsearch instance you have already installed.

docker run --name jmeter-elastic  --net jmeter \-p 9200:9200 -p 9300:9300 \-e"ES_JAVA_OPTS=-Xms1024m -Xmx1024m" \-e"xpack.security.enabled=false" \-e"discovery.type=single-node" \docker.elastic.co/elasticsearch/elasticsearch:8.4.1

3. Start Kibana and connect it to elastic search Using environnement variableELASTICSEARCH_HOSTS.

docker run --name jmeter-kibana  --net jmeter -p 5601:5601 -e"ELASTICSEARCH_HOSTS=http://jmeter-elastic:9200" docker.elastic.co/kibana/kibana:8.4.1

1. In the project folder create a folder named 'input' or you can use any input folder in your machine.
2. If you choose to use a different input folder, you should change"${PWD}/input" on the following command by yourinput folder.

#Run Imagedocker run --rm -it --net jmeter -e"ELASTICSEARCH_HOSTS=http://jmeter-elastic:9200" -v${PWD}/input:/input/ anasoid/jmeter-logstash

Download Dashboards fromKibana Dashboards and go to Stack management kibana ->saved object for import.

Main Dashboard	Compare Dashboard

1. To exit after all files parsed use (-e "FILE_EXIT_AFTER_READ=true") should be used with (-e "FILE_READ_MODE=read") ..
2. To not remove container logstash after execution not use --rm from arguments.
3. Logstash keep information on position f last line parsed in a file sincedb, this file by default is on the path/usr/share/logstash/data/plugins/inputs/file, if you use the same container this file will be persisted even yourestart logstash cotainer, and if you need to maintain this file even you remove container you can mount volumefolder in the path (/usr/share/logstash/data/plugins/inputs/file)
4. To have relative time for comparison test start time should be logged :in user.properties file (sample_variables=TESTSTART.MS,...) or add properties with file using -q argument or directlyin command line with (-Jsample_variables=TESTSTART.MS,..)seeFull list of command-line options

Run Logstash without remove container after stop.

docker run   -it --net jmeter -e"ELASTICSEARCH_HOSTS=http://jmeter-elastic:9200" \-v${PWD}/input:/input/ \anasoid/jmeter-logstash

Run Logstash and remove file after the end of reading.

docker run --rm -it --net jmeter -e"ELASTICSEARCH_HOSTS=http://jmeter-elastic:9200" \-v${PWD}/input:/input/ \-e"FILE_READ_MODE=read" \-e"FILE_COMPLETED_ACTION=delete" \anasoid/jmeter-logstashRun Logstash with external sincedb folder.```shelldocker run --rm -it --net jmeter -e"ELASTICSEARCH_HOSTS=http://jmeter-elastic:9200" \-v${PWD}/input:/input/ \-v${PWD}/.sincedb:/usr/share/logstash/data/plugins/inputs/file \anasoid/jmeter-logstash

Common field are integrated during file parsing for both JTL ans statistics file.

Fields for JTL File.For csv field see documentation onCSV Log format.

For additional fields see documentationonResults file configuration.

Fields for Statistics

For percentiles configuration (pc1,pc2,pc3)seePercentiles configuration.

1. Logstash instance can't parse CSV file with different header Format, as first header will be used for all file, ifyou have files with different format you should use each time a new instance or restart the instance.
2. Change sincedb file can't done on logstash with Elasticsearch without building image.
3. Label with suffix '-{number}' will be considered as subresult, so don't prefix label with '-{number}' or disablesubresult flag with PARSE_WITH_FLAG_SUBRESULT.