How to use Artillery to perform automate testing for Amity Chat

  • This is some text inside of a div block.
  • This is some text inside of a div block.
  • This is some text inside of a div block.
  • This is some text inside of a div block.

How to use Artillery to perform automate testing for Amity Chat

Automate testing is a Software testing technique to test and compare the actual outcome with the expected outcome. This can be achieved by writing test scripts or using any automation testing tool. Test automation is used to automate repetitive tasks and other testing tasks which are difficult to perform manually.

by : https://www.softwaretestinghelp.com/automation-testing-tutorial-1/

What is objectives of automate testing ?

  • To reduce Testing Cost and Time.
  • To reduce Redundancy.
  • To speed up the Testing Process.
  • To help improve Quality.
  • To improve Test coverage.
  • To reduce Manual Intervention.

by : http://www.qatutorial.com/Software_Test_Automation#:~:text=Goals%20and%20Objectives%20of%20Software,To%20help%20improve%20Quality.

How to automate testing ?

In this topic I will write by myself ! (lol) and we’ll follow by Topic

  • Why Artillery ?
  • Prepare Artillery
  • Test Artillery
  • Create scenario
  • Run automate testing

Why Artillery ?

link information here for decide your way to automate testing. But for me because it’s free and easy to use and you can load-test with this tool too.

By the way you can use whatever you want to use but for this content we’ll use Artillery for example.

Prepare Artillery

First of all we must to have a product for automate testing and we’ll use Artillery for automate testing. Product I will use Amity Chat (1 of Amity product) If you don’t have product for test I’m sorry you must have. In this time let install Nodejs I recommend version 12+ (Artillery require)

I have 2 choices for install Nodejs ! for long term I suggest solution nvm. *** if you want to use more than one version I recommend install “nvm” !!!

1. Solution NVM

link download : https://github.com/coreybutler/nvm-windows

( If installed nvm you can use command for install nodejs v.12 )

{% c-line %}nvm install 12.0.0{% c-line-end %}

and use version 12

{% c-line %}nvm use 12.0.0{% c-line-end %}

2. Solution Install only Nodejs v12

link download : https://nodejs.org/en/download/

and then install Artillery with this command (more information)

{% c-line %}RUN npm install -g artillery --allow-root --unsafe-perm=true{% c-line-end %}

Now test this command

{% c-line %}artillery{% c-line-end %}

If result look like below image we’re done on process prepare.

run command artillery

Test artillery

Create file name test.yaml and copy code to the file then save it.

{% c-block language="javascript" %}
config:  
     target: "https://google.com"  
     phases:    
          - duration: 200      
             arrivalCount: 1000
scenarios:
     - flow:    
          - get:          
                url: "/"
{% c-block-end %}

test.yaml

and run command here.

{% c-line %}artillery run test.yaml{% c-line-end %}

Your terminal must me like this.

terminal

and then when Artillery finish it’ll show like this but maybe result not same.

What we just did it ?

We just test GET www.google.com with 2000 VUs (Visual Users) in 200 seconds ! If you want to know command of Artillery more Click Here! But I will always explain what I do.

1. If you want to have report you can use -o like this

{% c-line %}artillery run test.yaml -o result.json{% c-line-end %}

Artillery will create result JSON for you

2. If you think JSON very hard to read Artillery can create HTML for visualize report

{% c-line %}artillery report result.json -o result.html{% c-line-end %}

3. How to generate VUs ?

duration for how many time to generate VUs

arrivalCount such as…

{% c-block language="javascript" %}
duration : 10
arrivalCount : 10
{% c-block-end %}

is mean create 10 VUs in 10 sec (1 VU/s)

arrivalRate such as…

{% c-block language="javascript" %}
duration : 10
arrivalRate : 10
{% c-block-end %}

this means create 10 VUs/s

and in arrivalRate you can use rampTo such as…

{% c-block language="javascript" %}
duration : 10
arrivalRate : 10
rampTo : 20
{% c-block-end %}

this means 1st second will create 10 VUs/s until 10th second will create 20VUs/s

You can adapt phase like this

{% c-block language="javascript" %}
config:  
     target: "URL"  
     phases:    
          - duration: 20      
             arrivalCount: 10    
          - duration: 20      
             arrivalRate: 10      
             rampTo: 20    
          - duration: 30      
             arrivalRate: 20      
             rampTo: 10
scenarios:  
     #scenarios
{% c-block-end %}

This means we create 10 VUs in 20 second then create 10VUs/sec and ramp to 20VUs/s in 20 second then create 20VUs/s and ramp to 10 VUs/second in 30 second

4. How to create scenarios ?

Artillery can GET / POST / PUT / PATCH / DELETE requests and also use socketio , websocket

I sorry for I don’t have example because it’s so long story but you can check it here !

I have a little bit example for you ! ;)

{% c-block language="javascript" %}
config:  
     target: "https://youweb.com"  
     path: "../data/users.csv"    
          fields:      
             - "user"    
             - "password"    
          order: "sequence"  
     http:    
          timeout: 10  
     defaults:    
          headers:      
             x-api-key: "{{ $processEnvironment.SERVICE_API_KEY }}"  
     phases:    
             - duration: 20      
                arrivalCount: 10    
             - duration: 20      
                arrivalRate: 10      
                rampTo : 20    
             - duration: 20      
                arrivalRate: 10      
                rampTo : 0
scenarios:  
     - flow:    
          - log: "New virtual user running"    
          - get:                
                url: "/"        
                qs:          
                   search_keyword: "coffee"          
                   page_size: 25    
          - post:        
                url: "/signin"        
                json:          
                   username: "{{ user }}"          
                   password : "{{ password }}"        
                capture:          
                   json: "$.id"          
                   as: "id"    
          - post:        
                url: "/buy"        
                json:          
                   id: "{{ id }}"          
                   product : "snack"        
                capture:        
                   json: "$.message"          
                   as: "message"    
          - log : "Show message => {{ message }}"
{% c-block-end %}

Create scenario

Amity Social Cloud mainly utilizes Socket.io as the main communication engine, however socket.io engine of Artillery does not support the scenario we want to create, therefore we will use plugin-expect and acknowledge to check status code.

We can create 2 scenarios on one script and use weight to split Virtual users up into multiple groups. For example if I want 15,000 VUs run on first scenario and 5,000 VUs run on second scenario the calculation should be as followed:

=> 20,000/4 = 5,000 VUs

therefore weight of first scenario is 3

weight of second scenario is 1

{% c-block language="javascript" %}
config:  
     variables:    
          apiKey: "{{ myApiKey }}"  
     target: "{{ MyHost }}"  
     engines:    
          quiwsth-socket: {}  
     SocketDynamic:    
          transports: ["websocket"]  
     phases:        
          - duration: 1000      
             arrivalCount: 20000  
     plugins:    
          expect: {}  
     processor: 'functional.js'
scenarios:  
     - name : "connect - query - leave"    
        engine: "quiwsth-socket"    
        weight: 999    
        flow:      
            - function: "genUser"      
            - think : 10      
            - log : "{{ userId }} - {{ deviceId }} - {{ channel }}"      
            - post:          
                  url: "/v1/device"          
                  headers:            
                      x-api-key : "{{ apiKey }}"          
                  json:            
                      userId: "{{ userId }}"            
                      deviceId: "{{ deviceId }}"            
                      deviceInfo: {              
                          kind: "web",              
                          model: ""            
                      }          
                  afterResponse: "printStatus"          
                  capture:            
                      - json: "$.accessToken"              
                        as: "accessToken"          
                  expect:            
                      - statusCode: 200      
            - think : 10      
            - emit:          
                  channel: "channel.getInactiveChannelIds"          
                  data: {}      
            - think : 10      
            - emit:          
                  channel: "channel.query"          
                  data: {}          
                  acknowledge:            
                      match:              
                          json: "$.0.status"              
                          value: "success"      
            - think: 10      
            - emit:          
                  channel: "channel.join"          
                  data: {            
                          channelId: "{{ channel }}",            
                          type: "standard",          
                  }          
                  acknowledge:            
                      match:              
                          json: "$.0.status"              
                          value: "success"      
            - think: 10      
            - emit:          
                  channel: "message.query"          
                  data: {            
                       channelId: "{{ channel }}"        
                  }          
                  acknowledge:            
                      match:              
                          json: "$.0.status"              
                          value: "success"      
            - think: 200      
            - think: 10      
            - emit:          
                  channel: "channel.leave"          
                  data: {            
                       channelId: "{{ channel }}",          
                  }          
                  acknowledge:            
                       match:              
                          json: "$.0.status"              
                          value: "success"  

     - name : "connect - query - send message - leave"    
        engine: "quiwsth-socket"    
        weight: 1    flow:      
            - function: "genUser"      
            - think : 10      
            - log : "{{ userId }} - {{ deviceId }} - {{ channel }}"      
            - post:          
                  url: "/v1/device"          
                  headers:            
                       x-api-key : "{{ apiKey }}"          
                  json:            
                       userId: "{{ userId }}"            
                       deviceId: "{{ deviceId }}"            
                       deviceInfo: {              
                           kind: "web",              
                           model: ""            
                       }          
                  # afterResponse: "printStatus"          
                  capture:            
                        - json: "$.accessToken"              
                          as: "accessToken"          
                  expect:            
                        - statusCode: 200      
            - think : 10      
            - emit:          
                  channel: "channel.getInactiveChannelIds"          
                  data: {}      
            - think : 10      
            - emit:          
                  channel: "channel.query"          
                  data: {}          
                  acknowledge:            
                        match:              
                           json: "$.0.status"              
                           value: "success"      
            - think: 10      
            - emit:          
                  channel: "channel.join"          
                  data: {            
                           channelId: "{{ channel }}",            
                           type: "standard",          
                  }          
                  acknowledge:            
                           match:              
                               json: "$.0.status"              
                               value: "success"      
            - think: 10      
            - emit:          
                  channel: "message.query"          
                  data: {            
                           channelId: "{{ channel }}"          
                  }          
                  acknowledge:            
                           match:              
                               json: "$.0.status"              
                               value: "success"      
            - think: 200      
            - function: "genMessageId"      
            - emit:          
                  channel: "message.create"          
                  data: {            
                           channelId: "{{ channel }}",            
                           type: "text",            
                           data: {              
                               text: "{{ userId }} I send message from artillery !"            
                           },            
                           messageId: "{{ msgId }}"          
                  }          
                  acknowledge:            
                           match:              
                               json: "$.0.status"              
                               value: "success"      
            - think: 10      
            - emit:          
                  channel: "channel.leave"          
                  data: {            
                           channelId: "{{ channel }}",          
                  }          
                  acknowledge:            
                           match:              
                               json: "$.0.status"              
                               value: "success"
{% c-block-end %}

and then Run !!!

Check your result.