Order: sequence not working to loop over .csv line by line

[i-farooq]

Given the following .yml my understanding is that artillery will load the first 2 lines from the .csv for the respective user count.

Using version 1.6.2 this worked exactly as expected. I recently installed a newer version 2.0.0.-5 and this no longer appears to work. The 1st line from the .csv is used for both virtual users. As it stands I am unable to test with an arrivalRate greater than 1 user per second. Am I missing something?

config:
  http:  
    timeout: 120  
  target: "https:/"  
  payload:  
    path: "data/users.csv"  
    fields: ["email"]  
    order: sequence  
  phases:  
    - arrivalRate: 2  
      duration: 1  
scenarios:  
    - flow:  
      - log: "{{email}}"

[hassy]

hi @i-farooq 👋 Your understanding is correct as far as Artillery v1 goes. Artillery v2 no longer makes that guarantee with sequence when more than one VU is created.

The reason is that in order to make sequence work, VUs need to have a way to coordinate with each other, through some sort of a global shared state mechanism. This can work well in a single Artillery process (i.e. when run on a single machine), but does not scale well for distributed load testing.

It's one of a few things we're deprecating in Artillery v2 to make it work better in those distributed testing scenarios.

The intended use for sequence in Artillery v2 is for running smoke tests, which usually create just a single VU (e.g. with the new --solo flag)

My apologies, the current docs are behind when it comes to v2 at the moment.

[i-farooq]

Thank you for the rapid response! Glad to know it's not something I was doing wrong.

Is there an alternative way then to simulate greater than 1 user arriving per second?

We have a scenario to test concurrent sign in where we were pulling user credentials in from the .csv
We are unable to simulate sign in using the same credentials every time as our logic prevents multiple sign in until a session has expired.

[jared-christensen]

Is this why the Playwright smoke test example isn’t working as expected for me? I’ve made a new discussion describing the challenges I’m facing.

[mnylensc]

I have a very similar use case load testing our sign in endpoints. I understand that order: sequence doesn't work across multiple processes/workers, but it should be relatively simple to coordinate within one process who gets the next line from CSV, and this is kind of what you would expect from the documentation.

My initial plan was to work around the lack of synchronization between processes by splitting the payload CSV into multiple files like data-1.csv, data-2.csv and so on. Then specify payload as:

payload:
  path: "data-{{ worker }}.csv"
  order: sequence
  fields: ["...", "..."]

And then run artillery with artillery run --variables '{"worker":1}'.

This way I could assign a CSV file with unique set of test data to each worker and have each worker process the CSV file in sequence.

But due to this problem, that plan needs revisiting now.