How does the Linx REST Service perform under load?

Apr 2, 2021

We recently ran load tests for the Linx Host Service. In these tests, the CPU load and memory were monitored under heavy load to provide better insights into how the REST API Request Limits perform for the different virtual machines (Azure B-Series) that run Linx.



About Load Testing: Load testing simulates real-world conditions and examines how the system behaves during normal and high loads. They will determine if your system or software can handle high loads given the high demand of end-users. This test is typically applied when a software development project nears completion.

 

Method

Load testing was done using JMeter. A web service with four endpoints with empty tasks was tested to determine the behaviour of the system as the load increases from zero-load situation to a situation until the server`s CPU usage reached 100 % without the API returning any errors or timing out.

Four endpoints were tested for Load as described in the following table

 

Label Method Headers Parameters Body Data Response
DummyGET Get None None None None
DummyPOST Post Content-type and Accept None None None
GetWithResponse Get None None None String
PostWithResponseAndContetn Post Content-type and Accept None Yes String

 

We started by trying to find the maximum throughput that our API can sustain. We define this metric as the number of requests per second that will bring our API server to the maximum CPU utilization. The test was carried out in 2 phases; first, requests were done consecutively on each of the four endpoints, and then requests were done in parallel on four endpoints.


Result

To find each endpoint limit on each machine, traffic was generated by increasing the number of users slowly until errors were noted in JMeter. In the consecutive test, traffic was simulated on four different endpoints using linear injection of users for a fixed time period of sixty seconds (60 sec). Each user would send 10 requests on each endpoint consecutively.


Consecutive Test Results

  B1ms (1
vcpus, 2 GiB
memory)
B2ms (2
vcpus, 4 GiB
memory)
B2ms (2
vcpus, 8 GiB
memory)
B4ms (4
vcpus, 16 GiB
memory)
B8ms (8
vcpus, 32 GiB
memory)
No. of requests 80,000 120,000 120,000 320,000 320,000
Mean response time (ms) 15 5 5 6 6
Average requests/s 1250 2653 2823 4064 6524

 



To test how increasing the requests on the four endpoints simultaneously would affect performance, traffic was simulated on all four different endpoints at the same time. Requests were sent on all four endpoints at the same time using linear injection of users for fixed time period of sixty seconds (60 sec). Each user would send 10 requests.

Parallel Test Results

  B1ms (1
vcpus, 2 GiB
memory)
B2ms (2
vcpus, 4 GiB
memory)
B2ms (2
vcpus, 8 GiB
memory)
B4ms (4
vcpus, 16 GiB
memory)
B8ms (8
vcpus, 32 GiB
memory)
No. of requests 48,000 140,000 120,000 240,000 240,000
Mean response time (ms) 9 17 18 351 2041
Average requests/s 811 1736 1848 3212 5510



Conclusion

REST API requests of all types and to all endpoints are limited by the number of CPU of the virtual machines. As we increased the number of virtual CPU, we could see that Linx RESTHost performance increased to serve more requests and the mean response time was acceptable.  As the number of requests increased above approximately 5500 requests per second, we started having errors with 500/Internal Server Error.



Challenges

  • In parallel tests, the actual number of concurrent users and transactions per seconds depends on the server response time.
  • In parallel tests, only four endpoints were tested. The result may vary with the number of endpoints.
  • Users were added during a fixed period of time. 60 seconds was chosen as the ‘spread over time’ in all tests, which is not a real-life case scenario.
  • The example does not accurately reflect a normal user’s usage pattern as the Linx server deployment and load injector were in the same data center.