I tested an empty $4 server to see how much load it can take

TL;DR: I bought a $4 server, wrote the simplest possible API on it, and sent it as many requests as I could. With Node.js it handled 15,366 requests per second. With Go it handled 28,122. With nginx it handled 40,805. With Python Flask it handled only 910. The machine was the same every time. Only the language changed.

I was wondering how anyone decides which server to buy. How much CPU, how much RAM, which plan. I asked around and I did not get a clear answer. So I decided to find out myself.

I did a very basic thing. I went to DigitalOcean and bought two $4 servers. Each one has 1 vCPU, 512 MB RAM, 10 GB SSD and 500 GB transfer. I named one “target” and the other “client”.

I bought two servers in the same region for a reason. If I send requests from my laptop, the requests travel through my WiFi and my internet provider. Then I am testing my internet connection, not the server. Two servers in the same data center talk to each other directly, so the result shows only what the server can do.

On the target server I wrote a very simple Node.js API. It accepts a request and returns 200. It does nothing else.

On the client server I installed a load testing tool called wrk. I asked it to open 100 parallel connections and keep sending requests for 30 seconds.

The result was 15,366 requests per second. While the test was running, I watched htop on the target. The CPU was at 100%. The one core was fully busy.

Then I increased the load to 500 parallel connections for 30 seconds. I expected the requests per second to go up. Instead it dropped to 2,271 requests per second. This surprised me a lot.

The output also showed 566,898 socket errors. That number made me dig deeper.

I learned that Node.js has a default rule. If a connection stays silent for 5 seconds, Node closes it. With 500 connections the server was so busy that some requests waited in the queue for more than 5 seconds. Node thought those connections were idle and closed them. The client had to reconnect. Each reconnect takes CPU time. So the CPU was busy closing and reopening connections instead of serving requests.

I changed that timeout from 5 seconds to 65 seconds with one line of code. I ran the 500 connection test again. The result was 17,188 requests per second with zero errors.

This number is almost the same as the 100 connection test. Only the latency went up. Each request waited longer. This told me that I had reached the maximum this server can handle with Node. Adding more connections does not add more requests per second. It only makes each request wait longer.

I also ran the plain Node test one more time later and got 18,772. This is a shared server, so a 15 to 20% difference between runs is normal noise. I ran each test only once, so take the exact numbers lightly.

Then another question came to my mind. Does the language I choose change anything? I wrote the same simple API in Go, in Python Flask, and in Node Express. I also tried nginx, which is a web server written in C, and asked it to just return 200.

To my surprise, the language changed everything.

Same machine. Same simple API. The fastest was 45 times faster than the slowest.

Express was much slower than I expected. I could not find out why. I am noting it honestly instead of guessing.

So this is what I understood. This $4 machine can handle at least 41,000 requests per second, because nginx reached that number. Even nginx uses some CPU, so the real hardware limit is somewhere above it. The language you choose takes its share first, before you write any code. Node leaves you with 15,000. Python leaves you with 900. Then your real application adds database calls and business logic. Each request takes more time, so latency goes up, and the requests per second go down even more. The number you get from a simple empty API is the best you will ever see on that server. Your real app will always be below it.

This was a very basic test done with the knowledge I have. I am sure there are things I missed. If you know something I should have checked, please tell me. I would be happy to learn.