How to send many requests in parallel in ASP.Net Core

I want to make 1000 requests! How can I make it really fast? Let’s have a look at 4 approaches and compare their speed.


In order to test different methods of handling requests, I created a very simple ASP.Net Core API, that return user by his id. It fetches them from plain old MSSQL database.

I deployed it quickly to Azure using App services and it was ready for testing in less than two hours. It’s amazing how quickly a .net core app can be deployed and tested in a real hosting environment. I was also able to debug it remotely and check it’s work in Application Insights.

Here is my post on how to build an app and deploy it to Azure:

And a post about custom data source in Application Insights:

API in a swagger looks like this:

So the task here is to write a method, that would call this endpoint and fetch 1000 users by their ids as fast as possible.

I wrapped a single call in a UsersClient class:

#1 Let’s use asynchronous programming

Asynchronous programming in C# is very simple, you just use async / await keywords in your methods and magic happens.

Score: 4 minutes 51 seconds

This is because although it is asynchronous programming, it doesn’t mean requests are done in parallel. Asynchronous means requests will not block the main thread, that can go further with the execution. If you look at how requests are executed in time, you will see something like this:

Let’s run requests in parallel

Running in parallel is the key here because you can make many requests and use the same time that one request takes. The code can look like this:

WhenAll is a beautiful creation that waits for tasks with the same type and returns a list of results. A drawback here would be an exception handling because when something goes wrong you will get an AggregatedException with possibly multiple exceptions, but you would not know which task caused it.

Score: 28 seconds

This is way better than before, but it’s not impressive. The thing that slows down the process is thread handling. Executing 1000 requests at the same time will try to create or utilize 1000 threads and managing them is a cost. Timeline looks like this:

Let’s run requests in parallel, but smarter

The idea here is to do parallel requests, but not all at the same time. Let’s do it batches for 100.

Score: 20 seconds

This is the slightly better result because framework needs to handle fewer threads at the same time and therefore it is more effective. You can manipulate the batch size and figure out what is best for you. Timeline looks like this:

The proper solution

The proper solution needs some modifications in the API. You won’t always have the ability to change the API you are calling, but only changes on both sides can get you even further. It is not effective to fetch users one by one when we need to fetch thousands of them. To further enhance performance we need to create a specific endpoint for our use. In this case – fetching many users at once. Now swagger looks like this:

and code for fetching users:

Notice that endpoint for getting multiple users is a POST. This is because payload we send can be big and might not fit in a query string, so it is a good practice to use POST in such a case.

Code that would fetch users in batches in parallel looks like this:

Score: 0,38 seconds

Yes, less than one second! On a timeline it looks like this:

Comparing to other methods on a chart, it’s not even there:

How to optimize your requests

Have in mind, that every case is different and what works for one service, does not necessarily need to work with the next one. Try different things and approaches, find methods to measure your efforts.

Here are a few tips from me:

  • Remember that the biggest cost is not processor cycles, but rather IO operations. This includes SQL queries, network operations, message handling. Find improvements there.
  • Don’t start with parallel processing in the beginning as it brings complexity. Try to optimize your service by using hashsets or dictionaries instead of lists
  • Use smallest Dtos possible, serialize only those fields you actually use
  • Implement an endpoint suited to your needs
  • Use caching if applicable
  • Try different serializers instead of Json, for example ProfoBuf
  • When it is still not enough… – try different architecture, like push model architecture or maybe actor-model programming, like Microsoft Orleans:

You can find all code posted here in my github repo:

Optimize and enjoy 🙂

19 thoughts on “How to send many requests in parallel in ASP.Net Core

  1. You may want to check out ServicePointManager.DefaultConnectionLimit, since your requests are to the same point they are most likely only 2 at a time actually running. This is most likely the reason for ~20s to run them, not the thread and context switches because those usually take microseconds. That’s assuming your api can handle many concurrent requests.

  2. “… A drawback here would be an exception handling because when something goes wrong you will get an AggregatedException with possibly multiple exceptions, but you would not know which task caused it…”

    Actually ‘Task.WhenAll()’ won’t return the AggregateException (you can read here: and here Short explanation is that:
    “…Now the team in Microsoft could have decided that really you should catch AggregateException and iterate over all the exceptions contained inside the exception, handling each of them separately. (…) They decided to simply extract the first exception from the AggregateException within a task, and throw that instead…”
    So the output of the ‘…await Task.WhenAll()…” will be basic Exception (first task which failed). The way to handle exceptions from other task can be, eg.:
    “…public static async Task AwaitGracefully(this IList taskList, Action logger)
    await Task.WhenAll(taskList);
    catch (Exception)
    foreach (var task in taskList.Where(t => t.IsFaulted))
    logger(task.Exception?.InnerException ?? task.Exception);

  3. I am really impressed with your writing skills and also with
    the layout on your blog. Is this a paid theme or did you customize it yourself?

    Either way keep up the excellent quality writing, it’s rare to see a nice blog like this one these days.

  4. Hello, would it be possible to add the code of the visual studio project on your github ?
    I would like to see the UserDto.cs file.

    thanks for the great article.

  5. In your last example you modify your Api to support return list of users instead returning single user. Actually this is not our concern in this case because we might don’t access to modify Api, It maybe another vendor Api and we can not access it.

    1. Saeid,
      That’s correct. In this example, I wanted to show, that speeding up communication between micro-services can be done on one side just to a certain point. In order to go a step forward, both sides need to be involved.
      Thanks for feedback 🙂

  6. In the case of parallel execution where we don’t do batches, and send 1000 requests in parallel, do we have a guarantee do receive responses in the same order we sent the requests?

  7. Hello,

    Thank you very much for your article, it helps me a lot.

    I’ve implemented the “Let’s run requests in parallel, but smarter”, but I have a problem :

    Here is my code :
    var tasks = currentIds.Select(product => client.PutAsync(“url”, “json”, Encoding.UTF8, “application/json”)));
    response = await Task.WhenAll(tasks);

    But I do need associate each response to each product, I co not arrived to do this.

    Actually I’m just having a list of response, but I do not know which product is concerned for each response.

    Thank you

    1. Maciej,
      It’s a very good point. However, the full answer would have a small ‘but’. In ASP.Net Core there’s no SynchronizationContext, so there’s no point in using ConfigureAwait(false), however, it is advised to use when writing a library, that could be used in old ASP.Net.

      Have a look here for more info.

  8. Thanks very much, I’ve modified your code a bit to make it a generic extension and added an optional delay in between batches.

  9. Thanks very much, I’ve modified your code a bit to make it a generic extension and added an optional delay in between batches.
    public static class TaskExtentions
    public async static Task<IEnumerable> WhenAllInBatches(this Task task, IEnumerable<Task> tasks, int batchSize, int? batchPauseMiliSeconds = null )
    var results = new List();
    int numberOfBatches = (int)Math.Ceiling((double)tasks.Count() / batchSize);

    for (int i = 0; i < numberOfBatches; i++)
    var currentIds = tasks.Skip(i * batchSize).Take(batchSize);
    results.AddRange(await Task.WhenAll(tasks));
    if (batchPauseMiliSeconds.HasValue) await Task.Delay(batchPauseMiliSeconds.Value);

    return results;


    1. oops should be

      for (int i = 0; i < numberOfBatches; i++)
      var currentBatch = tasks.Skip(i * batchSize).Take(batchSize);
      results.AddRange(await Task.WhenAll(currentBatch));
      if (batchPauseMiliSeconds.HasValue) await Task.Delay(batchPauseMiliSeconds.Value);

Leave a Reply

Your email address will not be published. Required fields are marked *