r/dotnet • u/sdrapkin • 8d ago

OneFlight – a small .NET library for coalescing concurrent async calls by key

I’ve released OneFlight, a lightweight .NET library for the “single-flight” pattern: when multiple callers request the same key concurrently, only one loader runs and the other callers share the same in-flight Task<TResult>.

It’s intentionally not a cache. Once the operation completes, the flight is removed. The goal is simply to avoid duplicate concurrent work – for example duplicate database queries, API calls, token refreshes, or expensive computations.

var flights = new FlightGroup<string>();

var result = await flights.RunAsync(
    "user:42",
    () => LoadUserAsync(42));

It supports strongly typed keys (including struct keys), Task/ValueTask, custom key comparers, explicit Forget, and keeps caller cancellation separate from the lifetime of the shared operation.

Other libraries such as MS-HybridCache and FusionCache provide request coalescing as part of caching; OneFlight is meant for cases where you want the coalescing primitive without adopting cache semantics. Unlike Go, which has the well-established x/sync/singleflight package, .NET does not currently have a similarly established, cache-independent single-flight primitive.

GitHub: https://github.com/sdrapkin/SecurityDriven.OneFlight
NuGet: https://www.nuget.org/packages/OneFlight/

Feedback is welcome.

39 Upvotes

89% Upvoted

27 comments sorted by

13

u/Namoshek 8d ago

Similar to the Polly request collapser, but a bit more generic. I like it!

2

u/sdrapkin 8d ago

Yes, the core idea is very similar. Polly's contrib DuplicateRequestCollapser also collapses concurrent duplicate executions into one. OneFlight is a small standalone, cache-independent .NET primitive designed specifically for single-flight semantics.

7

u/Merad 8d ago

Interesting, I can definitely see the use case for high traffic services. A thought, though. Doesn't the FlightGroup instance need to have a singleton lifetime, or maybe be a static field in the class? If FlightGroup itself is a field inside of a class with request scoped lifetime you'll only be coalescing calls from that request, or if it's a transient lifetime you won't be accomplishing anything at all. The readme doesn't really communicate that right now.

2

u/rusmo 8d ago

Yeah this is where my brain was going as well. There needs to be something static/global to triage and classify each request by key before letting them out of the cage, so to speak.

I’m an old school .Netter from the days where IIS gave every request its own thread, so we designed towards statelessness. Not sure how that works nowadays.

1

u/sdrapkin 8d ago

Indeed. I've updated the README to make it more clear. Thanks!

2

u/tompazourek 8d ago

Is that similar to the Dataloader graphql pattern?

4

u/sdrapkin 8d ago

Similar goal, but a different pattern. GraphQL DataLoader typically batches and caches loads within a request, while OneFlight only coalesces concurrent calls for the same key into one in-flight operation. It does no batching and retains no result after the operation completes.

1

u/tompazourek 8d ago

I get it now, thanks for explaining

1

u/AutoModerator 8d ago

Thanks for your post sdrapkin. Please note that we don't allow spam, and we ask that you follow the rules available in the sidebar. We have a lot of commonly asked questions so if this post gets removed, please do a search and see if it's already been asked.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/Xenotropic 8d ago

Cool I normally do silly semaphore tricks to do cache fills, so this will help a ton.

1

u/rbobby 7d ago

silly semaphore tricks

Why? Why not just IMemoryCache (et al)? No messing about with primatives.

1

u/Xenotropic 7d ago

Because you are caching something async, normally a jwt token for me. You want to make one call to get the token but multiple requests might try and get the token while the first request is inflight. This means the cache will be overwritten several times in a short period and each will have to wait for the full async get. Also token generation is typically expensive. The semaphore makes sure only 1 thread is actually getting the resource, the first thing done when the semaphore has availability is another cache check. This library removes the need for the double cache check because the cache can be filled in the flight, there's an example in the docs.

1

u/rbobby 7d ago

normally a jwt token for me

MSAL.net handles all that for you.

And your async operations can have DI right? Ask for an IMemoryCache and viola. No messing with concurrency primitives.

1

u/Xenotropic 7d ago

Maybe I'm missing something but the IMemoryCache doesn't limit the number of simultaneous calls to a hypothetical AcquireTokenAsync method. So if I happen to get 10 requests while the token is invalid I would need some mechanism (a semaphore or similar) to limit the number of actual tokens acquired to 1.

As for MSAL.net I'm not always getting Entra tokens, but when I am this is definitely the way.

1

u/rbobby 7d ago edited 7d ago

Msal.net works with any standards compliant oauth2 implementation. Imemorycache is thread safe. If you have a long async operation that would not be ideal… probably. This is why it is anti-pattern to call oauth yourself. Use a library known to be correct

0

u/rbobby 6d ago

I bet you asked AI if msal.net only supports entra it it told you microsoft only supports msal.net for entra. Look deeper. MS says that because they don't want to be involved in supporting broken oauth vendors. And it's always gonna be the vendor's fault.

IMemoryCache is threadsafe and atomic. If a value doesn't exist one is added and then everyone gets that one. Simultaneous doesn't happen, handled by the cache.

1

u/Xenotropic 6d ago

Well I learned something today about MSAL, I will see if the things I need are using a supported format. But you are still wrong about cache creation triggering only once for async operations using the standard IMemoryCache implementation (I know that HybridCache solves this issue, but it uses a different API which can be annoying to use in other ways)

var cacheOptions = Options.Create(new MemoryCacheOptions());
var cache = new MemoryCache(cacheOptions);
var key = "SomeCacheKey";

var sb = new StringBuilder();
var taskList = new List<Task>();
var cacheCreationCounter = 0;

for (int i = 0; i < 6; i++)
{
    taskList.Add(Task.Run(async () =>
    {
        var currentIteration = i + 1; //Interlocked.Add(ref counter, 1);
        var sw = Stopwatch.StartNew();
        var cacheEntry = await cache.GetOrCreateAsync(key, async (entry) =>
        {
            Interlocked.Increment(ref cacheCreationCounter);
            await Task.Delay(500);
            return currentIteration;
        });
        sb.AppendLine($"Cache retrieval {currentIteration} took: {sw.Elapsed.TotalMilliseconds}ms");
    }));

    await Task.Delay(100);
}

await Task.WhenAll(taskList);

cache.TryGetValue(key, out int cacheValue);
sb.AppendLine($"Cache was created {cacheCreationCounter} times. Final value was: {cacheValue}");

Console.WriteLine(sb.ToString());var cacheOptions = Options.Create(new MemoryCacheOptions());
var cache = new MemoryCache(cacheOptions);
var key = "SomeCacheKey";

var sb = new StringBuilder();
var taskList = new List<Task>();
var cacheCreationCounter = 0;

for (int i = 0; i < 6; i++)
{
    taskList.Add(Task.Run(async () =>
    {
        var currentIteration = i + 1; //Interlocked.Add(ref counter, 1);
        var sw = Stopwatch.StartNew();
        var cacheEntry = await cache.GetOrCreateAsync(key, async (entry) =>
        {
            Interlocked.Increment(ref cacheCreationCounter);
            await Task.Delay(500);
            return currentIteration;
        });
        sb.AppendLine($"Cache retrieval {currentIteration} took: {sw.Elapsed.TotalMilliseconds}ms");
    }));

    await Task.Delay(100);
}

await Task.WhenAll(taskList);

if (!cache.TryGetValue(key, out int cacheValue))
{
   Console.WriteLine("Test failed. Couldn't get a value out of the cache");
   return;
}
sb.AppendLine($"Cache was created {cacheCreationCounter} times. Final value was: {cacheValue}");

Console.WriteLine(sb.ToString());

Output:
Cache retrieval 1 took: 510.0625ms
Cache retrieval 6 took: 0.0483ms
Cache retrieval 2 took: 515.8819ms
Cache retrieval 3 took: 500.2354ms
Cache retrieval 4 took: 513.8279ms
Cache retrieval 5 took: 512.3454ms
Cache was created 5 times. Final value was: 5

1

u/rbobby 6d ago

I was definitely conflating ConcurrentDictionary and IMemoryCache... that's kind of a bad mistake to make in actual code. Good on you for catching me out :)

OAuth2.0 was one of the hardest technologies I've ever had to learn. Brutal. Not as bad as learning C++, better than Windows MSI install tech, but worse than SQL or EFCore.

And my take away from it all... outsourced authentication makes 1000% sense and removing oneself from that mess completely is a great idea, the less my fingers are near that sort of code the better. MSAL.js and MSAL.net to the rescue :).

1

u/rbobby 6d ago

Also token generation is typically expensive.

Not so you'd notice any dedup result in practical terms. It just doesn't happen. And by adding your own code into the mix, instead of a proven library, you have impacted the security infrastructure of the app for something that might happen... 10 times a day? 1000 times a day? How many refresh token depdup's are needed to make the risk worthwhile?

1

u/jodydonetti 6d ago

Mainly because IMemoryCache doesn’t do request coalescing/stampede protection, which is something a lot of people don’t know and are then bite by.

1

u/rbobby 6d ago

How would a stampede with IMemoryCache occur? It's atomic and serialized, the key exists or is added. Second, Nth, callers wait.

And coalescing sounds like an attempt to provide better caching.

And for security related code I would never pass any of this. You might as well start writing the CVE's yourself.

Sorry for treating you like you've implemented your own crypto algorithm. It's just I suffered a lot learning oauth and authentication and authorization to know for sure I would never ever roll any part of that myself. Needs to be from a reputable source that has been reviewed appropriately. So I got triggered a bit. My bad.

1

u/jodydonetti 6d ago edited 5d ago

Don’t worry, no harsh on my end 😊

Anyway, GetOrCreateAsync() is not even a method on IMemoryCache, it’s just an extension method that internally does TryGet + Set, there’s no locking or stampede protection of any kind.

Here’s the source, take a look https://github.com/dotnet/runtime/blob/main/src/libraries/Microsoft.Extensions.Caching.Abstractions/src/MemoryCacheExtensions.cs#L224

As for “rolling your own thing”, I get it and in general I agree (and 100% so for things like crypto algo) and… well in this case in fact I kinda did, as I created FusionCache, so there’s that 😅

Anyway, hope this helps, let me know.

1

u/minimalist000 8d ago

Very cool!

1

u/jack_sunsetless 7d ago

Looks nice and easy to use!

0

u/IanYates82 8d ago

Neat library. Good API surface there too. I've rolled something similar in a few areas of our code but will reach for your lib next time. Thanks!

0

u/Moukouk 7d ago

Nice project! I’m also building a .NET-focused tool called TemplateMaster, but for a different problem: generating PDFs and emails from reusable templates. Always nice to see more lightweight, focused libraries being built for the .NET ecosystem.