How to Implement Semantic Caching for LLM Responses in ASP.NET Core 10 Using Redis and C# (2026)

A chatbot can answer “When can I return this item?” and “What is the return window?” with the same public policy. Reusing that answer can avoid another model call. Reusing “returns for international orders” against a domestic-order answer is a correctness failure, even when the embeddings are close.

This guide builds a Redis semantic cache for LLM responses in ASP.NET Core 10. It checks exact keys first, searches a Redis vector index on misses, and accepts a nearest neighbor only within an explicit distance threshold.

In this guide

1. What makes a semantic cache different?
2. Create the Redis index and C# app
3. Verify exact, semantic and miss behavior
4. Prevent false positives and leakage
5. FAQ and references

What makes a semantic cache different?

The API tries an exact hash, searches the scoped vector index, and uses a cached answer only under a tested distance threshold.

An exact cache requires an identical key. A semantic cache compares embeddings to find a related prompt. The nearest vector can still be the wrong answer, so the cache has a miss threshold and an eligibility policy. This fixture uses supplied three-component vectors to make results reproducible. A production caller should generate both vectors using the same trusted embedding model and never accept an unverified embedding alongside a private prompt.

Match	Example	When safe
Exact	Same prompt and compatible context	Scope and policy versions match
Semantic	Two public-policy paraphrases	Evaluation demonstrates acceptable false-positive rate
Miss	Different topic or unsupported scope	Send to the model only if authorized

Create the Redis index and C# app

Install .NET 10 and a Redis deployment with vector search commands. Redis 8 supports the Query Engine. For an older server, verify that Redis Search is installed. A plain server without FT.CREATE and FT.SEARCH cannot run this sample. See Redis vector search.

Start Redis 8 and create a cosine vector index

docker run --rm -p 127.0.0.1:6379:6379 redis:8
redis-cli FT.CREATE answers ON HASH PREFIX 1 llm: SCHEMA scope TAG version TAG \
  embedding VECTOR FLAT 6 TYPE FLOAT32 DIM 3 DISTANCE_METRIC COSINE

The fields must match the C# hash schema. Each vector contains three little-endian FLOAT32 values. The FLAT index favors simple, exact nearest-neighbor search for the fixture. Evaluate HNSW separately when the index is large. The cache index projects only the answer and distance. Redis’s cosine distance is smaller for closer vectors. Here a distance of 0.02 is a demonstration threshold, not a production recommendation.

Lab.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup><TargetFramework>net10.0</TargetFramework><ImplicitUsings>enable</ImplicitUsings><Nullable>enable</Nullable></PropertyGroup>
  <ItemGroup><PackageReference Include="StackExchange.Redis" Version="3.4.31" /></ItemGroup>
</Project>

Program.cs

using StackExchange.Redis;
using System.Buffers.Binary;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;

var b = WebApplication.CreateBuilder(args);
var config = b.Configuration["Redis"] ?? "127.0.0.1:6379";
var version = b.Configuration["CacheVersion"] ?? "v1";
if (version.Length > 32 || !version.All(char.IsAsciiLetterOrDigit))
    throw new InvalidOperationException("Use an alphanumeric cache version.");
b.Services.AddSingleton<IConnectionMultiplexer>(_ => ConnectionMultiplexer.Connect(config));
b.WebHost.ConfigureKestrel(o => o.Limits.MaxRequestBodySize = 8192);
var app = b.Build();
app.MapPost("/cache/lookup", async (Lookup q, IConnectionMultiplexer conn, CancellationToken ct) =>
{
    if (!Valid(q.Prompt, q.Vector)) return Results.BadRequest();
    var db = conn.GetDatabase();
    var exact = await db.HashGetAsync(Key(q.Prompt), "answer").WaitAsync(ct);
    if (!exact.IsNull) return Results.Ok(new { hit = "exact", answer = exact.ToString() });

    // Loopback fixture has a fixed scope. A real API derives scope from validated identity.
    var res = await db.ExecuteAsync("FT.SEARCH", "answers",
        $"(@scope:{{public}} @version:{{{version}}})=>[KNN 1 @embedding $vector AS dist]",
        "PARAMS", "2", "vector", Bytes(q.Vector),
        "SORTBY", "dist", "RETURN", "2", "answer", "dist", "DIALECT", "2").WaitAsync(ct);
    var list = (RedisResult[])res!;
    if (list.Length < 3) return Results.NotFound();
    var values = (RedisResult[])list[2]!;
    string? answer = null;
    double distance = double.PositiveInfinity;
    for (var i = 0; i + 1 < values.Length; i += 2)
    {
        var field = values[i].ToString();
        if (field == "answer") answer = values[i + 1].ToString();
        if (field == "dist" && double.TryParse(values[i + 1].ToString(),
            System.Globalization.CultureInfo.InvariantCulture, out var d)) distance = d;
    }
    return distance <= 0.02 && answer is not null
        ? Results.Ok(new { hit = "semantic", answer, distance }) : Results.NotFound();
});
app.MapPut("/cache", async (Entry value, IConnectionMultiplexer conn, CancellationToken ct) =>
{
    if (!Valid(value.Prompt, value.Vector) || string.IsNullOrWhiteSpace(value.Answer) ||
        value.Answer.Length > 2000) return Results.BadRequest();
    var tx = conn.GetDatabase().CreateTransaction();
    var write = tx.HashSetAsync(Key(value.Prompt),
        [new("scope","public"), new("version",version), new("embedding",Bytes(value.Vector)),
         new("answer",value.Answer)]);
    var ttl = tx.KeyExpireAsync(Key(value.Prompt), TimeSpan.FromMinutes(5));
    if (!await tx.ExecuteAsync().WaitAsync(ct)) return Results.Problem(statusCode: 503);
    await write.WaitAsync(ct);
    await ttl.WaitAsync(ct);
    return Results.NoContent();
});
await app.RunAsync();

string Key(string prompt) => $"llm:public:{version}:" +
    Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(prompt)));
static bool Valid(string? p, float[]? v) => p is { Length: >= 1 and <= 1000 } &&
    v is { Length: 3 } && v.All(x => float.IsFinite(x) && Math.Abs(x) <= 1) &&
    v.Any(x => Math.Abs(x) >= 0.0001f);
static byte[] Bytes(float[] v)
{
    var bytes = new byte[12];
    for (int i = 0; i < v.Length; i++)
        BinaryPrimitives.WriteSingleLittleEndian(bytes.AsSpan(i * 4, 4), v[i]);
    return bytes;
}
record Lookup(string Prompt, float[] Vector);
record Entry(string Prompt, float[] Vector, string Answer);

The API is a single-scope loopback fixture. The hash key includes a version and SHA-256 digest of the prompt. The KNN query filters on scope and version before choosing the nearest answer. The hash and its TTL are set in one Redis transaction. Changing CacheVersion causes a logical invalidation, while old entries expire. A raw prompt hash is still susceptible to guessing if prompts are predictable. Avoid storing sensitive data, and consider a scoped keyed digest where appropriate.

Verify exact, semantic and miss behavior

Store one answer and try a similar prompt

dotnet run --no-launch-profile -- --urls http://127.0.0.1:5080

# In another terminal:
curl -i -X PUT http://127.0.0.1:5080/cache -H 'Content-Type: application/json' \
  -d '{"prompt":"What is the return window?","vector":[1,0,0],"answer":"30 days."}'
curl -i http://127.0.0.1:5080/cache/lookup -H 'Content-Type: application/json' \
  -d '{"prompt":"When can I return this?","vector":[1,0.01,0]}'

Expect 204 for storage and a semantic hit for the near vector. Use the original prompt to verify an exact hit, then try a distinct prompt with [0,1,0] and expect 404. Repeat after expiration or after changing the version. Check that invalid vectors and oversized answers are rejected. An indexing lag after writing can briefly affect vector search. Test that behavior for your deployment.

The C# project compiled on .NET 10. The Redis Query Engine and the example hit and miss requests were not executed here.

Prevent false positives and leakage

The real cache scope should include authorized tenant, retrieval-policy version, prompt template version, model and output contract, locale, and any other context that changes the answer. Use safe encoded scope identifiers in Redis queries. Never share private answers through the public cache. A matching embedding does not authorize a data read. Avoid caching time-dependent answers, private account results or actions.

Evaluate threshold precision on positive paraphrases and difficult negative pairs. Include negation, different dates, geographic scope and almost identical product names. Choose an acceptable false-hit rate before optimizing hit rate. Measure latency of embedding plus lookup against a fresh model call. Provide a safe miss when the cache is unavailable. Do not allow a lookup cancellation to imply a submitted Redis mutation was rolled back.

Symptom	Check
Unknown FT command	Redis Query Engine availability
Vector mismatch	DIM, FLOAT32 layout and model version
Wrong tenant answer	Scope filter and authenticated identity
Outdated answer	TTL and policy-version invalidation
Similar text gives a bad hit	Negative examples and threshold policy

FAQ and references

Can I cache every model answer? Restrict reuse to eligible tasks with stable inputs and public or properly isolated data.

Does a higher similarity threshold always help? It trades fewer false hits for more misses. Measure using domain-specific pairs.

Is exact cache invalidation enough? Both exact keys and vector queries must use the new scope and version.

• Redis FT.SEARCH.
• Redis FT.CREATE.
• ASP.NET PB Redis distributed caching guide.

Start with stable public FAQs. Add semantic reuse only after the negative fixture shows the wrong answer is reliably rejected.