Improve this page

Global Last Update Index

The global last-update index tracks the timestamp of the most recent change for every entity in a cache, grouped by a property you nominate. It powers incremental sync APIs: "give me everything that changed since t." Every block on this page runs against the sample domain and is asserted in samples/Prague.Samples/Advanced/GlobalLastUpdateIndex.cs.

Why it's separate

Prague's normal indices are scoped to one cache. The global index sits above caches: one IDataCacheGlobalLastUpdateIndex<TGroupKey> instance can collect timestamps from many caches whose entities share a TGroupKey. In the sample domain Customer and Order both carry a customer id; one global index keyed by it becomes the single source of truth for "what changed for customer 42 since last sync" — a profile edit and a new order alike.

Declaring the index

Define a partial class that implements IDataCacheGlobalLastUpdateIndex<TGroupKey>. The interface is implemented for you by the generator:

// docs: the global last-update index shared by Customer (group key = its primary key) and Order
// (group key = the CustomerId foreign key). "Everything that changed for customer 42 since T."
namespace Prague.Samples.Models;

using Prague.Core;

public partial class CustomerActivityIndex : IDataCacheGlobalLastUpdateIndex<int>;

Then mark the group-key property on each contributing cache with [DataCacheGlobalLastUpdateIndex<TIndex>]. On Customer the group key is the primary key; on Order it is the CustomerId foreign key:

// docs: the parent side of the Order → Customer join. Carries a Unique index (Email), two Many
// indices used as join-filter and Or examples (Country, Tier), and is the primary-key group of
// CustomerActivityIndex, which gives OrderCache/CustomerCache the generated UpdatedAfter(...) verb.
namespace Prague.Samples.Models;

using Prague.Core;

public enum CustomerTier {
    Standard,
    Gold,
}

[DataCache]
public partial class Customer {
    [DataCacheKey]
    [DataCacheGlobalLastUpdateIndex<CustomerActivityIndex>]        // group key == primary key → customerCache.Query().UpdatedAfter(...)
    public required int Id { get; init; }

    [DataCacheIndex(DataCacheIndexType.Unique)]                    // 1:1 — alternate key
    public required string Email { get; init; }

    [DataCacheIndex(DataCacheIndexType.Many)]
    public required string Country { get; init; }

    [DataCacheIndex(DataCacheIndexType.Many)]
    public required CustomerTier Tier { get; init; }

    public required string Name { get; init; }
}

public enum OrderStatus {
    Pending,
    Paid,
    Shipped,
    Cancelled,
}

[DataCache]                                                     // emits OrderCache; [DataCache("OrdersCache")] renames it
[DataCacheTopic("orders.[v:env]")]                              // Kafka topic; [v:env] resolves from KafkaCachesOptions.Vars
[DataCacheSort("ByCreatedAt", "CreatedAtUnixMs:-1", "OrderId:1")] // emits OrderCache.ByCreatedAtComparer (newest first)
public partial class Order {                                    // must be partial
    [DataCacheKey]
    public required string OrderId { get; init; }

    [DataCacheForeignKey<Customer>(DataCacheJoinType.ManyToOne)]   // many orders → one customer; also indexes CustomerId
    [DataCacheGlobalLastUpdateIndex<CustomerActivityIndex>]        // every order write stamps its customer's activity
    public required int CustomerId { get; init; }

    [DataCacheIndex(DataCacheIndexType.Range)]                     // sorted — Gte / Lt / … queries
    public required long CreatedAtUnixMs { get; init; }

    [DataCacheIndex(DataCacheIndexType.Many)]                      // 1:N — many orders share a status
    public required OrderStatus Status { get; init; }

    [DataCacheHasValueIndex]                                       // key set: WithSettledAtUnixMs() = settled (non-null)
    public long? SettledAtUnixMs { get; init; }

    public required decimal Total { get; init; }
}

Both caches now report into CustomerActivityIndex on every write.

A filtered variant, [DataCacheGlobalLastUpdateIndex<TIndex, TFilter>], tracks only the entities a readonly struct : IDataCacheGlobalLastUpdateFilter<TEntity> admits (a static abstract bool Include(TEntity)); membership is re-evaluated on every update, so an entity that stops qualifying leaves the index.

Timestamp source

By default the index uses the timestampMs argument passed to AddOrUpdate(value, timestampMs) (which, for Kafka-backed caches, is the message timestamp; the single-argument AddOrUpdate(value) stamps the current time). If you'd rather drive it from an entity property, name it in the attribute — [DataCacheGlobalLastUpdateIndex<CustomerActivityIndex>(nameof(UpdatedAtUnixMs))] — and give it type long (Unix milliseconds), DateTime or DateTimeOffset.

Resolving the index

The index is a singleton: resolve it from DI when the caches come from AddKafkaCaches, or from the registry otherwise.

var index = _registry.GetGlobalIndex<CustomerActivityIndex>();   // from DI: serviceProvider.GetRequiredService<CustomerActivityIndex>()

Its public surface is IDataCacheGlobalLastUpdateIndex<TGroupKey>:

var hasAny = index.TryGetMax(out var newestUnixMs, out var newestCustomerId);   // the most recent change and its group
var oldest = index.TryGetMin(out var oldestUnixMs);
var rowsOf42 = index.GetEntitiesCount(42);                                      // entities tracked under the group, across caches
var since = index.Index.GetValuesBetween(T0 + 1_000, T0 + 2_000);               // long / DateTime / DateTimeOffset forms

TryGetMin / TryGetMax come with and without the out TGroupKey key parameter and return false on an empty index. GetEntitiesCount(key) counts the entities tracked under a group across every contributing cache. Index is the underlying LastUpdatedIndex<TGroupKey> with GetValuesGt / GetValuesGte / GetValuesLt / GetValuesBetween in long, DateTime and DateTimeOffset forms; they return the group keys that changed in the window.

Querying

When the group key is the cache's primary key (Customer above), the generator emits UpdatedAfter(...) on the query builder — the same thing as UseIndex(globalIndex, …) on the raw cache:

using var changed = _customers.Query()
    .UpdatedAfter(lastSyncUnixMs)                               // strictly greater than lastSyncUnixMs
    .ExecutePooled();

The customer 42 row comes back although nothing was written to the customer itself after the sync point — its order was, and the order's write stamps the group. The out long max form reports the highest timestamp among the rows returned, which is the watermark to commit for the next call:

using var window = _customers.Query()
    .UpdatedAfter(lastSyncUnixMs, out var maxObservedUnixMs)    // highest timestamp among the rows returned
    .ExecutePooled();

CommitSyncPoint(maxObservedUnixMs);                             // the safe watermark for the next call

On the eager builder UpdatedAfter takes a long or a DateTimeOffset, each with the out long max twin. The prepared builder adds the bound (after, untilInclusive) window form in long, DateTime and DateTimeOffset, and the argument forms UpdatedAfter(static a => a.Since) and UpdatedAfter(static a => a.Since, static a => a.Until). It binds only when the global index is keyed by the cache's primary key type.

When the group key is not the primary key (Order above), read the index itself: which groups changed, then query each group's rows through its own index.

foreach (var customerId in index.Index.GetValuesGt(lastSyncUnixMs)) {        // which groups changed…
    using var orders = _orders.Query().WithCustomerId(customerId).ExecutePooled();   // …then each group's rows via its own index
    foreach (var order in orders)
        touched.Add(order.OrderId);
}

Watermark semantics

• UpdatedAfter(t) returns entities with timestamp strictly greater than t.
• There is no snapshot isolation: a write landing during the query may or may not be included. The out long max form reports the highest timestamp among the rows actually returned, which is the safe watermark to commit.
• The index is not transactional across caches: if you query several caches separately, the watermarks you observe may differ. Take the minimum of the per-cache watermarks as your sync point.

When to use it

The global index has a per-write cost (a sorted-index update per contributing cache). It pays off when:

• You serve incremental APIs (/api/changes?since=...).
• You need to forward changes to a downstream system on a polling cadence.
• You feed an external search index that wants delta updates.

For "tail the topic in real time" use-cases, ICacheAfterHandler is the cheaper hook.

Next

• Performance Tuning.