Intent Metadata and Context Propagation

Introduction

The intent pattern focuses on specifying the ‘what’ (goal) rather than the ‘how’ (process), and in distributed systems, achieving this goal often requires careful consideration of consistency and tracing. This section delves into the nuances of enriching intent payloads with distributed tracing IDs and consistency floors, ensuring that the execution of intents is both traceable and consistent with the desired system behavior.

Consistency Floors and System Behavior

Consistency floors are metadata attributes within an intent payload that define the minimum required consistency level for the execution of that specific intent. The choice of consistency floor has significant implications on system behavior, particularly in the presence of network partitions. The following table illustrates the trade-offs associated with different consistency floors:

Distributed Tracing and Context Propagation

Distributed tracing is crucial for understanding the flow of intents through a distributed system. OpenTelemetry, with its W3C Trace Context standard, provides a robust framework for propagating tracing context across service boundaries. The traceparent header is instrumental in this process, carrying the TraceID, SpanID, and trace flags necessary for reconstructing the intent flow. To integrate distributed tracing with intent payloads, header injection is used. This involves inserting the tracing context into the transport-layer headers of a message or request. In Kafka-based systems, this can be achieved through the use of TextMapPropagator, an OpenTelemetry interface designed for injecting and extracting cross-cutting concerns into carriers like Kafka headers.

Implementation Considerations

Implementing intent metadata and context propagation requires careful consideration of the underlying system architecture. Java Records, with their canonical constructors and serialization safety, are well-suited for representing intent payloads. The following Java Record example demonstrates how to capture intent metadata and tracing context during header injection:

public record IntentHeader(
    String traceId,
    String spanId,
    ConsistencyLevel consistencyFloor
) {
    public static IntentHeader fromMetadata(Metadata metadata) {
        // Logic to extract W3C traceparent and custom consistency floor
        return new IntentHeader(
            metadata.getTraceId(),
            metadata.getSpanId(),
            metadata.getConsistencyFloor()
        );
    }
}

enum ConsistencyLevel {
    EVENTUAL, CAUSAL, SEQUENTIAL, LINEARIZABLE
}

This example illustrates how intent headers can be constructed from metadata, incorporating both tracing context and consistency floors.

Conclusion

Enriching intent payloads with distributed tracing IDs and consistency floors is essential for ensuring the traceability and consistency of intent execution in distributed systems. By understanding the trade-offs associated with different consistency floors and leveraging distributed tracing standards like OpenTelemetry, developers can design and implement robust intent-based architectures that meet the demands of modern distributed applications.

Sources

[1] https://stackoverflow.com/questions/74443714/context-propagation-opentelemetry-get-traceid-from-kafka-record [2] https://last9.io/blog/opentelemetry-context-propagation/ [3] https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/1113