Event-Driven Workflow Architecture¶

This guide explains the event-driven workflow architecture in the Flow Framework, as implemented in the temperature monitoring example.

Overview¶

The event-driven workflow pattern is designed to handle asynchronous, unpredictable events and process them through a directed graph of nodes. Unlike traditional request-response workflows, event-driven workflows continue processing indefinitely until a termination condition is met.

Architecture Diagram¶

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────────┐
│                 │     │                 │     │                     │
│  Event Sources  │────▶│  Event-Driven   │────▶│  Handler Workflows  │
│  (Sensors)      │     │  Workflow       │     │  (Actions)          │
│                 │     │                 │     │                     │
└─────────────────┘     └─────────────────┘     └─────────────────────┘
                              │                           │
                              │                           │
                              ▼                           ▼
                        ┌──────────────┐          ┌─────────────────┐
                        │              │          │                 │
                        │   Context    │◀─────────│    Feedback     │
                        │  Management  │          │     Loop        │
                        │              │          │                 │
                        └──────────────┘          └─────────────────┘

Temperature Monitoring Example¶

In the temperature monitoring example, the architecture is implemented as follows:

1. Event Sources¶

Multiple temperature sensors (simulated)
Send events to a channel-based event source
Events are asynchronous and unpredictable

// Create the event source with a buffer capacity of 100 events
let (source, sender) = ChannelEventSource::new(100);

2. Event-Driven Workflow¶

Receives events from the event source
Processes events through a chain of nodes
Maintains state between events using context
Handles errors and retries gracefully

let workflow = EventDrivenWorkflow::new()
    .source(source)
    .node("process_temperature", process_temperature)
    .node("check_threshold", check_threshold)
    .node("alert", alert_handler);

3. Event Classification¶

Analyzes temperature events
Categorizes them based on thresholds
Returns appropriate actions

// The classifier processes events and returns actions
let classifier = TemperatureClassifier::new(30.0, 10.0, 40.0);

4. Handler Workflows¶

Execute specific actions based on event processing
Can spawn additional workflows if needed
Report results back to the main workflow

// Add nodes for different temperature classifications
let normal_handler = NormalTempHandler::new();
let high_handler = HighTempHandler::new();
let low_handler = LowTempHandler::new();
let critical_handler = CriticalTempHandler::new();

5. Context Management¶

The context in event-driven workflows needs to handle:

Event history and aggregation
State persistence between events
Configuration and thresholds
Error tracking and recovery state

// The context maintains state across the workflow
struct MonitoringContext {
    temperature_history: HashMap<String, Vec<f32>>,
    alerts: Vec<String>,
    average_temperatures: HashMap<String, f32>,
}

6. Feedback Loop¶

The feedback loop enables:

Dynamic threshold adjustments
Learning from historical data
Adaptive event processing
System health monitoring

Flow of Events¶

Event Generation: Sensors generate temperature readings
Event Transmission: Readings are sent to the event-driven workflow
Event Classification: The classifier node categorizes the temperature
Action Routing: Based on the classification, the workflow routes to the appropriate handler
Action Execution: The handler performs the necessary actions
State Update: The context is updated with new information
Continuation/Termination: The workflow continues or terminates based on conditions

Implementation Considerations¶

When implementing an event-driven workflow, consider the following:

Event Source Design: How events are generated and retrieved
Event Classification Logic: Rules for categorizing events
Action Types: The set of possible actions that can result from events
Routing Logic: How events are routed through the workflow
Termination Conditions: When and how the workflow should terminate
Timeout Handling: How to handle hanging or slow event sources
Context Management: What state needs to be maintained across events

Best Practices¶

Error Handling
Implement proper error recovery mechanisms
Use retries with exponential backoff
Log errors for debugging and monitoring
State Management
Keep state minimal and focused
Consider persistence for critical state
Use atomic operations when updating state
Performance Optimization
Buffer events appropriately
Use async processing where beneficial
Implement backpressure mechanisms
Testing
Test event sources independently
Mock events for workflow testing
Verify error handling paths
Test state persistence and recovery
Monitoring
Track event processing latency
Monitor queue depths
Set up alerting for anomalies
Log important state transitions
Timeout Handling
Set appropriate timeouts for event processing
Implement deadletter queues
Handle slow event sources gracefully
Clean up resources on timeout
Context Management
Define clear boundaries for context data
Implement proper serialization
Handle context versioning
Clean up stale context data

Integration with Standard Workflows¶

Event-driven workflows can be integrated with standard request-response workflows:

Use event sources as triggers for standard workflows
Convert workflow results into events
Implement hybrid patterns for complex use cases

Event-driven workflows can be integrated with standard workflows using adapters:

// Create a nested event-driven workflow adapter
let nested_workflow = NestedEventDrivenWorkflow::new(
    workflow.clone(),
    TempAction::Complete,
    TempAction::Timeout,
);

This allows for combining both synchronous and asynchronous processing patterns.