How data moves through the ObjectStack system — from defineStack() configuration to database queries, realtime events, and metadata resolution

Data Flow Diagrams

This guide provides detailed diagrams showing how data moves through every layer of the ObjectStack system — from developer configuration to database operations and back to the client.

Architecture Overview: For a high-level view of how protocol layers connect, see the Protocol Relationship Diagram.

1. Configuration Flow: defineStack() → Database

How your objectstack.config.ts becomes a running application.

sequenceDiagram
    participant Dev as Developer
    participant Cfg as objectstack.config.ts
    participant CLI as CLI / Build
    participant Man as Manifest
    participant Ker as Kernel
    participant Plu as Plugins
    participant Drv as Driver
    participant DB as Database

    Dev->>Cfg: defineStack({ objects, views, flows })
    Cfg->>CLI: Export configuration
    CLI->>Man: Compile & validate (Zod)

    alt Validation Failed
        Man-->>CLI: Schema errors
        CLI-->>Dev: Error output with suggestions
    end

    CLI->>Ker: Boot Kernel with Manifest
    Ker->>Plu: Initialize plugins
    Plu-->>Ker: Contracts registered

    Ker->>Drv: Connect to datasource
    Drv->>DB: Run migrations (if needed)
    DB-->>Drv: Schema ready

    Ker-->>CLI: Server ready
    CLI-->>Dev: Listening on http://localhost:3000

Stage	What Happens
`defineStack()`	Developer declares objects, fields, views, flows in TypeScript (array or map format)
Compile	CLI validates all definitions against Zod schemas
Manifest	Compiled metadata package containing all configuration
Kernel Boot	Kernel loads the manifest and initializes the plugin graph
Plugin Init	Each plugin registers its contracts (DataEngine, AuthService, etc.)
Driver Connect	Database driver connects and runs any pending migrations

2. Read Flow: Client → API → Database → Response

How a GET request for a single record flows through the system.

sequenceDiagram
    participant C as Client
    participant API as REST / GraphQL
    participant Auth as Auth Middleware
    participant K as Kernel
    participant Sec as Security
    participant QE as Query Engine
    participant H as Hooks
    participant D as Driver
    participant DB as Database

    C->>API: GET /api/v1/data/task/tsk_123
    API->>Auth: Verify JWT token
    Auth-->>API: AuthUser resolved

    API->>K: read('task', 'tsk_123', context)

    K->>Sec: checkPermission('task', 'read', user)
    Sec-->>K: ✓ Allowed

    K->>QE: buildQuery('task', { id: 'tsk_123' })
    QE->>Sec: applyRLS(query, user)
    Sec-->>QE: Filtered query

    QE->>D: execute(filteredQuery)
    D->>DB: SELECT * FROM tasks WHERE id = 'tsk_123' AND ...
    DB-->>D: Row data
    D-->>QE: Typed record

    QE-->>K: DataRecord

    K->>H: afterRead hooks
    H-->>K: Transformed record

    K->>Sec: applyFieldSecurity(record, user)
    Sec-->>K: Fields filtered by profile

    K-->>API: Final record
    API-->>C: 200 OK { data: { ... } }

Row-Level Security (RLS): Security filters are applied at the query level, not after fetching. This means the database only returns rows the user is allowed to see — efficient even on large tables.

3. Write Flow: Client → Validation → Hooks → Database → Response

How a POST (create) request is processed with full validation and hook execution.

sequenceDiagram
    participant C as Client
    participant API as REST API
    participant K as Kernel
    participant V as Validator
    participant BH as Before Hooks
    participant D as Driver
    participant DB as Database
    participant AH as After Hooks
    participant EB as Event Bus
    participant WS as WebSocket

    C->>API: POST /api/v1/data/task { data: {...} }
    API->>K: create('task', data, context)

    K->>V: validate(schema, data)

    alt Validation Failed
        V-->>K: ZodError
        K-->>API: 400 VALIDATION_ERROR
        API-->>C: { error: { details: [...] } }
    end

    V-->>K: ✓ Valid data

    K->>BH: beforeCreate hooks
    Note over BH: Transform data, enforce rules,<br/>set defaults, check permissions

    alt Hook Throws Error
        BH-->>K: BusinessError
        K-->>API: 4xx/5xx Error
        API-->>C: Error Response
    end

    BH-->>K: Transformed data

    K->>D: insert('task', transformedData)
    D->>DB: INSERT INTO tasks (...) VALUES (...)
    DB-->>D: Inserted row with ID
    D-->>K: DataRecord { id: 'tsk_456', ... }

    K->>AH: afterCreate hooks
    Note over AH: Send notifications,<br/>update counters,<br/>trigger workflows

    K->>EB: emit('task.created', record)
    EB->>WS: Broadcast to subscribers

    K-->>API: Created record
    API-->>C: 201 Created { data: { id: 'tsk_456', ... } }

4. Metadata Flow: defineStack() → IDE Autocomplete

How object definitions flow from configuration to runtime API and IDE tooling.

flowchart LR
    subgraph "Development"
        A["defineStack()"] -->|"TypeScript"| B[objectstack.config.ts]
        B -->|"compile"| C[Zod Validation]
    end

    subgraph "Build Time"
        C -->|"generate"| D[JSON Schema]
        C -->|"generate"| E[TypeScript Types]
        D -->|"feed"| F[IDE Autocomplete]
        E -->|"feed"| F
    end

    subgraph "Runtime"
        C -->|"bundle"| G[Manifest]
        G -->|"load"| H[Schema Registry]
        H -->|"serve"| I["GET /api/v1/meta/objects"]
        I -->|"discover"| J[Client SDK]
        J -->|"generate"| K[Typed Client]
    end

Output	Used By	Purpose
JSON Schema	VS Code, IntelliJ	Autocomplete and validation in `*.object.ts` files
TypeScript Types	Plugin developers	Type-safe access to object definitions
Manifest	Kernel	Runtime metadata for query validation and execution
Metadata API	Client SDK	Dynamic object/field discovery
Typed Client	Frontend apps	`client.task.create({ title: '...' })`

5. Realtime Flow: Database Change → Client

How data changes propagate to connected clients in real time.

sequenceDiagram
    participant W as Writer Client
    participant API as API Layer
    participant K as Kernel
    participant DB as Database
    participant EB as Event Bus
    participant Sub as Subscription Manager
    participant R1 as Reader Client 1
    participant R2 as Reader Client 2

    Note over R1,R2: Clients subscribe via WebSocket

    R1->>Sub: subscribe('task', { filter: { project: 'prj_1' } })
    R2->>Sub: subscribe('task', { filter: { assigned_to: 'usr_2' } })
    Sub-->>R1: Subscribed (channelId: ch_1)
    Sub-->>R2: Subscribed (channelId: ch_2)

    W->>API: PATCH /api/v1/data/task/tsk_123 { status: 'done' }
    API->>K: update(...)
    K->>DB: UPDATE tasks SET status = 'done' WHERE id = 'tsk_123'
    DB-->>K: Updated record

    K->>EB: emit('task.updated', { record, changes })

    EB->>Sub: Route event to matching subscriptions
    Sub->>Sub: Check filters

    Note over Sub: tsk_123.project = 'prj_1' ✓ → ch_1
    Note over Sub: tsk_123.assigned_to = 'usr_2' ✓ → ch_2

    Sub->>R1: { event: 'task.updated', data: { id: 'tsk_123', ... } }
    Sub->>R2: { event: 'task.updated', data: { id: 'tsk_123', ... } }

    K-->>API: Updated record
    API-->>W: 200 OK { data: { ... } }

Subscription Filtering: The Subscription Manager applies each client's filter server-side. Clients only receive events for records that match their subscription criteria — no over-fetching.

6. Query Execution Flow

How a structured query is translated to SQL and executed.

flowchart TD
    A[QueryInput] -->|parse| B[Query Parser]
    B -->|validate| C[Schema Validator]
    C -->|optimize| D[Query Optimizer]
    D -->|secure| E[RLS Filter Injection]
    E -->|translate| F[SQL Generator]
    F -->|execute| G[Database]
    G -->|map| H[Result Mapper]
    H -->|paginate| I[QueryResult]

Stage	Description
Query Parser	Parses the JSON `QueryInput` into an AST
Schema Validator	Verifies field names and operators against the object schema
Query Optimizer	Reorders filters, selects indexes, applies batch strategies
RLS Filter Injection	Appends row-level security conditions based on the user's profile
SQL Generator	Translates the AST to the target database dialect (PostgreSQL, MySQL, SQLite)
Database	Executes the query and returns raw rows
Result Mapper	Converts raw rows to typed `DataRecord` objects
Pagination	Calculates `total`, `hasMore`, `offset` for the response

7. Hook Execution Order

The precise order in which hooks fire during each operation.

flowchart TD
    subgraph "Create Operation"
        C1[beforeValidate] --> C2[validate]
        C2 --> C3[afterValidate]
        C3 --> C4[beforeCreate]
        C4 --> C5["DB INSERT"]
        C5 --> C6[afterCreate]
        C6 --> C7[afterCommit]
    end

    subgraph "Update Operation"
        U1[beforeValidate] --> U2[validate]
        U2 --> U3[afterValidate]
        U3 --> U4[beforeUpdate]
        U4 --> U5["DB UPDATE"]
        U5 --> U6[afterUpdate]
        U6 --> U7[afterCommit]
    end

    subgraph "Delete Operation"
        D1[beforeDelete] --> D2["DB DELETE"]
        D2 --> D3[afterDelete]
        D3 --> D4[afterCommit]
    end

Hook	Phase	Can Modify Data?	Can Abort?
`beforeValidate`	Pre-validation	✅ Yes	✅ Yes
`validate`	Validation	❌ No	✅ Yes
`afterValidate`	Post-validation	✅ Yes	✅ Yes
`beforeCreate/Update`	Pre-persist	✅ Yes	✅ Yes
`afterCreate/Update/Delete`	Post-persist	❌ No (record is saved)	❌ No
`afterCommit`	Post-transaction	❌ No	❌ No

Side Effects: Place side effects (emails, notifications, external API calls) in afterCommit hooks. These run after the database transaction is committed, so you won't send a notification for a record that was rolled back.

Data Flow Diagrams

On this page