Skip to main content

Component Ecosystem Overview

This document summarizes how BubuStack's component ecosystem fits together, including SDK usage, contracts, packaging/registry patterns, and reliability semantics. It links to the detailed docs and repo references you'll use day-to-day.

Who this is for​

  • Component authors building Engrams and Impulses.
  • SDK users wiring triggers, inputs, and outputs.
  • Platform operators packaging and distributing components.

What you'll get​

  • Where the SDKs live and what they cover.
  • The core contracts that define inputs, outputs, and errors.
  • How components are packaged and discovered.
  • The reliability guarantees you must design around.

The rule: do one thing and do it well​

An Engram does one job. A "fetch-and-summarize" Engram is two Engrams. A "summarize-and-notify" Engram is two Engrams. If you can describe what it does with "and", split it.

Stories compose Engrams into pipelines. This is the same principle as Unix: curl fetches, jq transforms, notify-send alerts. Each program is small, testable, and reusable. The shell (Story) wires them together.

Impulses follow the same rule. A cron-impulse fires on a schedule. A github-webhook-impulse fires on GitHub events. A kubernetes-impulse fires on cluster events. Each trigger does one thing.

When you follow this, every component becomes independently testable, swappable, and reusable across Stories. When you don't, you get monoliths with implicit coupling that break when requirements change.

Quick start: authoring flow​

  1. Create a component repo (see Building Engrams for structure).
  2. Implement an Engram or Impulse using bubu-sdk-go.
  3. Define schemas in EngramTemplate or ImpulseTemplate.
  4. Validate with bubu-sdk-go/testkit and bubu-sdk-go/conformance.
  5. Build a Docker image. Until the registry release lands, publish the template manifest as a GitHub Release asset and have users kubectl apply it directly before they reference it from an Engram or Impulse.

Repo map (where to look)​

RepositoryPurpose
tractatusProtobuf contracts for gRPC transport.
coreShared contracts, templating engine, transport protocol/env helpers, and connector runtime helpers.
bobrapetCRDs, controllers, webhooks, and the runtime contract docs.
bubu-sdk-goGo SDK, testkit, and conformance helpers.
bobravoz-grpcStreaming transport operator: gRPC hub, topology analysis, connector lifecycle.
bubuilderWeb console and REST API server.
bubu-registryPre-release registry work and bubu CLI for future scaffolding, discovery, install, and publishing flows.
engrams/*Engram implementations (batch and streaming data processors).
impulses/*Impulse implementations (event-driven workflow triggers).
examplesSample Stories and workflows.

For the full module dependency graph and layering rules, see Architecture.

Component SDKs​

The Go SDK is the primary authoring surface for components. It provides three entry points:

Entry pointUse caseKubernetes workload
sdk.StartBatch[C, I]Finite tasks with clear start/endJob
sdk.StartStreaming[C]Continuous processing with gRPC bidirectional streamingDeployment
sdk.RunImpulse[C]Long-running trigger that submits durable StoryTrigger requests from external eventsDeployment

The SDK also provides:

  • testkit — Local harness for testing component behavior without a cluster (testkit.BatchHarness, testkit.StreamHarness).
  • conformance — Contract test suites that all components should pass (conformance.BatchSuite, conformance.StreamSuite).

Use the SDK to:

  • Parse runtime context (story, run, step, transport settings).
  • Emit outputs, logs, and structured errors.
  • Integrate with streaming transports when enabled.
  • Handle storage ref resolution (automatic for inputs; transparent offloading for large outputs).

For a full development guide with interfaces, code examples, testing patterns, and template definitions, see Building Engrams and Go SDK.

Available components​

Engrams (data processors)​

Engrams process data — they receive inputs, execute logic, and produce outputs. The current public catalog in this workspace is 13 Engrams and 4 Impulses.

EngramPatternDescription
conversation-memory-engramBatchIn-memory per-session conversation history with TTL-based eviction
http-request-engramBatchHTTP requests with pagination, retries, and response parsing
json-filter-engramBatchFilter JSON payloads (inline or offloaded) with JSONPath
map-reduce-adapter-engramBatchDynamic fan-out with child StoryRuns and result aggregation
materialize-engramBatchEvaluate Go templates with Sprig against a provided context
mcp-adapter-engramBothModel Context Protocol adapter (streamable HTTP and stdio)
openai-chat-engramBothOpenAI chat completions (batch and streaming)
openai-stt-engramStreamingOpenAI speech-to-text with real-time transcription events
openai-tts-engramStreamingOpenAI text-to-speech with audio chunk streaming
silero-vad-engramStreamingVoice activity detection using Silero ONNX models
livekit-bridge-engramStreamingLiveKit room participant bridge for audio/data channels
livekit-turn-detector-engramStreamingConversational turn detection for voice pipelines
text-emitter-engramBatchEmits a one-time text message when the stream starts

Impulses (event triggers)​

Impulses trigger workflows — they listen for external events and submit StoryTrigger requests that resolve to StoryRuns.

ImpulseTrigger sourceDescription
cron-impulseCron scheduleTime-based triggers with cron expressions
github-webhook-impulseGitHub webhooksPR, push, issue, and other GitHub events
kubernetes-impulseKubernetes eventsPod crashes, deployments, resource changes
livekit-webhook-impulseLiveKit webhooksRoom and participant lifecycle events

Browse all components: Engrams · Impulses

Contracts and schemas​

Contracts live in two places:

  • CRD schemas define the API surface for Stories, Runs, Engrams, and Impulses. These are authored in bobrapet/api/ and documented in CRD Design.
  • Runtime contracts define environment variables, labels, annotations, and structured error payloads for SDKs. See core/contracts and Error Contract.

Streaming components also follow the streaming message contract:

The latest structured streaming contract is:

  • inbound packets arrive as engram.InboundMessage and must be acknowledged with Done()
  • structured JSON outputs keep canonical bytes in Payload and mirror them into Binary with MimeType: application/json
  • raw Binary without Payload is reserved for opaque media or non-JSON blobs

Packaging and registry​

BubuStack has registry work in bubu-registry, but until the public registry release lands, the supported user path is GitHub Release based: each component repo publishes a versioned Engram.yaml or Impulse.yaml asset, and users install those templates directly with kubectl apply.

The bubu CLI and registry repo are the direction of travel for discovery and publishing, but public catalog coverage, SemVer-aware resolution, signing, provenance, digest pinning, and stronger package guarantees are still release work. See Installing Components for the current release-asset flow and Roadmap for the next registry hardening steps.

Reliability semantics​

Reliability guarantees are defined in Durable Semantics. Key themes:

  • StoryTrigger provides the durable trigger-admission boundary for external events.
  • EffectClaim provides the durable reservation authority for cross-process effect execution.
  • Step execution is at-least-once; components must handle retries.
  • Redrive and rerun behavior is driven by annotations and controller logic.

When designing components, assume retries, partial failure, and replays. Use idempotency keys and structured errors to preserve correctness.