Architecture Overview

Osmedeus is a workflow engine for security automation. It executes YAML-defined workflows with support for multiple execution environments, distributed processing, and extensive customization.

Layered Architecture

+-------------------------------------------------------------+
|                     CLI / REST API                          |
|              (pkg/cli, pkg/server)                          |
+-------------------------------------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                       Executor                              |
|              (internal/executor)                            |
|     Coordinates workflow execution and manages state        |
+-------------------------------------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                   Step Dispatcher                           |
|                                                             |
|  +----------+ +----------+ +----------+ +----------+       |
|  |   Bash   | | Function | | Parallel | | Foreach  |       |
|  | Executor | | Executor | | Executor | | Executor |       |
|  +----------+ +----------+ +----------+ +----------+       |
|                                                             |
|  +----------+ +----------+ +----------+ +----------+       |
|  |  Remote  | |   HTTP   | |   LLM    | | Fragment |       |
|  |   Bash   | | Executor | | Executor | | Executor |       |
|  +----------+ +----------+ +----------+ +----------+       |
|                                                             |
+-------------------------------------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                        Runner                               |
|              (internal/runner)                              |
|                                                             |
|   +------------+  +------------+  +------------+           |
|   |    Host    |  |   Docker   |  |    SSH     |           |
|   |   Runner   |  |   Runner   |  |   Runner   |           |
|   +------------+  +------------+  +------------+           |
|                                                             |
+-------------------------------------------------------------+

Core Packages

Package	Purpose
`internal/core`	Type definitions: Workflow, Step, Trigger, RunnerConfig, ExecutionContext
`internal/parser`	YAML parsing, validation, and caching (Loader)
`internal/executor`	Workflow execution engine with step dispatching
`internal/runner`	Execution environments implementing Runner interface
`internal/template`	`{{Variable}}` interpolation engine (Engine, ShardedEngine)
`internal/functions`	Utility functions via Goja JavaScript runtime pool
`internal/scheduler`	Cron, event, and file-watch triggers
`internal/database`	SQLite/PostgreSQL via Bun ORM
`internal/linter`	Workflow validation and linting
`pkg/cli`	Cobra CLI commands
`pkg/server`	Fiber REST API
`internal/snapshot`	Workspace export/import as ZIP archives
`internal/installer`	Binary installation (direct-fetch and Nix)
`internal/state`	Run state export for debugging
`internal/updater`	Self-update via GitHub releases

Workflow Execution Flow

+-------------------------------------------------------------+
| 1. CLI parses arguments                                     |
|    osmedeus run -f general -t example.com                   |
+-------------------------------------------------------------+
                           |
                           v
+-------------------------------------------------------------+
| 2. Load config from ~/osmedeus-base/osm-settings.yaml       |
+-------------------------------------------------------------+
                           |
                           v
+-------------------------------------------------------------+
| 3. Parser loads YAML workflow                               |
|    - Validates schema                                       |
|    - Resolves includes (fragments)                          |
|    - Caches in Loader                                       |
+-------------------------------------------------------------+
                           |
                           v
+-------------------------------------------------------------+
| 4. Executor initializes context                             |
|    - Injects built-in variables (Target, Output, etc.)      |
|    - Loads params with defaults                             |
|    - Creates execution context                              |
+-------------------------------------------------------------+
                           |
                           v
+-------------------------------------------------------------+
| 5. For each step:                                           |
|    +-----------------------------------------------------+  |
|    | a. Check depends_on (wait for dependencies)         |  |
|    | b. Evaluate pre_condition                           |  |
|    | c. Render templates                                 |  |
|    | d. Dispatch to appropriate executor                 |  |
|    | e. Execute via runner                               |  |
|    | f. Capture output and exports                       |  |
|    | g. Evaluate decision routing                        |  |
|    | h. Handle on_success/on_error                       |  |
|    +-----------------------------------------------------+  |
+-------------------------------------------------------------+
                           |
                           v
+-------------------------------------------------------------+
| 6. Export results                                           |
|    - Save run state                                         |
|    - Update database                                        |
|    - Generate artifacts                                     |
+-------------------------------------------------------------+

Step Type Routing

The Step Dispatcher routes steps to the appropriate executor based on type:

+-------------------------------------------------------------+
|                    Step Dispatcher                          |
|                                                             |
|   Step Type                    Executor                     |
|   -------------------------------------------------         |
|   "bash"           ---------> BashExecutor --> Runner       |
|   "function"       ---------> FunctionExecutor --> Goja     |
|   "parallel-steps" ---------> ParallelExecutor              |
|   "foreach"        ---------> ForeachExecutor               |
|   "remote-bash"    ---------> RemoteBashExecutor --> SSH/Docker |
|   "http"           ---------> HTTPExecutor                  |
|   "llm"            ---------> LLMExecutor                   |
|   "fragment-step"  ---------> FragmentStepExecutor --> Inline |
|                                                             |
+-------------------------------------------------------------+

Key Types

WorkflowKind

const (
    KindModule   WorkflowKind = "module"   // Single unit workflow
    KindFlow     WorkflowKind = "flow"     // Orchestrates modules
    KindFragment WorkflowKind = "fragment" // Reusable step collection
)

StepType

const (
    StepTypeBash         StepType = "bash"
    StepTypeFunction     StepType = "function"
    StepTypeParallel     StepType = "parallel-steps"
    StepTypeForeach      StepType = "foreach"
    StepTypeRemoteBash   StepType = "remote-bash"
    StepTypeHTTP         StepType = "http"
    StepTypeLLM          StepType = "llm"
    StepTypeFragmentStep StepType = "fragment-step"
)

RunnerType

const (
    RunnerTypeHost   RunnerType = "host"   // Local execution
    RunnerTypeDocker RunnerType = "docker" // Docker container
    RunnerTypeSSH    RunnerType = "ssh"    // Remote SSH
)

TriggerType

const (
    TriggerCron   TriggerType = "cron"   // Cron schedule
    TriggerEvent  TriggerType = "event"  // Event-driven
    TriggerWatch  TriggerType = "watch"  // File system watch
    TriggerManual TriggerType = "manual" // CLI execution
)

Step Executors

Executor	Description	Runner
`BashExecutor`	Execute shell commands	Host/Docker/SSH
`FunctionExecutor`	Execute utility functions	Goja runtime
`ParallelExecutor`	Concurrent step execution	Multiple
`ForeachExecutor`	Iterate with parallelism	Multiple
`RemoteBashExecutor`	Remote command execution	Docker/SSH
`HTTPExecutor`	HTTP requests	Built-in
`LLMExecutor`	LLM API calls	Built-in
`FragmentStepExecutor`	Inline fragment execution	Dispatcher

Template Engine

The template engine provides {{variable}} interpolation:

+-------------------------------------------------------------+
|                    Template Engine                          |
|                                                             |
|   Input: "nuclei -l {{Output}}/urls.txt -o {{Output}}/out"  |
|                           |                                 |
|                           v                                 |
|   Context: { Output: "/workspaces/example_com" }            |
|                           |                                 |
|                           v                                 |
|   Output: "nuclei -l /workspaces/example_com/urls.txt ..."  |
|                                                             |
|   Features:                                                 |
|   - Standard templates: {{variable}}                        |
|   - Secondary (foreach): [[variable]]                       |
|   - Generators: $rand(16), $uuid()                          |
|   - Sharded caching for high concurrency                    |
|   - Pre-compiled templates for workflows                    |
|                                                             |
+-------------------------------------------------------------+

Function Runtime

Functions are executed via the Goja JavaScript runtime with VM pooling:

+-------------------------------------------------------------+
|                    Goja Runtime Pool                        |
|                                                             |
|   +---------+  +---------+  +---------+  +---------+       |
|   |   VM1   |  |   VM2   |  |   VM3   |  |   VM4   |       |
|   | (idle)  |  | (busy)  |  | (idle)  |  | (busy)  |       |
|   +---------+  +---------+  +---------+  +---------+       |
|                                                             |
|   Features:                                                 |
|   - Pool size based on CPU cores                            |
|   - No global mutex for parallel execution                  |
|   - Lazy variable loading for conditions                    |
|   - Context isolation per execution                         |
|                                                             |
+-------------------------------------------------------------+

Database Schema

+-----------------+     +-----------------+
|   Workspaces    |     |     Assets      |
+-----------------+     +-----------------+
| id              |     | id              |
| name            |     | workspace       |--+
| target          |     | asset_value     |  |
| created_at      |     | asset_type      |  |
| total_subs      |     | url             |  |
| total_urls      |     | status_code     |  |
| ...             |     | created_at      |  |
+-----------------+     | updated_at      |  |
                        +-----------------+  |
+-----------------+                          |
| Vulnerabilities |                          |
+-----------------+                          |
| id              |                          |
| workspace       |--------------------------+
| asset_value     |
| vuln_info       |
| severity        |
| template_id     |
| created_at      |
+-----------------+

Scheduler

The scheduler manages automated workflow triggers:

+-------------------------------------------------------------+
|                       Scheduler                             |
|                                                             |
|   Trigger Types:                                            |
|   +------------------------------------------------------+  |
|   |  Cron        | Time-based scheduling (cron syntax)   |  |
|   |  Event       | System events (assets.new, etc.)      |  |
|   |  Watch       | File system changes (fsnotify)        |  |
|   |  Manual      | CLI invocation                        |  |
|   +------------------------------------------------------+  |
|                                                             |
|   Event Topics:                                             |
|   - assets.new         New asset discovered                 |
|   - vulnerabilities.new New vulnerability found             |
|   - webhook.received   External webhook received            |
|   - db.change          Database change event                |
|   - watch.files        File system change event             |
|                                                             |
+-------------------------------------------------------------+

Decision Routing

Steps support conditional branching using switch/case syntax:

decision:
  switch: "{{variable}}"
  cases:
    "value1": { goto: step-a }
    "value2": { goto: step-b }
  default: { goto: fallback }

Use goto: _end to terminate the workflow.

Plugin Registry Pattern

Step executors are registered in a plugin registry for extensibility:

type StepExecutor interface {
    Name() string
    StepTypes() []core.StepType
    Execute(ctx context.Context, step *core.Step, execCtx *core.ExecutionContext) (*core.StepResult, error)
    CanHandle(stepType core.StepType) bool
}

// Registration
dispatcher.RegisterExecutor(NewBashExecutor())
dispatcher.RegisterExecutor(NewFunctionExecutor())
dispatcher.RegisterExecutor(NewFragmentStepExecutor())
// ...

Configuration

Configuration is loaded from ~/osmedeus-base/osm-settings.yaml:

general:
  base_folder: ~/osmedeus-base
  workspaces_path: ~/workspaces-osmedeus
  binaries_path: ~/osmedeus-base/external-binaries

database:
  type: sqlite  # or postgres
  path: ~/osmedeus-base/osmedeus.db

notification:
  telegram:
    bot_token: ""
    chat_id: ""
  webhooks: []

cdn:
  enabled: false
  provider: s3
  bucket: ""

Adding New Features

New Step Type

Add constant in internal/core/types.go:
StepTypeCustom StepType = "custom"

Create executor implementing StepExecutor in internal/executor/:

type CustomExecutor struct {}
func (e *CustomExecutor) Name() string { return "custom" }
func (e *CustomExecutor) StepTypes() []core.StepType { return []core.StepType{core.StepTypeCustom} }
func (e *CustomExecutor) Execute(...) (*core.StepResult, error) { ... }

dispatcher.RegisterExecutor(NewCustomExecutor())

New Runner

Implement Runner interface in internal/runner/:

type CustomRunner struct {}
func (r *CustomRunner) Execute(ctx context.Context, cmd string) (string, error) { ... }
func (r *CustomRunner) Close() error { ... }

Add type constant and register in runner factory.

New Utility Function

Add Go implementation in internal/functions/:

func (vf *vmFunc) customFunc(call goja.FunctionCall) goja.Value { ... }

Add constant in constants.go:
FnCustomFunc = "custom_func"
Register in goja_runtime.go:
vm.Set(FnCustomFunc, vf.customFunc)

New CLI Command

Create in pkg/cli/:

var customCmd = &cobra.Command{
    Use:   "custom",
    Short: "Custom command",
    RunE:  func(cmd *cobra.Command, args []string) error { ... },
}

Add to rootCmd in init().

New API Endpoint

Add handler in pkg/server/handlers/:

func CustomHandler(cfg *config.Config) fiber.Handler { ... }

Register route in server.go.
Document in docs/api/.

Getting Started

Advanced Usage

Understand Architecture

Workflows

Functions

Cloud

Advanced Topics & Extending Osmedeus

Reference

Architecture Overview

Architecture Overview

Layered Architecture

Core Packages

Workflow Execution Flow

Step Type Routing

Key Types

WorkflowKind

StepType

RunnerType

TriggerType

Step Executors

Template Engine

Function Runtime

Database Schema

Scheduler

Decision Routing

Plugin Registry Pattern

Configuration

Adding New Features

New Step Type

New Runner

New Utility Function

New CLI Command

New API Endpoint

​Architecture Overview

​Layered Architecture

​Core Packages

​Workflow Execution Flow

​Step Type Routing

​Key Types

​WorkflowKind

​StepType

​RunnerType

​TriggerType

​Step Executors

​Template Engine

​Function Runtime

​Database Schema

​Scheduler

​Decision Routing

​Plugin Registry Pattern

​Configuration

​Adding New Features

​New Step Type

​New Runner

​New Utility Function

​New CLI Command

​New API Endpoint

Architecture Overview

Layered Architecture

Core Packages

Workflow Execution Flow

Step Type Routing

Key Types

WorkflowKind

StepType

RunnerType

TriggerType

Step Executors

Template Engine

Function Runtime

Database Schema

Scheduler

Decision Routing

Plugin Registry Pattern

Configuration

Adding New Features

New Step Type

New Runner

New Utility Function

New CLI Command

New API Endpoint