Workflow Architecture

Workflows are the core abstraction in Osmedeus, defining automated security tasks through YAML configuration. This document covers the workflow system architecture, including fragments and the linting system.

Workflow Kinds

Osmedeus supports three workflow kinds:

Kind	Purpose	Contains
`module`	Single execution unit	Steps array
`flow`	Orchestrate modules	Modules array
`fragment`	Reusable step collection	Steps array

+-------------------------------------------------------------+
|                       Flow                                  |
|  +-------------+  +-------------+  +-------------+          |
|  |  Module A   |  |  Module B   |  |  Module C   |          |
|  |  +-------+  |  |  +-------+  |  |  +-------+  |          |
|  |  | Step  |  |  |  | Step  |  |  |  |Fragment|  |          |
|  |  | Step  |  |  |  | Step  |  |  |  | Step   |  |          |
|  |  |Fragment|  |  |  +-------+  |  |  | Step   |  |          |
|  |  +-------+  |  |              |  |  +-------+  |          |
|  +-------------+  +-------------+  +-------------+          |
+-------------------------------------------------------------+

Workflow Structure

Workflow Type

type Workflow struct {
    Kind         WorkflowKind  // module, flow, fragment
    Name         string        // Unique identifier
    Description  string        // Human-readable description
    Tags         TagList       // Comma-separated tags
    Params       []Param       // Input parameters
    Triggers     []Trigger     // Automated triggers
    Dependencies *Dependencies // External tool requirements
    Reports      []Report      // Output reports

    // Execution preferences
    Preferences  *Preferences  // Optional execution settings

    // Runner configuration (module-kind only)
    Runner       RunnerType    // host, docker, ssh
    RunnerConfig *RunnerConfig // Runner-specific config

    // Module-specific fields
    Steps    []Step           // Execution steps
    Includes []FragmentInclude // Fragment includes

    // Flow-specific fields
    Modules  []ModuleRef      // Module references

    // Internal metadata
    FilePath string           // Source file path
    Checksum string           // Content checksum
}

Step Type

type Step struct {
    Name         string      // Unique step identifier
    Type         StepType    // Step type
    DependsOn    []string    // Step dependencies
    StepRunner   RunnerType  // Per-step runner override
    PreCondition string      // Skip if false
    Log          string      // Log file path
    Timeout      StepTimeout // Execution timeout

    // Bash step fields
    Command          string
    Commands         []string
    ParallelCommands []string
    StdFile          string    // Stdout capture file

    // Structured argument fields
    SpeedArgs  string
    ConfigArgs string
    InputArgs  string
    OutputArgs string

    // Function step fields
    Function          string
    Functions         []string
    ParallelFunctions []string

    // Parallel step fields
    ParallelSteps []Step

    // Foreach step fields
    Input    string
    Variable string
    Threads  StepThreads
    Step     *Step

    // Remote-bash step fields
    StepRunnerConfig *StepRunnerConfig
    StepRemoteFile   string
    HostOutputFile   string

    // HTTP step fields
    URL         string
    Method      string
    Headers     map[string]string
    RequestBody string

    // LLM step fields
    Messages       []LLMMessage
    Tools          []LLMTool
    LLMConfig      *LLMStepConfig
    IsEmbedding    bool
    EmbeddingInput []string

    // Fragment-step fields
    FragmentName string            // Fragment to execute
    Override     map[string]string // Override parameters

    // Common fields
    Exports   map[string]string
    OnSuccess []Action
    OnError   []Action
    Decision  *DecisionConfig
}

Workflow Inheritance

Workflows support inheritance through the extends field, allowing child workflows to inherit and override parent configurations.

Inheritance Architecture

+-------------------------------------------------------------+
|                  InheritanceResolver                        |
|                                                             |
|   Child Workflow                                            |
|   +---------------------------------------------------------+
|   | extends: parent-workflow                                |
|   | override: { params: ..., steps: ... }                   |
|   +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|   +---------------------------------------------------------+
|   | 1. Check for circular dependency                        |
|   | 2. Load parent workflow                                 |
|   | 3. Recursively resolve parent's inheritance             |
|   | 4. Validate kind compatibility                          |
|   | 5. Merge parent -> child with overrides                 |
|   +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|   Merged Workflow                                           |
|   +---------------------------------------------------------+
|   | All parent fields + child overrides                     |
|   | ResolvedFrom: "parent-workflow"                         |
|   +---------------------------------------------------------+
+-------------------------------------------------------------+

InheritanceResolver Type

type InheritanceResolver struct {
    loader    *Loader
    resolving map[string]bool  // Track workflows being resolved (circular detection)
    childPath string           // Directory of current child (for relative resolution)
}

Resolution Process

Circular Detection: Track workflows being resolved to detect circular inheritance
Parent Loading: Load parent by name (same directory) or path (relative/absolute)
Recursive Resolution: If parent also extends, resolve recursively
Kind Validation: Child and parent must have matching kind (module/flow)
Merge: Apply child’s direct fields and override section

Override Modes

const (
    OverrideModeReplace OverrideMode = "replace"   // Replace parent items entirely
    OverrideModePrepend OverrideMode = "prepend"   // Add child items before parent
    OverrideModeAppend  OverrideMode = "append"    // Add child items after parent (default)
    OverrideModeMerge   OverrideMode = "merge"     // Match by name, replace/remove/append
)

WorkflowOverride Type

type WorkflowOverride struct {
    Params       map[string]*ParamOverride  // Override parameter properties
    Steps        *StepsOverride             // Steps override (modules only)
    Modules      *ModulesOverride           // Modules override (flows only)
    Triggers     []Trigger                  // Replace triggers entirely
    Dependencies *Dependencies              // Merge with parent dependencies
    Preferences  *Preferences               // Child overrides parent
    RunnerConfig *RunnerConfig              // Child overrides parent
    Runner       *RunnerType                // Override runner type
}

StepsOverride Type

type StepsOverride struct {
    Mode    OverrideMode  // replace, prepend, append, merge
    Steps   []Step        // Steps to add/match
    Remove  []string      // Step names to remove (merge mode)
    Replace []Step        // Steps to replace by name (merge mode)
}

Merge Priority

Priority: Child direct fields > Child override > Parent

Start with clone of parent workflow
Apply child's direct fields (name, description, tags)
Apply override section by mode
Clear extends field to prevent re-resolution

Parent Resolution Order

Same directory as child (name + .yaml/.yml)
Relative path from child’s directory
Workflows directory search by name
Absolute path

Fragments

Fragments are reusable step collections that can be embedded in modules.

Fragment Definition

kind: fragment
name: notification-fragment
description: Common notification steps
params:
  - name: channel
    type: string
    default: "#security-alerts"
steps:
  - name: notify
    type: bash
    command: notify-send "{{channel}}" "{{message}}"

Fragment Include

Modules can include fragments using the includes field:

kind: module
name: subdomain-enum
includes:
  - name: notification-fragment
    as: notify
    params:
      channel: "#recon"
steps:
  - name: run-amass
    type: bash
    command: amass enum -d {{target}}
  - name: send-notification
    type: fragment
    fragment_name: notify
    override:
      message: "Subdomain enumeration complete"

FragmentInclude Type

type FragmentInclude struct {
    Name   string            // Fragment workflow name
    As     string            // Local alias
    Params map[string]string // Parameter overrides
}

Fragment Resolution

Loading: Fragments are loaded during workflow parsing
Validation: Fragment kind must be fragment
Parameter Binding: Include params merged with fragment defaults
Step Expansion: Fragment steps are embedded at execution time

Fragment Step Type

type Step struct {
    // ... other fields
    Type         StepType // "fragment"
    FragmentName string   // Alias from includes
    Override     map[string]string // Runtime parameter overrides
}

Fragment Execution

When a fragment step is executed:

Resolve fragment from includes by alias
Merge override params with include params
Execute fragment steps in sequence
Return combined results

Linting System

The workflow linter validates YAML workflows for correctness and best practices.

Linting Architecture

+-------------------------------------------------------------+
|                        Linter                               |
|                                                             |
|   YAML Source                                               |
|   +---------------------------------------------------------+
|   | kind: module                                            |
|   | name: my-workflow                                       |
|   | steps: ...                                              |
|   +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|  +---------------------------------------------------------+
|  |                   WorkflowAST                           |
|  |  - Workflow: *core.Workflow                             |
|  |  - Source: []byte                                       |
|  |  - Root: ast.Node                                       |
|  |  - NodeMap: map[string]ast.Node                         |
|  +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|  +---------------------------------------------------------+
|  |                    Rules                                |
|  |  +------------------+  +------------------+              |
|  |  | MissingRequired  |  | DuplicateStepName|              |
|  |  +------------------+  +------------------+              |
|  |  +------------------+  +------------------+              |
|  |  |   EmptyStep      |  | UnusedVariable   |              |
|  |  +------------------+  +------------------+              |
|  |  +------------------+  +------------------+              |
|  |  |  InvalidGoto     |  |InvalidDependsOn  |              |
|  |  +------------------+  +------------------+              |
|  |  +------------------+  +------------------+              |
|  |  |CircularDependency|  |UndefinedVariable |              |
|  |  +------------------+  +------------------+              |
|  +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|  +---------------------------------------------------------+
|  |                   LintResult                            |
|  |  - Issues: []LintIssue                                  |
|  |  - Errors: int                                          |
|  |  - Warnings: int                                        |
|  |  - Infos: int                                           |
|  +---------------------------------------------------------+
+-------------------------------------------------------------+

Built-in Rules

Rule	Severity	Description
`missing-required-field`	warning	Required fields (name, kind, type) missing
`duplicate-step-name`	warning	Multiple steps with same name
`empty-step`	warning	Step has no executable content
`unused-variable`	info	Variable exported but never used
`undefined-variable`	warning	Variable referenced but not defined
`invalid-goto`	warning	Decision goto references non-existent step
`invalid-depends-on`	warning	depends_on references non-existent step
`circular-dependency`	warning	Circular step dependencies detected

LinterRule Interface

type LinterRule interface {
    Name() string                    // Unique identifier
    Description() string             // Human-readable description
    Severity() Severity              // Default severity
    Check(ast *WorkflowAST) []LintIssue
}

LintIssue Type

type LintIssue struct {
    Rule       string   // Rule name
    Severity   Severity // Issue severity
    Message    string   // Human-readable description
    Suggestion string   // Fix suggestion
    Line       int      // 1-based line number
    Column     int      // 1-based column number
    Field      string   // YAML path (e.g., "steps[0].bash")
}

Running the Linter

CLI Usage

# Validate by name
osmedeus workflow validate subdomain-enum

# Validate file
osmedeus workflow lint ./my-workflow.yaml

# Validate folder
osmedeus workflow validate /path/to/workflows/

# CI mode
osmedeus workflow lint . --check --format json

Output Formats

Pretty (default):

workflows/test.yaml:15:12 warning undefined-variable
  Variable 'unknown_var' is not defined
  Suggestion: Check that the variable is defined in params or a previous step's exports

JSON:

{
  "file_path": "workflows/test.yaml",
  "issues": [
    {
      "rule": "undefined-variable",
      "severity": "warning",
      "message": "Variable 'unknown_var' is not defined",
      "line": 15,
      "column": 12,
      "field": "steps[2].command"
    }
  ]
}

GitHub Actions:

::warning file=workflows/test.yaml,line=15,col=12::undefined-variable: Variable 'unknown_var' is not defined

Disabling Rules

# Disable specific rules
osmedeus workflow lint . --disable unused-variable,empty-step

Custom Rules

Implement the LinterRule interface:

type MyCustomRule struct{}

func (r *MyCustomRule) Name() string { return "my-custom-rule" }

func (r *MyCustomRule) Description() string {
    return "Description of what this rule checks"
}

func (r *MyCustomRule) Severity() Severity { return SeverityWarning }

func (r *MyCustomRule) Check(wast *WorkflowAST) []LintIssue {
    var issues []LintIssue
    w := wast.Workflow

    // Implement your validation logic
    for i, step := range w.Steps {
        if /* condition */ {
            line, col := wast.FindStepPosition(step.Name)
            issues = append(issues, LintIssue{
                Rule:       r.Name(),
                Severity:   r.Severity(),
                Message:    "Issue description",
                Suggestion: "How to fix",
                Line:       line,
                Column:     col,
                Field:      fmt.Sprintf("steps[%d]", i),
            })
        }
    }

    return issues
}

// Register the rule
linter := linter.NewDefaultLinter()
linter.RegisterRule(&MyCustomRule{})

Decision Routing

Steps support conditional branching:

decision:
  switch: "{{status}}"
  cases:
    "critical": { goto: alert-step }
    "high": { goto: process-high }
    "none": { goto: _end }
  default: { goto: continue-step }

DecisionConfig Type

type DecisionConfig struct {
    Switch  string                  // Variable to evaluate
    Cases   map[string]DecisionCase // Case mappings
    Default *DecisionCase           // Default case
}

type DecisionCase struct {
    Goto string // Target step name or "_end"
}

Workflow Execution Context

type ExecutionContext struct {
    RunID         string
    Target        string
    Workspace     string
    Output        string
    Variables     map[string]interface{}
    Exports       map[string]interface{}
    StepResults   map[string]*StepResult
    Config        *config.Config
    Runner        runner.Runner
    TemplateEngine template.TemplateEngine
    Runtime       *functions.GojaRuntime
}

Best Practices

Workflow Design

Use fragments for reusable step collections
Keep modules focused - Single responsibility
Use flows to orchestrate complex pipelines
Leverage decision routing for adaptive workflows
Define dependencies with depends_on for DAG execution

Performance

Use parallel_commands for independent commands
Use parallel_functions for independent function calls
Set appropriate timeouts to prevent hanging
Use foreach with appropriate thread counts

Validation

Run linter before deploying workflows
Use CI integration with --check --format github
Enable all rules during development
Use type annotations in params for validation

Error Handling

Use on_error handlers for graceful failures
Use pre_condition to skip steps safely
Export meaningful error information
Log appropriately for debugging

​Workflow Architecture

​Workflow Kinds

​Workflow Structure

​Workflow Type

​Step Type

​Workflow Inheritance

​Inheritance Architecture

​InheritanceResolver Type

​Resolution Process

​Override Modes

​WorkflowOverride Type

​StepsOverride Type

​Merge Priority

​Parent Resolution Order

​Fragments

​Fragment Definition

​Fragment Include

​FragmentInclude Type

​Fragment Resolution

​Fragment Step Type

​Fragment Execution

​Linting System

​Linting Architecture

​Built-in Rules

​LinterRule Interface

​LintIssue Type

​Running the Linter

​CLI Usage

​Output Formats

​Disabling Rules

​Custom Rules

​Decision Routing

​DecisionConfig Type

​Workflow Execution Context

​Best Practices

​Workflow Design

​Performance

​Validation

​Error Handling