Skip to main content

​
Workflow Architecture

Workflows are the core abstraction in Osmedeus, defining automated security tasks through YAML configuration. This document covers the workflow system architecture, including fragments and the linting system.

​
Workflow Kinds

Osmedeus supports three workflow kinds:
KindPurposeContains
moduleSingle execution unitSteps array
flowOrchestrate modulesModules array
fragmentReusable step collectionSteps array
+-------------------------------------------------------------+
|                       Flow                                  |
|  +-------------+  +-------------+  +-------------+          |
|  |  Module A   |  |  Module B   |  |  Module C   |          |
|  |  +-------+  |  |  +-------+  |  |  +-------+  |          |
|  |  | Step  |  |  |  | Step  |  |  |  |Fragment|  |          |
|  |  | Step  |  |  |  | Step  |  |  |  | Step   |  |          |
|  |  |Fragment|  |  |  +-------+  |  |  | Step   |  |          |
|  |  +-------+  |  |              |  |  +-------+  |          |
|  +-------------+  +-------------+  +-------------+          |
+-------------------------------------------------------------+

​
Workflow Structure

​
Workflow Type

type Workflow struct {
    Kind         WorkflowKind  // module, flow, fragment
    Name         string        // Unique identifier
    Description  string        // Human-readable description
    Tags         TagList       // Comma-separated tags
    Params       []Param       // Input parameters
    Triggers     []Trigger     // Automated triggers
    Dependencies *Dependencies // External tool requirements
    Reports      []Report      // Output reports

    // Execution preferences
    Preferences  *Preferences  // Optional execution settings

    // Runner configuration (module-kind only)
    Runner       RunnerType    // host, docker, ssh
    RunnerConfig *RunnerConfig // Runner-specific config

    // Module-specific fields
    Steps    []Step           // Execution steps
    Includes []FragmentInclude // Fragment includes

    // Flow-specific fields
    Modules  []ModuleRef      // Module references

    // Internal metadata
    FilePath string           // Source file path
    Checksum string           // Content checksum
}

​
Step Type

type Step struct {
    Name         string      // Unique step identifier
    Type         StepType    // Step type
    DependsOn    []string    // Step dependencies
    StepRunner   RunnerType  // Per-step runner override
    PreCondition string      // Skip if false
    Log          string      // Log file path
    Timeout      StepTimeout // Execution timeout

    // Bash step fields
    Command          string
    Commands         []string
    ParallelCommands []string
    StdFile          string    // Stdout capture file

    // Structured argument fields
    SpeedArgs  string
    ConfigArgs string
    InputArgs  string
    OutputArgs string

    // Function step fields
    Function          string
    Functions         []string
    ParallelFunctions []string

    // Parallel step fields
    ParallelSteps []Step

    // Foreach step fields
    Input    string
    Variable string
    Threads  StepThreads
    Step     *Step

    // Remote-bash step fields
    StepRunnerConfig *StepRunnerConfig
    StepRemoteFile   string
    HostOutputFile   string

    // HTTP step fields
    URL         string
    Method      string
    Headers     map[string]string
    RequestBody string

    // LLM step fields
    Messages       []LLMMessage
    Tools          []LLMTool
    LLMConfig      *LLMStepConfig
    IsEmbedding    bool
    EmbeddingInput []string

    // Fragment-step fields
    FragmentName string            // Fragment to execute
    Override     map[string]string // Override parameters

    // Common fields
    Exports   map[string]string
    OnSuccess []Action
    OnError   []Action
    Decision  *DecisionConfig
}

​
Workflow Inheritance

Workflows support inheritance through the extends field, allowing child workflows to inherit and override parent configurations.

​
Inheritance Architecture

+-------------------------------------------------------------+
|                  InheritanceResolver                        |
|                                                             |
|   Child Workflow                                            |
|   +---------------------------------------------------------+
|   | extends: parent-workflow                                |
|   | override: { params: ..., steps: ... }                   |
|   +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|   +---------------------------------------------------------+
|   | 1. Check for circular dependency                        |
|   | 2. Load parent workflow                                 |
|   | 3. Recursively resolve parent's inheritance             |
|   | 4. Validate kind compatibility                          |
|   | 5. Merge parent -> child with overrides                 |
|   +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|   Merged Workflow                                           |
|   +---------------------------------------------------------+
|   | All parent fields + child overrides                     |
|   | ResolvedFrom: "parent-workflow"                         |
|   +---------------------------------------------------------+
+-------------------------------------------------------------+

​
InheritanceResolver Type

type InheritanceResolver struct {
    loader    *Loader
    resolving map[string]bool  // Track workflows being resolved (circular detection)
    childPath string           // Directory of current child (for relative resolution)
}

​
Resolution Process

  1. Circular Detection: Track workflows being resolved to detect circular inheritance
  2. Parent Loading: Load parent by name (same directory) or path (relative/absolute)
  3. Recursive Resolution: If parent also extends, resolve recursively
  4. Kind Validation: Child and parent must have matching kind (module/flow)
  5. Merge: Apply child’s direct fields and override section

​
Override Modes

const (
    OverrideModeReplace OverrideMode = "replace"   // Replace parent items entirely
    OverrideModePrepend OverrideMode = "prepend"   // Add child items before parent
    OverrideModeAppend  OverrideMode = "append"    // Add child items after parent (default)
    OverrideModeMerge   OverrideMode = "merge"     // Match by name, replace/remove/append
)

​
WorkflowOverride Type

type WorkflowOverride struct {
    Params       map[string]*ParamOverride  // Override parameter properties
    Steps        *StepsOverride             // Steps override (modules only)
    Modules      *ModulesOverride           // Modules override (flows only)
    Triggers     []Trigger                  // Replace triggers entirely
    Dependencies *Dependencies              // Merge with parent dependencies
    Preferences  *Preferences               // Child overrides parent
    RunnerConfig *RunnerConfig              // Child overrides parent
    Runner       *RunnerType                // Override runner type
}

​
StepsOverride Type

type StepsOverride struct {
    Mode    OverrideMode  // replace, prepend, append, merge
    Steps   []Step        // Steps to add/match
    Remove  []string      // Step names to remove (merge mode)
    Replace []Step        // Steps to replace by name (merge mode)
}

​
Merge Priority

Priority: Child direct fields > Child override > Parent

1. Start with clone of parent workflow
2. Apply child's direct fields (name, description, tags)
3. Apply override section by mode
4. Clear extends field to prevent re-resolution

​
Parent Resolution Order

  1. Same directory as child (name + .yaml/.yml)
  2. Relative path from child’s directory
  3. Workflows directory search by name
  4. Absolute path

​
Fragments

Fragments are reusable step collections that can be embedded in modules.

​
Fragment Definition

kind: fragment
name: notification-fragment
description: Common notification steps
params:
  - name: channel
    type: string
    default: "#security-alerts"
steps:
  - name: notify
    type: bash
    command: notify-send "{{channel}}" "{{message}}"

​
Fragment Include

Modules can include fragments using the includes field: 
kind: module
name: subdomain-enum
includes:
  - name: notification-fragment
    as: notify
    params:
      channel: "#recon"
steps:
  - name: run-amass
    type: bash
    command: amass enum -d {{target}}
  - name: send-notification
    type: fragment
    fragment_name: notify
    override:
      message: "Subdomain enumeration complete"

​
FragmentInclude Type

type FragmentInclude struct {
    Name   string            // Fragment workflow name
    As     string            // Local alias
    Params map[string]string // Parameter overrides
}

​
Fragment Resolution

  1. Loading: Fragments are loaded during workflow parsing
  2. Validation: Fragment kind must be fragment
  3. Parameter Binding: Include params merged with fragment defaults
  4. Step Expansion: Fragment steps are embedded at execution time

​
Fragment Step Type

type Step struct {
    // ... other fields
    Type         StepType // "fragment"
    FragmentName string   // Alias from includes
    Override     map[string]string // Runtime parameter overrides
}

​
Fragment Execution

When a fragment step is executed:
  1. Resolve fragment from includes by alias
  2. Merge override params with include params
  3. Execute fragment steps in sequence
  4. Return combined results

​
Linting System

The workflow linter validates YAML workflows for correctness and best practices.

​
Linting Architecture

+-------------------------------------------------------------+
|                        Linter                               |
|                                                             |
|   YAML Source                                               |
|   +---------------------------------------------------------+
|   | kind: module                                            |
|   | name: my-workflow                                       |
|   | steps: ...                                              |
|   +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|  +---------------------------------------------------------+
|  |                   WorkflowAST                           |
|  |  - Workflow: *core.Workflow                             |
|  |  - Source: []byte                                       |
|  |  - Root: ast.Node                                       |
|  |  - NodeMap: map[string]ast.Node                         |
|  +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|  +---------------------------------------------------------+
|  |                    Rules                                |
|  |  +------------------+  +------------------+              |
|  |  | MissingRequired  |  | DuplicateStepName|              |
|  |  +------------------+  +------------------+              |
|  |  +------------------+  +------------------+              |
|  |  |   EmptyStep      |  | UnusedVariable   |              |
|  |  +------------------+  +------------------+              |
|  |  +------------------+  +------------------+              |
|  |  |  InvalidGoto     |  |InvalidDependsOn  |              |
|  |  +------------------+  +------------------+              |
|  |  +------------------+  +------------------+              |
|  |  |CircularDependency|  |UndefinedVariable |              |
|  |  +------------------+  +------------------+              |
|  +---------------------------------------------------------+
|                           |                                 |
|                           v                                 |
|  +---------------------------------------------------------+
|  |                   LintResult                            |
|  |  - Issues: []LintIssue                                  |
|  |  - Errors: int                                          |
|  |  - Warnings: int                                        |
|  |  - Infos: int                                           |
|  +---------------------------------------------------------+
+-------------------------------------------------------------+

​
Built-in Rules

RuleSeverityDescription
missing-required-fieldwarningRequired fields (name, kind, type) missing
duplicate-step-namewarningMultiple steps with same name
empty-stepwarningStep has no executable content
unused-variableinfoVariable exported but never used
undefined-variablewarningVariable referenced but not defined
invalid-gotowarningDecision goto references non-existent step
invalid-depends-onwarningdepends_on references non-existent step
circular-dependencywarningCircular step dependencies detected

​
LinterRule Interface

type LinterRule interface {
    Name() string                    // Unique identifier
    Description() string             // Human-readable description
    Severity() Severity              // Default severity
    Check(ast *WorkflowAST) []LintIssue
}

​
LintIssue Type

type LintIssue struct {
    Rule       string   // Rule name
    Severity   Severity // Issue severity
    Message    string   // Human-readable description
    Suggestion string   // Fix suggestion
    Line       int      // 1-based line number
    Column     int      // 1-based column number
    Field      string   // YAML path (e.g., "steps[0].bash")
}

​
Running the Linter

​
CLI Usage

# Validate by name
osmedeus workflow validate subdomain-enum

# Validate file
osmedeus workflow lint ./my-workflow.yaml

# Validate folder
osmedeus workflow validate /path/to/workflows/

# CI mode
osmedeus workflow lint . --check --format json

​
Output Formats

Pretty (default): 
workflows/test.yaml:15:12 warning undefined-variable
  Variable 'unknown_var' is not defined
  Suggestion: Check that the variable is defined in params or a previous step's exports
JSON: 
{
  "file_path": "workflows/test.yaml",
  "issues": [
    {
      "rule": "undefined-variable",
      "severity": "warning",
      "message": "Variable 'unknown_var' is not defined",
      "line": 15,
      "column": 12,
      "field": "steps[2].command"
    }
  ]
}
GitHub Actions: 
::warning file=workflows/test.yaml,line=15,col=12::undefined-variable: Variable 'unknown_var' is not defined

​
Disabling Rules

# Disable specific rules
osmedeus workflow lint . --disable unused-variable,empty-step

​
Custom Rules

Implement the LinterRule interface: 
type MyCustomRule struct{}

func (r *MyCustomRule) Name() string { return "my-custom-rule" }

func (r *MyCustomRule) Description() string {
    return "Description of what this rule checks"
}

func (r *MyCustomRule) Severity() Severity { return SeverityWarning }

func (r *MyCustomRule) Check(wast *WorkflowAST) []LintIssue {
    var issues []LintIssue
    w := wast.Workflow

    // Implement your validation logic
    for i, step := range w.Steps {
        if /* condition */ {
            line, col := wast.FindStepPosition(step.Name)
            issues = append(issues, LintIssue{
                Rule:       r.Name(),
                Severity:   r.Severity(),
                Message:    "Issue description",
                Suggestion: "How to fix",
                Line:       line,
                Column:     col,
                Field:      fmt.Sprintf("steps[%d]", i),
            })
        }
    }

    return issues
}

// Register the rule
linter := linter.NewDefaultLinter()
linter.RegisterRule(&MyCustomRule{})

​
Decision Routing

Steps support conditional branching: 
decision:
  switch: "{{status}}"
  cases:
    "critical": { goto: alert-step }
    "high": { goto: process-high }
    "none": { goto: _end }
  default: { goto: continue-step }

​
DecisionConfig Type

type DecisionConfig struct {
    Switch  string                  // Variable to evaluate
    Cases   map[string]DecisionCase // Case mappings
    Default *DecisionCase           // Default case
}

type DecisionCase struct {
    Goto string // Target step name or "_end"
}

​
Workflow Execution Context

type ExecutionContext struct {
    RunID         string
    Target        string
    Workspace     string
    Output        string
    Variables     map[string]interface{}
    Exports       map[string]interface{}
    StepResults   map[string]*StepResult
    Config        *config.Config
    Runner        runner.Runner
    TemplateEngine template.TemplateEngine
    Runtime       *functions.GojaRuntime
}

​
Best Practices

​
Workflow Design

  1. Use fragments for reusable step collections
  2. Keep modules focused - Single responsibility
  3. Use flows to orchestrate complex pipelines
  4. Leverage decision routing for adaptive workflows
  5. Define dependencies with depends_on for DAG execution

​
Performance

  1. Use parallel_commands for independent commands
  2. Use parallel_functions for independent function calls
  3. Set appropriate timeouts to prevent hanging
  4. Use foreach with appropriate thread counts

​
Validation

  1. Run linter before deploying workflows
  2. Use CI integration with --check --format github
  3. Enable all rules during development
  4. Use type annotations in params for validation

​
Error Handling

  1. Use on_error handlers for graceful failures
  2. Use pre_condition to skip steps safely
  3. Export meaningful error information
  4. Log appropriately for debugging