Template Engine Architecture

The template engine provides {{variable}} interpolation for workflows. It uses the pongo2 template library (Django-compatible) with custom optimizations for high-concurrency scenarios.

Template Syntax

Standard Variables

Standard template variables use {{variable}} syntax:

command: "nuclei -l {{Output}}/urls.txt -o {{Output}}/nuclei.json"

Secondary Variables (Foreach)

Foreach loops use [[variable]] syntax to avoid conflicts:

step:
  type: bash
  command: "httpx -u [[subdomain]] >> {{Output}}/httpx.txt"

Generator Functions

Generator functions provide dynamic values:

exports:
  random_id: "$rand(16)"
  unique_id: "$uuid()"

Engine Types

The template system provides two engine implementations:

Engine	Description	Use Case
`Engine`	Standard single-threaded engine	Simple workflows
`ShardedEngine`	High-performance sharded cache	High concurrency

TemplateEngine Interface

type TemplateEngine interface {
    // Render renders a template string with the given context
    Render(template string, ctx map[string]any) (string, error)

    // RenderMap renders all template values in a map
    RenderMap(m map[string]string, ctx map[string]any) (map[string]string, error)

    // RenderSlice renders all template values in a slice
    RenderSlice(s []string, ctx map[string]any) ([]string, error)

    // RenderSecondary renders templates using [[ ]] delimiters
    RenderSecondary(template string, ctx map[string]any) (string, error)

    // HasSecondaryVariable checks if template contains [[ ]]
    HasSecondaryVariable(template string) bool

    // ExecuteGenerator executes a generator function expression
    ExecuteGenerator(expr string) (string, error)

    // RegisterGenerator registers a custom generator function
    RegisterGenerator(name string, fn GeneratorFunc)
}

BatchRenderer Interface

For high-throughput scenarios, the BatchRenderer interface extends TemplateEngine:

type BatchRenderer interface {
    TemplateEngine

    // RenderBatch renders multiple templates in a single operation
    // Reduces lock contention by grouping templates
    RenderBatch(requests []RenderRequest, ctx map[string]any) (map[string]string, error)
}

type RenderRequest struct {
    Key      string // Identifier (e.g., field name)
    Template string // Template string to render
}

ShardedEngine Architecture

The ShardedEngine distributes templates across multiple shards to reduce lock contention:

+-------------------------------------------------------------+
|                     ShardedEngine                           |
|                                                             |
|   Template Hash (FNV-1a)                                    |
|        |                                                    |
|        v                                                    |
|   +-----------------------------------------------------+   |
|   |  Shard Selection: hash & shardMask                  |   |
|   +-----------------------------------------------------+   |
|        |                                                    |
|        +--------+--------+--------+--------+                |
|        v        v        v        v        v                |
|   +--------++--------++--------++--------+...               |
|   |Shard 0 ||Shard 1 ||Shard 2 ||Shard 3 |                  |
|   | RWMutex|| RWMutex|| RWMutex|| RWMutex|                  |
|   | LRU    || LRU    || LRU    || LRU    |                  |
|   +--------++--------++--------++--------+                  |
|                                                             |
+-------------------------------------------------------------+

Configuration

type ShardedEngineConfig struct {
    ShardCount     int  // Number of shards (power of 2)
    ShardCacheSize int  // LRU cache size per shard
    EnablePooling  bool // Use pooled context maps
}

// Defaults
const DefaultShardCount = 16
const DefaultShardCacheSize = 64

Performance Features

Sharded Caching: Templates are distributed across shards using FNV-1a hash, reducing lock contention.
RWMutex per Shard: Read-heavy workloads benefit from reader-writer locks.
Quick Path: Templates without {{ are returned immediately without processing.
Lock-Free Execution: Compiled pongo2 templates are thread-safe; execution happens outside locks.
Context Pooling: Reusable context maps reduce GC pressure.

Render Flow

func (e *ShardedEngine) Render(template string, ctx map[string]any) (string, error) {
    // Quick path: no template variables
    if !strings.Contains(template, "{{") {
        return template, nil
    }

    // Get shard based on template hash
    shard := e.getShard(template)

    // Try cache with read lock
    shard.mu.RLock()
    tpl, ok := shard.cache.Get(template)
    shard.mu.RUnlock()

    if !ok {
        // Cache miss - parse and cache
        shard.mu.Lock()
        // Double-check after write lock
        tpl, ok = shard.cache.Get(template)
        if !ok {
            tpl, err = pongo2.FromString(template)
            if err != nil {
                shard.mu.Unlock()
                return "", err
            }
            shard.cache.Add(template, tpl)
        }
        shard.mu.Unlock()
    }

    // Execute outside lock (pongo2 is thread-safe)
    return tpl.Execute(pongo2.Context(ctx))
}

Precompiled Templates

The PrecompiledRegistry stores pre-compiled templates for workflows:

type PrecompiledRegistry struct {
    mu        sync.RWMutex
    workflows map[string]*WorkflowTemplates
}

type WorkflowTemplates struct {
    // Key format: "stepName:fieldName"
    Templates map[string]*pongo2.Template
}

Precompiler Interface

type Precompiler interface {
    // PrecompileWorkflow scans and pre-compiles all templates
    PrecompileWorkflow(workflowName string, templates map[string]string) error

    // GetPrecompiled retrieves a pre-compiled template
    GetPrecompiled(workflowName, key string) any

    // ClearPrecompiled removes pre-compiled templates
    ClearPrecompiled(workflowName string)
}

Usage

// At workflow load time
registry := NewPrecompiledRegistry()
templates := map[string]string{
    "scan:command": "nuclei -l {{Output}}/urls.txt",
    "scan:exports:result": "{{Output}}/nuclei.json",
}
registry.PrecompileWorkflow("my-workflow", templates)

// At execution time
if tpl := registry.GetPrecompiled("my-workflow", "scan:command"); tpl != nil {
    // Use pre-compiled template
}

Context Pooling

Context maps are pooled to reduce memory allocations:

var contextPool = sync.Pool{
    New: func() interface{} {
        return make(map[string]any, 32)
    },
}

func GetContext() map[string]any {
    return contextPool.Get().(map[string]any)
}

func PutContext(ctx map[string]any) {
    clear(ctx)
    contextPool.Put(ctx)
}

Normalized Boolean Handling

Boolean values are normalized for template compatibility:

func NormalizeBoolsToPooled(ctx map[string]any) map[string]any {
    pooled := GetContext()
    for k, v := range ctx {
        switch val := v.(type) {
        case bool:
            pooled[k] = val
        case string:
            if val == "true" {
                pooled[k] = true
            } else if val == "false" {
                pooled[k] = false
            } else {
                pooled[k] = v
            }
        default:
            pooled[k] = v
        }
    }
    return pooled
}

Generator Functions

Generator functions produce dynamic values:

type GeneratorFunc func(args ...string) (string, error)

// Built-in generators
var builtinGenerators = map[string]GeneratorFunc{
    "rand": func(args ...string) (string, error) {
        length := 8
        if len(args) > 0 {
            length, _ = strconv.Atoi(args[0])
        }
        return generateRandomString(length), nil
    },
    "uuid": func(args ...string) (string, error) {
        return uuid.New().String(), nil
    },
    "timestamp": func(args ...string) (string, error) {
        return strconv.FormatInt(time.Now().Unix(), 10), nil
    },
}

Usage

exports:
  random_id: "$rand(16)"
  unique_id: "$uuid()"
  time: "$timestamp()"

Batch Rendering

For high-throughput scenarios, batch rendering reduces lock acquisitions:

func (e *ShardedEngine) RenderBatch(requests []RenderRequest, ctx map[string]any) (map[string]string, error) {
    results := make(map[string]string, len(requests))

    // Prepare context once
    processedCtx := NormalizeBoolsToPooled(ctx)
    defer PutContext(processedCtx)

    // Group by shard
    shardGroups := make(map[uint32][]RenderRequest)
    for _, req := range requests {
        if !strings.Contains(req.Template, "{{") {
            results[req.Key] = req.Template
            continue
        }
        idx := e.getShardIndex(req.Template)
        shardGroups[idx] = append(shardGroups[idx], req)
    }

    // Process each shard group
    for idx, reqs := range shardGroups {
        shard := e.shards[idx]
        if err := e.renderShardBatch(shard, reqs, processedCtx, results); err != nil {
            return nil, err
        }
    }

    return results, nil
}

Secondary Template Rendering

Foreach loops use [[variable]] to avoid conflicts with pre-rendered {{variables}}:

func (e *ShardedEngine) RenderSecondary(template string, ctx map[string]any) (string, error) {
    if !strings.Contains(template, "[[") {
        return template, nil
    }

    // Convert [[ ]] to {{ }} for pongo2
    converted := strings.ReplaceAll(template, "[[", "{{")
    converted = strings.ReplaceAll(converted, "]]", "}}")

    return e.Render(converted, ctx)
}

Performance Benchmarks

Typical performance characteristics:

Operation	Single-threaded	10 Concurrent	100 Concurrent
Simple render	500ns	600ns	800ns
Cached render	200ns	250ns	400ns
Batch render (10)	1.5µs	2µs	3µs
Pre-compiled	150ns	200ns	350ns

Best Practices

Use pre-compilation for frequently executed workflows
Enable pooling for high-concurrency scenarios
Use batch rendering when rendering multiple templates with the same context
Avoid unnecessary templates - only use {{}} when variable substitution is needed
Use secondary syntax [[]] for foreach loop variables to avoid double-rendering issues

Getting Started

Advanced Usage

Understand Architecture

Workflows

Functions

Cloud

Advanced Topics & Extending Osmedeus

Reference

Template Engine Architecture

Template Engine Architecture

Template Syntax

Standard Variables

Secondary Variables (Foreach)

Generator Functions

Engine Types

TemplateEngine Interface

BatchRenderer Interface

ShardedEngine Architecture

Configuration

Performance Features

Render Flow

Precompiled Templates

Precompiler Interface

Usage

Context Pooling

Normalized Boolean Handling

Generator Functions

Usage

Batch Rendering

Secondary Template Rendering

Performance Benchmarks

Best Practices

​Template Engine Architecture

​Template Syntax

​Standard Variables

​Secondary Variables (Foreach)

​Generator Functions

​Engine Types

​TemplateEngine Interface

​BatchRenderer Interface

​ShardedEngine Architecture

​Configuration

​Performance Features

​Render Flow

​Precompiled Templates

​Precompiler Interface

​Usage

​Context Pooling

​Normalized Boolean Handling

​Generator Functions

​Usage

​Batch Rendering

​Secondary Template Rendering

​Performance Benchmarks

​Best Practices

Template Engine Architecture

Template Syntax

Standard Variables

Secondary Variables (Foreach)

Generator Functions

Engine Types

TemplateEngine Interface

BatchRenderer Interface

ShardedEngine Architecture

Configuration

Performance Features

Render Flow

Precompiled Templates

Precompiler Interface

Usage

Context Pooling

Normalized Boolean Handling

Generator Functions

Usage

Batch Rendering

Secondary Template Rendering

Performance Benchmarks

Best Practices