Build System

Opal Editor’s build system transforms markdown and template files into static websites. It supports multiple build strategies, template engines, and runs entirely in the browser with zero server dependencies.

Architecture

┌─────────────────────────────────────────┐
│        BuildRunnerFactory               │
│   (Strategy Selection)                  │
├─────────────────────────────────────────┤
│  ┌──────────────┬──────────────────┐    │
│  │  Freeform    │   Eleventy       │    │
│  │ BuildRunner  │  BuildRunner     │    │
│  └──────────────┴──────────────────┘    │
├─────────────────────────────────────────┤
│         BuildRunner (Base)              │
│  - DataflowGraph (DAG execution)        │
│  - TemplateManager (rendering)          │
│  - ObservableRunner (state)             │
├─────────────────────────────────────────┤
│  Source Disk  →  Process  →  Output     │
└─────────────────────────────────────────┘

Build Strategies

Freeform Strategy

Location: ~/workspace/source/src/services/build/strategies/FreeformBuildRunner.ts Simple build process:

Index source files
Copy static assets
Process templates and markdown

Eleventy Strategy

Location: ~/workspace/source/src/services/build/EleventyBuildRunner.ts Eleventy-inspired build with:

Data cascade (global, directory, template, frontmatter)
Layout inheritance
Custom permalinks
Configurable directories

Core Class: BuildRunner

Location: ~/workspace/source/src/services/build/BuildRunner.ts

Class Definition

export abstract class BuildRunner extends ObservableRunner<BuildDAO> {
  // Source and output
  get sourceDisk(): Disk;
  get outputDisk(): Disk;
  get outputPath(): AbsPath;
  get sourcePath(): AbsPath;
  
  // Build configuration
  get strategy(): BuildStrategy;
  get buildId(): string;
  
  // Template rendering
  protected templateManager?: TemplateManager;
  
  // Lifecycle
  async run(options?: { abortSignal?: AbortSignal }): Promise<BuildDAO>;
  cancel(): void;
  
  // Must implement
  protected abstract createBuildGraph(): DataflowGraph<any>;
}

Build Strategy Types

type BuildStrategy = "freeform" | "eleventy";

Creating Builds

Using BuildRunnerFactory

import { BuildRunnerFactory } from "@/services/build/BuildRunnerFactory";
import { Workspace } from "@/workspace/Workspace";

// Create new build
const runner = BuildRunnerFactory.Create({
  workspace,
  label: "My Site Build",
  strategy: "freeform" // or "eleventy"
});

// Run build
const result = await runner.run();

Restore Existing Build

// By build ID
const runner = await BuildRunnerFactory.Recall({
  buildId: "build-guid-123",
  workspace
});

// Show build (read-only)
const runner = BuildRunnerFactory.Show({ build, workspace });

Running Builds

Basic Build Execution

const runner = BuildRunnerFactory.Create({
  workspace,
  label: "Production Build",
  strategy: "freeform"
});

const buildDAO = await runner.run();

console.log(buildDAO.status);    // "success" | "error" | "pending"
console.log(buildDAO.fileCount); // Number of files generated
console.log(buildDAO.logs);      // Build log messages

With Abort Signal

const controller = new AbortController();

const buildPromise = runner.run({ 
  abortSignal: controller.signal 
});

// Cancel build
controller.abort();
// or
runner.cancel();

Monitor Build Progress

// Listen to log messages
runner.target.logs.forEach(log => {
  console.log(`[${log.level}] ${log.message}`);
});

// Check status
if (runner.target.status === "pending") {
  console.log("Build in progress...");
}

Freeform Build Strategy

Build Graph

protected createBuildGraph(): DataflowGraph<FreeformBuildContext> {
  return new DataflowGraph<FreeformBuildContext>()
    .node("init", [], async () => {})
    .node("indexSourceFiles", [], async () => {
      await this.sourceDisk.triggerIndex();
      return { sourceFilesIndexed: true };
    })
    .node("ensureOutputDirectory", [], async () => {
      await this.ensureOutputDirectory();
      return { outputDirectoryReady: true };
    })
    .node("copyAssets", ["indexSourceFiles", "ensureOutputDirectory"], async () => {
      await this.copyAssets();
      return { assetsReady: true };
    })
    .node("processTemplatesAndMarkdown", ["copyAssets"], async () => {
      await this.processTemplatesAndMarkdown();
      return { templatesProcessed: true };
    });
}

File Processing

Assets: Files that aren’t templates or markdown are copied as-is. Templates: .mustache, .ejs, .njk, .liquid files are rendered to HTML. Markdown: .md files are:

Parsed for frontmatter
Converted to HTML with marked.js
Wrapped in layout template
Output as .html

Example Usage

const runner = FreeformBuildRunner.Create({
  workspace,
  label: "Blog Build"
});

await runner.run();

Eleventy Build Strategy

Configuration

const runner = EleventyBuildRunner.Create({
  workspace,
  label: "Eleventy Build",
  config: {
    dir: {
      input: ".",          // Source directory
      output: "_site",     // Build output
      includes: "_includes", // Templates/layouts
      data: "_data",       // Global data files
      layouts: "_layouts"  // Optional: separate layouts dir
    }
  }
});

Data Cascade

Data is merged in priority order (lowest to highest):

Global data - JSON files from _data/ directory
Directory data - directory.json files
Template data - template.json files
Front matter - YAML/JSON in file header

// _data/site.json
{
  "title": "My Blog",
  "author": "Alice"
}

// posts/posts.json (directory data)
{
  "layout": "post",
  "tags": ["blog"]
}

// posts/hello.md
---
title: Hello World
date: 2024-01-15
---
# Hello

// Final data for posts/hello.md:
{
  "title": "Hello World",      // frontmatter (highest priority)
  "date": "2024-01-15",         // frontmatter
  "layout": "post",             // directory data
  "tags": ["blog"],            // directory data
  "author": "Alice",            // global data
  "page": {                     // auto-generated
    "url": "/posts/hello.html",
    "inputPath": "/posts/hello.md",
    "outputPath": "_site/posts/hello.html",
    "date": "2024-01-15T00:00:00.000Z"
  }
}

Layouts

Layouts are templates in the _includes/ directory. Layout file: _includes/post.mustache

<!DOCTYPE html>
<html>
<head>
  <title>{{title}}</title>
</head>
<body>
  <article>
    <h1>{{title}}</h1>
    <time>{{date}}</time>
    {{{content}}}
  </article>
</body>
</html>

Markdown file with layout:

---
layout: post.mustache
title: My Post
---

Post content here.

Layout Chaining

Layouts can inherit from other layouts:

<!-- _includes/base.mustache -->
<!DOCTYPE html>
<html>
<head><title>{{title}}</title></head>
<body>{{{content}}}</body>
</html>

<!-- _includes/post.mustache -->
---
layout: base.mustache
---
<article>
  <h1>{{title}}</h1>
  {{{content}}}
</article>

Permalinks

Customize output URLs with frontmatter:

---
permalink: /blog/my-custom-url/
---

Example Eleventy Build

// Create workspace with Eleventy structure
const workspace = await Workspace.CreateNew({
  name: "eleventy-site",
  files: {
    "/_data/site.json": JSON.stringify({ title: "My Site" }),
    "/_includes/base.mustache": baseTemplate,
    "/index.md": "---\ntitle: Home\n---\n# Welcome",
    "/posts/post1.md": post1Content,
    "/posts/post2.md": post2Content,
  },
  diskType: "IndexedDbDisk"
});

// Build
const runner = EleventyBuildRunner.Create({
  workspace,
  label: "Production"
});

const result = await runner.run();

// Output structure:
// _site/
//   index.html
//   posts/
//     post1.html
//     post2.html

Template Rendering

TemplateManager Integration

BuildRunner uses TemplateManager for rendering:

const html = await this.templateManager.renderTemplate(
  absPath("/template.mustache"),
  {
    title: "Page Title",
    content: markdownHtml,
    globalCssPath: "/styles/global.css"
  }
);

Supported Template Engines

Mustache (.mustache) - Default
EJS (.ejs)
Nunjucks (.njk, .nunchucks)
Liquid (.liquid)

Template Helpers

TemplateManager provides built-in helpers:

<!-- Format date -->
{{formatDate date "YYYY-MM-DD"}}

<!-- Format number -->
{{formatNumber 1234.56 "0,0.00"}}

<!-- Slugify -->
{{slugify "Hello World"}} <!-- hello-world -->

Build Pipeline

DataflowGraph Execution

Builds use a directed acyclic graph (DAG) for execution:

const graph = new DataflowGraph()
  .node("step1", [], async () => {
    // No dependencies
    return { result: "data" };
  })
  .node("step2", ["step1"], async (ctx) => {
    // Runs after step1
    console.log(ctx.result); // "data"
    return { more: "results" };
  })
  .node("step3", ["step1", "step2"], async (ctx) => {
    // Runs after both step1 and step2
    return {};
  });

await graph.run({});

Build Context

Context object tracks build state:

interface BuildContext {
  outputDirectoryReady?: boolean;
  sourceFilesIndexed?: boolean;
  assetsReady?: boolean;
  templatesProcessed?: boolean;
  pages?: PageData[];
  posts?: PageData[];
}

File Processing Methods

Copy Assets

protected async copyAssets(): Promise<void> {
  for (const node of this.sourceDisk.fileTree.iterator(
    (node) => node.isTreeFile() && FilterOutSpecialDirs(node.path)
  )) {
    if (this.shouldCopyAsset(node)) {
      await this.copyFileToOutput(node);
    }
  }
}

protected shouldCopyAsset(node: TreeNode): boolean {
  const path = relPath(node.path);
  return !path.startsWith("_") && 
         !this.isTemplateFile(node) && 
         !this.isMarkdownFile(node);
}

Process Markdown

protected async processMarkdown(node: TreeNode): Promise<void> {
  const content = String(await this.sourceDisk.readFile(node.path));
  const { data: frontMatter, content: markdownContent } = matter(content);
  
  // Convert markdown to HTML
  const htmlContent = await marked(markdownContent);
  
  // Load layout
  const layout = frontMatter.layout 
    ? await this.loadTemplate(relPath(`_layouts/${frontMatter.layout}`))
    : DefaultPageLayout;
  
  // Render with layout
  const html = mustache.render(layout, {
    content: htmlContent,
    title: frontMatter.title,
    ...frontMatter
  });
  
  // Write output
  const outputPath = this.getOutputPathForMarkdown(relPath(node.path));
  await this.writeFile(outputPath, await prettifyMime("text/html", html));
}

Process Templates

protected async processTemplate(node: TreeNode): Promise<void> {
  const content = String(await this.sourceDisk.readFile(node.path));
  const outputPath = this.getOutputPathForTemplate(relPath(node.path));
  
  // Render template
  const html = await this.templateManager.renderTemplate(
    node.path,
    {
      globalCssPath: await this.getGlobalCssPath(),
      date: new Date().toISOString()
    }
  );
  
  await this.writeFile(outputPath, await prettifyMime("text/html", html));
}

Build Output

BuildDAO

Build results are stored as BuildDAO:

interface BuildDAO {
  guid: string;
  label: string;
  strategy: BuildStrategy;
  status: "idle" | "pending" | "success" | "error";
  error: string | null;
  fileCount: number;
  logs: Array<{
    level: "info" | "warning" | "error";
    message: string;
    timestamp: number;
  }>;
  timestamp: number;
  workspaceId: string;
  sourcePath: AbsPath;
  buildPath: AbsPath;
  
  // Methods
  save(): Promise<void>;
  hydrate(): Promise<BuildDAO>;
  getOutputPath(): AbsPath;
  getSourceDisk(): Disk;
}

Reading Build Results

const build = await BuildDAO.FetchFromGuid(buildId);

console.log(build.status);      // "success"
console.log(build.fileCount);   // 42
console.log(build.logs);        // Build logs

// Access output files
const outputDisk = build.getSourceDisk();
const outputPath = build.getOutputPath();

const indexHtml = await outputDisk.readFile(
  joinPath(outputPath, relPath("index.html"))
);

Logging

// Log from within BuildRunner
this.log("Processing files...", "info");
this.log("Warning: missing layout", "warning");
this.log("Build failed", "error");

// Logs are stored in build.logs
build.logs.forEach(({ level, message, timestamp }) => {
  console.log(`[${level}] ${message}`);
});

Advanced Features

Custom Build Strategy

Extend BuildRunner to create custom strategies:

import { BuildRunner } from "@/services/build/BuildRunner";
import { DataflowGraph } from "@/lib/DataFlow";

class CustomBuildRunner extends BuildRunner {
  protected createBuildGraph(): DataflowGraph<any> {
    return new DataflowGraph()
      .node("init", [], async () => {})
      .node("customStep", ["init"], async () => {
        // Your custom logic
        return {};
      });
  }
}

Pre-process Files

// Override methods to customize processing
protected async processMarkdown(node: TreeNode): Promise<void> {
  let content = String(await this.sourceDisk.readFile(node.path));
  
  // Custom preprocessing
  content = await myCustomPreprocessor(content);
  
  // Continue with normal processing
  await super.processMarkdown(node);
}

Best Practices

Use Appropriate Strategy

// Simple sites - use Freeform
const runner = FreeformBuildRunner.Create({ workspace, label: "Simple" });

// Complex sites with data - use Eleventy
const runner = EleventyBuildRunner.Create({ 
  workspace, 
  label: "Complex",
  config: { dir: { /* ... */ } }
});

Handle Build Errors

try {
  const result = await runner.run();
  
  if (result.status === "error") {
    console.error("Build failed:", result.error);
    console.error("Logs:", result.logs);
  }
} catch (error) {
  console.error("Build crashed:", error);
}

Monitor Progress

const runner = BuildRunnerFactory.Create({ workspace, label: "Build" });

// Observable state
const build = runner.target;

setInterval(() => {
  console.log(`Status: ${build.status}`);
  console.log(`Files: ${build.fileCount}`);
}, 1000);

await runner.run();

Optimize Large Builds

// Use skipListeners for build disks
const buildDisk = new MemDisk("build-output");
await buildDisk.init({ skipListeners: true });

// Batch write operations
const files = await Promise.all(
  nodes.map(async node => [node.path, await process(node)])
);
await buildDisk.newFiles(files);

Common Patterns

Build and Deploy

const runner = BuildRunnerFactory.Create({
  workspace,
  label: "Production Build",
  strategy: "eleventy"
});

const build = await runner.run();

if (build.status === "success") {
  // Deploy build output
  const outputDisk = build.getSourceDisk();
  const outputPath = build.getOutputPath();
  
  await deployToNetlify(outputDisk, outputPath);
}

Preview Build

// Create temp disk for preview
const previewDisk = new MemDisk("preview-" + nanoid());
await previewDisk.init({ skipListeners: true });

const runner = FreeformBuildRunner.Create({
  workspace,
  label: "Preview",
  build: BuildDAO.CreateNew({
    label: "Preview",
    workspaceId: workspace.guid,
    disk: previewDisk,
    sourceDisk: workspace.disk,
    strategy: "freeform"
  })
});

await runner.run();

// Serve preview from previewDisk

Workspace System - Source workspace
Storage System - Read source, write output
Git System - Version built files

Documentation Index

​Build System

​Architecture

​Build Strategies

​Freeform Strategy

​Eleventy Strategy

​Core Class: BuildRunner

​Class Definition

​Build Strategy Types

​Creating Builds

​Using BuildRunnerFactory

​Restore Existing Build

​Running Builds

​Basic Build Execution

​With Abort Signal

​Monitor Build Progress

​Freeform Build Strategy

​Build Graph

​File Processing

​Example Usage

​Eleventy Build Strategy

​Configuration

​Data Cascade

​Layouts

​Layout Chaining

​Permalinks

​Example Eleventy Build

​Template Rendering

​TemplateManager Integration

​Supported Template Engines

​Template Helpers

​Build Pipeline

​DataflowGraph Execution

​Build Context

​File Processing Methods

​Copy Assets

​Process Markdown

​Process Templates

​Build Output

​BuildDAO

​Reading Build Results

​Logging

​Advanced Features

​Custom Build Strategy

​Pre-process Files

​Best Practices

​Use Appropriate Strategy

​Handle Build Errors

​Monitor Progress

​Optimize Large Builds

​Common Patterns

​Build and Deploy

​Preview Build

​Related Systems