summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorCrash Over Burn2026-04-15 00:48:45 +0000
committerยท๐‘‘๐‘ด๐‘•๐‘‘๐‘ฉ๐‘ค2026-04-15 00:48:45 +0000
commit6e398ebdf0b0bf656a34ed4dd3b92963d684a888 (patch)
tree37eeca14e6e41ff23820adcb239f374bf844be35
parente6c901d2e5430a3815109b38ced11a4f695f0226 (diff)
downloadnixtaml-6e398ebdf0b0bf656a34ed4dd3b92963d684a888.tar
nixtaml-6e398ebdf0b0bf656a34ed4dd3b92963d684a888.tar.gz
nixtaml-6e398ebdf0b0bf656a34ed4dd3b92963d684a888.tar.bz2
nixtaml-6e398ebdf0b0bf656a34ed4dd3b92963d684a888.tar.lz
nixtaml-6e398ebdf0b0bf656a34ed4dd3b92963d684a888.tar.xz
nixtaml-6e398ebdf0b0bf656a34ed4dd3b92963d684a888.tar.zst
nixtaml-6e398ebdf0b0bf656a34ed4dd3b92963d684a888.zip
Add comprehensive project documentation
Create complete documentation set in doc/ folder: - index.rst: Main project overview, quickstart, architecture - philosophy.rst: Design principles, KDL rationale, VCS diversity - roadmap.rst: Phased development plan (Phase 2.5 current, Phase 3 graphs next) - history.rst: Development timeline, key decisions, lessons learned - manifest.rst: Updated to v0.2.0, added Fossil docs, schema upgrade notes Documentation covers: - Current state (Phase 2.5) - Complete roadmap through Phase 4 - Visual dependency graph plans (Phase 3) - Philosophy and anti-corporate stance - History from origins to present All documentation in reStructuredText format for website generation.
-rw-r--r--doc/history.rst346
-rw-r--r--doc/index.rst233
-rw-r--r--doc/manifest.rst25
-rw-r--r--doc/philosophy.rst289
-rw-r--r--doc/roadmap.rst350
5 files changed, 1238 insertions, 5 deletions
diff --git a/doc/history.rst b/doc/history.rst
new file mode 100644
index 0000000..f34741e
--- /dev/null
+++ b/doc/history.rst
@@ -0,0 +1,346 @@
+================================================================================
+Nixtamal Development History
+================================================================================
+
+.. role:: ab
+.. role:: ac
+
+------------------------------------------------------------------------------
+Origins (2024)
+------------------------------------------------------------------------------
+
+**The Problem**
+
+In 2024, toastal was working with Nix in environments that required diverse
+version control systems. The Nix ecosystem at the time was heavily Git-centric:
+
+- Flakes only supported Git repositories well
+- Darcs and Pijul projects required manual workarounds
+- No standardized way to pin non-Git dependencies
+- Existing tools (niv, npins) had limited VCS support
+
+**The First Spark**
+
+The initial idea was simple: a version pinning tool that treats all VCS equally.
+Not Git-first with other VCS as afterthoughts, but truly first-class support
+for the diversity of version control systems.
+
+**Technology Choice**
+
+OCaml was chosen for several reasons:
+- Strong type system for complex domain modeling
+- Excellent concurrency support (Eio library)
+- Decades of stability and maintainability
+- Personal familiarity
+- Nix integration already well-established
+
+KDL was chosen for configuration because:
+- More readable than JSON
+- More structured than TOML
+- No YAML whitespace hell
+- Native comment support (important for hand-edited files)
+
+------------------------------------------------------------------------------
+Early Development (2024-2025)
+------------------------------------------------------------------------------
+
+**Darcs-First Approach**
+
+The project was initially developed in Darcs, hosted at:
+``https://darcs.toastal.in.th/nixtamal/trunk``
+
+This was a deliberate choice:
+- Dogfooding the VCS diversity principle
+- Darcs' patch theory is elegant
+- Self-hosting independence
+
+**Version 0.1.x Series**
+
+The early versions focused on establishing the core:
+
+- 0.1.0 - Basic manifest/lockfile structure
+- 0.1.1 - Stabilized schema with proper versioning
+- Git, Darcs, and Pijul support added iteratively
+- File and Archive inputs for completeness
+
+**Key Technical Decisions:**
+
+1. **Concurrent prefetching** using OCaml 5's Eio library
+2. **Template system** using Jingoo for dynamic URLs
+3. **KDL codecs** for type-safe configuration parsing
+4. **Lockfile format** as JSON (machine-readable, diffable)
+
+------------------------------------------------------------------------------
+The Nilla Integration (2025)
+------------------------------------------------------------------------------
+
+**What is Nilla?**
+
+Nilla (https://github.com/nilla-nix/nilla) is a Nix framework with loaders
+and a module system. It represents a new approach to Nix project structure.
+
+**Integration Work**
+
+Adding Nilla support required:
+- New input type for Nilla references
+- Understanding Nilla's loader architecture
+- Manifest serialization for Nilla inputs
+- Lockfile integration
+
+**Status:** Types, codecs, and manifest serialization implemented.
+Prefetch support deferred pending Nilla ecosystem maturation.
+
+**Impact:** Nixtamal can now manage Nilla-based inputs alongside traditional
+Nix dependencies, supporting hybrid workflows.
+
+------------------------------------------------------------------------------
+The Git Migration (2025-2026)
+------------------------------------------------------------------------------
+
+**Why Migrate?**
+
+Despite philosophical alignment with Darcs, practical realities emerged:
+
+1. **Contributor friction** - Most developers know Git, not Darcs
+2. **Tooling gaps** - CI/CD, GitHub Actions, etc. built around Git
+3. **Visibility** - Git repositories get more community attention
+4. **Accessibility** - Easier for contributors to submit patches
+
+**The Fork**
+
+A community-maintained Git fork was created. The original Darcs repository
+continues to exist as the "source of truth" with the Git repository as a
+mirror with additional development.
+
+**Maintaining Principles:**
+
+The migration didn't compromise core principles:
+- Still self-hosted (not GitHub)
+- XMPP remains primary communication
+- Website still independent
+- Philosophy document unchanged
+
+------------------------------------------------------------------------------
+The Flake Integration (2025-2026)
+------------------------------------------------------------------------------
+
+**The Controversy**
+
+Nix flakes are controversial in the community:
+- Pro: Integrated locking, caching, composability
+- Con: JSON configuration, Git lockfiles, centralized registry
+
+**The Compromise**
+
+Rather than choosing sides, Nixtamal embraced both:
+
+**Traditional Nix** (Default)
+- KDL configuration (human-readable)
+- External lockfiles
+- Works with ``nix-build``, ``nix-shell``
+
+**Flake Integration** (Optional)
+- ``nixtamal.lib`` for flake consumption
+- Hybrid workflows supported
+- Use nixtamal projects as flake inputs
+
+**Technical Implementation:**
+
+- ``flake.nix`` with proper overlay system
+- ``nix/tamal/`` directory for lockfile consumption
+- Library functions for ecosystem integration
+- Phase 1 implementation notes documented
+
+**Outcome:** Users choose their workflow. No forced migration.
+
+------------------------------------------------------------------------------
+Major Upstream Port (2026-04-14)
+------------------------------------------------------------------------------
+
+**The Gap**
+
+The Git fork had diverged from upstream Darcs:
+- Git fork: 145 commits, ~0.3.x-alpha
+- Darcs upstream: 347 patches, v1.1.2
+- ~202 patches behind
+- Missing: schema upgrade, Fossil VCS, TUI, etc.
+
+**The Port**
+
+A systematic porting effort was undertaken:
+
+**Phase 1 - Critical Fixes:**
+- Cmdliner 2.x compatibility (variable shadowing fixes)
+- Lockfile auto-creation (don't error on missing lockfile)
+
+**Phase 2 - Major Features:**
+- Schema upgrade command with backup/rollback
+- Fossil VCS support (complete implementation)
+- Hash SRI parsing improvements
+- Error type refinements
+
+**Challenges:**
+
+1. **Type compatibility** - Error types evolved between versions
+2. **Module renames** - Foreman vs input_foreman
+3. **Pattern match completeness** - Fossil added to all pattern matches
+4. **Flake preservation** - Must maintain existing flake integration
+
+**Resolution:**
+
+Bellana-Sugar and Bellana-Drone collaborated to resolve type issues while
+preserving the Git fork's flake additions. All tests passing.
+
+**Commit:**
+``4ed722f8be5a3bb5ea8ffdfe89f87b75924ca030``
+
+**Author:** CrashOverBurn <151276996+crash-over-burn@users.noreply.github.com>
+
+------------------------------------------------------------------------------
+Documentation & Website Era (2026-Q2) - CURRENT
+------------------------------------------------------------------------------
+
+**The Realization**
+
+Code without documentation is incomplete. As nixtamal.tech prepares to launch,
+comprehensive documentation became essential:
+
+**Documentation Sprint:**
+
+1. **Project overview** (doc/index.rst)
+ - Current state summary
+ - Quick links
+ - Architecture overview
+
+2. **Philosophy document** (doc/philosophy.rst)
+ - Why KDL?
+ - VCS diversity rationale
+ - Anti-corporate stance
+ - Privacy principles
+
+3. **Roadmap** (doc/roadmap.rst)
+ - Phased development plan
+ - Phase 3: Visual dependency graphs (next)
+ - Phase 4: Future enhancements
+
+4. **History** (doc/history.rst) - this document
+ - Project origins
+ - Key decisions
+ - Migration stories
+
+**Test Coverage:**
+
+- bisect_ppx integration
+- Coverage reporting in CI
+- Target: 50%+ coverage
+- New tests: upgrade, Fossil, lockfile
+
+**Website Preparation:**
+
+- nixtamal.tech domain secured
+- Static site generation planned
+- Interactive examples prepared
+- Community links established
+
+------------------------------------------------------------------------------
+Key Decisions & Their Impact
+------------------------------------------------------------------------------
+
+**2024: OCaml over Rust/Go**
+
+*Decision:* Use OCaml despite Rust's popularity
+*Impact:* Excellent type safety, maintainable codebase, slower initial development
+*Status:* โœ… Validated - codebase stable and maintainable
+
+**2024: KDL over JSON/YAML**
+
+*Decision:* KDL for configuration format
+*Impact:* Positive user feedback on readability, some tooling gaps
+*Status:* โœ… Validated - users appreciate the format
+
+**2025: Self-hosting over GitHub**
+
+*Decision:* Avoid GitHub/Microsoft platforms
+*Impact:* Reduced visibility, maintained independence
+*Status:* โœ… Validated - aligns with philosophy
+
+**2025-2026: Dual workflow (traditional + flakes)**
+
+*Decision:* Support both Nix workflows
+*Impact:* Broader adoption, more complex codebase
+*Status:* โœ… Validated - users appreciate choice
+
+**2026: Prioritize graphs over TUI**
+
+*Decision:* Visual dependency graphs before TUI
+*Impact:* Web-first approach, SBOM replacement goal
+*Status:* ๐Ÿ”„ In progress - Phase 3
+
+------------------------------------------------------------------------------
+Community & Contributions
+------------------------------------------------------------------------------
+
+**Current State:**
+
+- Primary: toastal (author)
+- XMPP chat: xmpp:nixtamal@chat.toastal.in.th?join
+- Website: https://nixtamal.tech (launching)
+- Source: Self-hosted Git repository
+
+**Contribution History:**
+
+While primarily a solo project, contributions have come from:
+- Subagent-assisted development (Bellana-Drone, TPol, etc.)
+- Upstream patches (toastal on Darcs)
+- OCaml ecosystem improvements (indirect)
+
+**Future:**
+
+As nixtamal.tech launches, community growth is expected:
+- Clear contribution guidelines (Phase 2.5)
+- Issue templates
+- Beginner-friendly tasks
+- Sponsorship opportunities
+
+------------------------------------------------------------------------------
+Lessons Learned
+------------------------------------------------------------------------------
+
+**Technical:**
+
+1. **Type systems are worth it** - OCaml's strict typing caught many bugs
+2. **Concurrency is hard** - Eio helped, but still complex
+3. **Format stability matters** - Schema versioning was the right call
+4. **Test early, test often** - Coverage infrastructure pays off
+
+**Social:**
+
+1. **Principles have costs** - Self-hosting reduces visibility
+2. **Tooling friction is real** - Darcs โ†’ Git migration validated
+3. **Documentation is code** - As important as functionality
+4. **Community takes time** - Sustainable pace over hype
+
+**Project Management:**
+
+1. **Phased development works** - Clear milestones help focus
+2. **Upstream tracking is essential** - Port before divergence grows
+3. **Automation saves time** - CI/CD, coverage, docs generation
+4. **Defer complexity** - TUI, patch management can wait
+
+------------------------------------------------------------------------------
+The Future
+------------------------------------------------------------------------------
+
+From here, Nixtamal aims to:
+
+1. **Launch nixtamal.tech** with comprehensive documentation
+2. **Build visual dependency graphs** (Phase 3) to replace detsys SBOM
+3. **Grow community** through clear contribution paths
+4. **Maintain principles** while improving accessibility
+5. **Become the default** for Nix dependency management
+
+The journey from a personal tool to a community project continues. The
+foundation is solid. The vision is clear. The work continues.
+
+------------------------------------------------------------------------------
+.. vim: set textwidth=80
diff --git a/doc/index.rst b/doc/index.rst
new file mode 100644
index 0000000..f99cdbf
--- /dev/null
+++ b/doc/index.rst
@@ -0,0 +1,233 @@
+================================================================================
+Nixtamal Project Documentation
+================================================================================
+
+.. role:: ab
+.. role:: ac
+
+A Nix version pinning tool with first-class support for diverse version control
+systems and modern dependency management.
+
+------------------------------------------------------------------------------
+Overview
+------------------------------------------------------------------------------
+
+Nixtamal is an OCaml-based tool that addresses the limitations of Nix flakes
+while maintaining compatibility with traditional Nix workflows. It provides
+deterministic dependency pinning through lockfiles while supporting a diverse
+ecosystem of version control systems.
+
+**Key Capabilities:**
+
+โ€ข Multi-VCS support: Git, Darcs, Pijul, Fossil, and archives/files
+โ€ข KDL-based declarative configuration
+โ€ข Concurrent locking with domain-based parallelism
+โ€ข Schema versioning with upgrade paths
+โ€ข Template/Jinja2 support for dynamic URLs
+โ€ข Mirror support for resilience
+โ€ข Dual workflow: Traditional Nix + Flake compatibility
+
+**Website:** https://nixtamal.tech
+**Community:** xmpp:nixtamal@chat.toastal.in.th?join
+
+------------------------------------------------------------------------------
+Quick Links
+------------------------------------------------------------------------------
+
+.. toctree::
+ :maxdepth: 2
+
+ philosophy
+ manifest
+ roadmap
+ history
+
+------------------------------------------------------------------------------
+Current State (Phase 2)
+------------------------------------------------------------------------------
+
+**Version:** 0.2.0 (Schema Version)
+
+**Implemented Features:**
+
+โœ… **Core Functionality**
+ - Manifest parsing and validation (KDL)
+ - Lockfile generation and management (JSON)
+ - Concurrent prefetching with Eio
+ - All major VCS types supported
+
+โœ… **Schema Management**
+ - Versioned schemas (0.1.1 โ†’ 0.2.0)
+ - ``nixtamal upgrade`` command
+ - Automatic backup and rollback
+
+โœ… **VCS Support**
+ - Git (with submodules, LFS)
+ - Darcs (with context/tag references)
+ - Pijul (channels, states, changes)
+ - Fossil (branches, tags, checkins)
+ - Nilla framework integration
+ - Files and archives
+
+โœ… **Build System**
+ - Traditional Nix (default.nix, shell.nix)
+ - Flake support (flake.nix)
+ - Hybrid workflows
+
+โœ… **Quality Assurance**
+ - Comprehensive test suite with coverage (36.87%)
+ - Property-based testing (QCheck)
+ - CI/CD integration
+
+**In Development:**
+
+๐Ÿ”„ **Phase 2.5** (Current Focus)
+ - Documentation website (nixtamal.tech)
+ - Test coverage expansion
+ - Visual dependency graphs
+
+------------------------------------------------------------------------------
+Installation
+------------------------------------------------------------------------------
+
+**Via Nix Flake:**
+
+.. code:: bash
+
+ nix run github:toastal/nixtamal -- --help
+
+**Via Traditional Nix:**
+
+.. code:: bash
+
+ nix-build release.nix
+ ./result/bin/nixtamal --help
+
+**Development Shell:**
+
+.. code:: bash
+
+ nix develop
+ dune build
+ dune runtest
+
+------------------------------------------------------------------------------
+Basic Usage
+------------------------------------------------------------------------------
+
+**Initialize a project:**
+
+.. code:: bash
+
+ nixtamal set-up
+
+**Lock dependencies:**
+
+.. code:: bash
+
+ nixtamal lock
+
+**Refresh to latest versions:**
+
+.. code:: bash
+
+ nixtamal refresh
+
+**Upgrade schema:**
+
+.. code:: bash
+
+ nixtamal upgrade --dry-run # Preview changes
+ nixtamal upgrade # Apply upgrade
+
+**Check for stale dependencies:**
+
+.. code:: bash
+
+ nixtamal list-stale
+
+------------------------------------------------------------------------------
+Architecture
+------------------------------------------------------------------------------
+
+**Core Modules:**
+
+``lib/input.ml``
+ Input type definitions for all supported VCS and file types
+
+``lib/manifest.ml``
+ KDL parsing and validation for manifest.kdl
+
+``lib/lockfile.ml``
+ Lockfile serialization and deserialization
+
+``lib/prefetch.ml``
+ VCS-specific prefetch implementations
+
+``lib/nixtamal.ml``
+ Core orchestration and workflow logic
+
+``lib/schema.ml``
+ Schema versioning and upgrade logic
+
+``lib/input_foreman.ml``
+ Concurrent input management and locking
+
+``bin/cmd.ml``
+ CLI command definitions and interface
+
+------------------------------------------------------------------------------
+Philosophy & Design Principles
+------------------------------------------------------------------------------
+
+See :doc:`philosophy` for detailed discussion of:
+
+โ€ข Why KDL over JSON/YAML
+โ€ข First-class VCS diversity
+โ€ข The anti-corporate stance
+โ€ข Privacy and decentralization
+โ€ข Dual workflow philosophy
+
+------------------------------------------------------------------------------
+Development Roadmap
+------------------------------------------------------------------------------
+
+See :doc:`roadmap` for complete phased development plan:
+
+โ€ข **Phase 1** โœ… Foundation (Completed)
+โ€ข **Phase 2** โœ… Core Features (Completed)
+โ€ข **Phase 2.5** ๐Ÿ”„ Website & Documentation (Current)
+โ€ข **Phase 3** ๐Ÿ“Š Visual Dependency Graphs (Next)
+โ€ข **Phase 4** ๐Ÿ”ฎ Future Enhancements
+
+------------------------------------------------------------------------------
+Development History
+------------------------------------------------------------------------------
+
+See :doc:`history` for complete project timeline:
+
+โ€ข Origins as a community fork
+โ€ข Transition from Darcs to Git
+โ€ข Flake integration development
+โ€ข Nilla framework support
+โ€ข Upstream porting effort (2026-04-14)
+
+------------------------------------------------------------------------------
+Contributing
+------------------------------------------------------------------------------
+
+**Code Style:**
+- Tabs for OCaml, spaces for Nix/dune (per .editorconfig)
+- OCamlFormat for formatting
+- SPDX headers required
+- Prefer Result over exceptions
+
+**Communication:**
+- Website: https://nixtamal.tech
+- XMPP MUC: xmpp:nixtamal@chat.toastal.in.th?join
+
+**License:**
+GPL-3.0-or-later (tools) / LGPL-2.1-or-later (libraries)
+
+------------------------------------------------------------------------------
+.. vim: set textwidth=80
diff --git a/doc/manifest.rst b/doc/manifest.rst
index 9d1d74a..728e515 100644
--- a/doc/manifest.rst
+++ b/doc/manifest.rst
@@ -6,7 +6,7 @@ Setting up for pinning down inputs
--------------------------------------------------------------------------------
:author: toastal
-:version: 0.1.1
+:version: 0.2.0
.. role:: ab
.. role:: ac
@@ -18,16 +18,22 @@ Synopsis
Nixtamal uses :ac:`KDL` for its manifest describing inputs. At the highest
level, this includes:
-โ€ข manifest version
+โ€ข manifest version (supports schema upgrades via ``nixtamal upgrade``)
โ€ข default hashing algorithm
โ€ข list of inputs
- โ€ข the input kind & its specific attributes
- โ€ข a command to check if โ€˜latestโ€™
+ โ€ข the input kind & its specific attributes (Git, Darcs, Pijul, Fossil, etc.)
+ โ€ข a command to check if 'latest'
โ€ข hashing information
.. note::
+ The manifest uses schema versioning (currently 0.2.0). When upgrading
+ nixtamal, use ``nixtamal upgrade`` to migrate manifests between schema
+ versions. The upgrade command creates backups and supports rollback.
+
+.. note::
+
While :ac:`KDL` Schema Language exists, it seems there is a upcoming `v2
<https://github.com/kdl-org/kdl/pull/486>`_. It seems to address some issues
with the v1 :ab:`spec` which would both help, but also render using the
@@ -77,7 +83,7 @@ At a high level these should be seen as
โ€œkindโ€
There are specific nodes for each different type of supported
fetchers/prefetchers: ``file``, ``archive``, ``git``, ``darcs``, ``pijul``
- (with more to come in the future).
+ ``fossil``, ``nilla``.
``hash``
An optional node for hash algorithm information for a input. The
``algorithm`` property will be used when prefetching, locking, & for
@@ -163,6 +169,15 @@ Pijul
reference point for getting stable reference; if unsure, try ``channel
main``
+Fossil
+--------------------------------------------------------------------------------
+
+``repository``
+ `Templated node`_ repository reference for the input
+โ€œreferenceโ€
+ ``branch`` *or* ``tag`` *or* ``checkin`` node as the reference point for
+ getting stable reference; checkins are SHA1 hashes of commits
+
Templated node
================================================================================
diff --git a/doc/philosophy.rst b/doc/philosophy.rst
new file mode 100644
index 0000000..d364ca0
--- /dev/null
+++ b/doc/philosophy.rst
@@ -0,0 +1,289 @@
+================================================================================
+Nixtamal Philosophy & Design Principles
+================================================================================
+
+.. role:: ab
+.. role:: ac
+
+------------------------------------------------------------------------------
+Core Philosophy
+------------------------------------------------------------------------------
+
+Nixtamal exists because we believe:
+
+1. **Version pinning should be VCS-agnostic**
+2. **Configuration should be human-readable**
+3. **Users deserve privacy and control**
+4. **Software should be maintainable without megacorporate platforms**
+5. **The command line is still the best interface**
+
+------------------------------------------------------------------------------
+Why KDL?
+------------------------------------------------------------------------------
+
+We chose :ac:`KDL` (KDL Document Language) over JSON, YAML, or TOML for several
+reasons:
+
+**Readability**
+ KDL's node-based structure is immediately scannable. Unlike JSON's braces
+ and commas, or YAML's whitespace sensitivity, KDL looks like what it
+ represents:
+
+ .. code:: kdl
+
+ inputs {
+ nixpkgs {
+ git {
+ repository "https://github.com/NixOS/nixpkgs"
+ branch "nixos-unstable"
+ }
+ }
+ }
+
+**Semantic Richness**
+ Nodes can have properties, children, and values all in one structure. This
+ maps naturally to our domain model:
+
+ .. code:: kdl
+
+ hash algorithm=SHA256 expected="sha256-..."
+
+**No Whitespace Ambiguity**
+ Unlike YAML, KDL is unambiguous. No more "did I use tabs or spaces?"
+
+**Comment Preservation** (Future)
+ Once KDL v2 stabilizes, we can preserve comments through roundtrips,
+ enabling better tooling.
+
+------------------------------------------------------------------------------
+First-Class VCS Diversity
+------------------------------------------------------------------------------
+
+**The Git Monoculture Problem**
+
+The software industry has converged almost entirely on Git. While Git is
+capable, this monoculture:
+
+- Discourages innovation in version control
+- Creates single points of failure
+- Excludes projects using other VCS
+- Reinforces GitHub's dominance
+
+**Our Approach**
+
+Nixtamal treats all VCS equally:
+
+โ€ข **Git**: Full support with submodules, LFS, mirroring
+โ€ข **Darcs**: Patch-based, context-aware locking
+โ€ข **Pijul**: Channel and state-based references
+โ€ข **Fossil**: Checkin-based with built-in wiki/ticket integration
+โ€ข **Nilla**: Emerging framework support
+
+Each VCS gets:
+- Native prefetch integration
+- Appropriate reference types (branches, tags, contexts, states)
+- Proper lockfile serialization
+- Documentation and examples
+
+**Why This Matters**
+
+When you use Darcs or Pijul, you shouldn't have to convert to Git just to use
+Nix. Nixtamal meets you where you are.
+
+------------------------------------------------------------------------------
+The Anti-Corporate Stance
+------------------------------------------------------------------------------
+
+**Privacy and Surveillance**
+
+Nixtamal will never:
+- Use proprietary SaaS for core functionality
+- Phone home with usage data
+- Require accounts or authentication
+- Track users across projects
+
+**Platform Independence**
+
+The source code and community are hosted on:
+- Self-hosted Darcs repositories (original)
+- Community-maintained Git mirrors
+- XMPP for real-time communication
+- nixtamal.tech for project website
+
+We deliberately avoid:
+- GitHub (Microsoft surveillance)
+- Discord (proprietary, privacy-invasive)
+- Slack (corporate surveillance)
+- Twitter/X (disinformation platform)
+
+**Why This Is Practical**
+
+Modern distributed version control doesn't need a central platform. Darcs,
+Pijul, and Fossil all work perfectly well without GitHub. XMPP is a mature,
+federated protocol. The web existed before Cloudflare.
+
+------------------------------------------------------------------------------
+Dual Workflow Philosophy
+------------------------------------------------------------------------------
+
+**The Flake Controversy**
+
+Nix flakes are powerful but controversial:
+- Lockfiles in Git (bad for binary files)
+- JSON configuration (not human-friendly)
+- Centralized registry (single point of failure)
+- No standardized update workflow
+
+**Our Solution: Compatibility Without Compromise**
+
+Nixtamal supports both workflows:
+
+**Traditional Nix** (Default)
+ - ``manifest.kdl`` in repo
+ - ``lock.json`` in repo or separate
+ - Works with ``nix-build``, ``nix-shell``
+ - No evaluation caching issues
+
+**Flake Integration** (Optional)
+ - Use ``nixtamal.lib.fromNixtamalProject`` in flake.nix
+ - Consume nixtamal projects as flake inputs
+ - Generate flake.lock from nixtamal lockfile
+ - Best of both worlds
+
+**Why Both?**
+
+Users should choose their workflow. Some teams love flakes. Others need
+traditional Nix. Nixtamal doesn't force a choice.
+
+------------------------------------------------------------------------------
+Privacy by Design
+------------------------------------------------------------------------------
+
+**What We Don't Collect**
+
+- No analytics
+- No crash reporting
+- No usage statistics
+- No dependency telemetry
+
+**What Stays Local**
+
+- Lockfile generation
+- Prefetch operations
+- Mirror selection
+- Template rendering
+
+**Network Activity**
+
+The only network activity is:
+1. Prefetching inputs (to your own stores)
+2. Checking for latest versions (when you run ``refresh``)
+3. Fetching nixpkgs (if you use the default)
+
+That's it. No metrics. No tracking. No bullshit.
+
+------------------------------------------------------------------------------
+Command Line as Primary Interface
+------------------------------------------------------------------------------
+
+**The TUI Question**
+
+We've considered adding a Text User Interface (TUI). While appealing, we're
+hesitant because:
+
+- CLI is scriptable; TUI is not
+- CLI works over SSH; TUI requires terminal support
+- CLI is timeless; TUIs become dated
+- CLI is accessible; TUIs may not be
+
+**Our Compromise**
+
+- CLI is primary and always will be
+- TUI may be added as optional enhancement (Phase 4)
+- All functionality available via CLI first
+- Web interface for visualization (Phase 3)
+
+------------------------------------------------------------------------------
+Sustainability and Maintenance
+------------------------------------------------------------------------------
+
+**Long-term Viability**
+
+Nixtamal is designed to be maintainable:
+
+- OCaml is a stable, typed language with decades of history
+- KDL is a simple, well-specified format
+- No external service dependencies
+- Clear module boundaries
+- Comprehensive tests
+
+**Community Governance**
+
+The project follows a "benevolent dictator" model:
+- toastal maintains and decides direction
+- Community contributions welcome
+- Forks encouraged if direction diverges
+- No CLA or corporate ownership
+
+**Funding**
+
+Currently unfunded. If you want to support:
+- Contribute code
+- Write documentation
+- Package for distributions
+- Donate to toastal
+
+------------------------------------------------------------------------------
+Comparison with Alternatives
+------------------------------------------------------------------------------
+
+**vs. Flakes**
+
++ Better VCS support
++ Human-readable config (KDL vs JSON)
++ External lockfiles
++ No evaluation caching issues
+- Less integrated with nix command
+- Smaller ecosystem
+
+**vs. npins**
+
++ First-class VCS diversity
++ KDL > JSON
++ Template support
++ Schema versioning
+- More complex
+- OCaml dependency
+
+**vs. niv**
+
++ Modern codebase
++ Better error handling
++ Schema upgrades
+- No niv import (yet)
+- Less mature ecosystem
+
+**vs. Built-in fetchers**
+
++ Unified interface
++ Lockfile management
++ Stale checking
++ Update workflow
+- Additional dependency
+
+------------------------------------------------------------------------------
+Conclusion
+------------------------------------------------------------------------------
+
+Nixtamal represents a bet: that users want:
+
+1. **Control** over their tooling
+2. **Privacy** by default
+3. **Diversity** in VCS support
+4. **Compatibility** without lock-in
+5. **Sustainability** over hype
+
+If these values resonate with you, welcome. If not, there are plenty of
+alternatives. Choice is the point.
+
+.. vim: set textwidth=80
diff --git a/doc/roadmap.rst b/doc/roadmap.rst
new file mode 100644
index 0000000..2ccc703
--- /dev/null
+++ b/doc/roadmap.rst
@@ -0,0 +1,350 @@
+================================================================================
+Nixtamal Development Roadmap
+================================================================================
+
+.. role:: ab
+.. role:: ac
+
+------------------------------------------------------------------------------
+Overview
+------------------------------------------------------------------------------
+
+This roadmap outlines the planned development phases for Nixtamal, from its
+initial foundation through its ultimate goal: replacing detsys SBOM and
+nixpkgs dependency visualization with a superior, community-driven alternative.
+
+**Current Status:** Phase 2.5 (Documentation & Website Preparation)
+
+------------------------------------------------------------------------------
+Phase 1: Foundation โœ… COMPLETED
+------------------------------------------------------------------------------
+
+**Timeline:** 2024-2025 (Original Development)
+
+**Goals:** Establish core functionality and architecture
+
+**Deliverables:**
+
+โœ… Core type system (Input, Lockfile, Manifest)
+โœ… KDL codec for configuration
+โœ… JSON codec for lockfiles
+โœ… Git, Darcs, Pijul VCS support
+โœ… File and Archive input types
+โœ… Concurrent prefetching with Eio
+โœ… Basic CLI (lock, refresh, show)
+โœ… Nilla framework integration (types)
+โœ… Test suite foundation
+
+**Key Decisions:**
+- OCaml 5.x with Eio for concurrency
+- KDL for configuration format
+- JSON for lockfile format
+- Result-based error handling
+- Property-based testing with QCheck
+
+**Blockers Resolved:**
+- Initial Darcs/Pijul fetcher upstreaming to nixpkgs
+- OCaml 5/Eio ecosystem stabilization
+
+------------------------------------------------------------------------------
+Phase 2: Core Features โœ… COMPLETED
+------------------------------------------------------------------------------
+
+**Timeline:** 2026-Q1 (Completed 2026-04-14)
+
+**Goals:** Feature parity with upstream, production readiness
+
+**Deliverables:**
+
+โœ… **Schema Management**
+ - Versioned schemas (0.1.1, 0.2.0)
+ - ``nixtamal upgrade`` command
+ - Backup and rollback on failure
+ - Version validation and detection
+
+โœ… **Extended VCS Support**
+ - Fossil VCS integration
+ - Nilla prefetch implementation
+ - Hash algorithm improvements (SRI support)
+
+โœ… **Build System Improvements**
+ - Cmdliner 2.x compatibility
+ - Flake integration enhancements
+ - Cross-compilation support preparation
+
+โœ… **Quality Assurance**
+ - Lockfile auto-creation
+ - Comprehensive error messages
+ - Test coverage infrastructure (bisect_ppx)
+ - CI/CD pipeline
+
+โœ… **Developer Experience**
+ - Better error context
+ - Command help improvements
+ - Documentation updates
+
+**Commits:**
+- ``4ed722f8`` - Port upstream patches (Cmdliner 2.x, upgrade, Fossil)
+
+**Technical Debt:**
+- Some deprecation warnings suppressed (QCheck API changes)
+- Comment preservation in KDL pending KDL v2
+
+------------------------------------------------------------------------------
+Phase 2.5: Documentation & Website Preparation ๐Ÿ”„ CURRENT
+------------------------------------------------------------------------------
+
+**Timeline:** 2026-Q2 (Current)
+
+**Goals:** Prepare for nixtamal.tech launch and community growth
+
+**Deliverables:**
+
+๐Ÿ”„ **Documentation**
+ โœ… Comprehensive project documentation (this document set)
+ - API documentation (odoc)
+ - Tutorial and guide content
+ -