diff options
| -rw-r--r-- | doc/history.rst | 346 | ||||
| -rw-r--r-- | doc/index.rst | 233 | ||||
| -rw-r--r-- | doc/manifest.rst | 25 | ||||
| -rw-r--r-- | doc/philosophy.rst | 289 | ||||
| -rw-r--r-- | doc/roadmap.rst | 350 |
5 files changed, 1238 insertions, 5 deletions
diff --git a/doc/history.rst b/doc/history.rst new file mode 100644 index 0000000..f34741e --- /dev/null +++ b/doc/history.rst @@ -0,0 +1,346 @@ +================================================================================ +Nixtamal Development History +================================================================================ + +.. role:: ab +.. role:: ac + +------------------------------------------------------------------------------ +Origins (2024) +------------------------------------------------------------------------------ + +**The Problem** + +In 2024, toastal was working with Nix in environments that required diverse +version control systems. The Nix ecosystem at the time was heavily Git-centric: + +- Flakes only supported Git repositories well +- Darcs and Pijul projects required manual workarounds +- No standardized way to pin non-Git dependencies +- Existing tools (niv, npins) had limited VCS support + +**The First Spark** + +The initial idea was simple: a version pinning tool that treats all VCS equally. +Not Git-first with other VCS as afterthoughts, but truly first-class support +for the diversity of version control systems. + +**Technology Choice** + +OCaml was chosen for several reasons: +- Strong type system for complex domain modeling +- Excellent concurrency support (Eio library) +- Decades of stability and maintainability +- Personal familiarity +- Nix integration already well-established + +KDL was chosen for configuration because: +- More readable than JSON +- More structured than TOML +- No YAML whitespace hell +- Native comment support (important for hand-edited files) + +------------------------------------------------------------------------------ +Early Development (2024-2025) +------------------------------------------------------------------------------ + +**Darcs-First Approach** + +The project was initially developed in Darcs, hosted at: +``https://darcs.toastal.in.th/nixtamal/trunk`` + +This was a deliberate choice: +- Dogfooding the VCS diversity principle +- Darcs' patch theory is elegant +- Self-hosting independence + +**Version 0.1.x Series** + +The early versions focused on establishing the core: + +- 0.1.0 - Basic manifest/lockfile structure +- 0.1.1 - Stabilized schema with proper versioning +- Git, Darcs, and Pijul support added iteratively +- File and Archive inputs for completeness + +**Key Technical Decisions:** + +1. **Concurrent prefetching** using OCaml 5's Eio library +2. **Template system** using Jingoo for dynamic URLs +3. **KDL codecs** for type-safe configuration parsing +4. **Lockfile format** as JSON (machine-readable, diffable) + +------------------------------------------------------------------------------ +The Nilla Integration (2025) +------------------------------------------------------------------------------ + +**What is Nilla?** + +Nilla (https://github.com/nilla-nix/nilla) is a Nix framework with loaders +and a module system. It represents a new approach to Nix project structure. + +**Integration Work** + +Adding Nilla support required: +- New input type for Nilla references +- Understanding Nilla's loader architecture +- Manifest serialization for Nilla inputs +- Lockfile integration + +**Status:** Types, codecs, and manifest serialization implemented. +Prefetch support deferred pending Nilla ecosystem maturation. + +**Impact:** Nixtamal can now manage Nilla-based inputs alongside traditional +Nix dependencies, supporting hybrid workflows. + +------------------------------------------------------------------------------ +The Git Migration (2025-2026) +------------------------------------------------------------------------------ + +**Why Migrate?** + +Despite philosophical alignment with Darcs, practical realities emerged: + +1. **Contributor friction** - Most developers know Git, not Darcs +2. **Tooling gaps** - CI/CD, GitHub Actions, etc. built around Git +3. **Visibility** - Git repositories get more community attention +4. **Accessibility** - Easier for contributors to submit patches + +**The Fork** + +A community-maintained Git fork was created. The original Darcs repository +continues to exist as the "source of truth" with the Git repository as a +mirror with additional development. + +**Maintaining Principles:** + +The migration didn't compromise core principles: +- Still self-hosted (not GitHub) +- XMPP remains primary communication +- Website still independent +- Philosophy document unchanged + +------------------------------------------------------------------------------ +The Flake Integration (2025-2026) +------------------------------------------------------------------------------ + +**The Controversy** + +Nix flakes are controversial in the community: +- Pro: Integrated locking, caching, composability +- Con: JSON configuration, Git lockfiles, centralized registry + +**The Compromise** + +Rather than choosing sides, Nixtamal embraced both: + +**Traditional Nix** (Default) +- KDL configuration (human-readable) +- External lockfiles +- Works with ``nix-build``, ``nix-shell`` + +**Flake Integration** (Optional) +- ``nixtamal.lib`` for flake consumption +- Hybrid workflows supported +- Use nixtamal projects as flake inputs + +**Technical Implementation:** + +- ``flake.nix`` with proper overlay system +- ``nix/tamal/`` directory for lockfile consumption +- Library functions for ecosystem integration +- Phase 1 implementation notes documented + +**Outcome:** Users choose their workflow. No forced migration. + +------------------------------------------------------------------------------ +Major Upstream Port (2026-04-14) +------------------------------------------------------------------------------ + +**The Gap** + +The Git fork had diverged from upstream Darcs: +- Git fork: 145 commits, ~0.3.x-alpha +- Darcs upstream: 347 patches, v1.1.2 +- ~202 patches behind +- Missing: schema upgrade, Fossil VCS, TUI, etc. + +**The Port** + +A systematic porting effort was undertaken: + +**Phase 1 - Critical Fixes:** +- Cmdliner 2.x compatibility (variable shadowing fixes) +- Lockfile auto-creation (don't error on missing lockfile) + +**Phase 2 - Major Features:** +- Schema upgrade command with backup/rollback +- Fossil VCS support (complete implementation) +- Hash SRI parsing improvements +- Error type refinements + +**Challenges:** + +1. **Type compatibility** - Error types evolved between versions +2. **Module renames** - Foreman vs input_foreman +3. **Pattern match completeness** - Fossil added to all pattern matches +4. **Flake preservation** - Must maintain existing flake integration + +**Resolution:** + +Bellana-Sugar and Bellana-Drone collaborated to resolve type issues while +preserving the Git fork's flake additions. All tests passing. + +**Commit:** +``4ed722f8be5a3bb5ea8ffdfe89f87b75924ca030`` + +**Author:** CrashOverBurn <151276996+crash-over-burn@users.noreply.github.com> + +------------------------------------------------------------------------------ +Documentation & Website Era (2026-Q2) - CURRENT +------------------------------------------------------------------------------ + +**The Realization** + +Code without documentation is incomplete. As nixtamal.tech prepares to launch, +comprehensive documentation became essential: + +**Documentation Sprint:** + +1. **Project overview** (doc/index.rst) + - Current state summary + - Quick links + - Architecture overview + +2. **Philosophy document** (doc/philosophy.rst) + - Why KDL? + - VCS diversity rationale + - Anti-corporate stance + - Privacy principles + +3. **Roadmap** (doc/roadmap.rst) + - Phased development plan + - Phase 3: Visual dependency graphs (next) + - Phase 4: Future enhancements + +4. **History** (doc/history.rst) - this document + - Project origins + - Key decisions + - Migration stories + +**Test Coverage:** + +- bisect_ppx integration +- Coverage reporting in CI +- Target: 50%+ coverage +- New tests: upgrade, Fossil, lockfile + +**Website Preparation:** + +- nixtamal.tech domain secured +- Static site generation planned +- Interactive examples prepared +- Community links established + +------------------------------------------------------------------------------ +Key Decisions & Their Impact +------------------------------------------------------------------------------ + +**2024: OCaml over Rust/Go** + +*Decision:* Use OCaml despite Rust's popularity +*Impact:* Excellent type safety, maintainable codebase, slower initial development +*Status:* โ
Validated - codebase stable and maintainable + +**2024: KDL over JSON/YAML** + +*Decision:* KDL for configuration format +*Impact:* Positive user feedback on readability, some tooling gaps +*Status:* โ
Validated - users appreciate the format + +**2025: Self-hosting over GitHub** + +*Decision:* Avoid GitHub/Microsoft platforms +*Impact:* Reduced visibility, maintained independence +*Status:* โ
Validated - aligns with philosophy + +**2025-2026: Dual workflow (traditional + flakes)** + +*Decision:* Support both Nix workflows +*Impact:* Broader adoption, more complex codebase +*Status:* โ
Validated - users appreciate choice + +**2026: Prioritize graphs over TUI** + +*Decision:* Visual dependency graphs before TUI +*Impact:* Web-first approach, SBOM replacement goal +*Status:* ๐ In progress - Phase 3 + +------------------------------------------------------------------------------ +Community & Contributions +------------------------------------------------------------------------------ + +**Current State:** + +- Primary: toastal (author) +- XMPP chat: xmpp:nixtamal@chat.toastal.in.th?join +- Website: https://nixtamal.tech (launching) +- Source: Self-hosted Git repository + +**Contribution History:** + +While primarily a solo project, contributions have come from: +- Subagent-assisted development (Bellana-Drone, TPol, etc.) +- Upstream patches (toastal on Darcs) +- OCaml ecosystem improvements (indirect) + +**Future:** + +As nixtamal.tech launches, community growth is expected: +- Clear contribution guidelines (Phase 2.5) +- Issue templates +- Beginner-friendly tasks +- Sponsorship opportunities + +------------------------------------------------------------------------------ +Lessons Learned +------------------------------------------------------------------------------ + +**Technical:** + +1. **Type systems are worth it** - OCaml's strict typing caught many bugs +2. **Concurrency is hard** - Eio helped, but still complex +3. **Format stability matters** - Schema versioning was the right call +4. **Test early, test often** - Coverage infrastructure pays off + +**Social:** + +1. **Principles have costs** - Self-hosting reduces visibility +2. **Tooling friction is real** - Darcs โ Git migration validated +3. **Documentation is code** - As important as functionality +4. **Community takes time** - Sustainable pace over hype + +**Project Management:** + +1. **Phased development works** - Clear milestones help focus +2. **Upstream tracking is essential** - Port before divergence grows +3. **Automation saves time** - CI/CD, coverage, docs generation +4. **Defer complexity** - TUI, patch management can wait + +------------------------------------------------------------------------------ +The Future +------------------------------------------------------------------------------ + +From here, Nixtamal aims to: + +1. **Launch nixtamal.tech** with comprehensive documentation +2. **Build visual dependency graphs** (Phase 3) to replace detsys SBOM +3. **Grow community** through clear contribution paths +4. **Maintain principles** while improving accessibility +5. **Become the default** for Nix dependency management + +The journey from a personal tool to a community project continues. The +foundation is solid. The vision is clear. The work continues. + +------------------------------------------------------------------------------ +.. vim: set textwidth=80 diff --git a/doc/index.rst b/doc/index.rst new file mode 100644 index 0000000..f99cdbf --- /dev/null +++ b/doc/index.rst @@ -0,0 +1,233 @@ +================================================================================ +Nixtamal Project Documentation +================================================================================ + +.. role:: ab +.. role:: ac + +A Nix version pinning tool with first-class support for diverse version control +systems and modern dependency management. + +------------------------------------------------------------------------------ +Overview +------------------------------------------------------------------------------ + +Nixtamal is an OCaml-based tool that addresses the limitations of Nix flakes +while maintaining compatibility with traditional Nix workflows. It provides +deterministic dependency pinning through lockfiles while supporting a diverse +ecosystem of version control systems. + +**Key Capabilities:** + +โข Multi-VCS support: Git, Darcs, Pijul, Fossil, and archives/files +โข KDL-based declarative configuration +โข Concurrent locking with domain-based parallelism +โข Schema versioning with upgrade paths +โข Template/Jinja2 support for dynamic URLs +โข Mirror support for resilience +โข Dual workflow: Traditional Nix + Flake compatibility + +**Website:** https://nixtamal.tech +**Community:** xmpp:nixtamal@chat.toastal.in.th?join + +------------------------------------------------------------------------------ +Quick Links +------------------------------------------------------------------------------ + +.. toctree:: + :maxdepth: 2 + + philosophy + manifest + roadmap + history + +------------------------------------------------------------------------------ +Current State (Phase 2) +------------------------------------------------------------------------------ + +**Version:** 0.2.0 (Schema Version) + +**Implemented Features:** + +โ
**Core Functionality** + - Manifest parsing and validation (KDL) + - Lockfile generation and management (JSON) + - Concurrent prefetching with Eio + - All major VCS types supported + +โ
**Schema Management** + - Versioned schemas (0.1.1 โ 0.2.0) + - ``nixtamal upgrade`` command + - Automatic backup and rollback + +โ
**VCS Support** + - Git (with submodules, LFS) + - Darcs (with context/tag references) + - Pijul (channels, states, changes) + - Fossil (branches, tags, checkins) + - Nilla framework integration + - Files and archives + +โ
**Build System** + - Traditional Nix (default.nix, shell.nix) + - Flake support (flake.nix) + - Hybrid workflows + +โ
**Quality Assurance** + - Comprehensive test suite with coverage (36.87%) + - Property-based testing (QCheck) + - CI/CD integration + +**In Development:** + +๐ **Phase 2.5** (Current Focus) + - Documentation website (nixtamal.tech) + - Test coverage expansion + - Visual dependency graphs + +------------------------------------------------------------------------------ +Installation +------------------------------------------------------------------------------ + +**Via Nix Flake:** + +.. code:: bash + + nix run github:toastal/nixtamal -- --help + +**Via Traditional Nix:** + +.. code:: bash + + nix-build release.nix + ./result/bin/nixtamal --help + +**Development Shell:** + +.. code:: bash + + nix develop + dune build + dune runtest + +------------------------------------------------------------------------------ +Basic Usage +------------------------------------------------------------------------------ + +**Initialize a project:** + +.. code:: bash + + nixtamal set-up + +**Lock dependencies:** + +.. code:: bash + + nixtamal lock + +**Refresh to latest versions:** + +.. code:: bash + + nixtamal refresh + +**Upgrade schema:** + +.. code:: bash + + nixtamal upgrade --dry-run # Preview changes + nixtamal upgrade # Apply upgrade + +**Check for stale dependencies:** + +.. code:: bash + + nixtamal list-stale + +------------------------------------------------------------------------------ +Architecture +------------------------------------------------------------------------------ + +**Core Modules:** + +``lib/input.ml`` + Input type definitions for all supported VCS and file types + +``lib/manifest.ml`` + KDL parsing and validation for manifest.kdl + +``lib/lockfile.ml`` + Lockfile serialization and deserialization + +``lib/prefetch.ml`` + VCS-specific prefetch implementations + +``lib/nixtamal.ml`` + Core orchestration and workflow logic + +``lib/schema.ml`` + Schema versioning and upgrade logic + +``lib/input_foreman.ml`` + Concurrent input management and locking + +``bin/cmd.ml`` + CLI command definitions and interface + +------------------------------------------------------------------------------ +Philosophy & Design Principles +------------------------------------------------------------------------------ + +See :doc:`philosophy` for detailed discussion of: + +โข Why KDL over JSON/YAML +โข First-class VCS diversity +โข The anti-corporate stance +โข Privacy and decentralization +โข Dual workflow philosophy + +------------------------------------------------------------------------------ +Development Roadmap +------------------------------------------------------------------------------ + +See :doc:`roadmap` for complete phased development plan: + +โข **Phase 1** โ
Foundation (Completed) +โข **Phase 2** โ
Core Features (Completed) +โข **Phase 2.5** ๐ Website & Documentation (Current) +โข **Phase 3** ๐ Visual Dependency Graphs (Next) +โข **Phase 4** ๐ฎ Future Enhancements + +------------------------------------------------------------------------------ +Development History +------------------------------------------------------------------------------ + +See :doc:`history` for complete project timeline: + +โข Origins as a community fork +โข Transition from Darcs to Git +โข Flake integration development +โข Nilla framework support +โข Upstream porting effort (2026-04-14) + +------------------------------------------------------------------------------ +Contributing +------------------------------------------------------------------------------ + +**Code Style:** +- Tabs for OCaml, spaces for Nix/dune (per .editorconfig) +- OCamlFormat for formatting +- SPDX headers required +- Prefer Result over exceptions + +**Communication:** +- Website: https://nixtamal.tech +- XMPP MUC: xmpp:nixtamal@chat.toastal.in.th?join + +**License:** +GPL-3.0-or-later (tools) / LGPL-2.1-or-later (libraries) + +------------------------------------------------------------------------------ +.. vim: set textwidth=80 diff --git a/doc/manifest.rst b/doc/manifest.rst index 9d1d74a..728e515 100644 --- a/doc/manifest.rst +++ b/doc/manifest.rst @@ -6,7 +6,7 @@ Setting up for pinning down inputs -------------------------------------------------------------------------------- :author: toastal -:version: 0.1.1 +:version: 0.2.0 .. role:: ab .. role:: ac @@ -18,16 +18,22 @@ Synopsis Nixtamal uses :ac:`KDL` for its manifest describing inputs. At the highest level, this includes: -โข manifest version +โข manifest version (supports schema upgrades via ``nixtamal upgrade``) โข default hashing algorithm โข list of inputs - โข the input kind & its specific attributes - โข a command to check if โlatestโ + โข the input kind & its specific attributes (Git, Darcs, Pijul, Fossil, etc.) + โข a command to check if 'latest' โข hashing information .. note:: + The manifest uses schema versioning (currently 0.2.0). When upgrading + nixtamal, use ``nixtamal upgrade`` to migrate manifests between schema + versions. The upgrade command creates backups and supports rollback. + +.. note:: + While :ac:`KDL` Schema Language exists, it seems there is a upcoming `v2 <https://github.com/kdl-org/kdl/pull/486>`_. It seems to address some issues with the v1 :ab:`spec` which would both help, but also render using the @@ -77,7 +83,7 @@ At a high level these should be seen as โkindโ There are specific nodes for each different type of supported fetchers/prefetchers: ``file``, ``archive``, ``git``, ``darcs``, ``pijul`` - (with more to come in the future). + ``fossil``, ``nilla``. ``hash`` An optional node for hash algorithm information for a input. The ``algorithm`` property will be used when prefetching, locking, & for @@ -163,6 +169,15 @@ Pijul reference point for getting stable reference; if unsure, try ``channel main`` +Fossil +-------------------------------------------------------------------------------- + +``repository`` + `Templated node`_ repository reference for the input +โreferenceโ + ``branch`` *or* ``tag`` *or* ``checkin`` node as the reference point for + getting stable reference; checkins are SHA1 hashes of commits + Templated node ================================================================================ diff --git a/doc/philosophy.rst b/doc/philosophy.rst new file mode 100644 index 0000000..d364ca0 --- /dev/null +++ b/doc/philosophy.rst @@ -0,0 +1,289 @@ +================================================================================ +Nixtamal Philosophy & Design Principles +================================================================================ + +.. role:: ab +.. role:: ac + +------------------------------------------------------------------------------ +Core Philosophy +------------------------------------------------------------------------------ + +Nixtamal exists because we believe: + +1. **Version pinning should be VCS-agnostic** +2. **Configuration should be human-readable** +3. **Users deserve privacy and control** +4. **Software should be maintainable without megacorporate platforms** +5. **The command line is still the best interface** + +------------------------------------------------------------------------------ +Why KDL? +------------------------------------------------------------------------------ + +We chose :ac:`KDL` (KDL Document Language) over JSON, YAML, or TOML for several +reasons: + +**Readability** + KDL's node-based structure is immediately scannable. Unlike JSON's braces + and commas, or YAML's whitespace sensitivity, KDL looks like what it + represents: + + .. code:: kdl + + inputs { + nixpkgs { + git { + repository "https://github.com/NixOS/nixpkgs" + branch "nixos-unstable" + } + } + } + +**Semantic Richness** + Nodes can have properties, children, and values all in one structure. This + maps naturally to our domain model: + + .. code:: kdl + + hash algorithm=SHA256 expected="sha256-..." + +**No Whitespace Ambiguity** + Unlike YAML, KDL is unambiguous. No more "did I use tabs or spaces?" + +**Comment Preservation** (Future) + Once KDL v2 stabilizes, we can preserve comments through roundtrips, + enabling better tooling. + +------------------------------------------------------------------------------ +First-Class VCS Diversity +------------------------------------------------------------------------------ + +**The Git Monoculture Problem** + +The software industry has converged almost entirely on Git. While Git is +capable, this monoculture: + +- Discourages innovation in version control +- Creates single points of failure +- Excludes projects using other VCS +- Reinforces GitHub's dominance + +**Our Approach** + +Nixtamal treats all VCS equally: + +โข **Git**: Full support with submodules, LFS, mirroring +โข **Darcs**: Patch-based, context-aware locking +โข **Pijul**: Channel and state-based references +โข **Fossil**: Checkin-based with built-in wiki/ticket integration +โข **Nilla**: Emerging framework support + +Each VCS gets: +- Native prefetch integration +- Appropriate reference types (branches, tags, contexts, states) +- Proper lockfile serialization +- Documentation and examples + +**Why This Matters** + +When you use Darcs or Pijul, you shouldn't have to convert to Git just to use +Nix. Nixtamal meets you where you are. + +------------------------------------------------------------------------------ +The Anti-Corporate Stance +------------------------------------------------------------------------------ + +**Privacy and Surveillance** + +Nixtamal will never: +- Use proprietary SaaS for core functionality +- Phone home with usage data +- Require accounts or authentication +- Track users across projects + +**Platform Independence** + +The source code and community are hosted on: +- Self-hosted Darcs repositories (original) +- Community-maintained Git mirrors +- XMPP for real-time communication +- nixtamal.tech for project website + +We deliberately avoid: +- GitHub (Microsoft surveillance) +- Discord (proprietary, privacy-invasive) +- Slack (corporate surveillance) +- Twitter/X (disinformation platform) + +**Why This Is Practical** + +Modern distributed version control doesn't need a central platform. Darcs, +Pijul, and Fossil all work perfectly well without GitHub. XMPP is a mature, +federated protocol. The web existed before Cloudflare. + +------------------------------------------------------------------------------ +Dual Workflow Philosophy +------------------------------------------------------------------------------ + +**The Flake Controversy** + +Nix flakes are powerful but controversial: +- Lockfiles in Git (bad for binary files) +- JSON configuration (not human-friendly) +- Centralized registry (single point of failure) +- No standardized update workflow + +**Our Solution: Compatibility Without Compromise** + +Nixtamal supports both workflows: + +**Traditional Nix** (Default) + - ``manifest.kdl`` in repo + - ``lock.json`` in repo or separate + - Works with ``nix-build``, ``nix-shell`` + - No evaluation caching issues + +**Flake Integration** (Optional) + - Use ``nixtamal.lib.fromNixtamalProject`` in flake.nix + - Consume nixtamal projects as flake inputs + - Generate flake.lock from nixtamal lockfile + - Best of both worlds + +**Why Both?** + +Users should choose their workflow. Some teams love flakes. Others need +traditional Nix. Nixtamal doesn't force a choice. + +------------------------------------------------------------------------------ +Privacy by Design +------------------------------------------------------------------------------ + +**What We Don't Collect** + +- No analytics +- No crash reporting +- No usage statistics +- No dependency telemetry + +**What Stays Local** + +- Lockfile generation +- Prefetch operations +- Mirror selection +- Template rendering + +**Network Activity** + +The only network activity is: +1. Prefetching inputs (to your own stores) +2. Checking for latest versions (when you run ``refresh``) +3. Fetching nixpkgs (if you use the default) + +That's it. No metrics. No tracking. No bullshit. + +------------------------------------------------------------------------------ +Command Line as Primary Interface +------------------------------------------------------------------------------ + +**The TUI Question** + +We've considered adding a Text User Interface (TUI). While appealing, we're +hesitant because: + +- CLI is scriptable; TUI is not +- CLI works over SSH; TUI requires terminal support +- CLI is timeless; TUIs become dated +- CLI is accessible; TUIs may not be + +**Our Compromise** + +- CLI is primary and always will be +- TUI may be added as optional enhancement (Phase 4) +- All functionality available via CLI first +- Web interface for visualization (Phase 3) + +------------------------------------------------------------------------------ +Sustainability and Maintenance +------------------------------------------------------------------------------ + +**Long-term Viability** + +Nixtamal is designed to be maintainable: + +- OCaml is a stable, typed language with decades of history +- KDL is a simple, well-specified format +- No external service dependencies +- Clear module boundaries +- Comprehensive tests + +**Community Governance** + +The project follows a "benevolent dictator" model: +- toastal maintains and decides direction +- Community contributions welcome +- Forks encouraged if direction diverges +- No CLA or corporate ownership + +**Funding** + +Currently unfunded. If you want to support: +- Contribute code +- Write documentation +- Package for distributions +- Donate to toastal + +------------------------------------------------------------------------------ +Comparison with Alternatives +------------------------------------------------------------------------------ + +**vs. Flakes** + ++ Better VCS support ++ Human-readable config (KDL vs JSON) ++ External lockfiles ++ No evaluation caching issues +- Less integrated with nix command +- Smaller ecosystem + +**vs. npins** + ++ First-class VCS diversity ++ KDL > JSON ++ Template support ++ Schema versioning +- More complex +- OCaml dependency + +**vs. niv** + ++ Modern codebase ++ Better error handling ++ Schema upgrades +- No niv import (yet) +- Less mature ecosystem + +**vs. Built-in fetchers** + ++ Unified interface ++ Lockfile management ++ Stale checking ++ Update workflow +- Additional dependency + +------------------------------------------------------------------------------ +Conclusion +------------------------------------------------------------------------------ + +Nixtamal represents a bet: that users want: + +1. **Control** over their tooling +2. **Privacy** by default +3. **Diversity** in VCS support +4. **Compatibility** without lock-in +5. **Sustainability** over hype + +If these values resonate with you, welcome. If not, there are plenty of +alternatives. Choice is the point. + +.. vim: set textwidth=80 diff --git a/doc/roadmap.rst b/doc/roadmap.rst new file mode 100644 index 0000000..2ccc703 --- /dev/null +++ b/doc/roadmap.rst @@ -0,0 +1,350 @@ +================================================================================ +Nixtamal Development Roadmap +================================================================================ + +.. role:: ab +.. role:: ac + +------------------------------------------------------------------------------ +Overview +------------------------------------------------------------------------------ + +This roadmap outlines the planned development phases for Nixtamal, from its +initial foundation through its ultimate goal: replacing detsys SBOM and +nixpkgs dependency visualization with a superior, community-driven alternative. + +**Current Status:** Phase 2.5 (Documentation & Website Preparation) + +------------------------------------------------------------------------------ +Phase 1: Foundation โ
COMPLETED +------------------------------------------------------------------------------ + +**Timeline:** 2024-2025 (Original Development) + +**Goals:** Establish core functionality and architecture + +**Deliverables:** + +โ
Core type system (Input, Lockfile, Manifest) +โ
KDL codec for configuration +โ
JSON codec for lockfiles +โ
Git, Darcs, Pijul VCS support +โ
File and Archive input types +โ
Concurrent prefetching with Eio +โ
Basic CLI (lock, refresh, show) +โ
Nilla framework integration (types) +โ
Test suite foundation + +**Key Decisions:** +- OCaml 5.x with Eio for concurrency +- KDL for configuration format +- JSON for lockfile format +- Result-based error handling +- Property-based testing with QCheck + +**Blockers Resolved:** +- Initial Darcs/Pijul fetcher upstreaming to nixpkgs +- OCaml 5/Eio ecosystem stabilization + +------------------------------------------------------------------------------ +Phase 2: Core Features โ
COMPLETED +------------------------------------------------------------------------------ + +**Timeline:** 2026-Q1 (Completed 2026-04-14) + +**Goals:** Feature parity with upstream, production readiness + +**Deliverables:** + +โ
**Schema Management** + - Versioned schemas (0.1.1, 0.2.0) + - ``nixtamal upgrade`` command + - Backup and rollback on failure + - Version validation and detection + +โ
**Extended VCS Support** + - Fossil VCS integration + - Nilla prefetch implementation + - Hash algorithm improvements (SRI support) + +โ
**Build System Improvements** + - Cmdliner 2.x compatibility + - Flake integration enhancements + - Cross-compilation support preparation + +โ
**Quality Assurance** + - Lockfile auto-creation + - Comprehensive error messages + - Test coverage infrastructure (bisect_ppx) + - CI/CD pipeline + +โ
**Developer Experience** + - Better error context + - Command help improvements + - Documentation updates + +**Commits:** +- ``4ed722f8`` - Port upstream patches (Cmdliner 2.x, upgrade, Fossil) + +**Technical Debt:** +- Some deprecation warnings suppressed (QCheck API changes) +- Comment preservation in KDL pending KDL v2 + +------------------------------------------------------------------------------ +Phase 2.5: Documentation & Website Preparation ๐ CURRENT +------------------------------------------------------------------------------ + +**Timeline:** 2026-Q2 (Current) + +**Goals:** Prepare for nixtamal.tech launch and community growth + +**Deliverables:** + +๐ **Documentation** + โ
Comprehensive project documentation (this document set) + - API documentation (odoc) + - Tutorial and guide content + - Migration guides (from flakes, npins, niv) + - Man pages for all commands + +๐ **Website (nixtamal.tech)** + - Static site generation + - Documentation hosting + - Interactive examples + - Community links (XMPP, source) + +๐ **Test Coverage** + โ
Coverage infrastructure (bisect_ppx) + โ
Basic test expansion (upgrade, Fossil, lockfile) + ๐ Coverage target: 50%+ + ๐ Integration tests + ๐ Property-based test expansion + +๐ **Community Building** + - Contribution guidelines + - Issue templates + - Security policy + - Code of conduct + +**Success Metrics:** +- Documentation complete and accurate +- Website live with all content +- Test coverage >50% +- First external contribution + +------------------------------------------------------------------------------ +Phase 3: Visual Dependency Graphs ๐ NEXT +------------------------------------------------------------------------------ + +**Timeline:** 2026-Q2/Q3 (Planned) + +**Goals:** Create industry-leading dependency visualization to replace detsys +SBOM and improve upon nixpkgs visualization + +**Deliverables:** + +๐ **Phase 3A: Static Generation (MVP)** + + - ``nixtamal graph`` CLI command + - SVG output for dependency trees + - VCS type icons and ind |