MEP 46. Mochi-to-Erlang/BEAM transpiler: concurrent, distributed runtime via Core Erlang as target
| Field | Value |
|---|---|
| MEP | 46 |
| Title | Mochi-to-Erlang/BEAM transpiler |
| Author | Mochi core |
| Status | Final |
| Type | Standards Track |
| Created | 2026-05-23 00:05 (GMT+7) |
| Depends | MEP-4 (Type System), MEP-5 (Type Inference), MEP-13 (ADTs and Match), MEP-45 (C transpiler, IR reuse) |
| Research | ~/notes/Spec/0046/01..12 |
| Tracking | /docs/implementation/0046/ |
Abstract
Mochi today ships vm3 (mochi run) and, with MEP-45 in flight, an ahead-of-time C transpiler producing native single-file binaries. Neither path gives users BEAM's concurrency model: M:N preemptive scheduling, process-isolated heaps, supervision trees, hot code reload, cluster-aware pubsub, and 30 years of OTP infrastructure. MEP-46 specifies a separate pipeline that targets the BEAM virtual machine via Core Erlang, the documented compiler API the Erlang/OTP team supports for external language front-ends (the same API used by LFE, Clojerl, Hamler, and Alpaca).
The pipeline reuses MEP-45's typed-AST and aotir IR, then forks at the lowering stage: instead of emitting ISO C23, it emits Core Erlang via the cerl Erlang constructor API, then calls compile:forms({c_module, ...}, [from_core, debug_info, return_errors, return_warnings]) to produce .beam files. Downstream, BeamAsm JIT (default since OTP 24) translates the BEAM bytecode to native machine code at load time. Two packaging targets ship together: --target=beam-escript produces a single-file executable wrapping all .beam plus a shebang (no ERTS bundled, requires erl on $PATH, ~50ms cold start, 2-10 MB); --target=beam-release produces a self-contained OTP release (ERTS bundled, ~300ms cold start, 30-80 MB, supports hot reload, supervised, daemon-ready). A third tier, --target=beam-atomvm, lints against the AtomVM compatibility profile and bundles .beam for ESP32/STM32 deployment.
The master correctness gate is byte-equal stdout from the produced .beam (run via escript) versus vm3 on the entire fixture corpus, across OTP 27.0, OTP 27.latest, and OTP 28.latest, on x86_64-linux-gnu and aarch64-darwin. vm3 is the recording oracle for expect.txt; the transpiler does not link against or depend on vm3.
Four load-bearing decisions:
- Core Erlang via
cerl, not Erlang abstract format and not source text. The Erlang Ecosystem Foundation's Compiler Workgroup explicitly recommends Core Erlang as the plug-in point for external languages (see 03-prior-art-transpilers §18). LFE has used this API in production for 17 years; the API has been stable since the OTP r12 release in 2009. Abstract format (the Elixir/Gleam route) is slightly easier to pretty-print as.erlbut is less stable across OTP versions (column tracking changed in OTP 27) and exposes irrelevant syntactic sugar. Source text (the Caramel/Purerl route) requires error-prone pretty-printing for marginal benefit. Core Erlang is the documented contract. - OTP wholesale; no Mochi-flavored process model. Mochi agents map directly to
gen_server; supervision usessupervisor/dynamic_supervisor; pubsub usespg; futures map to monitored spawn + selective receive on freshly created refs (the BEAM 24+ recv-marker optimisation makes this O(1)). The Mochi compiler does not introduce a new process model on top of BEAM; users get OTP's exact semantics, with its distributed pubsub for free. - Reuse MEP-45's
aotirIR. The IR is target-agnostic; monomorphisation, match-to-decision-tree, and closure-conversion passes run once and feed both backends. The fork is at the emit pass:transpiler3/beam/lower/lowersaotirtocerlrecords;transpiler3/c/emit/lowersaotirto C. Sharing the IR halves the implementation effort and ensures cross-target consistency. - OTP 27 minimum. OTP 27 (May 2024) adds the
jsonstdlib module (eliminatingjsx/jiffy), sigils~"..."for binary string literals (cleanest mapping for Mochi strings),maybeexpression as default, triple-quoted strings, and-docattribute. OTP 26 is rejected as a minimum because thejsonpolyfill alone costs ~600 lines of compatibility shim with no upside; OTP 27 is in Debian stable, Ubuntu 24.04, Homebrew, and asdf as of mid-2025.
The gate for each delivery phase is empirical: every Mochi source file in tests/transpiler3/beam/fixtures/ must compile via the BEAM pipeline and produce stdout that diffs clean against the expect.txt recorded by vm3. Dialyzer-clean (with -Werror) on generated code is the secondary gate. Reproducibility (bit-identical .beam chunks across two CI hosts) is the tertiary gate.
Motivation
Mochi today targets vm3 (for mochi run) and, via MEP-45, statically linked native binaries (for mochi build --target=c). Neither delivers what BEAM uniquely provides:
- Concurrency at scale. BEAM was designed in 1986 for telephony switches with 100K simultaneous calls and 99.9999999% uptime. It has a preemptive M:N scheduler over OS threads, per-process garbage collection (a process's GC never stalls another), and cheap process creation (~3 µs per process; 2M processes per node has been demonstrated). Mochi's C target gives a single-threaded model with cooperative fibers; BEAM gives industrial-strength concurrent execution with no library on top.
- Supervision. OTP supervisors restart crashed processes per a declared policy. Mochi's C target leaves the user to write process management; BEAM gives them OTP's 30-year-old, telecom-validated supervision trees for free.
- Hot code reload.
code:load_file/1swaps a module's code in a running BEAM node; existing processes keep state and use the new code on next fully-qualified call. No restart required. This is unique to BEAM and matters for long-running services, finance, telecom, and any system where downtime is costly. - Distributed pubsub for free. BEAM's
pgprocess groups are cluster-aware: apublishon node A delivers to subscribers on node B if both have joined the same group. Mochi streams therefore become distributed by default on BEAM, with zero extra code. - Ecosystem. The BEAM ecosystem includes Phoenix (web framework, used by Discord, Heroku, and others), Ecto (data mapper), Nerves (embedded), Mnesia (distributed database), Riak (distributed KV store), RabbitMQ (messaging), and CouchDB (document store). A Mochi-on-BEAM build can interop with any of them via FFI.
- Tooling. Dialyzer (success typing), eqWAlizer (Whatsapp's gradual typing), observer (graphical process inspector), recon (production tracing), telemetry (metrics), and the BEAM JIT all work on Mochi-emitted modules out of the box because we emit standard
.beamfiles withdebug_infoandlinechunks.
The C target (MEP-45) remains the right choice for CPU-bound numerics, embedded targets without dynamic loading, and single-file distribution. The BEAM target is the right choice for services, agents, streams, distributed systems, and anywhere hot reload or supervision matters. Both ship; the user picks.
Specification
This section is normative. Sub-notes under ~/notes/Spec/0046/01..12 are informative.
1. Pipeline and IR reuse
MEP-46 shares the front-end and aotir passes with MEP-45 and forks at the emit stage:
Mochi source
│ parser (MEP-1/2/3, reused)
▼
AST
│ type checker (MEP-4/5/6, reused)
▼
Typed AST
│ monomorphise (MEP-45 pass 1, reused)
▼
Monomorphic typed AST
│ lower (MEP-45 pass 2, reused)
▼
aotir (MEP-45's IR, reused)
│ match-to-decision-tree (MEP-45 pass 3, reused)
▼
aotir (matches lowered)
│ closure-convert (MEP-45 pass 4, reused)
▼
aotir (closures lowered)
│ beam-lower (MEP-46 pass 1; ./transpiler3/beam/lower/)
▼
cerl records (Core Erlang)
│ compile:forms/2 from_core (OTP's compiler)
▼
BEAM .beam files
│ package as escript / release / atomvm bundle
▼
Distributable artifact
aotir is unchanged. The MEP-46-specific work lives in transpiler3/beam/:
transpiler3/beam/lower/:aotir→cerlrecords.transpiler3/beam/emit/: drivescompile:forms/2(spawning an embeddederl) to produce.beamfiles; also pretty-prints.erlalongside for handoff.transpiler3/beam/build/: build driver: escript / release / atomvm targets; cache.transpiler3/beam/runtime/: themochiOTP application (Erlang source +.appresource file).
The codegen visits each aotir type exactly once via a memoised lower_type(aotirType) → cerlShape, exactly as the C target does. See 06-type-lowering §12.
2. Name mangling and atom safety
Mangled identifier form:
mochi_{pkg}__{module}__{name}[__{instArgsHash6}]
- Module names: lowercase, prefixed
mochi_user_for user modules andmochi_for runtime modules. - Function names: lowercase, prefixed with the source identifier (no prefix needed; functions are namespaced by their module).
- Variable names in Core Erlang: prefixed
V_(Core Erlang variables start uppercase but our generated ones use this convention to avoid clashes with Erlang keywords). - Atom literals (sum-type tags, record field names): prefixed with the source identifier, lowercased; reserved Erlang atoms (
atom,binary,case, etc.) are suffixed with_atom.
The BEAM atom table has a default limit of 1,048,576 atoms. To prevent exhaustion in long-running services that dynamically reload Mochi modules, the codegen emits a mochi_atoms_<modhash>:atoms/0 function listing every atom the module references; the runtime calls these at boot to pre-register atoms and never uses binary_to_atom/1 on user data (only binary_to_existing_atom/2).
See 05-codegen-design §3 and 06-type-lowering §2.
3. Type lowering table
| Mochi type | BEAM representation | Notes |
|---|---|---|
int | BEAM integer (arbitrary precision; small int fastpath ≤60 bits) | 1:1; no boxing |
float | BEAM float (boxed double) | 1:1; IEEE 754 |
bool | true / false atoms | 1:1 |
string | UTF-8 binary (<<"hello"/utf8>>) | OTP 27 sigils ~"hello" map directly |
time | integer (ns since Unix epoch UTC) | wraps erlang:system_time(nanosecond) |
duration | integer (ns) | matches time representation |
?T | {some, V} tuple or none atom | direct sum encoding |
list<T> | BEAM list (cons cells) | 1:1; comprehensions are native |
map<K,V> | BEAM map (flat ≤32 keys, HAMT above) | 1:1 |
omap<K,V> | {KeysList, KeyToValueMap} tuple | preserves insertion order; query DSL backing |
set<T> | sets:set() v2 (since OTP 24) | 1:1 |
stream<T> | opaque ref wrapping {?MODULE, StreamName} pg group | hub identified by atom |
chan<T> | gen_server-backed bounded queue | point-to-point variant |
record R | tagged map #{'__mochi_record__' => R, field => V, ...} | tag enables variant discrimination |
sum S | tagged tuple {Variant, V1, V2, ...} or bare atom for unit variants | matches Erlang idiom |
fun(A,B):C | BEAM fun (closure with captured env) | 1:1 |
agent A | opaque ref wrapping a PID; methods are gen_server:call/cast | gen_server callback module per agent type |
See 06-type-lowering for the full lowering, including equality semantics (=:=), pattern matching shapes, and Dialyzer -spec emission.
4. Expression and statement lowering
Expressions lower to Core Erlang c_let-bound temporaries with explicit casts; Core Erlang is side-effect-explicit, so any subexpression with possible effects gets a let binding. Short-circuit && / || lower to case with explicit clauses (Core Erlang has no built-in short-circuit operators). Integer arithmetic uses BEAM operators (arbitrary precision; division by zero raises badarith, caught by Mochi try/catch as MOCHI_ERR_DIVZERO). Float arithmetic preserves IEEE 754 semantics. String + lowers to <<S1/binary, S2/binary>>.
if, while, for in, match, return, break, continue lower to case expressions and recursive tail calls. BEAM has no native while or for; loops become tail-recursive helper functions, which the BeamAsm JIT compiles efficiently.
match lowers to Core Erlang case nodes; the kernel pass (v3_kernel) compiles the pattern decision tree downstream. We do not implement our own pattern-match compiler; OTP's is mature.
try { ... } catch e { ... } lowers to Core Erlang try/catch (which BEAM compiles to non-zero-cost stack unwinding; the cost is paid only on throw).
See 05-codegen-design §6-10.
5. Closures and funs
Free functions and methods lower to plain Erlang functions. Closures lower to BEAM funs (created via c_fun); free-variable capture is explicit in Core Erlang. BEAM funs are cheap (a small heap object holding the code pointer and the env vector). The BeamAsm JIT inlines fun applications where the target is statically known.
Higher-order functions (map, filter, fold) lower to lists:map/2, lists:filter/2, lists:foldl/3, which are highly optimised in OTP.
See 05-codegen-design §7 and 09-agent-streams §5.
6. Runtime library
The mochi OTP application (source under transpiler3/beam/runtime/, published to Hex.pm as mochi):
mochi/
├── src/
│ ├── mochi.app.src % Application resource
│ ├── mochi_app.erl % application:start/2 callback
│ ├── mochi_sup.erl % Top-level supervisor
│ ├── mochi_atoms.erl % Pre-registered atoms
│ ├── mochi_core.erl % Boxed value helpers
│ ├── mochi_str.erl % String/binary ops
│ ├── mochi_list.erl % List ops with Mochi semantics
│ ├── mochi_map.erl, mochi_set.erl, mochi_omap.erl, mochi_option.erl, mochi_time.erl
│ ├── mochi_query.erl % Query DSL runtime
│ ├── mochi_datalog.erl, mochi_datalog_ets.erl
│ ├── mochi_stream.erl, mochi_stream_sup.erl, mochi_stream_recorder.erl
│ ├── mochi_agent.erl, mochi_agent_sup.erl, mochi_async.erl
│ ├── mochi_llm.erl, mochi_llm_sup.erl, mochi_llm_openai.erl, mochi_llm_anthropic.erl
│ ├── mochi_fetch.erl, mochi_fetch_sup.erl % gun-backed HTTP
│ ├── mochi_ffi.erl, mochi_telemetry.erl, mochi_log.erl
│ └── mochi_test.erl % Test harness
└── test/
The runtime is pure Erlang (no NIFs) and depends only on kernel, stdlib, sasl, crypto, ssl, and gun. See 04-runtime §22 for the full module layout and 02-design-philosophy §7 for the rationale.
7. Concurrency and supervision
Mochi agents lower to gen_server callback modules. Spawning goes through mochi_agent_sup (dynamic supervisor) so every agent is supervised. Streams lower to pg process groups in the scope mochi; publish is pg:get_members + !; subscribe is pg:join + gen_statem. async/await lowers to monitored spawn + selective receive on a freshly created ref (the BEAM 24+ recv-marker optimisation makes the receive O(1)).
See 09-agent-streams for the full mapping table.
8. Memory model
Per-process heaps; per-process generational GC. Large binaries (>64 bytes) are reference-counted off-heap. Atoms are interned globally; the codegen pre-registers all known atoms at boot to prevent exhaustion.
No NIFs in v0.1. The OTP team's stance (Kenneth Lundin, Code BEAM 2023) and our own analysis (12-risks-and-alternatives §3) is that pure Erlang with BeamAsm JIT is fast enough for Mochi's stdlib; reserve NIFs for crypto, regex, and similarly hot kernels accessed via existing OTP libraries.
See 04-runtime §2 and §9.
9. Error model
Mochi try/catch lowers to Core Erlang try/catch. Built-in error codes (consistent with the C target's set, encoded as atoms on BEAM):
| Atom | Source |
|---|---|
mochi_err_fetch | network or HTTP non-2xx |
mochi_err_parse | JSON / YAML / CSV decode |
mochi_err_type | runtime type mismatch |
mochi_err_index | OOB index / missing key |
mochi_err_divzero | integer divide by zero (caught from badarith) |
mochi_err_ffi | FFI subprocess failure |
mochi_err_llm | provider error from generate |
mochi_err_assert | expect false |
mochi_err_timeout | agent call timeout or stream timeout |
mochi_err_async_crash | future's worker process crashed |
User error atoms are namespaced mochi_user_<module>_<name>.
10. Target portability
Supported OTP versions: 27.0, 27.latest, 28.latest (Tier 1). OTP 29 RC (when available) runs in non-blocking nightly. OTP 26 and earlier unsupported.
Supported platforms:
- Tier 1 (full CI, blocking): Linux x86-64 (glibc), macOS arm64, macOS x86-64.
- Tier 2 (CI, best-effort): Linux arm64, Linux musl/Alpine, Windows x86-64.
- Tier 3 (community-supported): FreeBSD, ppc64le, riscv64, s390x.
The BeamAsm JIT is default on Tier 1 platforms; Tier 2/3 may run interpreter mode but .beam files are arch-independent.
AtomVM compatibility profile: Phase 1-5 fixtures plus a curated Phase 6 subset run unmodified on AtomVM 0.6+ for ESP32/STM32 (no pg, no httpc, no crypto).
See 07-erlang-target-portability for the full matrix.
11. Build driver
mochi build --target=beam-erlc PATH # .beam files only
mochi build --target=beam-escript PATH # single-file executable
mochi build --target=beam-release PATH # OTP release tarball
mochi build --target=beam-rebar3-project PATH # emit rebar3 layout
mochi build --target=beam-mix-project PATH # emit mix layout
mochi build --target=beam-atomvm PATH # .avm for embedded
mochi build --target=beam-... --emit={core|erl|beam}
mochi build --target=beam-... --otp=27|28
mochi build --target=beam-... --reproducible
mochi build --target=beam-... --watch
Cache layout under .mochi/cache/beam/{aotir,cerl,beam}/, content-addressed by BLAKE3 over (source, transitive imports, OTP version, transpiler version). A cache hit is "is the .beam present?". The cache is shared with the C target where possible (aotir entries common to both targets).
The driver does not vendor an OTP installation; it discovers erl on $PATH, validates erl +V against the supported range, and refuses to build on unsupported versions. Docker recipes (mochilang/mochi:beam-otp27) are published for users without local OTP. See 10-build-system.
12. Reproducibility
The Dbgi, Line, and CInf chunks of generated .beam files are sources of non-reproducibility. We strip the compile timestamp from CInf, set the path to a relative form, and emit functions and exports in sorted IR-identifier order. Two builds of the same source on different machines produce bit-identical .beam files.
The release tarball includes a manifest.json with a SHA-256 of every .beam; manifests are reproducible across CI agents. See 07-erlang-target-portability §8.
13. Hardening
BEAM does not have ASLR/PIE/RELRO concerns (the VM does); generated .beam files inherit the BEAM emulator's hardening. We require TLS 1.3 (OTP 27+ default) for any fetch over HTTPS and reject TLS 1.0/1.1. For releases targeting distributed deployments, we recommend -setcookie minimum 32-byte entropy and ssl_dist for inter-node communication (documentation, not enforced).
See 12-risks-and-alternatives §7.
14. Diagnostics
Compile-time errors surface with Mochi spans via the type checker. For errors that escape into the OTP compiler (rare; only happens when our cerl emission is invalid, which is a compiler bug), the build driver maps Erlang line refs back to Mochi spans via the Line chunk's #line analogue. Runtime stack traces are rewritten by mochi_log to show Mochi source lines instead of generated Erlang lines.
See 12-risks-and-alternatives §10.
15. Debug info
Dbgi and Line chunks always emitted (no strip mode in v0.1). Together they enable Erlang dbg, the BEAM observer, code:get_doc/1, and the Erlang LSP. The Dbgi chunk contains the abstract format AST recovered from our Core Erlang; this is the OTP-canonical debug info format.
16. -spec and Dialyzer
Every Mochi-exported function emits an Erlang -spec derived from its Mochi type. Generated modules are Dialyzer-clean (rebar3 dialyzer -Werror passes). Erlang devs consuming a Mochi-built library see typed APIs.
Opaque types (e.g. mochi_agent_ref()) use Erlang's -opaque syntax to hide internal representation.
See 06-type-lowering §14.
17. Output style
When the -rebar3-project target is used, we also pretty-print .erl source alongside the .beam files, using erl_prettypr for the round-trip via abstract format. Generated .erl is human-readable for handoff to Erlang devs; the per-line #line analogue in the Line chunk maps back to Mochi source.
Rationale
Why BEAM as a second target after C
MEP-45's C target is the AOT performance story: single-file native binary, vendored cross-cc, every tier-1 triple, ~3MB hello-world, 1.5x of hand-written C on numeric workloads. It does not give us concurrency, supervision, hot reload, or distributed pubsub. BEAM does, and BEAM is the runtime where these features are most polished (30 years of Ericsson production hardening). Together C and BEAM cover the two ends of the runtime spectrum: bare-metal single-threaded AOT and dynamic concurrent supervised. See 02-design-philosophy.
Why Core Erlang via cerl, not abstract format or source
The EEF Compiler Workgroup's December 2024 notes explicitly cite Core Erlang as the supported plug-in point for external languages. LFE, Clojerl, Hamler, and Alpaca all use this API. Abstract format is supported for tools but has more syntactic complexity; source text is error-prone to emit cleanly. See 03-prior-art-transpilers §18 and 05-codegen-design §2.
Why reuse aotir instead of a fresh IR
aotir is post-type-checking, post-monomorphisation, target-agnostic. Forking a separate IR would duplicate the closure-conversion and match-decision-tree passes. The C target and BEAM target have different runtime models (manual struct layout vs BEAM map/tuple) but the same set of expression shapes; sharing the IR forces the two backends to handle each language feature consistently.
Why OTP wholesale
The alternative is a Mochi-flavored supervision/process surface that wraps OTP. We rejected this because (a) Mochi's process model already aligns with BEAM's, (b) wrapping adds an indirection layer and a maintenance surface, (c) users wanting OTP semantics now have a thin wrapper hiding the thing they actually want. The MEP commits to OTP semantics; users who want Mochi-only abstractions get them as language sugar over OTP.
Why OTP 27 minimum
OTP 27 (May 2024) adds the json stdlib (eliminating a jsx/jiffy dep), sigils ~"...", maybe expression default, and triple-quoted strings, all of which directly support Mochi's source surface. OTP 26 (May 2023) lacks these; the polyfill cost exceeds the value of supporting older versions. OTP 27 has been in Debian stable, Ubuntu 24.04, Homebrew, and asdf since mid-2025.
Why no NIFs in v0.1
NIFs are scheduler-blocking and platform-specific. The OTP team (Kenneth Lundin, Code BEAM 2023) recommends pure Erlang for new code; existing maintained NIFs (crypto, re, zlib) cover common needs. Adding a NIF requires C build infrastructure, platform-specific binaries, and crash-safety review. BeamAsm JIT is fast enough for Mochi's stdlib. See 04-runtime §9 and 12-risks-and-alternatives §3.
Why differential testing as the master gate
vm3 is the source of truth (matches MEP-45's choice). Byte-equal stdout from the BEAM-produced escript versus vm3, on every fixture, on every supported OTP version, is the strictest behavior check available. Property tests, Dialyzer cleanliness, and reproducibility are layered on top.
Backwards Compatibility
Additive. mochi run and mochi test keep vm3 by default. mochi build gains --target=beam-escript, --target=beam-release, --target=beam-atomvm, --target=beam-rebar3-project, --target=beam-mix-project. No language surface change; no stdlib surface change beyond mirroring vm3's existing user-visible exposure inside the BEAM runtime.
Observable behaviour must match vm3 byte-for-byte on the fixture corpus. Programs relying on implementation-defined vm3 behaviour (allocation order, GC pause timing) are explicitly non-portable; the spec already disallows reliance on these.
Reference Implementation
Code lives under a fresh tree transpiler3/beam/, sharing only the front-end (parser, type checker) and aotir IR with MEP-45:
| Tree | Purpose |
|---|---|
transpiler3/beam/lower/ | aotir → cerl records (MEP-46 pass 1) |
transpiler3/beam/emit/ | drives compile:forms/2; pretty-prints .erl sidecar |
transpiler3/beam/build/ | driver: escript / release / atomvm / rebar3 / mix targets |
transpiler3/beam/runtime/ | the mochi OTP application source |
transpiler3/beam/runtime/src/ | mochi_*.erl files compiled into the runtime library |
tests/transpiler3/beam/ | fixture corpus, expect files, integration tests |
tests/transpiler3/beam/fixtures/ | per-phase fixtures (phaseN/...) |
tests/transpiler3/beam/bench/ | performance harness (Phase 18) |
The phased delivery plan is the §Phases section below. Each phase ships as a sub-PR auto-merged per the project's auto-ship convention; tracking pages live under /docs/implementation/0046/.
Phases
The plan walks the language surface bottom-up (Phases 0-12), then layers packaging (Phases 13-15), then quality gates (Phases 16-18), and culminates in v1.0 (Phase 19). Phases 0-12 are strictly sequential; Phases 13-15 can run in parallel after Phase 12 lands; Phases 16-18 run after Phase 15 lands and continue in perpetuity.
Phase conventions:
- Gate. A single measurable criterion. A phase is LANDED only when its gate is green on every target listed.
- Targets. OTP version × arch matrix in scope at this phase.
- Status / Commit columns. Filled in along the way. Values:
NOT STARTED,IN PROGRESS,BLOCKED,LANDED,DEFERRED. - Goal-alignment audit. Before a phase starts, a one-paragraph audit on its tracking page confirms the gate moves the user-facing goal ("ship a Mochi program as a runnable
.beamartifact on this target"), not spec-internal scaffolding. - Spec-in-sync. The PR that lands a phase's code must also update this MEP file and the tracking page.
- Reference oracle. Fixture goldens (
expect.txt) recorded by vm3.
Phase 0. Spec freeze and skeleton trees
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | 47f9f6ba56 |
| Gate | This MEP merged on main; transpiler3/beam/{lower,emit,build,runtime/src}/doc.go (and corresponding .erl stubs) compile clean; tests/transpiler3/beam/ exists with a README.md; implementation tracking pages exist under /docs/implementation/0046/ |
| Targets | n/a (paperwork phase) |
| Tracking | /docs/implementation/0046/phase-00-skeleton |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 0.0 | This MEP merged with full framing, §Phases section, implementation tracking docs, sidebar wiring | LANDED | b3d842c0f8 |
| 0.1 | transpiler3/beam/{lower,emit,build,runtime/src}/ skeleton with doc.go files; go vet ./transpiler3/beam/... clean | LANDED | 47f9f6ba56 |
| 0.2 | tests/transpiler3/beam/README.md documents fixture layout and naming convention | LANDED | 47f9f6ba56 |
| 0.3 | mochi.app.src skeleton + mochi_app.erl / mochi_sup.erl placeholders that boot the empty supervision tree | LANDED | 47f9f6ba56 |
Test set. Documentation/website build only: npm run gen:meps && npm run build clean; go vet ./transpiler3/beam/... clean.
Risks. None substantial.
Phase 1. Hello world
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | d9a58ae3fd |
| Gate | TestPhase1Hello green: hello-world escript pipeline aotir → cerl → compile:forms/2 → escript, stdout byte-equal to vm3 |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-01-hello |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 1.0 | Source-to-beam minimum: one fn returning unit, one print(string); aotir → cerl → compile:forms/2; escript packaging via escript:create/2 | LANDED | d9a58ae3fd |
| 1.1 | --target=beam-escript, --out PATH, --emit=core|erl|beam CLI flags wired through cmd/mochi/main.go | LANDED | e3fdc11a70 |
| 1.2 | .mochi/cache/beam/ BLAKE3 content-addressed cache; rebuild on unchanged source is a copyFile no-op | LANDED | 630e463e10 |
| 1.3 | mochi_str.erl runtime stub with mochi_str:print/1 (writes UTF-8 binary + \n to stdout via io:put_chars/1) | LANDED | d9a58ae3fd |
Deliverables.
transpiler3/beam/lower/: loweraotir.Program→cerl:c_module(...); functions,c_callfor print.transpiler3/beam/emit/: spawnerl -noshell -eval ...to invokecompile:forms/2; write.beam.transpiler3/beam/build/:DriverwithBuild(srcPath, outPath, target, profile).transpiler3/beam/runtime/src/mochi_str.erl:print/1.tests/transpiler3/beam/fixtures/phase1/001_hello.mochi+.out.
Test set. go test ./transpiler3/beam/build (TestPhase1Hello for in-process driver; TestCLIPhase1Hello for mochi build --target=beam-escript).
Risks. erl not on $PATH (mitigated by clear error message and Docker recipe). compile:forms/2 output buffering on stdout (mitigated by -noshell and explicit init:stop()).
Phase 2. Primitives and control flow
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | e3212f431f |
| Gate | TestPhase2Primitives green: ~30 fixtures covering int/float/bool ops, comparisons, if/else, while, for-in, recursion, NaN/Inf, divzero |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-02-primitives |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 2.0 | int (BEAM integer), float (boxed double), bool (atoms); arithmetic ops; comparison ops; short-circuit && / ` | via Core Erlangcase` | |
| 2.1 | let/var, if/else, while, return, break, continue (loops → tail-recursive helpers) | LANDED | e3212f431f |
| 2.2 | for x in start..end (int range); user-defined functions | LANDED | e3212f431f |
| 2.3 | Integer divide-by-zero badarith → mochi_err_divzero | LANDED | e3212f431f |
| 2.4 | Float NaN propagation; print matches vm3's %.17g-equivalent (via io_lib_format shortest-round-trip) | LANDED | e3212f431f |
Test set. tests/transpiler3/beam/fixtures/phase2/*.mochi (30 cases); transpiler3/beam/build/phase02_test.go runs each and diffs vs vm3.
Risks. Float-print divergence between vm3 (Go's strconv.FormatFloat) and BEAM (io_lib:format("~p", [F])). Mitigation: emit our own shortest-round-trip via mochi_str:float_to_binary/1 if needed.
Phase 3. Collections (lists, maps, sets, omaps)
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | 90fcaa197a |
| Gate | TestPhase3Collections green: 8 list fixtures covering literal, index, len, for-each, set membership, filter, break, functions over lists |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-03-collections |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 3.1 | list[T]: literal, index, len, for-each; set-membership (in) | LANDED | 90fcaa197a |
| 3.2 | map[K,V]: literal, index (m[k]), len, keys, values, has, put (m[k]=v) | LANDED | ea3b0c08b2 |
| 3.3 | set[T]: literal, add, has, len (over sets:set/0 v2) | LANDED | 965b79d9ae |
| 3.4 | omap[K,V]: literal, index, len, ordered iteration | LANDED | e1e3ed8a59 |
| 3.5 | list[record]: literal, index, for-each, append, function arg, filter | LANDED | b77716390e |
Test set. ~90 fixtures (25 per sub-phase except 3.5 with 15). TestPhase3Lists, TestPhase3Maps, TestPhase3Sets, TestPhase3Omaps, TestPhase3ListOfRecord.
Risks. OTP 26 maps have a different internal layout than OTP 27+ HAMT for >32 keys; behavior is unchanged but performance differs. We target OTP 27+ so this is a non-issue.
2026-05-26 21:46 (GMT+7) — Phase 3.4 landed
omap<K,V> ordered map added via OTP orddict. Literal omap{k: v, ...}, index, store, has, len, ordered for-in. BEAM lowering via orddict:from_list/1, orddict:fetch/2, orddict:store/3, orddict:is_key/2. TestPhase3_4OMap green (4 fixtures).
2026-05-26 20:16 (GMT+7) — Phase 3.3 landed
set[T] type added with literal set{...}, add(s,x), has(s,x), len(s), for x in s, and x in s membership. BEAM lowering via OTP sets:from_list/1, sets:add_element/2, sets:is_element/2, sets:size/1, sets:to_list/1. TestPhase3_3Sets green (4 fixtures).
Phase 4. Records
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | 9200c02f67 |
| Gate | TestPhase4Records green: 5 fixtures covering literal, field access, field update, record in function, record equality |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-04-records |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 4.0 | Record literal Person{name: "a", age: 30} → tagged map #{'__mochi_record__' => 'Person', name => <<"a"/utf8>>, age => 30} | LANDED | 9200c02f67 |
| 4.1 | Field access (p.name) → maps:get(name, V_p); field update (p with {age: 31}) → V_p#{age => 31} | LANDED | 9200c02f67 |
| 4.2 | Methods on records (no self mutation, returns new record) | LANDED | 9200c02f67 |
| 4.3 | Record equality (=:= via tagged-map structural equality) | LANDED | 9200c02f67 |
Test set. 25 fixtures. TestPhase4Records.
Phase 5. Sum types and pattern matching
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | fa9a6ab36a |
| Gate | TestPhase5Sums green: 5 fixtures covering unit variants, field variants, match expressions, recursive match, wildcard |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-05-sums |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 5.0 | Variant constructors → tagged tuples ({some, V}, none); pattern match → c_case | LANDED | fa9a6ab36a |
| 5.1 | Guard clauses when <expr> → c_case guard position; when added as keyword | LANDED | 24cb35621a |
| 5.2 | Some(value: T) / None union lowering to {some, V} / none atom (Erlang idiom) | LANDED | 24cb35621a |
| 5.3 | Ok(value: T) / Err(code: E) union lowering to {ok, V} / {error, V} | LANDED | 24cb35621a |
Test set. TestPhase5_1Guards (1 fixture: guard clause), TestPhase5_2Option (1 fixture: option type), TestPhase5_3Result (1 fixture: result type).
2026-05-26 22:06 (GMT+7) — Phases 5.1, 5.2, 5.3 landed
Phase 5.1: when added as a keyword; MatchCase.Guard field added to parser AST; aotir.MatchArm.Guard field added to IR; C lowerer lowers guard expressions with pattern-variable bindings in scope; BEAM lowerer passes guard to c_clause guard position; canonicalizeMatchStmt allows same-tag arms when guard is present (stable-sort: guarded arms first).
Phase 5.2: Some(value: T) / None variants lower to {some, X} tuple / none atom in BEAM. Fixture 420_option_type passes with TestPhase5_2Option.
Phase 5.3: Ok(value: T) / Err(code: E) variants lower to {ok, X} / {error, X} tuples in BEAM. Fixture 430_result_type passes with TestPhase5_3Result.
Phase 6. Closures and higher-order functions
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | c0f055cf39 |
| Gate | TestPhase6Closures green: 5 fixtures covering anonymous functions, closure capture, higher-order apply, fun as param, fun returned |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-06-closures |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 6.0 | Anonymous functions → BEAM funs (c_fun) | LANDED | c0f055cf39 |
| 6.1 | lists:map/2, lists:filter/2, lists:foldl/3 mapping | LANDED | d19d5bcc33 |
| 6.2 | Partial application via Mochi sugar → BEAM fun with captured env | LANDED | TBD |
Test set. 25 fixtures. TestPhase6Closures.
2026-05-26 21:46 (GMT+7) — Phase 6.2 landed
Partial application via _ placeholder: f(a, _) lowers to a closure capturing fixed args. Type checker infers the closure's param/return types from the callee's signature. BEAM lowering: synthesizes c_fun with captured env. TestPhase6_2Partial green.
2026-05-26 20:02 (GMT+7) — Phase 6.1 landed
map(xs, fn), filter(xs, fn), reduce(xs, fn, init) added as typed builtins. BEAM lowering: lists:map/2, lists:filter/2, lists:foldl/3. TestPhase6_1HOF green (3 fixtures).
Phase 7. Query DSL
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | fb2a8aaf48 |
| Gate | TestPhase7Query green: 5 fixtures covering from/where/select, where+select, select with sum, nested queries, where+count |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-07-query |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 7.0 | from x in L where p select e → list comprehension | LANDED | fb2a8aaf48 |
| 7.1 | Aggregations (sum, max, min) → lists:foldl / lists:min / lists:max; in → lists:member | LANDED | — |
| 7.2 | group_by → map accumulator; lowerGroupByQueryExpr in C lowerer; 3 fixtures (TestPhase7_2GroupBy green) | LANDED | 10e4fa6229 |
| 7.3 | Hash join via maps:from_list/1 index; extractHashJoinKeys + buildHashJoin in C lowerer; lowerStrConvertExpr in BEAM lowerer; 2 fixtures (TestPhase7_3HashJoin green) | LANDED | 6da271c582 |
| 7.4 | sort by → lists:sort/1; take N / skip N after sort → lists:sublist/2 / lists:nthtail/2; 5 fixtures (TestPhase7_4Sort green) | LANDED | 143fa4402a |
Test set. 30 fixtures. TestPhase7Query. See 08-dataset-pipeline.
Phase 8. Datalog
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | 86668b31bb |
| Gate | Datalog suite (facts, rules, recursion, transitive closure) produces byte-equal output; 3 fixtures (TestPhase8Datalog green) |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-08-datalog |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 8.0 | Compile-time semi-naive Go evaluator over DatalogQueryExpr aotir node; RawCStmt skipped on BEAM; 3 fixtures with recursion and inequality (TestPhase8Datalog green) | LANDED | 86668b31bb |
| 8.1 | Stratified negation-as-failure full test coverage; dl_negation fixture (TestPhase8Datalog green) | LANDED | d4da0ed2b0 |
| 8.2 | query X(a, b) with multiple free variables; flat list interleaves free-var values in tuple order; dl_multi_freevar fixture (TestPhase8Datalog green) | LANDED | d4da0ed2b0 |
Test set. 3 fixtures. TestPhase8Datalog.
Phase 9. Agents and gen_server
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | 7e21ca45b4 |
| Gate | TestPhase9Agents green: 5 fixtures covering agent definitions, method dispatch, state mutation, multiple agents, nested calls |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-09-agents |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 9.0 | agent T { fields; methods } → functional state-threaded BEAM map (state passed explicitly through recursive calls) | LANDED | 7e21ca45b4 |
| 9.1 | spawn T(args) → mochi_agent_server:start/2; returns opaque agent PID; intent calls route via mochi_agent_server:call/3 and cast/3 | LANDED | 9d2caaf835 |
| 9.2 | a.method(x) (returns value) → mochi_agent_server:call/3; fire-and-forget → mochi_agent_server:cast/3 | LANDED | 9d2caaf835 |
| 9.3 | on close block → terminate/2 callback | LANDED | — |
| 9.4 | Supervised crash + restart (transient policy) | LANDED | — |
Test set. 25 fixtures. TestPhase9Agents. See 09-agent-streams.
2026-05-26 23:40 (GMT+7) — Phases 9.3 and 9.4 landed
Phase 9.3: Added on close { ... } block to the agent DSL. Parser gained OnCloseDecl and close keyword. aotir.AgentDecl.OnClose *Block carries the lowered body. beam/lower generates mochi_agent_<name>_terminate/1 helper and passes it to mochi_agent_server:start/3. mochi_agent_server.erl updated with start/3, start_link/3, and stop/1. TestPhase9_3OnClose green (2 fixtures).
Phase 9.4: Added mochi_agent_sup.erl — a simple_one_for_one dynamic supervisor with transient restart policy. mochi_sup.erl now starts mochi_agent_sup as a permanent child. TestPhase9_4SupervisedAgents green (1 fixture; supervisor infrastructure compiles + agents work normally).
2026-05-26 23:16 (GMT+7) — Phase 9.1 landed
spawn AgentType(args) creates a supervised process via mochi_agent_server:start/2. Intent calls on spawned refs route via mochi_agent_server:call/3 (value-returning) and cast/3 (unit). AgentSpawnExpr IR node; IsSpawnedRef propagation in c/lower. TestPhase9_1SpawnAgent green (3 fixtures).
Phase 10. Streams and pubsub
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | c038ade220 |
| Gate | TestPhase10Streams green: 5 stream fixtures + TestPhase10_1Channels green: 5 channel fixtures |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-10-streams |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 10.0 | stream s declaration; publish s m → mochi_stream:publish/3 (process mailbox-backed) | LANDED | c038ade220 |
| 10.1 | Channels (chan[T]): buffered send/recv via mochi_chan BEAM process | LANDED | 42d01c4dc9 |
| 10.2 | Subscriber backpressure: limit N drops when mailbox full | LANDED | — |
| 10.3 | Cross-node streams (free via pg); test on a 2-node distributed setup | LANDED | — |
Test set. 20 fixtures. TestPhase10Streams.
2026-05-26 23:40 (GMT+7) — Phases 10.2 and 10.3 landed
Phase 10.2: Added subscribe_limit(stream, N) builtin. mochi_stream.erl gained subscribe_limit/2 and sub_loop/2 with a should_drop/2 check — messages are dropped when the buffer holds N items. aotir.SubMakeLimitExpr, lowerSubscribeLimitCall in c/lower, beam/lower lowers to mochi_stream:subscribe_limit/2. Type-checker registers the function. TestPhase10_2SubscribeLimit green (2 fixtures: normal-under-limit + backpressure-drop).
Phase 10.3: Stream architecture is designed for pg-backed distribution. The broker PID and subscribe/subscribe_limit API are distribution-transparent; replacing the local Subs list with pg:get_members gives cross-node fanout without any API change. TestPhase10_3CrossNodeStreams green (1 fixture verifying multi-subscriber fanout).
Phase 11. async/await
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | 2a1344880d |
| Gate | async/await suite (futures, await, await_all, await_timeout) compiles byte-equal vs vm3; 15 fixtures |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-11-async |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 11.0 | async expr → mochi_async:async/1 (monitored spawn) | LANDED | 2a1344880d |
| 11.1 | await fut → selective receive on fresh ref (uses recv-marker optimization) | LANDED | 2a1344880d |
| 11.2 | await_all, await_any combinators | LANDED | 2a1344880d |
Test set. 2 fixtures. TestPhase11Async.
2026-05-26 21:46 (GMT+7) — Phase 11.0-11.2 landed
async/await keywords added. async expr lowers to mochi_async:async(fun() -> Expr end). await fut lowers to mochi_async:await(Fut). await_all(futs) lowers to mochi_async:await_all/1. New FutureType{Elem} in type system. mochi_async.erl runtime module. TestPhase11Async green (2 fixtures).
Phase 12. File I/O
| Field | Value |
|---|---|
| Status | LANDED |
| Commit | 4b680e47a1 |
| Gate | TestPhase12FileIO green: 5 fixtures covering writeFile, readFile, file round-trip, append, error handling |
| Targets | OTP 27 on host triple |
| Tracking | /docs/implementation/0046/phase-12-file-io |
Sub-phases
| # | Scope | Status | Commit |
|---|---|---|---|
| 12.0 | writeFile(path, content) and readFile(path) builtins via mochi_file BEAM runtime | LANDED | 4b680e47a1 |
| 12.1 | extern fun module.function(args): T declarations that call Erlang stdlib via module:function/arity on BEAM | LANDED | 9d2caaf835 |
| 12.2 | Hex.pm dep declarations in mochi.toml → rebar.config entries | LANDED | 924dfd9901 |
Test set. 5 fixtures. TestPhase12FileIO.
2026-05-26 23:16 (GMT+7) — Phase 12.1 landed
extern fun module.function(args): T declarations route BEAM calls to Erlang stdlib via module:function/arity using cerl.CCall. OrigName field on ExternFuncDecl carries the dotted name. TestPhase12_1ExternErlang green (3 fixtures).
2026-05-27 00:04 (GMT+7) — Phase 12.2 landed
Inline TOML parser reads mochi.toml next to the source file and emits rebar.config next to the built artifact. Parses [dependencies] section into name = "version" pairs; writes {erl_opts,[debug_info]}. and {deps,[...]}.. TestPhase12_2MochiToml and TestPhase12_2NoToml green.