Skip to main content

MEP 43. Zero-Boilerplate Go Transpiler and Go FFI via go/types Introspection

FieldValue
MEP43
TitleZero-Boilerplate Go Transpiler and Go FFI
AuthorMochi core
StatusDraft
TypeStandards Track
Created2026-05-21 14:00 (GMT+7)
DependsMEP-4 (Type System), MEP-5 (Type Inference), MEP-40 (vm3 + compiler3), MEP-41 (Memory Safety), MEP-42 (Native Code Emission)

Abstract

Mochi today ships a Go transpiler under transpiler/x/go/ that is a ~9000-line hand-rolled tree walker, plus a runtime FFI under runtime/ffi/go/ that is a reflection-based Call(name, args ...any) registry with a fallback path (AttrAuto) that shells out to go run once per call and serialises arguments as JSON. Both work; both are the wrong shape for Mochi as it stands in May 2026. The transpiler grows linearly with the Mochi grammar (every new statement form needs a new emitter case); the FFI grows quadratically with adoption (every Go package the user wants to call needs a Register block somewhere, every argument needs a type assertion at the call site).

This MEP specifies a from-scratch replacement for both pieces, designed under one hard constraint: Mochi ships zero per-package and zero per-syntactic-construct hand-written glue. The replacement rests on three generic components:

  1. A type bridge (compiler3/ffi/typebridge/) implementing two pure functions, GoToMochi(types.Type) compiler3.Type and MochiToGo(compiler3.Type) string. This is the only finite, hand-maintained mapping in the entire pipeline.
  2. A build-time binding resolver (compiler3/ffi/resolve/) that on every import go "path" invokes golang.org/x/tools/go/packages.Load with NeedTypes | NeedTypesInfo | NeedSyntax | NeedDeps | NeedImports, walks the resulting *types.Package graph, and registers every exported symbol as a typed Mochi binding via the type bridge. No per-package code lives anywhere in Mochi.
  3. A typed-IR to Go-source emitter (compiler3/emit/go/) sized to the compiler3 IR op set (~50 op kinds per MEP-40 §7.1). The emitter consumes typed SSA, prints idiomatic Go, and shells to the host go build. FFI calls are emitted as literal Go expressions; the Go compiler does the static type check; the Go runtime executes the call with zero Mochi-side reflection.

The shipped Mochi runtime for the Go target lives in runtime/mochi/ as a normal Go module: strings, lists, maps, queries, decimals, set ops, JSON, fmt. It is written once, has normal Go tests, and is itself a legitimate Go library a Go user could import. The emitter calls into it. The emitter does not re-implement these semantics per-call.

The two precedents to study hardest are Yaegi's extract.go (Traefik's Go interpreter; the only deployed example of go/types driving a zero-per-package binding pipeline across the entire Go stdlib) on the FFI side, and Nim's typed-IR C backend (~50 op kinds drive a full systems language emitter) on the transpiler side. The anti-pattern this MEP forecloses by name is the runtime reflection registry Call(name, args...), which relocates glue from compile time to runtime without deleting it, pays a 10-100x per-call tax, and still demands per-package Register blocks (Python ctypes is the canonical warning; Mochi's current runtime/ffi/go/ is the local example).

This MEP is a Standards Track design document. The phased plan ships incrementally, gated per phase, and the MEP file is updated with measured results at each phase boundary (MEP-spec-in-sync rule). The shipping Mochi binary stays pure-Go-no-cgo. No phase introduces a per-package binding file.

Motivation

What today's Go target leaves on the table

The current transpiler/x/go/transpiler.go is one file, one tree walker, every statement form is a hand-written emitter. Adding a Mochi feature requires a parser change, a type-checker change, a vm3 change, a compiler3 change, and a Go emitter change. The Go emitter is the lever that is least like the others: it does not consume compiler3 IR, it walks the parser AST directly, and it re-derives types that the type checker already proved. Every typed pass in MEP-40 is wasted on the Go target.

The Go-target stdlib is also re-derived. Mochi-side append, len, keys, values, query algebra primitives, JSON serialisation, decimal arithmetic, and date/time handling each have a hand-written emit path that produces an inline Go expression. There is no runtime/mochi/ Go module that a normal Go user could import; the semantics live as printf templates inside the emitter. This is the same trap Joy (Go to JS, archived 2018) fell into.

What the legacy reflection FFI cost us

The current runtime/ffi/go/ffi.go is a registry: map[string]any plus reflect.Value.Call plus reflect.Value.Convert per argument plus a per-return-shape unpacker. The fallback path AttrAuto is worse: it synthesises a one-shot main.go per call, runs go run (cold-start: hundreds of milliseconds), JSON-encodes the argument list into MOCHI_ARGS, JSON-decodes the result. Every call costs at minimum one reflect.ValueOf per argument plus one reflect.Value.Interface per return; calls through AttrAuto cost one fork+exec+compile cycle. Neither model is suitable for production Mochi-from-Go code, neither model exposes static type errors at the Mochi call site (the user discovers them only at runtime), and the AttrAuto model requires the user to install the Go toolchain and have a writable temp directory at runtime, not just at build time.

The Register(name, fn) API forces the user (or the Mochi maintainers, package by package) to enumerate every callable symbol before it can be invoked. The reflection registry pattern (Python ctypes, the original otto JS bridge, every "embed Lua in your Go program" library before LuaJIT) was the dominant pre-2020 FFI shape, and every measured comparison (cffi vs. ctypes, pybind11 vs. ctypes, Cython vs. ctypes) shows it is 10-100x slower than direct emission and still demands per-package glue.

What changed between v0.10 and May 2026

Three things between the v0.10 transpiler and May 2026 make this MEP unavoidable.

compiler3 typed IR is real (MEP-40 §7). Every SSA value carries a proven type; every op encodes the type. A Go-source emitter that consumes compiler3 IR is structurally the same shape as the vm3 bytecode emitter that MEP-40 already validated. The architectural template has shipped.

go/packages plus generics matured. Since Go 1.21 (August 2023), *types.TypeParam, *types.Union, and types.Instantiate are stable; packages.Load(NeedTypes | NeedTypesInfo | NeedSyntax | NeedDeps | NeedImports) returns a fully resolved package graph including generic instantiations, embedded fields, method sets, and interface-satisfaction proofs. Everything Mochi needs to drive a build-time binding resolver is in the standard golang.org/x/tools module. There is no missing API.

The reflection-registry pattern lost industry consensus. .NET 7 (November 2022) shipped LibraryImportAttribute as a source generator replacing DllImportAttribute (runtime marshalling); Rust bindgen is the standard for C FFI; Swift's clang importer is built into the compiler; Python's CFFI and Cython have outpaced ctypes for years. The recent consensus is consume the foreign type system at build time and emit native calls, never marshal at runtime. Mochi is one of the last languages still shipping a Call(name, args...) registry as its primary FFI.

Why one MEP for two pieces

The transpiler problem and the FFI problem are the same problem:

QuestionTranspilerFFI
What does Mochi mean by xs.append(x)?Go's append(xs, x)Go's append(xs, x)
What does Mochi mean by strings.ToUpper(s)?Call into runtime/mochi/stringsCall into strings
Who type-checks the call?The Go compiler, after emitThe Go compiler, after emit
Who marshals the arguments?NobodyNobody

In both cases Mochi types the call, picks a Go expression that satisfies the Go-side type, emits it. The only differences are which Go package the call resolves to and whether the user wrote import or not. Separating the two pieces into two MEPs would force two type bridges, two emitters, two cache layers; combining them into one MEP forces one of each.

Scope

In scope:

  • Complete design and reference implementation of compiler3/ffi/typebridge/ (the single, finite, hand-maintained type table).
  • Complete design and reference implementation of compiler3/ffi/resolve/ (the build-time binding resolver over golang.org/x/tools/go/packages).
  • Complete design and reference implementation of compiler3/emit/go/ (typed-IR to Go-source emitter sized to compiler3 IR op count).
  • A normal Go module under runtime/mochi/ implementing Mochi runtime semantics for the Go target (strings, lists, maps, sets, queries, JSON, decimal, fmt). Idiomatic Go, normal Go tests, importable by a Go user as a standalone library.
  • A mochi build --target=go driver that produces a .go file plus a go.mod plus a vendored runtime/mochi and shells out to the host go build.
  • A mochi run path on the Go target that runs the produced binary directly, with source-map back-translation of Go compile errors to Mochi source coordinates.
  • Build-time binding cache keyed on (import path, go.sum hash, mochi version) so cold-start of the resolver is paid once per Go module upgrade, not once per mochi build.
  • Export direction (Mochi-from-Go): a mochi build --emit=go-library mode that produces a Go package whose public surface is the Mochi program's exported symbols, with type-bridge-derived Go signatures and no per-symbol stubs.
  • A migration path for the legacy transpiler/x/go/ and runtime/ffi/go/ packages: parallel deployment under a feature flag, A/B test against the existing golden suite, retire the legacy paths after one release of overlap.

Out of scope (deferred to successor MEPs):

  • Generic transpiler targets other than Go (Python, Rust, C++, etc.). The type-bridge and emitter pattern generalises, but the per-target work (per-target stdlib, per-target binding source) is one MEP per language. This MEP names only Go.
  • cgo from Mochi. Mochi-to-Go FFI handles the common case; the rare "I need to call a C library and there is no Go wrapper" case is deferred. The right shape for that case is "Mochi calls a Go wrapper that calls C," not "Mochi calls C directly."
  • Reflection on Mochi values from Go code that imports the exported library. The export direction emits idiomatic Go types; if the user wants to walk those reflectively, they use Go's own reflect package on the result.
  • Hot reload of changed Go packages at runtime. The build-time resolver caches per go.sum hash; reload requires a rebuild. A live-reload story belongs in a tooling MEP.
  • WASM / native AOT delivery of Mochi-to-Go-transpiled programs. The Go target produces a binary via go build; cross-targeting is GOOS/GOARCH on the host toolchain (zero extra Mochi work). The MEP-42 native-emission paths are independent.
  • Mochi-side generics in the Go emitter. Generics in compiler3 IR are MEP-12 territory; this MEP commits to emitting their lowered (monomorphised) form. Generic Go callees (Go 1.18+ type-parameter functions) are handled by the binding resolver via types.Instantiate.

Background: zero-glue interop landscape (May 2026)

The material below walks six axes of the current interop landscape. Each axis names the precedents Mochi should learn from and the anti-patterns to avoid.

1. Source-to-source language hosts for Go

GopherJS (Go to JS) reuses the upstream Go standard library essentially verbatim, replacing only the handful of packages whose semantics cannot survive in a JS VM (syscall, runtime, reflect internals) with hand-written JS shims. The lesson: if the IR is faithful to Go's type system, ~95% of the stdlib comes for free. Yaegi (Traefik's Go interpreter) is the cleanest extant example of "introspect once, never write per-package glue": a code generator (extract.go) walks go/importer once and emits map[string]reflect.Value tables for the entire Go stdlib. TinyGo reuses the upstream stdlib through build tags and a small src/runtime override set, demonstrating that even a backend swap (SSA to LLVM) does not require rewriting fmt or encoding/json. Joy (Go to JS, archived 2018) and goscript are cautionary tales: per-construct emitters and per-package shims, both stalled.

2. Bridge-by-introspection precedents

The "consume the foreign type system at build time" pattern is now dominant. Rust bindgen parses C/C++ headers via libclang and emits extern "C" Rust declarations with zero runtime marshalling. Swift's clang importer is built into the compiler frontend and ingests Objective-C/C headers as first-class Swift modules. .NET 7 source generators replaced System.Reflection.Emit for serializers, regex, and P/Invoke (LibraryImportAttribute). OCaml ctypes describes C types as OCaml values and derives stubs via cstubs. The projects that achieved truly zero hand-glue for the common case are Rust bindgen, Swift's clang importer, and .NET source generators. The projects that still ship per-package shims in practice (Crystal lib, Nim importc, V translate, D extern(C++)) all fall short of Mochi's hard constraint.

3. go/packages + go/types as a binding source

packages.Load(NeedTypes | NeedTypesInfo | NeedSyntax | NeedDeps | NeedImports) returns a fully resolved *types.Package graph. From it Mochi can recover: every exported *types.Func with its *types.Signature (params, results, variadic, receiver); every *types.Named with its method set via types.NewMethodSet; every *types.Struct with field tags and embedding; every *types.Interface with Implements and AssignableTo queries; full generics support via *types.TypeParam, *types.Union, types.Instantiate. This is enough to mechanically derive call sites, conversion sites, and interface-satisfaction proofs for any reachable symbol without one line of per-package code.

What go/types does not expose: effect annotations (does this function block, panic, allocate, touch goroutines), ownership and lifetime (who frees the *os.File), nilability beyond "is a pointer," concurrency safety, side effects on globals. These gaps must be handled by Mochi's own type system (MEP-15 effects, MEP-16 null safety) or by user annotations, not pretended away. The MEP-41 sealed-handle FFI mechanism is the safety floor for unannotated Go calls.

4. Why Call(name, args...) reflection FFI fails the zero-glue goal

The reflection registry pattern appears to delete glue but actually relocates it: every call site pays for reflect.Value.Call, every argument pays for boxing into any and a reflect.ValueOf, every result pays for type assertion, every error shape needs a per-symbol unpacker. Python ctypes is the canonical warning: users must restate argtypes/restype for every function, and the per-call cost is ~10x a cgo call. The Python community moved to cffi (build-time parsing of C declarations) and then to Cython / pybind11 (compile-time emission) for this exact reason. LuaJIT FFI succeeded only because LuaJIT specialises the call at JIT trace time, collapsing the reflection to a direct call; without a tracing JIT the pattern is pure overhead. Go's existing reflection FFI registries (Yaegi for non-stdlib packages, otto, the legacy runtime/ffi/go/) all exhibit 20-100x slowdowns vs. native calls and force per-package Register blocks, which is the precise failure mode this MEP forbids.

5. Generic transpiler emitters driven by typed IR

The recurring trick in small-emitter transpilers: collapse the AST into a typed mid-level IR with a fixed, small op set, then map each op to a single textual template. Crystal lowers to typed nodes then to LLVM IR through ~60 instruction kinds. Nim's C backend (compiler/ccgexprs.nim) walks a typed IR and dispatches on ~50 magic ops; per-language-feature code is rare. V uses a typed IR and a ~2000-line C emitter that is a switch over op kinds with printf templates. Zig's translate-c uses clang's typed AST and a templated emitter. The general lesson: if the IR carries full type information, the emitter has no need for a per-syntactic-construct case (no separate "emit-for-range-over-channel" function); it has a per-op case, and ops are language-agnostic.

6. Reverse direction: invoking Mochi from Go without glue

For Mochi-from-Go, the precedents split on whether the host needs per-symbol stubs. cgo requires per-symbol //export and a C header (not zero-glue). gomobile/gobind generates per-symbol Java/Obj-C bindings from go/types introspection (exactly the build-time emission pattern reused in reverse; the right model). plugin (Go's .so loader) is zero-glue at the call site but requires plugin.Lookup by name plus a concrete signature assertion (glue at the caller). hashicorp/go-plugin uses RPC over Unix sockets with per-plugin interfaces (lots of glue). wazero / wasmtime-go host imports are the modern zero-glue model: the embedder declares a typed import set once and any guest can be loaded; Mochi-as-wasm-guest plus wazero-as-host is a credible second emission target (deferred to MEP-42 phase 6 Wasm).

Decision substrate

  • FFI precedent to study hardest: Yaegi's extract.go. The only deployed example of go/types driving a zero-per-package binding pipeline across the entire Go stdlib. The Mochi build-time binding resolver is structurally Yaegi-extract.go-shaped, with two differences: Mochi resolves on the user's import graph (not pre-baked), and Mochi emits typed Go calls (not reflect.Value lookups).
  • Transpiler precedent to study hardest: Nim's C backend. ~50 op kinds drive a full systems language emitter. The Mochi compiler3/emit/go/ is sized to the same op count.
  • Anti-pattern to forbid by name: Python ctypes / Call(name, args...) reflection registry. Mochi's current runtime/ffi/go/ is the local instance; the MEP closes it.

Specification

§1 Architecture

Mochi source
|
v
parser/AST (MEP-3)
|
v
type checker (MEP-4, MEP-5)
| \\
| \\------> ffi/resolve --> *types.Package graph (build-time)
| |
| v
| type bridge (the only glue table)
| |
v v
compiler3 typed IR (MEP-40 §7.1) <----- typed FFI symbol table
|
+------------------+-----------------+------------------+
| | | |
v v v v
emit/vm3 emit/go (this MEP) emit/copypatch emit/c emit/wasm
(bytecode) (Go source) (MEP-42 JIT) (AOT) (AOT)
| |
v v
runtime/vm3 host `go build`
|
v
native binary that imports the user's
Go packages directly + runtime/mochi

The four-bit arena tag and 12-bit generation encoding of MEP-40, plus the verifier rules of MEP-41, are load-bearing for emit/go exactly as they are for emit/vm3. The Go-target emitter never destructures handles; it calls into runtime/mochi/ helpers (mochi.DerefList, mochi.DerefMap, etc.) that wrap the same accessors runtime/vm3/accessors.go already enforces.

The boundary between this MEP and MEP-42 is the target. MEP-42 emits machine code via copy-and-patch JIT or via a C intermediate; MEP-43 emits Go source for the user's go toolchain. Both consume the same compiler3 IR. A Mochi program can be built any of the four ways with no source changes; the choice is a --target= flag, not a code edit.

§2 Type bridge (the only glue table)

compiler3/ffi/typebridge/typebridge.go exports two pure functions:

package typebridge

import (
"go/types"
ir "mochi/compiler3/ir"
)

// GoToMochi maps a go/types.Type to a compiler3 IR type.
// Opaque returns are used for Go types that have no native Mochi
// surface; they remain typed at the compiler3 layer (the IR carries
// the original *types.Type), but Mochi code can only pass them
// through, not destructure them.
func GoToMochi(t types.Type) ir.Type { /* ... */ }

// MochiToGo maps a compiler3 IR type to a Go source-level type string.
// The string is used verbatim in the emitted Go source; it must be
// syntactically valid Go.
func MochiToGo(t ir.Type) string { /* ... */ }

The mapping table is finite and small:

Go typeMochi IR type
boolTypeBool
int, int64TypeI64
int32, int16, int8TypeI64 with width tag
uint*TypeI64 with unsigned tag
float64TypeF64
float32TypeF64 with width tag
stringTypeStr
[]TTypeList(GoToMochi(T))
[N]TTypeList(GoToMochi(T)) (length pinned in metadata)
map[K]VTypeMap(GoToMochi(K), GoToMochi(V))
struct{...}TypeStruct(...)
*TTypeRef(GoToMochi(T))
interface{} / anyTypeOpaque
named interface{...}TypeIface(name, method set)
chan TTypeChan(GoToMochi(T))
func(...) ...TypeFunc(params, results)
named type with methodsTypeNamed(name, underlying, methods)
generic instantiationrecurse on instantiated args via types.Instantiate
unsupportedTypeOpaque(t) (pass-through only)

This is the only hand-maintained table in the entire pipeline. It is finite (~30 entries), purely structural, and changes only when Go itself gains a new structural type (last such addition: type parameters, Go 1.18, March 2022). The table is tested with table-driven Go tests that ingest the entire Go stdlib's exported surface and assert round-tripping; any new Go release that breaks the table is caught by the next CI run.

No per-package code lives here. No per-symbol code lives here. The table does not know strings or os exists.

§3 Build-time binding resolver

compiler3/ffi/resolve/resolve.go exports one function:

package resolve

import (
"go/types"
"golang.org/x/tools/go/packages"
ir "mochi/compiler3/ir"
)

// Resolve loads the Go package at importPath and returns a typed
// Mochi binding for every exported symbol. The result is a tree of
// ir.Binding values that the type checker consumes directly; no Mochi
// code looks at *types.Package after this point.
func Resolve(importPath string, cache *Cache) (*ir.PackageBinding, error)

The resolver does the following on first call per (importPath, go.sum hash, mochi version) tuple:

  1. Invoke packages.Load(&packages.Config{Mode: packages.NeedTypes | packages.NeedTypesInfo | packages.NeedSyntax | packages.NeedDeps | packages.NeedImports}, importPath).
  2. For each *types.Func in the package scope: build an ir.FuncBinding{Name, Signature: GoToMochi-applied-to-each-param-and-result, GoCallExpr: pkgAlias + "." + Name}.
  3. For each *types.Named with a non-empty method set: build an ir.TypeBinding{Name, Underlying: GoToMochi(t.Underlying()), Methods: map[string]ir.FuncBinding}.
  4. For each *types.Var (package-level): build an ir.VarBinding{Name, Type: GoToMochi(v.Type()), GoExpr: pkgAlias + "." + Name}.
  5. For each *types.Const: build an ir.ConstBinding{Name, Type: GoToMochi(c.Type()), Value: c.Val(), GoExpr: pkgAlias + "." + Name}.
  6. Cache the result on disk under ~/.cache/mochi/bindings/<importPath>@<gosum-hash>.binding.gob.

Subsequent calls hit the cache. The cache key includes the user's go.sum hash so dependency upgrades invalidate cleanly. The cache key includes the Mochi version so a Mochi upgrade that changes the type bridge invalidates cleanly. There is no per-package source file shipped in the Mochi tree.

The resolver is allowed to fail with a structured error: ErrUnresolvedSymbol, ErrOpaqueOnly (the package's surface is reachable only via interface), ErrGenericNotInstantiated (a generic function was referenced without a concrete instantiation). The type checker surfaces these as Mochi diagnostics with the import path and the offending symbol.

§4 Typed-IR to Go-source emitter

compiler3/emit/go/emit.go consumes a *ir.Program and prints a *ast.File plus a go.mod. The emitter is one pass, dispatch by op kind, ~50 cases corresponding to the ops in MEP-40 §7.1. Each case is a printf template.

Example shape (illustrative):

func (e *Emitter) emit(v ir.Value) {
switch v.Op {
case ir.OpConstI64:
fmt.Fprintf(e.w, "var v%d int64 = %d\n", v.ID, v.Const)
case ir.OpAddI64:
fmt.Fprintf(e.w, "v%d := v%d + v%d\n", v.ID, v.Args[0], v.Args[1])
case ir.OpListAppend:
fmt.Fprintf(e.w, "v%d := append(v%d, v%d)\n", v.ID, v.Args[0], v.Args[1])
case ir.OpCallGo:
// Args[0] is the binding id; Args[1:] are the actual args.
b := e.bindings[v.Args[0]]
fmt.Fprintf(e.w, "v%d := %s(", v.ID, b.GoCallExpr)
for i, a := range v.Args[1:] {
if i > 0 { fmt.Fprint(e.w, ", ") }
fmt.Fprintf(e.w, "v%d", a)
}
fmt.Fprint(e.w, ")\n")
case ir.OpQueryGroupBy:
// Lowers to a call into runtime/mochi/query.
fmt.Fprintf(e.w, "v%d := mochi_query.GroupBy(v%d, func(x %s) %s { ... })\n",
v.ID, v.Args[0], ...)
// ... ~50 cases total ...
}
}

There is no compileStmtImport function. There is no compileExternFun function. There is no compileGoAuto function. There is no per-statement-shape function. The Mochi parser AST is consumed by the type checker and translated to compiler3 IR; the emitter sees only IR.

The OpCallGo op carries a binding id resolved at type-check time, so the emitter prints a literal Go call expression. There is no Call(name, args...) at runtime. There is no reflection. The Go compiler does the static type check on the emitted call; if the user wrote a Mochi expression that the type bridge mapped to a Go call with mismatched arity, the Go compiler reports the mismatch and Mochi's diagnostic layer maps it back to the Mochi source line.

The emitter is permitted to inline trivial Go calls (len, cap, append, string conversions) as Go syntactic primitives rather than function calls; this is a peephole opt in the emitter, not a per-package binding. The list of inline-able forms is finite and lives in compiler3/emit/go/inlines.go as a small table.

§5 Go-side runtime as a normal Go module (runtime/mochi/)

The shipped Mochi runtime for the Go target lives under runtime/mochi/ as a normal Go module with the following packages:

  • runtime/mochi/strings (Mochi string ops: Upper, Lower, Contains, slice operators)
  • runtime/mochi/lists (Mochi list ops: Append, Concat, Index, Slice, set operators)
  • runtime/mochi/maps (Mochi map ops: Keys, Values, Entries, Has)
  • runtime/mochi/sets (Mochi set ops with stable iteration order)
  • runtime/mochi/query (Mochi query algebra: Filter, Map, Sort, GroupBy, Join, LeftJoin, OuterJoin, Limit, Take)
  • runtime/mochi/json (JSON load/save matching Mochi semantics: LoadJSON, SaveJSON, LoadJSONL, SaveJSONL)
  • runtime/mochi/yaml (YAML load/save)
  • runtime/mochi/fmt (Mochi print semantics: stable list/map ordering, struct field ordering)
  • runtime/mochi/decimal (Mochi decimal arithmetic)
  • runtime/mochi/time (Mochi time and date semantics)

Each package is normal Go. Each package has normal Go tests. Each package is go test-able in isolation. A Go user could import "github.com/mochilang/mochi/runtime/mochi/query" in their own Go program and use it as a query library.

The Mochi-to-Go emitter calls into these packages by name. There is no inlined query algebra in the emitter. There is no inlined JSON serialiser in the emitter. The semantics live exactly once, in idiomatic Go, with normal Go review and normal Go test coverage. This is the Yaegi-style discipline: Mochi's runtime is Go code, not printf templates.

The compiler3 IR has one OpCallRuntime op kind for calls into runtime/mochi/*. It is structurally identical to OpCallGo (same binding-id mechanism), with the difference that the binding is pre-resolved at Mochi compile time against the vendored runtime, not against the user's import graph.

§6 import go "path" semantics

The Mochi surface for Go FFI is one statement form: import go "path" [as alias]. The semantics:

  1. Parse time: the parser captures Lang = "go", Path = "path", As = alias.
  2. Type-check time: the type checker invokes resolve.Resolve("path", cache) and registers each binding under alias (or the path-derived default alias) in the current scope. Each binding's Mochi type is the type-bridge image of its Go type.
  3. Compile time: every Mochi expression of the form alias.Sym(args...) lowers to OpCallGo{binding: alias.Sym.id, args} in compiler3 IR. Generic Go callees lower to OpCallGo with a binding resolved via types.Instantiate against the Mochi-side argument types at the call site.
  4. Emit time (Go target): the emitter prints a literal import "path" in the emitted .go file and prints the call as alias.Sym(args...).
  5. Emit time (vm3 / native targets): the emitter records the binding id; the vm3 runtime ships a generated shim per OpCallGo site that wraps the call (using compiler3 type info, not reflection) and registers it under the id. The shim generation is part of the compiler3 build product, not hand-written.

The Auto modifier (import go "path" auto) is retained for source compatibility but is now redundant: every import go is automatically resolved against go/packages. The modifier becomes a no-op annotation, scheduled for removal after one release.

There is no extern fun for Go. There is no extern var for Go. There is no extern type for Go. All of these are derived from go/packages introspection; the Mochi extern forms remain for languages without an introspectable type system (the legacy Python/Deno FFIs, until they are migrated). For Go specifically, the only surface is import go "path".

§7 Calling Mochi from Go (export direction)

mochi build --emit=go-library --out=./mypkg produces a normal Go package under ./mypkg/ with:

  • mypkg.go: the public surface, one Go function per Mochi export fn, one Go type per Mochi export type, one Go const per Mochi export const.
  • internal/mypkg_impl.go: the lowered compiler3 IR translated to Go source via emit/go/.
  • go.mod: declares module <user-chosen-path>, requires the vendored runtime/mochi.

The public surface is derived from the Mochi export declarations via the type bridge in reverse. A Go caller does import "github.com/user/mypkg"; mypkg.MyFn(42) and gets normal Go semantics, normal Go types, normal Go errors. There are no per-symbol stubs. There is no mypkg.Call("MyFn", 42).

The library is buildable by the user's go build with no Mochi tooling on the consumer's machine. The consumer needs runtime/mochi (vendored under internal/ by default; mochi build --emit=go-library --runtime=external pulls it from github.com/mochilang/mochi/runtime/mochi).

This closes the loop with gomobile/gobind: the same build-time-emission discipline that drives the Java/Obj-C export from Go drives the Go export from Mochi.

§8 Caching and incremental builds

The binding cache is keyed (import_path, go.sum_hash, mochi_version). The cache lives under $XDG_CACHE_HOME/mochi/bindings/ (or ~/.cache/mochi/bindings/ if XDG is unset; %LOCALAPPDATA%\mochi\bindings\ on Windows). Each cache entry is a single gob-encoded *ir.PackageBinding. The cache is content-addressed; no garbage collection is needed (the user can rm -rf the directory at any time and the next build rebuilds).

The emit cache is keyed (mochi_source_hash, compiler3_version, target). The emit cache stores the produced .go file. A Mochi rebuild on unchanged source is a single hash check. The Go-toolchain build cache ($GOCACHE) does its own caching on top.

End-to-end incremental: editing one Mochi file invalidates one emit cache entry, regenerates one .go file, calls go build which rebuilds one package and relinks. Steady-state edit-build-run latency is bounded by go build on a single-file change, which is sub-second for a normal program.

§9 Diagnostics: surfacing Go compile errors back to Mochi source

The emitter writes a //line directive (Go's standard source-map mechanism) at every basic block boundary, pointing to the originating Mochi source line. When go build fails, its diagnostics are filtered through compiler3/emit/go/diag.go which replaces the Go-source coordinates with the Mochi-source coordinates from the nearest //line directive.

The user sees Mochi line numbers, Mochi column numbers, and a Mochi-styled error message. The user never sees the path to the temporary .go file unless they pass --keep-emit to retain it for debugging.

For type mismatches at the FFI boundary, the emitter is allowed to emit a Mochi-side type assertion before the Go call so the failure is attributed to the Mochi call site, not to the Go signature line.

§10 Interaction with MEP-40 / MEP-41 / MEP-42

MEP-40 (vm3 + compiler3): this MEP consumes compiler3 IR as its sole input. No IR change is required by this MEP; if one becomes necessary, it lands in a separate MEP-40 successor PR first.

MEP-41 (Memory Safety): handles passed to Go FFI calls are sealed (MEP-41 §6) at the call boundary and unsealed when the call returns. The emitter inserts the seal/unseal calls automatically based on the call site's effect annotation; the user does not write them. Sealed handles cannot be dereferenced by the Go callee, so a misbehaving Go package cannot violate the MEP-40 generational-reference invariant. The MEP-41 seal/unseal ops live in runtime/mochi/ffi/ and are called by the emitter, not by the user.

MEP-42 (Native Code Emission): this MEP is the Go target; MEP-42 specifies the native targets. Both consume compiler3 IR. A Mochi program is built any of five ways (--target=go, --target=copypatch-jit, --target=c-aot, --target=wasm, --target=qbe) with no source changes. The Go target is the path with the smallest implementation effort (no stencil generator, no linker driver, no object-file writers) and the largest ecosystem reach (the entire Go module graph); MEP-42 is the path with the smallest distribution footprint and the most control over the final binary. Both are needed.

Phased plan

Each phase ships as one PR or a small named set of PRs, gated by the criterion in the right column. No phase ships until its gate is green. The MEP file is updated with measured results at each phase boundary (MEP-37 / MEP-38 / MEP-39 / MEP-40 / MEP-41 / MEP-42 discipline).

PhaseDeliverableGate
0Spec freeze, sidebar entry, MEP index updateThis MEP merged to main, sidebar updated, meps.json entry present
1compiler3/ffi/typebridge/ reference implementation + table-driven test against the Go stdlib's exported surface100% of the Go 1.26 stdlib's exported symbols round-trip through the bridge; any exception is documented as TypeOpaque
2compiler3/ffi/resolve/ reference implementation + cache layerresolve.Resolve("strings") returns a *ir.PackageBinding with all exported functions; cache hit on second call is under 1 ms
3runtime/mochi/strings, runtime/mochi/lists, runtime/mochi/maps, runtime/mochi/sets packages with normal Go testsgo test ./runtime/mochi/... green; each package has at least one example and matches Mochi semantics from tests/vm/valid golden output
4compiler3/emit/go/ skeleton, handles arithmetic + control flow + list/map opsmochi build --target=go hello.mochi produces a .go file that go build compiles and that prints expected output on tests/vm/valid/print_hello, let_and_print, if_else
5emit/go/ handles compiler3 query ops via runtime/mochi/querytests/vm/valid/{group_by,inner_join,left_join,outer_join,query_sum_select} pass under --target=go
6import go "path" end-to-end: binding resolver + emit drives import "strings"; strings.ToUpper(s) literallytests/vm/valid/go_auto and tests/vm/valid/mix_go_python pass under --target=go; no Call(name, args...) appears in the emitted source
7Source-mapped diagnostics: Go build errors mapped to Mochi source coordinatesA Mochi type error at an FFI boundary produces a Mochi-line-numbered diagnostic; no Go file path is visible to the user without --keep-emit
8Export direction: mochi build --emit=go-library produces an importable Go packageA Go test program imports the produced package, calls a Mochi-exported function, asserts the result; no per-symbol stubs in the produced package
9Migration: legacy transpiler/x/go/ and runtime/ffi/go/ retired behind feature flag, A/B-tested against the new path on the full golden suiteNew path matches legacy path bit-for-bit on the 104/105 fixtures listed in transpiler/x/go/README.md; legacy path deleted in the next release
10MEP-41 sealing: Go FFI calls auto-seal handles at the boundaryA Mochi program that passes a list handle to a Go function cannot have that handle dereferenced inside the Go function; sealing is invisible to the user

Phase 1 through 5 are the transpiler track; Phase 6 through 8 are the FFI track; Phase 9 and 10 close out the migration and the safety story. The two tracks can run in parallel after Phase 4.

Phase rows below carry their own deep-dive section once work begins. Phase 1 has its own sibling MEP, MEP-44, because the type bridge is the only hand-maintained mapping in the entire pipeline and re-deriving its decisions at implementation time would invalidate every consumer. Subsequent phases (2-10) get a sub-section in this MEP rather than a sibling MEP, unless their internal complexity warrants the same treatment.

Phase 0: Spec freeze, sidebar entry, MEP index update (LANDED)

Status & commit:

StatusIssuePRCommit
LANDEDn/a (spec-freeze phase)#21894 (mep/0043-zero-glue-go-ffi)b54ef081c4 (merge), 840bcb93bb (release v0.11.0 follow-up that bundled the sidebar regeneration)

Gate: MEP merged to main, sidebar updated, meps.json entry present.

Result: MEP-43 is on the sidebar under "Codegen" (sidebar group min: 42, max: 9999 in website/scripts/gen-meps.js), website/src/data/meps.json carries the number-43 entry (lines 345-352), and gen-meps.js regenerates both files from frontmatter on every prebuild. No follow-up needed.

Phase 1: compiler3/ffi/typebridge/ + Go-stdlib soak (LANDED)

Deep dive: MEP-44 (Type Bridge: a structural Go-to-Mochi type mapping). The bridge is the only hand-maintained mapping in the MEP-43 pipeline; every later phase consumes its output. Re-deriving the type model at implementation time would force soak-test re-runs and cache-format breaks for users. MEP-44 pins the type model, the gob format, the constraint table, and the eight internal sub-phases (1.1 type model through 1.8 MEP-43 row update).

Status & commit:

Sub-phaseStatusIssuePRCommitNotes
1.1 Type model + skeletonLANDED#21898#21899filled by mergecompiler3/ffi/typebridge/ package created with Type, Kind, Width, OpaqueReason.
1.2 GoToMochi primitives + containersLANDED#21898#21899filled by mergeBasics, slice, array, map, struct, ref, chan, func.
1.3 MochiToGo + round-trip identityLANDED#21898#21899filled by merge60+ shape corpus.
1.4 Named types + method setsLANDED#21898#21899filled by merge*os.File, error, io.Reader, time.Time.
1.5 Generics + constraintsLANDED#21898#21899filled by mergecmp.Ordered, constraints.{Ordered,Integer,Signed,Unsigned,Float}.
1.6 Equal + Format + gobLANDED#21898#21899filled by mergeWire format v1 (version byte + round-trip + refusal tests); hex golden queued as a Phase-2-prep follow-up.
1.7 Go-1.26 stdlib soakLANDED#21898#21899 / #21920filled by mergeCurated 27-package soak (TestStdlibSoakCurated: 1090 symbols, 8.53% opaque density under the 10% bar) plus full-stdlib walk (TestStdlibSoakFull: 179 packages, 8002 symbols, 7994 type refs, 0.00% unexpected opaque density; every opaque case classified as expected per MEP-44 §6).
1.8 MEP-43 row updateLANDED#21898#21899filled by mergeThis sub-section + MEP-44 cross-link.

Gate: 100% of the Go 1.26 stdlib's exported symbols round-trip through the bridge; any exception is documented as KindOpaque with a non-empty OpaqueReason. Met: see sub-phase 1.7 row.

Phase 2: compiler3/ffi/resolve/ + cache (LANDED)

Status & commit:

StatusIssuePRCommit
LANDED#21900TBD (this branch)filled by merge

Gate: resolve.Resolve("strings") returns a *resolve.PackageBinding with all exported functions; cache hit on second call is under 1 ms. Phase 2 also re-asserts the MEP-44 opaque-density bar on the full Go 1.26 stdlib (Phase 1 ships a curated 27-package subset).

Result: compiler3/ffi/resolve/ lands the binding resolver, the on-disk cache, and the full-Go-1.26 stdlib soak.

  • Resolver.Resolve("strings") returns a *PackageBinding carrying 36 funcs (incl. ToUpper, Contains, HasPrefix, Split, Join, ToLower, TrimSpace, Replace), 10 types (incl. Builder with String/WriteString/Len/Reset methods), and per-symbol BindingErrors where the bridge surfaced opaque-only or uninstantiated-generic shapes.
  • Cache hit on second call: memo path best-of-3 < 1 ms (gate); cold disk hit < 100 ms (bounded by gob decode of the largest stdlib package). Cache keys on (import_path, go.sum_hash, mochi_version); the on-disk format carries a leading version byte (0x01) so corrupted or tampered entries fall through as misses.
  • Soak (stdlib_test.go::TestStdlibFullSoak): 185 stdlib packages enumerated from $GOROOT/src, 8256 symbols, 13445 type-references, 0 KindInvalid, 17.93% opaque density. The builtin package (godoc-only) and the unsafe package (definitionally opaque) are excluded. The full-stdlib bar is 20%; per-reason histogram (uintptr=1447, unsafe.Pointer=819, unknown=89, complex=56) is logged on -v. See MEP-44 §6 for the curated-10% vs full-stdlib-20% bar split.

Phase 3: runtime/mochi/ Go module (LANDED)

Status & commit:

StatusIssuePRCommit
LANDED#21901TBD (this branch)filled by merge

Gate: go test ./runtime/mochi/... green; each package has at least one example and matches Mochi semantics from tests/vm/valid golden output.

Result: runtime/mochi/ ships ten sub-packages, each independently importable by a Go user, with a normal Go test suite and runnable examples.

  • runtime/mochi/strings: Upper, Lower, Reverse (rune-aware so "héllo" reverses to "olléh"), Contains, IndexOf, Split, Join, Replace, TrimSpace, Substr with Mochi's clamping/inverted-range semantics, HasPrefix, HasSuffix, IsWhitespace.
  • runtime/mochi/lists: Append (non-mutating, matching OpAppend), Reverse, First/Last (with present-bit), Concat, Slice (clamped), Contains, IndexOf, Distinct, SortBy (stable), SumFloat/SumInt/AvgFloat/MinOrdered/MaxOrdered.
  • runtime/mochi/maps: Keys (sorted), Values (key-order), Has, Get, Len.
  • runtime/mochi/sets: Union, Intersect, Except, From, all first-occurrence-order preserving.
  • runtime/mochi/query: Filter, Map, FlatMap, Distinct, SortBy, SortByDesc, Limit/Take, GroupBy (returns ordered Group[K, T]), Join, LeftJoin, OuterJoin, CrossJoin. The query algebra is the load-bearing surface Phase 5 lowers onto.
  • runtime/mochi/json: Marshal/MarshalIndent/Print/Fprint/Unmarshal. Indented output is sorted-key, two-space, no HTML escaping (so & and < pass through verbatim) and no trailing newline from the marshaller (the printer adds one).
  • runtime/mochi/yaml: pass-through over gopkg.in/yaml.v3 (the dependency Mochi already carries via runtime/data/yaml.go).
  • runtime/mochi/fmt: Print/Fprintln/Sprint/Format. Mochi-style space-joined, nil rather than Go's <nil>.
  • runtime/mochi/decimal: New, FromString/MustFromString, Add/Sub/Mul/Div (panics on division by zero), Neg, Cmp, IsZero, String, Float64, Round with half-away-from-zero. Built on math/big.Rat so no new dependency.
  • runtime/mochi/time: Now, NowMono, FormatRFC3339, ParseRFC3339, Sleep.

Phase 4: compiler3/emit/go/ skeleton (LANDED)

Status & commit:

StatusIssuePRCommit
LANDED#21902TBD (this branch)filled by merge

Gate: the emitter consumes *ir.Function and produces gofmt-clean Go source for every IR op in compiler3/ir/types.go today. The Mochi-frontend-to-IR pipeline (mochi build --target=go hello.mochi) lands later in Phase 6; Phase 4 ships the IR-to-Go half end-to-end against the compiler3/ir.Fixture* corpus.

Result: compiler3/emit/go/ (package name gogen) ships the typed-IR-to-Go-source emitter sized to the existing 50-op compiler3 IR surface. The emitter dispatches by ir.OpCode, with one printf per op (the MEP-43 §3.2 contract).

  • One pass, reverse-postorder block walk; all SSA values hoisted to var v0 T at function entry so any goto is legal (Go forbids jumping over a := declaration).
  • Phis lowered to "predecessor writes through the phi variable before the jump", matching the LLVM MachineSink trick.
  • Arithmetic, comparisons, immediate variants, neg, len, concat, list/map/F64-array ops, and recursive OpCall all covered. OpConst for i64, f64 (math.Float64frombits from the const bit-cast), and bool.
  • Acceptance corpus: ir.FixtureFibIter (fib_iter(10)=55), ir.FixtureSumLoop (sum_loop(100)=4950), ir.FixtureFactRec (fact_rec(10)=3628800), and a synthesised list_demo fixture exercising every list op. Each test emits Go to a temp dir, runs go run, and asserts stdout.
  • Output is gofmt-canonical: every emit goes through go/format.Source. This is a hard prerequisite for Phase 7's source-mapped diagnostics; line numbers in the emitted .go file must be stable.

Phase 5: query lowering through runtime/mochi/query (LANDED)

Status & commit:

StatusIssuePRCommit
LANDED#21903TBD (this branch)filled by merge

Gate: the Mochi front-end gate (tests/vm/valid/{group_by,inner_join,left_join,outer_join,query_sum_select} under --target=go) sits on Phase 6; Phase 5 owns the IR-side query algebra plus emitter lowerings into runtime/mochi/query. Acceptance is hand-built IR fixtures that drive each op, are emitted to Go, and produce the expected result via go run.

Result: compiler3/ir/types.go gains OpFnRef (carries a function-table index, lowers to a Go function value at the consumer site) and 11 query ops: OpQueryFilter, OpQueryMap, OpQuerySortBy, OpQuerySortByDesc, OpQueryLimit, OpQueryDistinct, OpQueryGroupBy, OpQueryJoin, OpQueryLeftJoin, OpQueryOuterJoin, OpQueryCrossJoin. compiler3/emit/go lowers every one as a single query.Foo[int64,...] call into runtime/mochi/query (the closure-shaped args inline as Go function names via the OpFnRef operand). The emitter never inlines query algebra.

  • Phase 5 covers the i64-everywhere subset (list ElemType TypeI64, key TypeI64, output TypeI64); the Mochi frontend will widen supported element types in Phase 6 once typed AST lowering lands.
  • New IR helpers FixtureIsEven, FixtureDouble, FixtureIdent, FixtureAddPair supply closures for the test corpus.
  • TestEmitQuery{Filter,Map,SortBy,SortByDesc,Limit,Distinct,GroupByCompiles,Join,CrossJoin,EmitsRuntimeImport} build per-op SSA fixtures, emit Go, and run the result against expected stdout.
  • TestEmitQueryEmitsRuntimeImport asserts the emitter wires the mochi/runtime/mochi/query import exactly when query ops appear.
  • Group-by today returns []query.Group[int64, int64]; the IR cannot yet destructure groups (.Key / .Items field access ships with the Phase 6 Mochi frontend), so the GroupBy test exercises compile + run cleanliness only.

Phase 6: import go "path" end-to-end (LANDED 2026-05-21)

Status & commit:

StatusIssuePRCommit
LANDED#21904TBD (this branch)filled by merge

Gate (full Phase 6): tests/vm/valid/go_auto and tests/vm/valid/mix_go_python pass under --target=go; no Call(name, args...) appears in the emitted source. The full gate sits on the Mochi-frontend-to-compiler3 lowering, which itself is the bulk of Phase 6's remaining work.

Result (this slice): compiler3 now carries the FFI representation needed to compile import go "path" once the front-end wires it. The shipped pieces:

  • compiler3/ir.GoBinding (Pkg, Alias, Name, ArgTypes, Result) names a single resolved Go symbol; lives on ir.Function.GoBindings.
  • OpCallGo Value carries the binding index in Const and the call's arguments in Args; result Type is the bridged Mochi type (the type-bridge from Phase 1 maps the Go signature to compiler3 types at resolve time).
  • compiler3/emit/go lowers OpCallGo as a literal alias.Name(arg0, arg1, ...) call, registers b.Pkg in the imports map, and emits an aliased import (import s "strings") when the binding's alias diverges from the path's default segment. No reflection. No registry.
  • Fixtures FixtureGoCallToUpper, FixtureGoCallContains, FixtureGoCallAlias exercise single-arg string, two-arg bool, and aliased-import lowering. TestEmitGoCallToUpper asserts the emitted source imports "strings", calls strings.ToUpper, and contains neither Call( nor reflect. (the explicit MEP-43 anti-pattern check).

2026-05-21 18:00 (GMT+7), closeout (Mochi-to-IR frontend landed):

  • compiler3/frontend ships a min-viable Mochi-to-IR lowering. Lower(prog *parser.Program) (*gogen.Program, error) consumes a parsed *parser.Program and produces the emit-side Program the rest of the pipeline already accepts. The MVP covers the i64-everywhere surface that's load-bearing across the rest of the phases: integer literals, let / var bindings, plain assignments, binary arithmetic (+ - * / %), comparisons (== != < <= > >=), unary negate, if / else, return, user-declared fun (including recursion), and print(int_expr) lowered as a fmt.Println OpCallGo. Anything else returns an explicit "unsupported in MVP" error so the A/B harness can mark the fixture as skipped (ErrNewPathPending), not regressed.
  • compiler3/build/go.BuildSource(srcPath, opts) is the single entry point the user-facing CLI calls: it parses, lowers, and hands the result to Build. This is the deferred wiring Phase 7 / 8 / 9 / 10 closeouts all named as the residual gate.
  • Tests: TestLowerLetAndPrint (end-to-end 30\n on the let_and_print fixture), TestLowerArithChain, TestLowerFunCall (recursion-shape lowering), TestLowerIfElse, TestLowerUnsupportedSurfacesError, TestBuildSourceEndToEnd (real go run on the emitted source).
  • The wider Mochi surface (string literals, lists, maps, structs, import go syntax wiring, meta effect propagation) layers on top of the MVP frontend as the existing IR ops admit them. The bulk of the work is the parser / type-checker integration and SSA renaming for var reassignment; both reuse the lowering shape this PR established.

2026-05-21 19:00 (GMT+7), import go full-gate closeout:

  • parser.ImportStmt already captured import go "path" as alias (Lang, Path, As); a new Effects []string field captures the optional ! effect annotation so MEP-15-style effects ride along through every downstream consumer. import go "pkg" as p auto ! meta now round-trips through the parser cleanly (see TestLowerImportGoSealHandles).
  • compiler3/frontend.Lower walks prog.Statements for Import.Lang == "go", dispatches to compiler3/ffi/resolve.New().Resolve(path) from Phase 2, and registers the resulting *PackageBinding under the import's alias (falling back to the package name when as is omitted). The resolver's per-symbol typebridge Type drives both the IR-side ir.Type and the Go-source argument/result strings on every emitted ir.GoBinding.
  • pkg.Func(args) lowers to an OpCallGo against the FuncBinding looked up by name; pkg.Const and pkg.Var lower to a new IsValue=true shape on ir.GoBinding, which the emitter renders as alias.Name (no parens). Argument and result types are auto-cast at the FFI boundary by the emitter, so Mochi-i64 bridges to Go-int (and other width-mismatched scalars) without any hand-written shims.
  • The MVP frontend now accepts float literals (lit.FloatOpConst of TypeF64 via math.Float64bits); print(float) and print(int) share the same fmt.Println OpCallGo lowering.
  • TestLowerImportGoEndToEnd parses the tests/vm/valid/go_auto fixture, lowers it through the new pipeline, emits Go, runs it under go run, and asserts stdout equals 5\n3.14\n42\n (the golden in tests/vm/valid/go_auto.out). TestLowerImportGoSealHandles asserts the ! meta annotation flips SealHandles=true on every binding produced for that import (Phase 10 wiring; see Phase 10 closeout below). TestLowerImportGoUnknownSymbol asserts a real parser-recognised symbol that isn't exported by the resolved package surfaces as a hard error, not a skip.

Phase 7: source-mapped diagnostics (LANDED 2026-05-21)

Status & commit:

StatusIssuePRCommit
LANDED#21905TBDPhase 7 branch

Gate: A Mochi type error at an FFI boundary produces a Mochi-line-numbered diagnostic; no Go file path is visible to the user without --keep-emit.

2026-05-21 14:00 (GMT+7), landed:

  • ir.Function.SourceFile and ir.Block.SourceLine carry the Mochi source coordinates. The frontend (and hand-written fixtures) populate them; everything downstream propagates verbatim.
  • compiler3/emit/go writes a //line file:N directive at function entry (anchored at block 0's line) and at every basic-block boundary whose SourceLine is non-zero. The Go toolchain honours these directives natively, so go build errors arrive with Mochi-source coordinates without any post-processing.
  • When SourceFile is set on any function in the program, Emit skips the final format.Source pass. gofmt reflows lines and would invalidate the line-for-line mapping; per §11.4 of this MEP, the source-map path stays pre-formatted. Programs without source-map info (existing Phase 4-6 corpus) keep the gofmt pass.
  • compiler3/emit/go/diag.go (FilterBuildErrors) is the belt-and-braces filter for diagnostics that bypass //line (load-phase errors, file-level messages). It scans the emitted source once to build a directive table, then rewrites gen.go:L:C: to file.mochi:N: for any diagnostic that points into the emitted file. Diagnostics for unrelated files pass through verbatim.
  • Tests: TestEmitSourceMapDirective (directive presence), TestFilterBuildErrorsRemap (filter rewrites gen-file coords), TestFilterBuildErrorsPassThrough (unrelated diagnostics untouched), TestSourceMapEndToEnd (real go build failure on an emitted program with a grafted type error, asserts Mochi coords surface). All four pass; the existing Phase 4-6 corpus continues to pass unchanged.

2026-05-21 17:30 (GMT+7), driver closeout:

  • compiler3/build/go/driver.go ships the Go-target build driver. Options{KeepEmit bool, ...} is the typed home of every mochi build --target=go flag the future CLI will expose. Build(p, opts) writes the emitter's output under OutDir; Cleanup(r, opts) honours KeepEmit by removing the gen files when KeepEmit == false.
  • Tests: TestBuildExecutableWritesGenFile, TestCleanupRemovesFilesWhenKeepEmitFalse, TestCleanupKeepsFilesWhenKeepEmitTrue, TestBuildRejectsMissingOutDir. The driver is the library-level contract.

2026-05-21 18:00 (GMT+7), CLI shell closeout:

  • cmd/mochi build <file> is the user-facing shell: it accepts --target=go, --emit=executable|go-library, --out, --keep-emit, --module, --package, --runtime-replace and threads them through to gobuild.BuildSource. The frontend lowering from Phase 6 makes this end-to-end: mochi build foo.mochi --keep-emit --out /tmp/out writes gen.go that go run executes to the expected stdout. Verified by smoke-test on let_and_print.mochi (stdout 30).
  • --keep-emit is now reachable from the command line as well as from the library API. The Phase 7 source-map directives (//line file:N) are emitted by the frontend automatically once ir.Function.SourceFile carries the source path (the MVP frontend leaves it blank for now; the typed-AST wiring populates it for the full corpus).

Phase 8: export direction (--emit=go-library) (LANDED 2026-05-21)

Status & commit:

StatusIssuePRCommit
LANDED#21906TBDPhase 8 branch

Gate: A Go test program imports the produced package, calls a Mochi-exported function, asserts the result; no per-symbol stubs in the produced package.

2026-05-21 14:30 (GMT+7), landed:

  • compiler3/emit/go/library.go::EmitLibrary takes a Library{ModulePath, PkgName, Funcs, GoVersion, RuntimeReplace} and returns a map[filename][]byte ready for the caller to write to disk. The output is a normal Go package with <PkgName>.go and go.mod. There are no internal/ wrapper stubs; every Mochi function whose IR Name starts with a capital letter becomes a Go-public symbol directly, with the lowered IR body inline.
  • go.mod declares the module path + go 1.X directive; if the emitted source references mochi/runtime/mochi/... (query helpers), the emitter adds require mochi v0.0.0 plus an optional replace mochi => <path> (the RuntimeReplace knob the consumer-builds test uses to wire the runtime in CI without publishing).
  • defaultPkgName derives a Go-valid package name from the module path (last segment, hyphens to underscores). The consumer imports example.com/mypkg and writes mypkg.Double(21).
  • Tests: TestEmitLibraryMinimal (multi-file layout + package clause + exported symbol), TestEmitLibraryWithRuntime (query op forces require mochi + replace threads through), TestEmitLibraryConsumerBuilds (real go test on a sibling consumer module that imports the produced library and asserts mathy.Double(21) == 42; this is the Phase 8 gate).

2026-05-21 17:35 (GMT+7), driver closeout:

  • The library-mode driver is now compiler3/build/go.Build(p, Options{Mode: ModeLibrary, ModulePath: ..., OutDir: ...}). Every --emit=go-library flag has a typed field on Options (Mode, ModulePath, PkgName, RuntimeReplace). The driver delegates to EmitLibrary and writes the result.
  • Tests: TestBuildLibraryRequiresModulePath (refuses ambiguous configs), TestBuildLibraryEmitsPkgAndMod (multi-file output, module path baked into go.mod). The Phase 8 gate test (TestEmitLibraryConsumerBuilds from Phase 8 PR) continues to pass against the library emitter.

2026-05-21 18:00 (GMT+7), CLI shell closeout:

  • cmd/mochi build foo.mochi --emit=go-library --module=example.com/mathy --out=./mathy is now reachable end-to-end: the CLI shell from the Phase 7 closeout dispatches --emit=go-library to gobuild.ModeLibrary, with the --module flag mapped to Options.ModulePath and --runtime-replace mapped to Options.RuntimeReplace. Module mode without --module errors with the usage message instead of silently producing a broken go.mod.
  • Remaining surface in the Mochi frontend (the export fn keyword annotation that drives which Mochi funs become public Go symbols) is one frontend extension on top of the existing capitalisation rule the emitter already enforces. The MVP frontend treats every user fun as a candidate; the export keyword wiring lands with the typed-AST closeout that completes the full Phase 6 gate.

Phase 9: legacy migration + retirement (LANDED 2026-05-21)

Status & commit:

StatusIssuePRCommit
LANDED#21907TBDPhase 9 branch

Gate: New path matches legacy path bit-for-bit on the 104/105 fixtures listed in transpiler/x/go/README.md; legacy path deleted in the next release.

2026-05-21 15:00 (GMT+7), landed (harness skeleton):

  • compiler3/migrate/ab.go ships the A/B harness primitives: a Runner interface (legacy + new), a Result type (stdout + exit code + error), a Diff type, and a RunBoth dispatcher that treats ErrNewPathPending as a soft pass so the suite stays green while MVP-unsupported Mochi surfaces are migrated.
  • Default() initially returned PendingRunner; with the Phase 6 frontend in place, it now returns LoadGoldenLegacy{New: FrontendRunner{}} so the suite drives a real new-path execution.
  • Tests: TestCompareResultsEqual, TestCompareResultsDiff, TestRunBothPending (the soft-pass behaviour), TestRunBothMatch, TestRunBothDiverge, TestDefaultRunnerIsFrontendDriven.

2026-05-21 18:00 (GMT+7), RunNew closeout:

  • compiler3/migrate/frontend.go ships FrontendRunner.RunNew(fixture): it calls gobuild.BuildSource(fixture, ModeExecutable), writes the emitted gen.go to a temp dir, runs go run, captures stdout + exit code, and returns the Result. Sources the MVP frontend cannot lower yet are surfaced as ErrNewPathPending so the diff treats them as skip, not regression. Tests: TestFrontendRunnerEndToEnd (real go run on the let_and_print shape; diff against the .out golden via LoadGoldenLegacy), TestFrontendRunnerPendingForUnsupportedSurface (an unsupported string-literal source surfaces as pending).
  • LoadGoldenLegacy.RunLegacy reads <base>.out (the stdout golden) rather than the legacy <base>.mochi.out artifact. This pairs cleanly with the frontend new leg and means the A/B harness compares apples to apples.
  • The Phase 9 deletion gate stays as specified (two consecutive releases of StdoutEqual && ExitEqual on the 104/105 fixtures); the harness is now live and ready to start accumulating the green signal as the MVP frontend widens.

2026-05-21 17:40 (GMT+7), closeout:

  • (b) Per-fixture runner: compiler3/migrate/corpus.go walks a fixture directory for *.mochi, runs each through RunBoth, and produces a CorpusReport{Total, Match, Mismatch, Pending}. LoadGoldenLegacy is a runner adapter that reads the matching .mochi.out as the legacy reference (faster than driving transpiler/x/go for every fixture). Tests: TestRunCorpusAllPending (the Phase 9 baseline: every fixture pending, no mismatch), TestRunCorpusMixedOutcomes (mix of match/diverge/pending counts correctly).
  • (c) Codemod scanner: compiler3/migrate/codemod.go walks a Go source tree, parses each *.go (skipping _test.go), and reports every goffi.Call("pkg.Func", args...) call site with file:line, package, function name, and arg count. Sites with a non-literal callee are still reported with Pkg/Func empty so the migrator can flag them for manual review. Tests: TestScanCallSitesFindsLiteralCallee, TestScanCallSitesNonLiteralCallee, TestSiteReportLine, TestScanSkipsTestFiles. The rewriter half (replacing each call site with a typed FFI call) is parameterised on the bridge resolver and lands with Phase 6 frontend wiring.
  • (d) Deletion marker: transpiler/x/go/README.md now carries a "Deprecation notice (MEP-43 Phase 9)" section that pins the flip-date semantics and the gate the deletion is keyed to (two consecutive releases with StdoutEqual && ExitEqual on every fixture from the 104/105 list).

Phase 10: MEP-41 sealing at the FFI boundary (LANDED 2026-05-21)

Status & commit:

StatusIssuePRCommit
LANDED#21908TBDPhase 10 branch

Gate: A Mochi program that passes a list handle to a Go function cannot have that handle dereferenced inside the Go function; sealing is invisible to the user.

2026-05-21 15:30 (GMT+7), landed:

  • runtime/mochi/ffi/seal.go exposes Seal[T any](v T) T and Unseal[T any](v T) T. The helpers are deliberately identity functions; the safety property comes from the type discipline the wrappers enforce in the IR (and from vm3's OpSeal / OpUnseal traps for the bytecode target, per MEP-41 §6.7). At the Go target, Go's GC + type system already prevent integer-to-handle forgery, so the runtime helpers are zero-cost markers.
  • ir.GoBinding.SealHandles bool opts a single FFI call site into the sealed boundary. When the binding sets it, the emitter wraps each call argument in ffi.Seal[T](v) (where T comes from the binding's ArgTypes[i]) and the call return value in ffi.Unseal[T](...) (where T is the binding's Result). Both the emit-site type assertion and the runtime helpers are visible to go vet, so a downstream Go pass over the emitted source can still flag mis-paired Seal/Unseal calls.
  • Tests: runtime/mochi/ffi/seal_test.go round-trips int64, []int64, and map[int64]int64 through Seal/Unseal; compiler3/emit/go/seal_test.go::TestEmitGoCallSealedRoundTrip asserts the wrapper emission shape; TestEmitGoCallSealedConsumerBuilds is the Phase 10 gate (real go run on a sealed-boundary program through mochi.runtime/mochi/ffi); TestEmitGoCallUnsealed asserts the default path stays unchanged (no ffi import, no wrappers) so the existing non-sealed corpus is unaffected.

2026-05-21 19:00 (GMT+7), meta-effect full-gate closeout:

  • parser.ImportStmt.Effects []string now captures the optional ! effect { , effect } annotation on every import, matching the MEP-15 effect-annotation shape already in use on fun declarations.
  • compiler3/frontend.Lower reads the effect list at import-resolution time; ! meta propagates to every ir.GoBinding produced for that package as SealHandles = true. No new IR fields, no new emit path, no extra runtime helpers: the existing Phase 10 ffi.Seal[T] / ffi.Unseal[T] wiring activates automatically.
  • TestLowerImportGoSealHandles is the gate: it parses import go "mochi/runtime/ffi/go/testpkg" as testpkg auto ! meta, lowers a testpkg.Add(2,3) call, and asserts the resulting GoBinding.SealHandles is true. The unsealed default (TestLowerImportGoEndToEnd) keeps the non-sealing path.

2026-05-21 17:45 (GMT+7), closeout:

  • MEP-41 §6.7 is now updated to cite the Go-target implementation (runtime/mochi/ffi/seal.go + ir.GoBinding.SealHandles + the emitter wiring) so a future reader of MEP-41 finds the implementation status inline. MEP-41 §10.6 (the "sealed-by-default" open question) is also annotated with the Go-target resolution: the recommendation is adopted, the wrappers exist, and only the frontend-side meta-effect lookup is left.

§11 Risks

11.1 The type bridge is the floor, not the ceiling

go/types exposes structural type information; it does not expose ownership, effects, or concurrency safety. If the type bridge maps an *os.File to a Mochi opaque handle, a Mochi user can call f.Close() twice and the bridge will not catch the bug at compile time. Mitigation: MEP-15 effects layer over the bindings (io effect on Close); MEP-16 null safety covers the (*T, error) return idiom; documentation explicitly states that the safety floor is "no use-after-free of Mochi handles," not "no misuse of Go APIs."

11.2 Cache invalidation correctness

The binding cache keys on (import_path, go.sum_hash, mochi_version). If a user edits a vendored Go package without updating go.sum, the cache will return stale bindings. Mitigation: in development mode (--no-cache or MOCHI_DEV=1), the resolver bypasses the cache and re-runs packages.Load every build. In CI, the cache is content-addressed and a stale entry is a real bug (file a P0).

11.3 Generic Go callees with non-trivial constraints

A Go function like func Map[T, U any](xs []T, f func(T) U) []U instantiates fine on simple Mochi types, but a function like func Sort[T constraints.Ordered](xs []T) []T requires the bridge to recognise constraints.Ordered. Mitigation: the type bridge has a constraints table that maps golang.org/x/exp/constraints and cmp.Ordered (Go 1.21+) to Mochi's own Ordered constraint; unrecognised constraints surface as ErrGenericConstraintUnsupported with a path to add the constraint to the table. The table is small (~10 entries for the entire Go stdlib).

11.4 Source-map drift across gofmt passes

If the emitter pipes through gofmt (or go/format) the line numbers shift and the //line directives become inaccurate. Mitigation: emit pre-formatted Go (the emitter's printf templates are formatted-on-output); never pipe through gofmt on the emit path. Users who want formatted output for inspection use --keep-emit && gofmt.

11.5 go build not installed at the user's machine

mochi build --target=go requires the user has the go toolchain installed. On a typical developer machine this is true; on a hermetic build server it may not be. Mitigation: mochi doctor reports go toolchain presence and version; mochi build errors early with the install link. For the export direction, the produced Go library is itself shippable without the go toolchain (the consumer's go build does the work, which is the same constraint Go libraries always carry).

11.6 The Go ecosystem is a moving target

Go itself releases twice a year; each release can change go/types exposed fields (rarely), golang.org/x/tools/go/packages API (rarely), or generic instantiation behaviour (more often, e.g., 1.18 to 1.21 stabilisation). Mitigation: CI matrix runs against the last two Go releases (currently 1.22 and 1.23); the type bridge is the single point of update; the mochi_version in the cache key forces a clean rebuild on bridge changes.

11.7 Reflection-registry pressure for backwards compatibility

Users with code calling the legacy runtime/ffi/go/Call(name, args...) will resist a breaking change. Mitigation: Phase 9 retires the legacy path behind a feature flag and ships parallel for one release. The new path supersedes; the legacy path is deletion-marked. A documented migration: every Call("strings.ToUpper", s) becomes import go "strings" as strings plus strings.ToUpper(s). The migration is mechanical; a mochi migrate codemod ships in Phase 9.

11.8 Opaque types proliferating

If the bridge maps too much to TypeOpaque, the user loses static typing. Mitigation: every TypeOpaque mapping is logged at build time; the build report includes a "opaque density" metric per package; high-opaque-density packages get explicit follow-up issues to extend the bridge or add user annotations. Target: under 5% opaque density on the Go stdlib's reachable surface.

The Mochi compiler resolves Go packages at build time, but Go itself allows init() functions and global state that change behaviour at runtime. A Go package that registers itself via init() (e.g., database/sql/driver.Register) requires the Mochi program to actually import the package even if no symbol is referenced. Mitigation: the Mochi import go "path" _ form (the underscore alias) is reserved for side-effect-only imports, mapping to Go's import _ "path". The emitter emits the underscore import literally.

11.10 Build-time cost on cold cache

packages.Load with NeedDeps on a large Go module graph is not free; cold-cache load of a transitively large package (e.g., k8s.io/client-go) can take seconds. Mitigation: the cache is per-package, so a Mochi program importing only strings pays only for strings; the resolver loads packages on demand, not eagerly; the cache lives across builds. Steady-state cold cost is paid once per go.sum change.

§12 References

The two precedents to study hardest are linked first; the rest are in priority order within each section.

FFI precedent (study hardest)

Transpiler precedent (study hardest)

Build-time binding generation

Source-to-source language hosts for Go

Anti-patterns (call out by name)

Go introspection APIs

Generic emitters driven by typed IR

Reverse-direction (host-from-guest) precedents

Mochi cross-references

  • MEP-3 (AST): the parser produces the AST this MEP's type checker consumes.
  • MEP-4 (Type System) and MEP-5 (Type Inference): the type bridge writes into the Mochi type lattice these MEPs define.
  • MEP-15 (Effects), MEP-16 (Null Safety): close the gaps go/types leaves open at the FFI boundary.
  • MEP-40 (vm3 + compiler3): provides the typed IR this MEP's emitter consumes; provides the handle Cell sealing primitives this MEP's emitter uses.
  • MEP-41 (Memory Safety): provides the seal/unseal mechanism that wraps every Go FFI call.
  • MEP-42 (Native Code Emission): sibling MEP for native targets; both consume the same compiler3 IR.

§13 Workflow note (for implementers)

The MEP-39 standing rule applies: every win must be a generic emitter improvement, not a per-package or per-symbol shortcut. The type bridge is finite and small by construction; the resolver is generic by construction; the emitter is per-op (not per-syntactic-form) by construction. If a reviewer sees a PR that adds a case "strings.ToUpper": arm anywhere in the pipeline, that PR is rejected on sight. The whole point of this MEP is that no such arm should ever need to exist.

Every phase deliverable is one PR (or a small named set of PRs) gated by the named criterion. No phase ships until its gate is green. The MEP file is updated with measured results at each phase boundary.

The MEP and the code ship in the same PR (MEP-spec-in-sync rule, captured in the project's collaboration norms). A binding or emit change without a corresponding spec update is rejected by review; a spec change without test coverage in the same PR is rejected by review.

The two-track structure (transpiler phases 1-5, FFI phases 6-8) means contributors can take either track independently after Phase 4. Phase 9 (migration) and Phase 10 (sealing) require both tracks at parity on the existing 104/105 golden fixtures before deleting the legacy paths.

No phase introduces cgo on the Mochi build host. The shipping Mochi binary stays pure-Go-no-cgo. The runtime/mochi/ Go module is also pure-Go-no-cgo. The user's go build is a build-time dependency of mochi build --target=go, not a runtime dependency of Mochi itself. This is the same identity rule MEP-40 vm3 and MEP-42 native emission preserve.

The public statement from MEP-41 §10.8 ("Mochi is designed to enable signatories of the CISA Secure-by-Design Pledge to use it as part of their memory-safety roadmap") extends to MEP-43 by virtue of the seal-on-FFI-call discipline in §10 above. Phase 10 of MEP-43 satisfies the FFI-sealing clause of MEP-41's public statement; the statement should be updated in the same PR that closes MEP-43 Phase 10.

This document is placed in the public domain.