Phase 10A/10B Inc 0: build system, handle model, first module surface
Builds n3xd-ocp end to end and publishes 7.9.3.1.dev1 to the Gitea registry, where it installs anonymously and passes its suite. - occt/Dockerfile: OCCT 7.9.3 compiled once into a manylinux_2_28 builder image (base digest + tarball sha256 pinned), Draw/VTK/Tk/Xlib/OpenGL off, FreeType on, -O2 without fast-math or march=native. A final layer asserts TKService/TKV3d exist with no libGL/libX11 DT_NEEDED, which is what lets the app image drop libgl1/libx11-6. Mounted into, never built FROM. - scikit-build-core + nanobind STABLE_ABI -> one cp312-abi3 extension that registers every OCP.* submodule via PyImport_AddModule, so `import OCP.TopoDS` needs no shim and cls.__module__ is right. Version <occt>.N is asserted against the OCCT found, keeping occt_version() truthful. - occt_handle.h: type caster for opencascade::handle<T> over OCCT's intrusive refcount. Wrappers are non-owning instances holding exactly one handle in their keep-alive list, reusing an existing wrapper so identity survives a round trip. Transient constructors go through ocp_new (never nb::init<>, which would let OCCT delete nanobind's storage); the caster refuses a refcount-0 object rather than corrupt the heap. Verified under ASAN with no memory-safety errors, plus an RSS bound over 50k create/destroy cycles. - Sub-shapes are returned by value everywhere, making the TShape lifetime class that segfaulted a process-global face memo unrepresentable. - Standard_Failure derives RuntimeError, with ~20 concrete types dispatched on the dynamic OCCT type (cad_pool marshals failures home by type name). - Inc 0 surface: gp subset, TopAbs, TopoDS (+ downcasts), TopExp, TopLoc, TopTools, BRep, BinTools, Poly, Standard. 34 of the app's 139 symbols. - n3xd_ocp: additive APIs kept out of the OCP namespace so parity testing stays meaningful. bintools (shape <-> bytes, GIL-free, byte-identical) and _debug. Two findings worth the record, both verified against the stock wheel rather than assumed: upstream binds __hash__ but leaves __eq__ at identity, which is exactly what geom_memo.py's hash-bucket + IsSame scan is built around, so we match it instead of "fixing" it; and BinTools can release the GIL after all, by slurping the file-like object instead of bridging a streambuf that would call back into Python. Gate: BREP round-trips are byte-identical to cadquery-ocp-novtk across six fixtures (the generator asserts stock idempotency first). That matters beyond IPC — derive.py content-addresses BREP payloads by sha256 and stores the ref.
This commit is contained in:
195
docs/design.md
Normal file
195
docs/design.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Design record
|
||||
|
||||
Decisions that are expensive to revisit, and the evidence behind them. If you
|
||||
are adding symbols rather than changing the machinery, read
|
||||
[adding-symbols.md](adding-symbols.md) instead.
|
||||
|
||||
## What this is
|
||||
|
||||
A hand-written [nanobind](https://github.com/wjakob/nanobind) binding of the
|
||||
OCCT surface the N3XD backend actually uses — 139 symbols across 48 `OCP.*`
|
||||
modules, per `tools/inventory.py`, not all of OCCT. It installs as a top-level
|
||||
`OCP`, so it is a drop-in replacement for `cadquery-ocp-novtk` and the app's 442
|
||||
import sites stay untouched.
|
||||
|
||||
Binding *call* overhead was never the bottleneck — the CAD hotspots are inside
|
||||
the C++ kernel — so the payoff is version velocity, footprint, correctness at
|
||||
the ownership boundary, and the freedom to add APIs upstream cannot (GIL
|
||||
release, bulk array extraction).
|
||||
|
||||
## The handle model
|
||||
|
||||
`opencascade::handle<T>` is cast by `src/common/occt_handle.h`, modelled on
|
||||
nanobind's own `stl/shared_ptr.h`.
|
||||
|
||||
- **C++ → Python**: the wrapper is a *non-owning* nanobind instance pointing at
|
||||
the C++ object, plus one handle stored in its keep-alive list. An existing
|
||||
wrapper is reused (`is_new == false`), so identity holds while a wrapper is
|
||||
alive, and a long-lived object crossing the boundary repeatedly does not pile
|
||||
up redundant references. Transients are polymorphic, so `nb_type_put_p`
|
||||
downcasts on the dynamic type.
|
||||
- **Python → C++**: a plain handle copy, balanced when the caster dies after the
|
||||
call. Unlike `shared_ptr`, no Python reference is taken: OCCT's intrusive
|
||||
atomic refcount owns the object's memory, not the Python instance. That is
|
||||
precisely what makes releasing the GIL safe — OCCT may copy handles on its own
|
||||
worker threads without touching the interpreter.
|
||||
|
||||
**Transient constructors never use `nb::init<>`.** nanobind's normal
|
||||
constructor placement-news the object into the Python instance's storage, which
|
||||
OCCT would later `delete`. Use `ocp_new<T, Args...>()` (`occt_transient.h`),
|
||||
which heap-allocates and returns a handle. This is not hypothetical: the app
|
||||
builds a `Geom_BSplineCurve` in `sketch_builder/edges.py` and hands it to
|
||||
`BRepBuilderAPI_MakeEdge`, which keeps a handle past the call.
|
||||
|
||||
The caster enforces the rule rather than trusting it: a transient whose
|
||||
`GetRefCount()` is zero is not handle-owned, and converting it would hand OCCT
|
||||
the right to free a nanobind instance. `from_python` refuses instead — a
|
||||
TypeError beats heap corruption, and only a binding bug can reach it.
|
||||
|
||||
*Alternative considered*: `nb::intrusive_ptr` plus a side table mapping
|
||||
`Standard_Transient*` to `PyObject*` (OCCT objects have no self-py slot). It
|
||||
loses on the property that matters most here — nanobind's intrusive protocol
|
||||
unifies the C++ count with the PyObject refcount, so `Py_INCREF` from a
|
||||
GIL-free OCCT thread becomes a crash class. Kept as the documented fallback if
|
||||
the caster above ever proves unworkable.
|
||||
|
||||
*Verification*: `tests/test_handles.py`, run both normally and under
|
||||
`make test-asan`. It covers wrapper identity across a round trip, survival in
|
||||
both directions when one side drops its reference, refcount balance across 1000
|
||||
conversions and across 100 raising calls, null-handle ↔ `None`, and RSS growth
|
||||
across 50 000 create/destroy cycles. ASAN covers memory *safety*; the RSS
|
||||
assertion covers *growth* and is skipped under ASAN, whose quarantine retains
|
||||
freed memory (139 MB of it, which is what a naive reading would call a leak).
|
||||
|
||||
## Fidelity rules
|
||||
|
||||
Verified against the installed stock wheel, not assumed.
|
||||
|
||||
**`__hash__` is bound; `__eq__` is not.** Upstream binds `__hash__` (TShape ⊕
|
||||
Location) and leaves `__eq__` at Python's default identity comparison, so two
|
||||
re-extracted copies of one face hash equal but compare unequal. That pairing
|
||||
looks like a bug and is load-bearing: `cad/topology/geom_memo.py` buckets on
|
||||
`hash(face)` and disambiguates with `IsSame` *because* `==` cannot be trusted.
|
||||
Binding `__eq__` to `IsEqual` would quietly collapse entries that memo keeps
|
||||
apart. Hash *values* need not match upstream — only the semantics do.
|
||||
|
||||
**Sub-shapes are returned by value** (`OCP_RETURN_COPY`) from explorers,
|
||||
iterators, map lookups and history lists. A `TopoDS_Shape` is a small value
|
||||
holding a handle to its TShape, so a copy is one incref and owns what it points
|
||||
at. This makes the lifetime class that segfaulted a process-global face memo
|
||||
unrepresentable.
|
||||
|
||||
**Executing constructors are not bound.** `BRepAlgoAPI_*` gets a default
|
||||
constructor plus `SetArguments`/`SetTools`/`Build` — the two-argument forms run
|
||||
the algorithm immediately, which is how a latent double-execution survived in
|
||||
the app for a while.
|
||||
|
||||
**`_s` on every static**, via `OCP_DEF_S`. The rule is blanket rather than
|
||||
clash-driven, so no static can ship without it; the app calls 176 of them.
|
||||
|
||||
**Enums** use `nb::is_arithmetic()` + `export_values()`, reproducing pybind11's
|
||||
int comparison and module-scope members (`from OCP.TopAbs import TopAbs_FACE`).
|
||||
|
||||
**Exceptions**: `Standard_Failure` derives `RuntimeError`, which is what keeps
|
||||
the backend's `except RuntimeError` sites working — it never names an OCCT
|
||||
class. About 20 concrete types are bound under `OCP.Standard` / `OCP.StdFail`
|
||||
and dispatched on the dynamic OCCT type, because cad_pool's children marshal
|
||||
failures home as `f"{type(exc).__name__}: {exc}"`, making the name observable.
|
||||
|
||||
**`TopoDS` is a namespace in OCCT 7.9**, not a class. Upstream still presents it
|
||||
as a class carrying the `_s` statics, and the app calls `TopoDS.Face_s(...)`, so
|
||||
`mod_TopoDS.cpp` binds an empty carrier struct under that name.
|
||||
|
||||
## GIL policy
|
||||
|
||||
Released around calls that stay inside the kernel and cannot re-enter Python:
|
||||
`Build`/`Perform`, meshing, `BRepCheck_Analyzer`, file readers and writers, and
|
||||
every `n3xd_ocp` bulk API. Applied from an explicit list, never blanket.
|
||||
|
||||
`BinTools` is *also* GIL-free for the kernel half, which the original plan
|
||||
assumed impossible. Rather than bridging a `streambuf` that calls back into
|
||||
Python per chunk, `occt_stream.h` slurps the file-like object first and hands
|
||||
the kernel a pure C++ stream. That is correct regardless of how BinTools seeks,
|
||||
and costs one extra copy of the payload — which `n3xd_ocp.bintools` avoids
|
||||
entirely for the pool paths that care.
|
||||
|
||||
**None as an argument** is rejected before the caster for simple overloads, so
|
||||
handle parameters that legitimately accept a null handle need an explicit
|
||||
`nb::arg("x").none()`. Null *returns* map to `None` unconditionally. The app
|
||||
passes no null handles today; `inventory.py` plus the app suite are the guard.
|
||||
|
||||
## Packaging
|
||||
|
||||
**One extension** (`OCP/_OCP.abi3.so`) registering every `OCP.*` submodule via
|
||||
`PyImport_AddModule`, so `import OCP.TopoDS` works with no shim module per name
|
||||
and `cls.__module__` reads `OCP.TopoDS`. The whole surface traffics in
|
||||
`TopoDS_Shape`, `gp_*` and handles, so sharing types in-process is free here and
|
||||
would otherwise lean on nanobind's cross-extension registry; registration order
|
||||
stays an explicit sequence in `core.cpp`; and cad_pool's forkserver warms one
|
||||
dlopen. One `.cpp` per module keeps incremental compiles cheap — only the link
|
||||
is shared.
|
||||
|
||||
**abi3 (`cp312-abi3`)** from the first build, matching forge/assay, so the
|
||||
Python 3.13 bump (roadmap 10D) needs no rebuild. Escape hatch if a nanobind
|
||||
STABLE_ABI limitation ever bites: drop `STABLE_ABI` and `wheel.py-api`, since
|
||||
every deployed environment is 3.12. Stub generation works fine under abi3.
|
||||
|
||||
**Version `<occt>.N`**, asserted at configure time against the OCCT actually
|
||||
found, so `occt_version()` keeps reporting the truth for assay's goldens.
|
||||
Iteration builds carry `.devN`; the registry never allows republishing.
|
||||
|
||||
**No sdist is ever published** — it could not build without the builder image,
|
||||
and offering one invites the 4-core production host to try. That is enforced by
|
||||
only ever building wheels. Note `sdist.exclude` is *not* the way to do it:
|
||||
scikit-build-core feeds it into the wheel's package-file mapping too, so
|
||||
excluding `*` silently ships a wheel containing the compiled extension and none
|
||||
of the Python package. Relatedly, file selection is git-based, so the generated
|
||||
`.pyi` stubs are gitignored *and* re-included through `sdist.include`.
|
||||
|
||||
**Fork safety**: import starts no threads and creates no fork-hostile state, so
|
||||
cad_pool's forkserver keeps costing ~30 ms per job instead of a ~1.3 s spawn.
|
||||
Pinned by `tests/test_forksafety.py`.
|
||||
|
||||
## The `n3xd_ocp` module
|
||||
|
||||
`OCP.*` stays a symbol-for-symbol drop-in so parity testing means something;
|
||||
anything additive lives in `n3xd_ocp`, shipped in the same wheel and backed by
|
||||
the same OCCT build. Only the leaf submodules are registered from C++ — creating
|
||||
the parent would put a bare module in `sys.modules` and a later `import
|
||||
n3xd_ocp` would skip the package's `__init__.py`.
|
||||
|
||||
Shipped: `bintools` (shape ↔ `bytes`, GIL-free, byte-identical to
|
||||
`OCP.BinTools` and asserted so) and `_debug` (test-only ownership
|
||||
introspection).
|
||||
|
||||
Designed, landing with the increment that binds their types:
|
||||
|
||||
```python
|
||||
# with Inc 1 (BRepGProp/GProp) — attacks the measured 94 % of
|
||||
# face_candidate_anchors (0.99 s of 1.05 s for 690 faces) that is
|
||||
# BRepGProp.SurfaceProperties_s called once per face from Python
|
||||
def face_surface_props(shape, *, parallel=True) -> tuple[ndarray, ndarray]
|
||||
# areas[F], centroids[F,3], ordered by TopExp.MapShapes_s(FACE) index
|
||||
|
||||
# with Inc 2 (BRepMesh/Poly) — replaces the per-node and per-triangle Python
|
||||
# loops in cad/tessellation.py
|
||||
class FaceMesh: nodes, triangles, normals, uv, face_index
|
||||
def extract_meshes(shape, *, want_normals=True, want_uv=False,
|
||||
apply_location=True, flip_reversed=True) -> list[FaceMesh]
|
||||
|
||||
# with Inc 4 — opt-in experiment: OSD::SetSignal turns some native faults into
|
||||
# catchable Standard_Failure subclasses inside cad_pool children. Never called
|
||||
# at import; subprocess isolation stays regardless.
|
||||
def set_signal(arm_fpe: bool = False) -> None
|
||||
```
|
||||
|
||||
The backend adopts these after cutover, one call site at a time.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **S5** — whether STEP needs `CSF_*` resource files shipped in the wheel.
|
||||
Modern OCCT code-initialises most `Interface_Static` defaults; settle it when
|
||||
Inc 3 (I/O) lands, in a container without system OCCT resources.
|
||||
- **Byte-identity beyond 7.9.3** — the gate compares against the stock wheel, so
|
||||
it necessarily retires at the OCCT 8.0 bump (roadmap 10E), where the fixtures
|
||||
are re-blessed deliberately alongside assay's goldens.
|
||||
Reference in New Issue
Block a user