Phase 10A/10B Inc 0: build system, handle model, first module surface
Builds n3xd-ocp end to end and publishes 7.9.3.1.dev1 to the Gitea registry, where it installs anonymously and passes its suite. - occt/Dockerfile: OCCT 7.9.3 compiled once into a manylinux_2_28 builder image (base digest + tarball sha256 pinned), Draw/VTK/Tk/Xlib/OpenGL off, FreeType on, -O2 without fast-math or march=native. A final layer asserts TKService/TKV3d exist with no libGL/libX11 DT_NEEDED, which is what lets the app image drop libgl1/libx11-6. Mounted into, never built FROM. - scikit-build-core + nanobind STABLE_ABI -> one cp312-abi3 extension that registers every OCP.* submodule via PyImport_AddModule, so `import OCP.TopoDS` needs no shim and cls.__module__ is right. Version <occt>.N is asserted against the OCCT found, keeping occt_version() truthful. - occt_handle.h: type caster for opencascade::handle<T> over OCCT's intrusive refcount. Wrappers are non-owning instances holding exactly one handle in their keep-alive list, reusing an existing wrapper so identity survives a round trip. Transient constructors go through ocp_new (never nb::init<>, which would let OCCT delete nanobind's storage); the caster refuses a refcount-0 object rather than corrupt the heap. Verified under ASAN with no memory-safety errors, plus an RSS bound over 50k create/destroy cycles. - Sub-shapes are returned by value everywhere, making the TShape lifetime class that segfaulted a process-global face memo unrepresentable. - Standard_Failure derives RuntimeError, with ~20 concrete types dispatched on the dynamic OCCT type (cad_pool marshals failures home by type name). - Inc 0 surface: gp subset, TopAbs, TopoDS (+ downcasts), TopExp, TopLoc, TopTools, BRep, BinTools, Poly, Standard. 34 of the app's 139 symbols. - n3xd_ocp: additive APIs kept out of the OCP namespace so parity testing stays meaningful. bintools (shape <-> bytes, GIL-free, byte-identical) and _debug. Two findings worth the record, both verified against the stock wheel rather than assumed: upstream binds __hash__ but leaves __eq__ at identity, which is exactly what geom_memo.py's hash-bucket + IsSame scan is built around, so we match it instead of "fixing" it; and BinTools can release the GIL after all, by slurping the file-like object instead of bridging a streambuf that would call back into Python. Gate: BREP round-trips are byte-identical to cadquery-ocp-novtk across six fixtures (the generator asserts stock idempotency first). That matters beyond IPC — derive.py content-addresses BREP payloads by sha256 and stores the ref.
This commit is contained in:
172
tools/gen_fixtures.py
Normal file
172
tools/gen_fixtures.py
Normal file
@@ -0,0 +1,172 @@
|
||||
"""Generate the .brep byte-identity fixtures.
|
||||
|
||||
Runs under the STOCK cadquery-ocp-novtk wheel, not this binding — the fixtures
|
||||
are the reference the Inc 0 gate compares against. From the app checkout:
|
||||
|
||||
app/.venv/bin/python ../ocp/tools/gen_fixtures.py
|
||||
|
||||
Two properties are recorded, and the first is checked here rather than assumed:
|
||||
|
||||
* stock is idempotent — reading a shape and writing it back reproduces the
|
||||
same bytes. Without that the gate would be comparing against a moving
|
||||
target, and a mismatch would say nothing.
|
||||
* the digest of those bytes, which is what the gate reproduces. This matters
|
||||
beyond IPC: cad/derive.py content-addresses BREP payloads as
|
||||
payloads/derived/brep/<sha256>.brep and stores the ref in the document, so
|
||||
a binding that serialises differently rewrites every derived payload.
|
||||
|
||||
Face counts are recorded alongside, pinning the map ordering that face and edge
|
||||
identity depend on throughout the topology code.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
from OCP.BinTools import BinTools
|
||||
from OCP.BRep import BRep_Builder
|
||||
from OCP.BRepAlgoAPI import BRepAlgoAPI_Cut, BRepAlgoAPI_Fuse
|
||||
from OCP.BRepMesh import BRepMesh_IncrementalMesh
|
||||
from OCP.BRepPrimAPI import BRepPrimAPI_MakeBox, BRepPrimAPI_MakeCylinder
|
||||
from OCP.gp import gp_Ax2, gp_Dir, gp_Pnt, gp_Trsf, gp_Vec
|
||||
from OCP.TopAbs import TopAbs_FACE
|
||||
from OCP.TopExp import TopExp
|
||||
from OCP.TopLoc import TopLoc_Location
|
||||
from OCP.TopoDS import TopoDS_Compound, TopoDS_Shape
|
||||
from OCP.TopTools import TopTools_IndexedMapOfShape
|
||||
|
||||
OUT = pathlib.Path(__file__).resolve().parent.parent / "tests" / "data"
|
||||
|
||||
|
||||
def _box(dx=10.0, dy=20.0, dz=30.0) -> TopoDS_Shape:
|
||||
return BRepPrimAPI_MakeBox(dx, dy, dz).Shape()
|
||||
|
||||
|
||||
def _meshed_box() -> TopoDS_Shape:
|
||||
shape = _box()
|
||||
BRepMesh_IncrementalMesh(shape, 0.1, False, 0.5, True)
|
||||
return shape
|
||||
|
||||
|
||||
def _fused() -> TopoDS_Shape:
|
||||
op = BRepAlgoAPI_Fuse()
|
||||
from OCP.TopTools import TopTools_ListOfShape
|
||||
|
||||
args, tools = TopTools_ListOfShape(), TopTools_ListOfShape()
|
||||
args.Append(_box())
|
||||
tools.Append(
|
||||
BRepPrimAPI_MakeBox(gp_Pnt(5, 5, 5), 20.0, 20.0, 20.0).Shape()
|
||||
)
|
||||
op.SetArguments(args)
|
||||
op.SetTools(tools)
|
||||
op.Build()
|
||||
return op.Shape()
|
||||
|
||||
|
||||
def _cut_cylinder() -> TopoDS_Shape:
|
||||
op = BRepAlgoAPI_Cut()
|
||||
from OCP.TopTools import TopTools_ListOfShape
|
||||
|
||||
args, tools = TopTools_ListOfShape(), TopTools_ListOfShape()
|
||||
args.Append(_box())
|
||||
tools.Append(
|
||||
BRepPrimAPI_MakeCylinder(
|
||||
gp_Ax2(gp_Pnt(5, 10, 0), gp_Dir(0, 0, 1)), 3.0, 30.0
|
||||
).Shape()
|
||||
)
|
||||
op.SetArguments(args)
|
||||
op.SetTools(tools)
|
||||
op.Build()
|
||||
return op.Shape()
|
||||
|
||||
|
||||
def _located_compound() -> TopoDS_Shape:
|
||||
"""Exercises the location/TShape sharing part of the format."""
|
||||
builder = BRep_Builder()
|
||||
comp = TopoDS_Compound()
|
||||
builder.MakeCompound(comp)
|
||||
|
||||
base = _box()
|
||||
builder.Add(comp, base)
|
||||
|
||||
trsf = gp_Trsf()
|
||||
trsf.SetTranslation(gp_Vec(50.0, 0.0, 0.0))
|
||||
builder.Add(comp, base.Moved(TopLoc_Location(trsf)))
|
||||
return comp
|
||||
|
||||
|
||||
def _empty_compound() -> TopoDS_Shape:
|
||||
builder = BRep_Builder()
|
||||
comp = TopoDS_Compound()
|
||||
builder.MakeCompound(comp)
|
||||
return comp
|
||||
|
||||
|
||||
SHAPES = {
|
||||
"box": _box,
|
||||
"box_meshed": _meshed_box,
|
||||
"fused": _fused,
|
||||
"cut_cylinder": _cut_cylinder,
|
||||
"located_compound": _located_compound,
|
||||
"empty_compound": _empty_compound,
|
||||
}
|
||||
|
||||
|
||||
def write_bytes(shape: TopoDS_Shape) -> bytes:
|
||||
buf = io.BytesIO()
|
||||
BinTools.Write_s(shape, buf)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def read_shape(data: bytes) -> TopoDS_Shape:
|
||||
shape = TopoDS_Shape()
|
||||
BinTools.Read_s(shape, io.BytesIO(data))
|
||||
return shape
|
||||
|
||||
|
||||
def face_count(shape: TopoDS_Shape) -> int:
|
||||
faces = TopTools_IndexedMapOfShape()
|
||||
TopExp.MapShapes_s(shape, TopAbs_FACE, faces)
|
||||
return faces.Extent()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
import OCP
|
||||
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
manifest: dict = {
|
||||
"generated_by": "cadquery-ocp-novtk",
|
||||
"occt_version": getattr(OCP, "__version__", "unknown"),
|
||||
"shapes": {},
|
||||
}
|
||||
|
||||
for name, build in SHAPES.items():
|
||||
shape = build()
|
||||
data = write_bytes(shape)
|
||||
|
||||
# The gate is "our rewrite == stock rewrite". If stock itself is not
|
||||
# idempotent for a fixture, that fixture cannot serve as a reference.
|
||||
rewritten = write_bytes(read_shape(data))
|
||||
if rewritten != data:
|
||||
print(f"FAIL {name}: stock is not idempotent", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
(OUT / f"{name}.brep").write_bytes(data)
|
||||
manifest["shapes"][name] = {
|
||||
"sha256": hashlib.sha256(data).hexdigest(),
|
||||
"size": len(data),
|
||||
"faces": face_count(shape),
|
||||
}
|
||||
print(f"{name}: {len(data)} bytes, {manifest['shapes'][name]['faces']} faces")
|
||||
|
||||
(OUT / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
|
||||
print(f"\nwrote {len(SHAPES)} fixtures + manifest to {OUT}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
154
tools/inventory.py
Normal file
154
tools/inventory.py
Normal file
@@ -0,0 +1,154 @@
|
||||
"""Symbol inventory: what the app imports from OCP, and what this wheel provides.
|
||||
|
||||
Two jobs, one source of truth:
|
||||
|
||||
--emit parse the app's sources for every `from OCP.<mod> import <name>`
|
||||
plus the attributes reached on those names, and write inventory.json.
|
||||
That file is the coverage work-queue for the remaining increments
|
||||
and, re-run later, the drift detector for newly used symbols.
|
||||
|
||||
--check import the installed OCP and report what the inventory asks for but
|
||||
the wheel does not provide, grouped by module.
|
||||
|
||||
Static analysis only reaches names reached through an imported symbol
|
||||
(`TopExp.MapShapes_s`), not methods called on instances (`shape.IsSame(...)`) —
|
||||
those are covered by running the app's own suite in the parity venv, where a
|
||||
gap is a loud AttributeError. So a green --check is necessary, not sufficient.
|
||||
|
||||
python tools/inventory.py --emit --app ../app/backend
|
||||
python tools/inventory.py --check
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import collections
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
HERE = pathlib.Path(__file__).resolve().parent
|
||||
DEFAULT_INVENTORY = HERE.parent / "inventory.json"
|
||||
DEFAULT_APP = HERE.parent.parent / "app" / "backend"
|
||||
|
||||
|
||||
def scan(app_root: pathlib.Path) -> dict:
|
||||
"""Collect {module: {symbol: [attributes reached on it]}} from the app."""
|
||||
modules: dict[str, dict[str, set[str]]] = collections.defaultdict(
|
||||
lambda: collections.defaultdict(set)
|
||||
)
|
||||
files = 0
|
||||
|
||||
for path in sorted(app_root.rglob("*.py")):
|
||||
if "__pycache__" in path.parts:
|
||||
continue
|
||||
try:
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"))
|
||||
except (SyntaxError, UnicodeDecodeError):
|
||||
continue
|
||||
|
||||
# local alias -> (module, original name)
|
||||
local: dict[str, tuple[str, str]] = {}
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.ImportFrom) and node.module:
|
||||
if node.module == "OCP" or node.module.startswith("OCP."):
|
||||
mod = node.module[4:] or "OCP"
|
||||
for alias in node.names:
|
||||
local[alias.asname or alias.name] = (mod, alias.name)
|
||||
modules[mod][alias.name] # noqa: B018 — create the entry
|
||||
|
||||
if local:
|
||||
files += 1
|
||||
|
||||
for node in ast.walk(tree):
|
||||
if (
|
||||
isinstance(node, ast.Attribute)
|
||||
and isinstance(node.value, ast.Name)
|
||||
and node.value.id in local
|
||||
):
|
||||
mod, name = local[node.value.id]
|
||||
modules[mod][name].add(node.attr)
|
||||
|
||||
return {
|
||||
"app_root": str(app_root),
|
||||
"files_importing_ocp": files,
|
||||
"modules": {
|
||||
mod: {name: sorted(attrs) for name, attrs in sorted(syms.items())}
|
||||
for mod, syms in sorted(modules.items())
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def emit(app_root: pathlib.Path, out: pathlib.Path) -> int:
|
||||
data = scan(app_root)
|
||||
out.write_text(json.dumps(data, indent=2) + "\n")
|
||||
n_mod = len(data["modules"])
|
||||
n_sym = sum(len(v) for v in data["modules"].values())
|
||||
print(f"{n_sym} symbols across {n_mod} modules "
|
||||
f"({data['files_importing_ocp']} files) -> {out}")
|
||||
return 0
|
||||
|
||||
|
||||
def check(inventory: pathlib.Path) -> int:
|
||||
import importlib
|
||||
|
||||
data = json.loads(inventory.read_text())
|
||||
missing: dict[str, list[str]] = {}
|
||||
present = 0
|
||||
|
||||
for mod, symbols in data["modules"].items():
|
||||
try:
|
||||
m = importlib.import_module(f"OCP.{mod}")
|
||||
except ImportError:
|
||||
missing[mod] = [f"<module missing> ({len(symbols)} symbols)"]
|
||||
continue
|
||||
|
||||
for name, attrs in symbols.items():
|
||||
obj = getattr(m, name, None)
|
||||
if obj is None:
|
||||
missing.setdefault(mod, []).append(name)
|
||||
continue
|
||||
present += 1
|
||||
for attr in attrs:
|
||||
if not hasattr(obj, attr):
|
||||
missing.setdefault(mod, []).append(f"{name}.{attr}")
|
||||
|
||||
total = sum(len(v) for v in data["modules"].values())
|
||||
print(f"{present}/{total} imported symbols available")
|
||||
if missing:
|
||||
print(f"\nmissing, by module ({len(missing)} modules):")
|
||||
for mod in sorted(missing):
|
||||
print(f" {mod}: {', '.join(sorted(missing[mod]))}")
|
||||
return 1
|
||||
print("complete")
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("--emit", action="store_true", help="rewrite the inventory")
|
||||
ap.add_argument("--check", action="store_true",
|
||||
help="compare the installed OCP against the inventory")
|
||||
ap.add_argument("--app", type=pathlib.Path, default=DEFAULT_APP)
|
||||
ap.add_argument("--inventory", type=pathlib.Path, default=DEFAULT_INVENTORY)
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.emit:
|
||||
if not args.app.is_dir():
|
||||
print(f"app sources not found: {args.app}", file=sys.stderr)
|
||||
return 2
|
||||
return emit(args.app, args.inventory)
|
||||
if args.check:
|
||||
if not args.inventory.exists():
|
||||
print(f"no inventory at {args.inventory}; run --emit first",
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
return check(args.inventory)
|
||||
|
||||
ap.print_help()
|
||||
return 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
53
tools/parity_venv.sh
Executable file
53
tools/parity_venv.sh
Executable file
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env bash
|
||||
# Set up a side environment where the app runs against n3xd-ocp instead of the
|
||||
# stock wheel.
|
||||
#
|
||||
# The app's manifests are never touched: a swap is per-environment because both
|
||||
# distributions own the OCP/ import path and a process can hold only one OCCT
|
||||
# build. main therefore keeps resolving cadquery-ocp-novtk until the Phase 10C
|
||||
# cutover, and this venv is how parity is proven in the meantime.
|
||||
#
|
||||
# tools/parity_venv.sh install from the Gitea registry
|
||||
# tools/parity_venv.sh --local install the local wheelhouse build
|
||||
#
|
||||
# Then run whichever slice of the app suite the current increment claims:
|
||||
# cd ../app && UV_PROJECT_ENVIRONMENT=.venv-ocp-parity uv run pytest backend/tests/test_geom_memo.py
|
||||
set -euo pipefail
|
||||
|
||||
HERE=$(cd "$(dirname "$0")" && pwd)
|
||||
OCP_REPO=$(dirname "$HERE")
|
||||
APP="${APP_DIR:-$(dirname "$OCP_REPO")/app}"
|
||||
VENV_NAME="${VENV_NAME:-.venv-ocp-parity}"
|
||||
INDEX="https://git.stroblme.de/api/packages/N3XD/pypi/simple/"
|
||||
|
||||
SOURCE="registry"
|
||||
[ "${1:-}" = "--local" ] && SOURCE="local"
|
||||
|
||||
cd "$APP"
|
||||
export UV_PROJECT_ENVIRONMENT="$VENV_NAME"
|
||||
|
||||
echo "--- syncing the app's dependencies into $VENV_NAME ---"
|
||||
uv sync --all-groups
|
||||
|
||||
# Removal must precede installation: both distributions install a top-level
|
||||
# OCP/, so installing over the stock wheel would leave a half-overwritten mix.
|
||||
echo "--- removing the stock wheel ---"
|
||||
uv pip uninstall cadquery-ocp-novtk || true
|
||||
|
||||
echo "--- installing n3xd-ocp ($SOURCE) ---"
|
||||
if [ "$SOURCE" = "local" ]; then
|
||||
uv pip install "$OCP_REPO"/wheelhouse/*.whl
|
||||
else
|
||||
uv pip install --index-url "$INDEX" --prerelease=allow n3xd-ocp
|
||||
fi
|
||||
|
||||
echo "--- sanity ---"
|
||||
uv run python -c "
|
||||
import OCP
|
||||
print('OCP', OCP.__version__, '/ OCCT', OCP.__occt_version__)
|
||||
print('modules:', len(OCP._OCP.__all_modules__))
|
||||
"
|
||||
|
||||
echo "--- coverage against the app's symbol inventory ---"
|
||||
uv run python "$HERE/inventory.py" --check || \
|
||||
echo "(incomplete coverage is expected until the increments land)"
|
||||
Reference in New Issue
Block a user