Skip to content

Engineering Guidelines

This document defines the architectural principles, boundaries, and technical rules for the xovis-sdk. It serves as a deep-dive reference for engineers and agents building or extending the SDK's core capabilities.

Core Philosophy: The Quadrifurcated Architecture

To handle 12.5Hz Live-Push DataPushes without destabilizing, while simultaneously orchestrating complex multisensor graph topologies and cloud fleets, the xovis-sdk architecture is strictly quadrifurcated into four distinct operational planes. Never mix the design patterns of these planes.

1. The Data Plane (High-Frequency Telemetry Ingestion)

Path: src/xovis/datapush/ The objective is zero-copy, non-blocking maximum throughput for raw TCP, HTTP Webhook, UDP, and MQTT DataPush ingestion.

  • Core Engine: Pure native asyncio (servers like asyncio.start_server, asyncio.DatagramProtocol and active client connection loops) with orjson and json.JSONDecoder for lock-free deserialization.
  • Active vs. Passive Ingestion:
    • Passive Receivers (Servers): Run in the SDK's process to ingest telemetry pushed directly by the sensors (XovisTCPServer, XovisUDPServer, XovisHTTPServer).
    • Active Receivers (Clients): Connect or subscribe outward to retrieve telemetry from remote interfaces (XovisTCPClient connecting to sensors in SERVER mode, XovisMQTTClient subscribing to external brokers).
  • Unified Ingestion: Leverages the DataPlaneIngestor to standardize parsing and sink routing across all transport layers.
  • Parsing Strategy:
    • TCP Streaming Data (Server & Client): Uses a sliding string buffer combined with standard library json.JSONDecoder().raw_decode() to safely carve out concatenated, newline-free JSON frames. This is mandatory for TCP because of MTU fragmentation. Do NOT use orjson for this specific extraction step.
    • Discrete Messages (Server & Client): Uses orjson.loads() via the centralized DataPlaneIngestor.parse_frame helper for UDP datagrams, HTTP webhooks, and MQTT topic payloads to minimize deserialization overhead.
  • Binary Fallback: Automatically wraps non-JSON payloads (e.g. binary recordings) in a recording_data pseudo-frame.
  • Data Packing: STRICTLY NO Pydantic validation is allowed in this hot path to ensure zero-blocking of high-frequency streams.
  • Network I/O: Telemetry frames are delivered to sinks in real-time. Efficient batching and lock-free archiving are enforced.

2. The Control Plane (Low-Frequency REST API)

Path: src/xovis/api/ Focuses on structural robustness, strict schema adherence, and network resilience for configuring hardware and cloud services.

  • Networking: Uses httpx for asynchronous HTTP connection pooling and strict redirect handling.
  • Token Resilience: The HubAuth manager handles stateful token caching and autonomous refreshes via asyncio.Lock().
  • Proactive Hardware Probing: Utilizes a lazy, asynchronous _probe_capability cache to prevent fragile 403/404 handling for missing hardware features. Note that Xovis sensors return 403 HTML (mapped to EndpointNotFoundError) for missing/restricted endpoints.
  • Time Normalization (XovisTime): Normalizes all time-sensitive inputs (relative strings, ISO 8601, datetime) to UTC Unix milliseconds.
  • Data Validation: Employs Pydantic v2 heavily for structural validation of OpenAPI-generated models.
  • Configuration Nuances: In HTTPConfig, the uri must exclude the port. DataPushAgent instances MUST have enabled=True explicitly set to prevent deactivation via exclude_unset=True.

3. The State & Topology Plane (Fleet Engine)

Path: src/xovis/api/device/ and src/xovis/api/hub/ Maintains graph-aware context, stateful caching, and Desired State Configuration (DSC) workflows.

  • Ecosystem Topology: Segregates between Host (IP), Singlesensor (physical lens), and Multisensor (virtual stitched environment).
  • Spider Nodes: Intercepts physical lens requests for lensless Spider hardware and raises HardwareNotSupportedError.
  • Stateful Caching: ConfigCacheManager enforces context-isolated persistence by rooting contexts into the HostStateBucket.
  • String-Normalized ID Resolution: Guarantees consistency between SDK dictionary keys and hardware JSON types.
  • The GC Trap: Background tasks MUST be stored in a hard-referenced Set to prevent Python 3.11+ Garbage Collection from deleting tasks mid-execution.
  • Offline-First Persistence: Offloads disk I/O to threads via asyncio.to_thread to ensure zero blocking of the event loop.
  • The Dynamic Hub-to-Edge Bridge: HubClient.connect_device() spawns DeviceClient instances routed through the Cloud HUB using standardized OpenAPI tunnel routes.
  • Hub Tunnel Concurrency: Serializes parallel operations targeting a specific device through the Hub tunnel using a per-device asyncio.Lock.

4. The Agentic Layer (Universal Tool Adapter)

Path: src/xovis/skills/ Provides safe, pseudonymized, and strictly validated hardware orchestration for autonomous agents.

  • Privacy First: AIPrivacySession handles two-way mapping of sensitive identifiers to format-preserving hashes.
  • Safety Guardrails: Assigns a SafetyLevel (OPEN, RESTRICTED, CRITICAL, BLOCKED) to every tool.
  • Strict Mapping: Only explicitly mapped SDK methods are exposed as tools to the AI.

5. Hyper-Optimized Developer Experience (DX)

  • Smart Resolvers: Configuration methods accept IDs (fast-path) or names (cache fallback).
  • Runtime Autocomplete: Uses REPLAccessor to expose dynamic .by_name and .by_mac dot-notation in interactive environments.
  • Static Autocomplete: The xovis-cli generates local xovis_types.py files with partitioned Literal strings.

6. Enterprise Coding & Documentation Standards

  • Zero-Inline-Comment, Max-Docstring: Architectural intent, Pydantic constraints, and plane boundaries MUST be formalized into rigorous Google-style docstrings.
  • Clean Code: Code must explain itself; inline chatter is forbidden.

7. Enterprise Testing Standards (SDET Rules)

Path: tests/ Organized into four execution tiers (Smoke, Stateful, Data Plane, Endurance).

  • Idempotency: All E2E hardware tests must be fully idempotent with hard teardowns in finally blocks.
  • Teardown Sequence: Delete agents before parent connections.
  • Fixture Scopes: Session-scoped HubClient to avoid 429s; function-scoped DeviceClient to prevent event loop closure errors.
  • Strict Parameter Serialization: Boolean flags in query parameters MUST be passed as lowercase strings "true" or "false".

8. Deterministic Dependency Management (uv)

The SDK uses uv for ultra-fast, deterministic dependency resolution and environment management.

  • Source of Truth: The pyproject.toml defines the abstract requirements.
  • Locking: The uv.lock file is the absolute, cross-platform source of truth for exact versions used in development and CI.
  • Resolution: uv uses a high-performance SAT solver to prevent complex resolution loops.
  • Contributor Workflow:
    • Use uv sync to keep your environment aligned with the lockfile.
    • Always commit changes to uv.lock alongside pyproject.toml updates.