Telekinesis – a unified skill library for robotics, perception, and Physical AI Hi HN, We’ve been working on Telekinesis, a developer SDK aimed at reducing fragmentation in robotics, computer vision, and Physical AI. This started from a recurring problem we ran into while building real systems: even relatively simple robotics applications often require stitching together a large number of incompatible components — classical robotics libraries, perception pipelines, learned models, and increasingly foundation models like LLMs and VLMs. The problem we’re trying to address Robotics software development is highly fragmented: Perception, planning, control, and learning often live in separate ecosystems Each library comes with its own APIs, assumptions, and data formats Integration glue ends up dominating development time Mixing classical robotics with modern AI workflows is still painful As Physical AI and agent-based systems become more common, this gap between classical robotics workflows and modern AI tooling is becoming more acute. What Telekinesis is Telekinesis is a large-scale skill library for Physical AI, exposed through a single, consistent Python interface. Instead of being another robotics framework, it’s designed as a modular, composable set of skills that can be combined into complete systems. The SDK currently covers: 3D perception: detection, registration, filtering, clustering The SDK will also cover: 2D perception: image processing, detection, segmentation Synthetic data generation Model training tools Motion planning, kinematics, and control Physical AI agents Vision–Language Models (VLMs) The idea is that roboticists and Computer Vision engineers can access these capabilities without spending most of their time integrating fragmented libraries, and instead focus on system behavior and iteration. How it’s structured From the developer’s perspective, Telekinesis provides: A single Python interface Hundreds of modular, composable skills The ability to combine perception, planning, control, and AI components predictably The skills are hosted on the cloud by default, but the same architecture can also be run on-premise for teams that need full control over data and computation. This makes it possible to: Prototype quickly Reuse components across projects Scale from experiments to more industry-grade systems without changing APIs Who this is for The SDK is intended for: Robotics engineers working close to perception or control Computer vision developers building systems, not just models People experimenting with Physical AI and embodied agents In particular, for those who feel they spend more time integrating components than evaluating system behavior. What we’re still figuring out This is still early, and we’re actively questioning parts of the design: Where abstractions help vs. hide too much Which components should never be unified How to balance flexibility with predictability How this compares to existing robotics + ML workflows in practice Happy to hear critical perspectives from people who’ve built or maintained real systems. Here the documentation (still evolving): https://docs.telekinesis.ai/ Thanks for reading. |