Getting Started with NVIDIA Isaac for Healthcare Using the Telesurgery Workflow

TL;DR

• Telesurgery is increasingly essential in a world facing a surgeon shortage and rural access challenges.
• NVIDIA Isaac for Healthcare provides a production-ready, modular telesurgery workflow that covers video and sensor streaming, robot control, haptics, and simulation.
• The solution leverages a three-computer architecture (DGX, OVX, IGX/AGX) to unify the full development stack from simulation to clinical deployment.
• The workflow is containerized, enabling consistent performance and seamless transfer of skills from simulation to real procedures.
• Early pilots indicate telesurgery can form the foundation of a new healthcare delivery model, especially for remote and crisis scenarios. NVIDIA Dev Blog

Context and background

Telesurgery is no longer a distant dream. With a global shortage of surgeons projected to reach 4.5 million by 2030 and rural hospitals facing limited access to specialists, the ability for experts to perform procedures remotely is shifting from experimental to increasingly essential. This shift creates a need for robust, production-grade tools that can operate across training environments and clinical settings. NVIDIA responds with Isaac for Healthcare, a platform designed to provide a reliable, low-latency pipe that connects simulation work to the actual operating room, enabling developers to build, test, and deploy advanced surgical robotics capabilities. The problem space is multifaceted: you must stream video and sensor data quickly, translate operator inputs into precise robot motion, deliver tactile feedback (haptics) when appropriate, and simulate the entire workflow for training. Isaac for Healthcare addresses these concerns with a cohesive, modular telesurgery workflow you can adapt, extend, and deploy in both training and clinical contexts. The result is a development stack that supports the full path from sandboxed experimentation to real patient procedures, without forcing teams to rebuild from scratch at each stage. This continuity is a core promise of the platform and is a critical enabler of rapid iteration and safer, more repeatable procedures. NVIDIA Dev Blog

What’s new

NVIDIA Isaac for Healthcare introduces a production-ready, modular telesurgery workflow that tightly couples the most important elements of modern robotic surgery: video and sensor streaming, robot control, haptic feedback, and high-fidelity simulation. The architecture centers on a three-computer setup that brings together the performance needed for computation, simulation, and edge deployment while maintaining a unified software model. Key architectural elements include:

• A three-computer architecture consisting of NVIDIA DGX for heavy computation, NVIDIA OVX for edge/cloud-like orchestration and simulation, and NVIDIA IGX/NVIDIA AGX for patient-side robotics and real-time control. This triad unifies the full development stack from simulation to clinical deployment.
• A containerized workflow that ensures identical control schemes and networking protocols across deployment modes, so skills learned in simulation transfer directly to live procedures.
• A modular pipeline that integrates video, sensors, robot control, haptics, and simulation, allowing institutions to start with training and progressively transition to live surgery as readiness improves. Early pilot deployments have shown promise, reinforcing the view that telesurgery is not merely an experiment but a foundation for a new healthcare delivery model. NVIDIA Dev Blog From the operator’s perspective, the telesurgery workflow connects a surgeon’s control station to a patient-side surgical robot over a high-speed network. The result is the ability to perform procedures in crisis situations, remote hospitals, or across continents without compromising responsiveness. The platform’s design emphasizes a low-latency experience that is crucial for safe remote procedures. The distribution of tasks across the three machines also makes it feasible to scale workflows for training and for real-world deployment, while maintaining consistent control schemes and networking across environments. NVIDIA Dev Blog

Why it matters (impact for developers/enterprises)

• For developers, Isaac for Healthcare lowers the barrier to building next-generation surgical robotics by offering a ready-made, modular telesurgery workflow. This reduces the time and risk required to bring new imaging modalities, control devices, or software features from concept to clinical use. The same architecture supports both simulation-based training and actual procedures, enabling teams to validate ideas in a safe environment before live operation.
• For healthcare institutions and enterprises, the platform promises a more seamless path from research and simulation to clinical deployment. A containerized approach ensures that performance and control semantics are preserved across environments, so the same training and software stack can be used in universities, pilot hospitals, and full clinical facilities.
• The focus on low latency and robust hardware integration (e.g., sensor streams, camera data, haptics, and control signals) is designed to meet real-world clinical requirements. By addressing gaps in global healthcare delivery—especially in areas with limited access to surgical specialists—the telesurgery workflow aims to expand reach, improve outcomes, and enable new models of care delivery. NVIDIA Dev Blog

Technical details or Implementation

The telesurgery workflow relies on a carefully engineered pipeline that prioritizes latency, reliability, and modularity. A few specifics highlighted in the release provide a sense of the system’s scope and performance targets:

• Latency is a core concern; the documented benchmarks show that the main pathways achieve a latency of less than 50 milliseconds. This low-latency requirement is critical for safe remote procedures and is a central design criterion for the platform.
• Display and measurement setup: benchmarking used a G-Sync-enabled monitor with a 240 Hz refresh rate, operating in Vulkan exclusive display mode. Latency measurements were captured using NVIDIA Latency and Display Analysis Tool (LDAT). These choices reflect a focus on high-fidelity, low-latency visualization for the surgeon’s control station.
• Sensor and camera integration: the workflow mentions a Holoscan Sensor Bridge and the HSB with an imx274 camera. Institutions can source the Holoscan Sensor Bridge from ecosystem FPGA partners such as Lattice and Microchip, highlighting an ecosystem-ready approach to sensor integration.
• Containerized deployment: since the workflow is containerized, deployments across diverse environments share identical control schemes and networking protocols. This containerization is crucial for ensuring that skills and software behavior transfer smoothly from simulation to operating rooms.
• Modular design and extensibility: the platform emphasizes a modular pipeline that lets teams connect cameras, configure DDS (Data Distribution Service), and start experimenting with robot control. Developers are encouraged to fork the repository, explore new control devices, and benchmark latency under their own conditions.
• The architecture supports a transition path from simulation to live surgery, meaning organizations can begin with simulation-based training and progressively deploy the same stack in real procedures as readiness grows. This continuity is designed to reduce the gap between research and clinical practice. NVIDIA Dev Blog Technical details at a glance | Item | Details |Item | Details |Latency target |

References

• https://developer.nvidia.com/blog/getting-started-with-nvidia-isaac-for-healthcare-using-the-telesurgery-workflow/