NVIDIA releases Cosmos3-H-Dreams surgery simulator

NVIDIA has released Cosmos3-H-Dreams, an AI model that makes video of a gallbladder surgery and changes it as instrument actions come in. It runs interactively on one graphics card. NVIDIA calls it a research simulator, not a tool for patient care.

What happened

NVIDIA announced the model in a Hugging Face blog post dated 20 October 2026. You give it a starting video frame and a stream of instrument actions. It then generates video of a gallbladder removal, called a cholecystectomy, that responds to those actions.

The post says that, to NVIDIA's knowledge, it is the first real-time generative surgical simulator trained on clinical cholecystectomy recordings that supports live control of the instruments. An earlier release, Cosmos-H-Dreams, covered practice stitching (suturing) on a tabletop.

NVIDIA also released the code, the model files, a step-by-step recipe for training on your own data, and links to the existing dataset behind the base model.

Key facts

All of these come from the NVIDIA post. - Size: the model is built on Cosmos3-H-Embodied-Nano, which has 16 billion parameters, per the NVIDIA post. - Training video: clinical cholecystectomy video and instrument motion recorded on the Versius surgical platform from CMR Surgical. - Controls: 28 action dimensions, covering arm poses, grippers, energy controls and thumbsticks. - Video format: 832 by 480 pixels, with actions lined up at 10 times per second in training. - Hardware: it runs interactively on a single NVIDIA RTX PRO 6000 graphics card. - Tools: it supports two instruments, a left grasper and a right hook.

Background

Our take: simulators let a robot practice before it ever touches a person. This one learns from video and then makes new video.

NVIDIA says it first trained a slower "teacher" model, then distilled it into a faster "student" that makes video in short blocks as actions arrive. One version uses four steps per block. A second uses two steps, which is faster but gives lower picture quality.

The post also adds a caution. Video that looks real does not prove that tissue moves the way it would in a body. It reports no measured accuracy results, no speed benchmarks and no surgeon evaluation yet.

What it means for you

This mostly affects researchers who build surgical robots and training tools, and nothing more is stated about who will use it. If you are a patient, nothing changes today. The post says this is not a validated system for patient care.

For developers, the code and model are public, so teams can try it and train it on their own surgery video. The post does not state a license, so check the terms before you build on it.

Our take: this is a step toward robots that practice thousands of times in simulation before touching a person. But with no accuracy numbers and no surgeon review yet, treat it as early research.

What happens next

NVIDIA lists research directions in the post: modeling the full procedure, supporting more than two instruments, surgeon evaluation, and coupling the simulator with autonomous surgical policies. It gives no dates.

Our take: watch for surgeon reviews and accuracy numbers. Those will show whether the video is useful for real training.

FAQ

What is Cosmos3-H-Dreams?

It is NVIDIA's AI simulator that generates video of a gallbladder surgery and reacts to instrument actions, per the NVIDIA post.

What hardware does it need?

NVIDIA says it runs interactively on a single RTX PRO 6000 graphics card.

Is it ready to use on patients?

No. The post describes it as a research simulator and not a validated system for patient care.

Sources

• huggingface.co