X Ray AR Glasses to See Hidden Objects

MIT researchers have built an augmented reality headset that gives the wearer X-ray vision. The headset combines computer vision and wireless perception to automatically locate a specific item that is hidden from view, perhaps inside a box or under a pile, and then guide the user to retrieve it. The system utilizes radio frequency (RF) signals, which can pass through common materials like cardboard boxes, plastic containers, or wooden dividers, to find hidden items that have been labeled with RFID tags, which reflect signals sent by an RF antenna. The headset directs the wearer as they walk through a room toward the location of the item, which shows up as a transparent sphere in the augmented reality (AR) interface. Once the item is in the user’s hand, the headset, called X-AR, verifies that they have picked up the correct object. When the researchers tested X-AR in a warehouse-like environment, the headset could localize hidden items to within 9.8 centimeters, on average. And it verified that users picked up the correct item with 96 percent accuracy. X-AR could aid e-commerce warehouse workers in quickly finding items on cluttered shelves or buried in boxes, or by identifying the exact item for an order when many similar objects are in the same bin. It could also be used in a manufacturing facility to help technicians locate the correct parts to assemble a product.

Our whole goal with this project was to build an augmented reality system that allows you to see things that are invisible — things that are in boxes or around corners — and in doing so, it can guide you toward them and truly allow you to see the physical world in ways that were not possible before,” says Fadel Adib, who is an associate professor in the Department of Electrical Engineering and Computer Science, the director of the Signal Kinetics group in the Media Lab, and the senior author of a paper on X-AR.

Adib’s co-authors are research assistants Tara Boroushaki, who is the paper’s lead author; Maisy Lam; Laura Dodds; and former postdoc Aline Eid, who is now an assistant professor at the University of Michigan. The research will be presented at the USENIX Symposium on Networked Systems Design and Implementation.

Source: https://news.mit.edu/

VR Gives 3D Depiction Inside Blood Vessels

UW Medicine interventional radiologist Wayne Monsky first saw virtual reality’s vivid, 3D depiction of the inside of a phantom patient’s blood vessels, his jaw dropped in childlike wonder.

A virtual-reality depiction of a catheter navigating blood vessel. With a VR headset, this would be 3D (click on the image to enjoy video)

When you put the (VR) headset on … you have a giddy laugh that you can’t control – just sheer happiness and enthusiasm. (I’m) moving up to the mesenteric artery and I can’t believe what I’m seeing,” he recalled.

The experience reminds him of “Fantastic Voyage,” the ’60s-era sci-fi film about a submarine and crew that are miniaturized and injected into a scientist’s body to repair a blood clot.

As a child, and today, I’ve been amazed at the premise that one day you can swim around inside someone’s body. And really, that’s the sensation: You’re in it,” he said. Interventional radiologists use catheters, thin flexible tubes that are inserted into arteries and veins and steered to any organ in the body, guided by X-ray visuals. With this approach, they (and cardiologists, vascular surgeons, and neuro-interventionalists) treat an array of conditions: liver tumors, narrowed and bleeding arteries, uterine fibroids, and more.

Monsky and two collaborators have pioneered VR technology that puts the operator inside 3D blood vessels. By following an anatomically correct, dynamic, 3D map of a phantom patient’s vessels, Monsky navigates the catheter through junctions and angles. The catheter‘s tip is equipped with sensors that visually represent its exact location to the VR headset. It’s a sizable leap forward from the 2D, black-and-white X-ray perspective that has guided Monsky’s catheters through vessels for most of his career.

He recently presented study findings that underscore VR’s value: In tests of a phantom patient, VR guidance got him to the destination faster – about 40 seconds faster, on average, over 18 simulations – than was the case with X-ray guidance.

Source: https://newsroom.uw.edu/