Here at Facebook's F8 conference, Mike Schroepfer, the company's CTO, is presiding over a segment devoted to its recent advances in AI. After recapping past progress in understanding still images and video, he talked about a new Facebook technology called Mask R-CNN that is super-smart at figuring out what's happening in imagery.
Mask R-CNN can tell with a high degree of confidence that a photo of a kitchen includes a person, a bowl, a cup, a wineglass, and a dog. In video, it can detect moving objects–opening up applications such as erasing a sailboat from a water scene.
"Our teams have gotten really good at rapidly taking state of the art technology and putting it in your pocket, on your smartphone," Schroepfer said.
Figuring out what's going on in the real world also comes into play in Facebook's Oculus VR efforts, he added. The company is using four cameras to simultaneously track the movements of a headset-wearing user in a physical space. That's one of the most demanding applications, Schroepfer said, because if there's even a little bit of lag it destroys VR's immersive effect.
Object recognition is also critical for helping Facebook cope with the problem of livestreamed violence or porn. On Monday, the company said it was reviewing its procedures for moderation after a Facebook Live video of a deadly shooting spread across social media.