Eric Brachmann
I am a staff scientist at Niantic, working on the Lightship Visual Positioning System (VPS). I work at the intersection of machine learning and computer vision, 3D vision in particular. My research revolves around topics such as visual relocalisation, pose estimation, end-to-end learning, robust optimization and feature matching.
I publish my research in the top conferences in computer vision where I am also active as area chair and reviewer with several outstanding reviewer mentions. I co-organized several tutorials and workshops on visual relocalisation and object pose estimation.
E-Mail Google Scholar Twitter LinkedIn CVEric Brachmann, Jamie Wynn, Shuai Chen, Tommaso Cavallari, Áron Monszpart, Daniyar Turmukhambetov, Victor Adrian Prisacariu
TL;DR: self-supervised ACE = learning-based structure-from-motion, needs no pose priors, works on unordered image sets, efficiently handles thousands of images.
arXiv project page code videoAxel Barroso-Laguna, Sowmya Munukutla, Victor Adrian Prisacariu, Eric Brachmann
TL;DR: MicKey, a method that regresses and matches scale-metric 3D key points, trained end-to-end using differentiable RANSAC
arXiv project page codeEduardo Arnold, Jamie Wynn, Sara Vicente, Guillermo Garcia-Hernando, Aron Monszpart, Victor Prisacariu, Daniyar Turmukhambetov, Eric Brachmann
TL;DR: only one mapping image and one query, dataset with multiple hundred outdoor scenes, benchmark and online leaderboard
arXiv supplement project page code dataset videoAritra Bhowmik, Stefan Gumhold, Carsten Rother, Eric Brachmann
TL;DR: refine SuperPoint end-to-end for relative pose estimation, gradients of feature matching wrt feature descriptors and key point heatmap
arXiv code video
Eric Brachmann, Carsten Rother
TL;DR: NG-RANSAC + NG-DSAC, gradients of RANSAC-fitted model wrt quality of data points, applied to E/F matrix fitting, horizon line estimation and camera relocalization
arXiv project page F/E matrix code horizon line code relocalisation code videoTomas Hodan, Frank Michel, Eric Brachmann, Wadim Kehl, Anders Buch, Dirk Kraft, Bertram Drost, Joel Vidal, Stephan Ihrke , Xenophon Zabulis, Caner Sahin, Fabian Manhardt, Federico Tombari, Tae-Kyun Kim, Jiri Matas, Carsten Rother
TL;DR: de facto standard benchmark for instance pose estimation, unifying dataset formats and proposing evaluation metrics, ongoing competition with online leaderboard
arXiv project page
Eric Brachmann, Carsten Rother
TL;DR: DSAC++, first time training scene coordinate regression without depth, differentiable PnP
arXiv project page code videoEric Brachmann, Alexander Krull, Sebastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, Carsten Rother
TL;DR: gradients of a RANSAC-fitted model wrt the coordinates of the input points, using policy gradient on discrete hypothesis selection
arXiv project page toy code relocalisation code videoEric Brachmann, Alexander Krull, Frank Michel, Stefan Gumhold, Jamie Shotton, Carsten Rother
TL;DR: introduces dense image-to-object correspondences as a learnable intermediate representation, introduced the LINEMOD-Occlusion dataset
paper supplement project page dataset video 1 video 2Eric Brachmann, Jamie Wynn, Shuai Chen, Tommaso Cavallari, Áron Monszpart, Daniyar Turmukhambetov, Victor Adrian Prisacariu
TL;DR: self-supervised ACE = learning-based structure-from-motion, needs no pose priors, works on unordered image sets, efficiently handles thousands of images.
arXiv project page code videoShuai Chen, Tommaso Cavallari, Victor Adrian Prisacariu, Eric Brachmann
TL;DR: marepo, a scene-agnostic absolute pose regression transformer on top of a scene-specific ACE map representation, on-par with structure-based relocalizers in terms of accuracy and mapping time
arXiv project page codeAxel Barroso-Laguna, Sowmya Munukutla, Victor Adrian Prisacariu, Eric Brachmann
TL;DR: MicKey, a method that regresses and matches scale-metric 3D key points, trained end-to-end using differentiable RANSAC
arXiv project page codeTomas Hodan, Martin Sundermeyer, Yann Labbe, Van Nguyen Nguyen, Gu Wang, Eric Brachmann, Bertram Drost, Vincent Lepetit, Carsten Rother, Jiri Matas
TL;DR: results of BOP challenge 2023, accuracy is excellent if objects are known in advance, for unseen objects, still good but slow
arXiv project page video
Martin Sundermeyer, Tomas Hodan, Yann Labbe, Gu Wang, Eric Brachmann, Bertram Drost, Carsten Rother, Jiri Matas
TL;DR: results of BOP challenge 2022, deep neural networks beat everything else
arXiv project page video 1 video 2
Eduardo Arnold, Jamie Wynn, Sara Vicente, Guillermo Garcia-Hernando, Aron Monszpart, Victor Prisacariu, Daniyar Turmukhambetov, Eric Brachmann
TL;DR: only one mapping image and one query, dataset with multiple hundred outdoor scenes, benchmark and online leaderboard
arXiv supplement project page code dataset videoKarren Yang, Michael Firman, Eric Brachmann, Clement Godard
TL;DR: camera pose by echolocation, relative pose / absolute pose / image retrieval, vision is more accurate but sound helps when vision fails
paper
Tomas Hodan, Martin Sundermeyer, Bertram Drost, Yann Labbe, Eric Brachmann, Frank Michel, Carsten Rother, Jiri Matas
TL;DR: results of BOP challenge 2020, deep neural networks on par with point pair features
arXiv project page
Aritra Bhowmik, Stefan Gumhold, Carsten Rother, Eric Brachmann
TL;DR: refine SuperPoint end-to-end for relative pose estimation, gradients of feature matching wrt feature descriptors and key point heatmap
arXiv code video
Eric Brachmann, Carsten Rother
TL;DR: ESAC, end-to-end learning of mixture-of-experts and RANSAC, large scale scene coordinate regression
arXiv project page code video
Eric Brachmann, Carsten Rother
TL;DR: NG-RANSAC + NG-DSAC, gradients of RANSAC-fitted model wrt quality of data points, applied to E/F matrix fitting, horizon line estimation and camera relocalization
arXiv project page F/E matrix code horizon line code relocalisation code videoOmid Hosseini Jafari, Siva Karthik Mustikovela, Karl Pertsch, Eric Brachmann, Carsten Rother
TL;DR: instance segmentation + deep object coordinate prediction
arXiv
Tomas Hodan, Frank Michel, Eric Brachmann, Wadim Kehl, Anders Buch, Dirk Kraft, Bertram Drost, Joel Vidal, Stephan Ihrke , Xenophon Zabulis, Caner Sahin, Fabian Manhardt, Federico Tombari, Tae-Kyun Kim, Jiri Matas, Carsten Rother
TL;DR: de facto standard benchmark for instance pose estimation, unifying dataset formats and proposing evaluation metrics, ongoing competition with online leaderboard
arXiv project page
Eric Brachmann
TL;DR: summary of my work prior to 2018, learning object and scene coordinate regression using random forests and neural networks
thesis
Eric Brachmann, Carsten Rother
TL;DR: DSAC++, first time training scene coordinate regression without depth, differentiable PnP
arXiv project page code videoEric Brachmann, Alexander Krull, Sebastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, Carsten Rother
TL;DR: gradients of a RANSAC-fitted model wrt the coordinates of the input points, using policy gradient on discrete hypothesis selection
arXiv project page toy code relocalisation code videoFrank Michel, Alexander Kirillov, Eric Brachmann, Alexander Krull, Stefan Gumhold, Bogdan Savchynskyy, Carsten Rother
TL;DR: find pose inlier correspondences by optimizing the energy in a graphical model
arXiv project page
Alexander Krull, Eric Brachmann, Sebastian Nowozin, Frank Michel, Jamie Shotton, Carsten Rother
TL;DR: an RL agent chooses which RANSAC hypothesis to refine next
arXiv project page
Daniela Massiceti, Alexander Krull, Eric Brachmann, Carsten Rother, Philip H.S. Torr
TL;DR: mapping of random forests to NNs for optimization, and back again for efficiency
arXiv
Eric Brachmann, Frank Michel, Alexander Krull, Michael Ying Yang, Stefan Gumhold, Carsten Rother
TL;DR: first object/scene coordinate regression system for RGB, predict correspondence distributions and search for max likelihood pose
paper supplement project page videoAlexander Krull, Eric Brachmann, Frank Michel, Michael Ying Yang, Stefan Gumhold, Carsten Rother
TL;DR: substitute inlier counting pose score with a CNN that compares input image and renderings, trained via max likelihood
paper supplement project page video
Frank Michel, Alexander Krull, Eric Brachmann, Michael Ying Yang, Stefan Gumhold, Carsten Rother
TL;DR: only n+2 correspondences are needed to estimate pose of n-jointed objects
paper conference page project page
Alexander Krull, Frank Michel, Eric Brachmann, Stefan Gumhold, Stephan Ihrke, Carsten Rother
TL;DR: combines RANSAC-based hypothesis sampling with particle filter for real-time pose tracking
paper supplement project page video 1 video 2Eric Brachmann, Alexander Krull, Frank Michel, Stefan Gumhold, Jamie Shotton, Carsten Rother
TL;DR: introduces dense image-to-object correspondences as a learnable intermediate representation, introduced the LINEMOD-Occlusion dataset
paper supplement project page dataset video 1 video 2Eric Brachmann, Marcel Spehr, Stefan Gumhold
TL;DR: propagate visual words along image web edges to make a BoW image descriptors more robust
paper
Eric Brachmann, Gero Dittmann, Klaus-Dieter Schubert
TL;DR: an authentication scheme for company intranets where you may want to trade security for simplicity
paper