set 限制解除

# ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM Scopus CPU 工学学科一流ESI学科分类：工程科学简介JCI 1.79IF(5) 10.2SCU 计算机科学BEI检索SCI升级版 计算机科学1区SCI基础版 工程技术2区SCI Q1IF 7.8CCF BCUG 工程技术T2XJU 一区SWJTU A++

Publisher: IEEE
PDF
Carlos Campos; Richard Elvira; Juan J. Gómez Rodríguez; José M. M. Montiel; Juan D. Tardós
All Authors
Free

- Alerts

Abstract:
This article presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multimap SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. The first main novelty is a tightly integrated visual-inertial SLAM system that fully relies on maximum a posteriori (MAP) estimation, even during IMU initialization, resulting in real-time robust operation in small and large, indoor and outdoor environments, being two to ten times more accurate than previous approaches. The second main novelty is a multiple map system relying on a new place recognition method with improved recall that lets ORB-SLAM3 survive to long periods of poor visual information: when it gets lost, it starts a new map that will be seamlessly merged with previous maps when revisiting them. Compared with visual odometry systems that only use information from the last few seconds, ORB-SLAM3 is the first system able to reuse in all the algorithm stages all previous information from high parallax co-visible keyframes, even if they are widely separated in time or come from previous mapping sessions, boosting accuracy. Our experiments show that, in all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature and significantly more accurate. Notably, our stereo-inertial SLAM achieves an average accuracy of 3.5 cm in the EuRoC drone and 9 mm under quick hand-held motions in the room of TUM-VI dataset, representative of AR/VR scenarios. For the benefit of the community we make public the source code.
Published in: IEEE Transactions on Robotics ( Volume: 37, Issue: 6, December 2021)
Page(s): 1874 - 1890
Date of Publication: 25 May 2021
ISSN Information:
INSPEC Accession Number: 21440558
DOI: 10.1109/TRO.2021.3075644
Publisher: IEEE
Funding Agency:

SECTION I.

## Introduction

Intense research on visual simultaneous localization and mapping (SLAM) systems and visual odometry (VO), using cameras either alone or in combination with inertial sensors, has produced, during the last two decades, excellent systems, with increasing accuracy and robustness. Modern systems rely on maximum a posteriori (MAP) estimation which, in the case of visual sensors, corresponds to bundle adjustment (BA), either geometric BA that minimizes feature reprojection error, in feature-based methods, or photometric BA that minimizes the photometric error of a set of selected pixels, in direct methods.

With the recent emergence of VO systems that integrate loop closing techniques, the frontier between VO and SLAM is more diffuse. The goal of visual SLAM is to use the sensors on-board a mobile agent to build a map of the environment and compute in real time the pose of the agent in that map. In contrast, VO systems put their focus on computing the agent’s ego-motion and not on building a map. The big advantage of a SLAM map is that it allows matching and using in BA previous observations performing three types of data association (extending the terminology used in [1]).

1. Short-term data association: matching map elements obtained during the last few seconds. This is the only data association type used by most VO systems, which forget environment elements once they get out of view, resulting in continuous estimation drift even when the system moves in the same area.

2. Mid-term data association: matching map elements that are close to the camera whose accumulated drift is still small. These can be matched and used in BA in the same way than short-term observations and allow to reach zero drift when the systems move in mapped areas. They are the key to the better accuracy obtained by our system compared against VO systems with loop detection.

3. Long-term data association: matching observations with elements in previously visited areas using a place recognition technique, regardless of the accumulated drift (loop detection), the current area being previously mapped in a disconnected map (map merging), or the tracking being lost (relocalization). Long-term matching allows to reset the drift and to correct the map using pose-graph (PG) optimization or, more accurately, using BA. This is the key to SLAM accuracy in medium and large loopy environments.

In this work, we build on ORB-SLAM [2], [3] and ORB-SLAM visual–inertial [4], the first visual and visual–inertial systems able to take full profit of short-term, mid-term, and long-term data association, reaching zero drift in mapped areas. Here, we go one step further providing multimap data association, which allows us to match and use in BA map elements coming from previous mapping sessions, achieving the true goal of a SLAM system: building a map that can be used later to provide accurate localization.

This is essentially a system paper, whose most important contribution is the ORB-SLAM3 library itself [5], the most complete and accurate visual, visual–inertial, and multimap SLAM system to date (see Table I). The main novelties of ORB-SLAM3 are as follows.

TABLE I Summary of the Most Representative Visual (Top) and Visual–Inertial (bottom) Systems, in Chronological Order

1. A monocular and stereo visual–inertial SLAM system that fully relies on MAP estimation, even during the inertial measurement unit (IMU) initialization phase. The initialization method proposed was previously presented in [6]. Here, we add its integration with ORB-SLAM visual–inertial [4], the extension to stereo-inertial SLAM, and a thorough evaluation in public datasets. Our results show that the monocular and stereo visual–inertial systems are extremely robust and significantly more accurate than other visual–inertial approaches, even in sequences without loops.

2. Improved-recall place recognition. Many recent visual SLAM and VO systems [2], [7], [8] solve place recognition using the DBoW2 bag of words library [9]. DBoW2 requires temporal consistency, matching three consecutive keyframes to the same area, before checking geometric consistency, boosting precision at the expense of recall. As a result, the system is too slow at closing loops and reusing previously mapped areas. We propose a novel place recognition algorithm, in which candidate keyframes are first checked for geometrical consistency, and then for local consistency with three covisible keyframes, which in most occasions are already in the map. This strategy increases recall and densifies data association improving map accuracy, at the expense of a slightly higher computational cost.

3. ORB-SLAM Atlas. The first complete multimap SLAM system able to handle visual and visual–inertial systems in monocular and stereo configurations. The Atlas can represent a set of disconnected maps and apply to them all the mapping operations smoothly: place recognition, camera relocalization, loop closure, and accurate seamless map merging. This allows to automatically use and combine maps built at different times, performing incremental multisession SLAM. A preliminary version of ORB-SLAM Atlas for visual sensors was presented in [10]. Here we add the new place recognition system, the visual–inertial multimap system, and its evaluation on public datasets.

4. An abstract camera representation making the SLAM code agnostic of the camera model used and allowing to add new models by providing their projection, unprojection, and Jacobian functions. We provide the implementations of pin-hole [11] and fisheye [12] models.

All these novelties, together with a few code improvements, make ORB-SLAM3 the new reference visual and visual–inertial open-source SLAM library, being as robust as the best systems available in the literature and significantly more accurate, as shown by our experimental results in Section VII. We also provide comparisons between monocular, stereo, monocular-inertial, and stereo-inertial SLAM results that can be of interest for practitioners.

SECTION II.

## Related Work

Table I presents a summary of the most representative visual and visual–inertial systems, showing the main techniques used for estimation and data association. The qualitative accuracy and robustness ratings included in the table are based on the results presented in Section VII and the comparison between parallel tracking and mapping (PTAM), large-scale direct monocular SLAM (LSD-SLAM), and ORB-SLAM reported in [2].

### A. Visual SLAM

Monocular SLAM was first solved in MonoSLAM [13], [14], [52] using an extended Kalman filter (EKF) and Shi-Tomasi points that were tracked in subsequent images doing a guided search by correlation. Mid-term data association was significantly improved using techniques that guarantee that the feature matches used are consistent, achieving hand-held visual SLAM [53], [54].

In contrast, keyframe-based approaches estimate the map using only a few selected frames, discarding the information coming from intermediate frames. This allows to perform the more costly, but more accurate, BA optimization at keyframe rate. The most representative system was PTAM [16] that splits camera tracking and mapping into two parallel threads. Keyframe-based techniques are more accurate than filtering for the same computational cost [55], becoming the gold standard in visual SLAM and VO. Large-scale monocular SLAM was achieved in [56] using sliding-window BA and in [57] using a double-window optimization and a covisibility graph.

Building on these ideas, ORB-SLAM [2], [3] uses oriented fast and rotated brief (ORB) features, whose descriptor provides short-term and mid-term data association, builds a covisibility graph to limit the complexity of tracking and mapping, and performs loop closing and relocalization using the bag-of-words library DBoW2 [9], achieving long-term data association. To date, it is the only visual SLAM system integrating the three types of data association, which we believe is the key to its excellent accuracy. In this work, we improve its robustness in pure visual SLAM with the new Atlas system that starts a new map when tracking is lost and its accuracy in loopy scenarios with the new place recognition method with improved recall.

Direct methods do not extract features, but use directly the pixel intensities in the images, and estimate motion and structure by minimizing a photometric error. LSD-SLAM [20] was able to build large-scale semidense maps using high gradient pixels. However, map estimation was reduced to PG optimization, achieving lower accuracy than PTAM and ORB-SLAM [2]. The hybrid system semi-direct visual odometry (SVO) [23], [24] extracts FAST features, uses a direct method to track features and any pixel with nonzero intensity gradient from frame to frame, and optimizes camera trajectory and 3-D structure using reprojection error. SVO is extremely efficient, but, being a pure VO method, it only performs short-term data association, which limits its accuracy. Direct sparse odometry (DSO) [27] is able to compute accurate camera poses in situations where point detectors perform poorly, enhancing robustness in low textured areas or against blurred images. It introduces local photometric BA that simultaneously optimizes a window of seven recent keyframes and the inverse depth of the points. Extensions of this work include stereo [29], loop closing using features and DBoW2 [58], [59], and visual–inertial odometry [46]. Direct sparse mapping (DSM) [31] introduces the idea of map reusing in direct methods, showing the importance of mid-term data association. In all cases, the lack of integration of short-, mid-, and long-term data association results in lower accuracy than our proposal (see Section VII).

### B. Visual–Inertial SLAM

The combination of visual and inertial sensors provides robustness to poor texture, motion blur, and occlusions and, in the case of monocular systems, makes scale observable.

Research in tightly coupled approaches can be traced back to multi-state constraint kalman filter (MSCKF) [33] where the EKF quadratic cost in the number of features is avoided by feature marginalization. The initial system was perfected in [34] and extended to stereo in [35] and [36]. The first tightly coupled VO system based on keyframes and BA was OKVIS [38], [39] which is also able to use monocular and stereo vision. While these systems rely on features, ROVIO [41], [42] feeds an EFK with photometric error using direct data association.

ORB-SLAM-VI [4] presented for the first time a visual–inertial SLAM system able to reuse a map with short-term, mid-term, and long-term data association, using them in an accurate local visual–inertial BA based on IMU preintegration [60], [61]. However, its IMU initialization technique was too slow, taking 15 s, which harmed robustness and accuracy. Faster initialization techniques were proposed in [62] and [63] based on a closed-form solution to jointly retrieve scale, gravity, accelerometer bias and initial velocity, as well as visual features depth. Crucially, they ignore IMU noise properties and minimize the 3-D error of points in space, and not their reprojection errors, that is the gold standard in feature-based computer vision. Our previous work [64] shows that this results in large unpredictable errors.

VINS-Mono [7] is a very accurate and robust monocular-inertial odometry system with loop closing that uses DBoW2 and 4 degrees of freedom (DoF) PG optimization and map-merging. Feature tracking is performed with Lucas–Kanade tracker, being slightly more robust than descriptor matching. In VINS-Fusion [44], it has been extended to stereo and stereo-inertial.

VI-DSO [46] extends DSO to visual–inertial odometry, proposing a BA that combines inertial observations with the photometric error of selected high gradient pixels, which renders very good accuracy. As the information from high gradient pixels is successfully exploited, the robustness in scene regions with poor texture is also boosted. Their initialization method relies on visual–inertial BA and takes 20–30 s to converge within 1% scale error.

The recent BASALT [47] is a stereo-inertial odometry system that extracts nonlinear factors from visual–inertial odometry to use them in BA and closes loops matching ORB features, achieving very good to excellent accuracy. Kimera [8] is a novel outstanding metric-semantic mapping system, but its metric part consists in stereo-inertial odometry plus loop closing with DBoW2 and PG optimization, achieving similar accuracy to VINS-Fusion.

In this work, we build on ORB-SLAM-VI and extend it to stereo-inertial SLAM. We propose a novel fast initialization method based on MAP estimation that properly takes into account visual and inertial sensor uncertainties and estimates the true scale with 5% error in 2 s, converging to 1% scale error in 15 s. All other systems discussed above are visual–inertial odometry methods, some of them extended with loop closing, and lack the capability of using mid-term data associations. We believe that this, together with our fast and precise initialization, is the key to the better accuracy consistently obtained by our system, even in sequences without loops.

### C. Multimap SLAM

The idea of adding robustness to tracking losses during exploration by means of map creation and fusion was first proposed in [65] within a filtering approach. One of the first keyframe-based multimap systems was [66], but the map initialization was manual and the system was not able to merge or relate the different submaps. Multimap capability has been researched as a component of collaborative mapping systems, with several mapping agents and a central server that only receives information [67] or with bidirectional information flow as in C2TAM [68]. MOARSLAM [69] proposed a robust stateless client-server architecture for collaborative multidevice SLAM, but the main focus was the software architecture and did not report accuracy results.

More recently, CCM-SLAM [70], [71] proposes a distributed multimap system for multiple drones with bidirectional information flow, built on top of ORB-SLAM. Their focus is on overcoming the challenges of limited bandwidth and distributed processing, while ours is on accuracy and robustness, achieving significantly better results on the EuRoC dataset. SLAM [72] also proposes a multimap extension of ORB-SLAM2 but keeps submaps as separated entities while we perform seamless map merging, building a more accurate global map.

VINS-Mono [7] is a VO system with loop closing and multimap capabilities that rely on the place recognition library DBoW2 [9]. Our experiments show that ORB-SLAM3 is 2.6 times more accurate than VINS-Mono in monocular-inertial single-session operation on the EuRoC dataset, thanks to the ability to use mid-term data association. Our Atlas system also builds on DBoW2 but proposes a novel higher recall place recognition technique and performs a more detailed and accurate map merging using local BA, increasing the advantage to 3.2 times better accuracy than VINS-Mono in multisession operation on EuRoC.

SECTION III.

## System Overview

ORB-SLAM3 is built on ORB-SLAM2 [3] and ORB-SLAM-VI [4]. It is a full multimap and multisession system able to work in pure visual or visual–inertial modes with monocular, stereo, or RGB-D sensors, using pin-hole and fisheye camera models. Fig. 1 shows the main system components that are parallel to those of ORB-SLAM2 with some significant novelties, which are summarized next.

Fig. 1.

Main system components of ORB-SLAM3.

Show All

1. Atlas is a multimap representation composed of a set of disconnected maps. There is an active map where the tracking thread localizes the incoming frames and is continuously optimized and grown with new keyframes by the local mapping thread. We refer to the other maps in the Atlas as the nonactive maps. The system builds a unique DBoW2 database of keyframes that is used for relocalization, loop closing, and map merging.

2. Tracking thread processes sensor information and computes the pose of the current frame with respect to the active map in real time, minimizing the reprojection error of the matched map features. It also decides whether the current frame becomes a keyframe. In visual–inertial mode, the body velocity and IMU biases are estimated by including the inertial residuals in the optimization. When tracking is lost, the tracking thread tries to relocalize the current frame in all the Atlas’ maps. If relocalized, tracking is resumed, switching the active map if needed. Otherwise, after a certain time, the active map is stored as nonactive, and a new active map is initialized from scratch.

3. Local mapping thread adds keyframes and points to the active map, removes the redundant ones, and refines the map using visual or visual–inertial BA, operating in a local window of keyframes close to the current frame. Additionally, in the inertial case, the IMU parameters are initialized and refined by the mapping thread using our novel MAP-estimation technique.

4. Loop and map merging thread detects common regions between the active map and the whole Atlas at keyframe rate. If the common area belongs to the active map, it performs loop correction; if it belongs to a different map, both maps are seamlessly merged into a single one, which becomes the active map. After a loop correction, a full BA is launched in an independent thread to further refine the map without affecting real-time performance.

SECTION IV.

## Camera Model

ORB-SLAM assumed in all system components a pin-hole camera model. Our goal is to abstract the camera model from the whole SLAM pipeline by extracting all properties and functions related to the camera model (projection and unprojection functions, Jacobian, etc.) into separate modules. This allows our system to use any camera model by providing the corresponding camera module. In ORB-SLAM3 library, apart from the pin-hole model, we provide the Kannala–Brandt [12] fisheye model.

As most popular computer vision algorithms assume a pin-hole camera model, many SLAM systems rectify either the whole image or the feature coordinates to work in an ideal planar retina. However, this approach is problematic for fisheye lenses that can reach or surpass a field of view (FOV) of 180∘. Image rectification is not an option as objects in the periphery get enlarged and objects in the center lose resolution, hindering feature matching. Rectifying the feature coordinates requires using less than 180∘ FOV and causes trouble to many computer vision algorithms that assume uniform reprojection error along the image, which is far from true in rectified fisheye images. This forces to crop out the outer parts of the image, losing the advantages of large FOV: faster mapping of the environment and better robustness to occlusions. Next, we discuss how to overcome these difficulties.

### A. Relocalization

A robust SLAM system needs the capability of relocalizing the camera when tracking fails. ORB-SLAM solves the relocalization problem by setting a perspective-n-points solver based on the ePnP algorithm [73], which assumes a calibrated pin-hole camera along all its formulation. To follow up with our approach, we need a PnP algorithm that works independently of the camera model used. For that reason, we have adopted maximum likelihood perspective-n-point algorithm [74] that is completely decoupled from the camera model as it uses projective rays as input. The camera model just needs to provide an unprojection function passing from pixels to projection rays, to be able to use relocalization.

### B. Nonrectified Stereo SLAM

Most stereo SLAM systems assume that stereo frames are rectified, i.e., both images are transformed to pin-hole projections using the same focal length, with image planes coplanar, and are aligned with horizontal epipolar lines, such that a feature in one image can be easily matched by looking at the same row in the other image. However, the assumption of rectified stereo images is very restrictive and, in many applications, is neither suitable nor feasible. For example, rectifying a divergent stereo pair or a stereo fisheye camera would require severe image cropping, losing the advantages of a large FOV.

For that reason, our system does not rely on image rectification, considering the stereo rig as two monocular cameras having the following:

1. a constant relative SE(3) transformation between them;

2. optionally, a common image region that observes the same portion of the scene.

These constrains allow us to effectively estimate the scale of the map by introducing that information when triangulating new landmarks and in the BA optimization. Following up with this idea, our SLAM pipeline estimates a 6 DoF rigid body pose, whose reference system can be located in one of the cameras or in the IMU sensor, and represents the cameras with respect to the rigid body pose.

If both cameras have an overlapping area in which we have stereo observations, we can triangulate true scale landmarks the first time they are seen. The rest of both images still has a lot of relevant information that is used as monocular information in the SLAM pipeline. Features first seen in these areas are triangulated from multiple views, as in the monocular case.

SECTION V.

## Visual–Inertial SLAM

ORB-SLAM-VI [4] was the first true visual–inertial SLAM system capable of map reusing. However, it was limited to pin-hole monocular cameras, and its initialization was too slow, failing in some challenging scenarios. In this work, we build on ORB-SLAM-VI providing a fast and accurate IMU initialization technique and an open-source SLAM library capable of monocular-inertial and stereo-inertial SLAM, with pin-hole and fisheye cameras.

### A. Fundamentals

While, in pure visual SLAM, the estimated state only includes the current camera pose, in visual–inertial SLAM, additional variables need to be computed. These are the body pose T i=[R i, p i]∈SE(3) and velocity v i in the world frame, and the gyroscope and accelerometer biases, b g i and b a i, which are assumed to evolve according to a Brownian motion. This leads to the state vector

S i≐{T i, v i, b g i, b a i}.(1)
View Source \begin{equation*} \mathcal {S}_i \doteq \lbrace \mathbf {T}_i, \mathbf {v}_i, \mathbf {b}^g_i, \mathbf {b}^a_i\rbrace . \tag{1} \end{equation*}

For visual–inertial SLAM, we preintegrate IMU measurements between consecutive visual frames, i and i+1, following the theory developed in [60] and formulated on manifolds in [61]. We obtain preintegrated rotation, velocity, and position measurements, denoted as Δ R i,i+1, Δ v i,i+1, and Δ p i,i+1, as well as a covariance matrix Σ I i,i+1 for the whole measurement vector. Given these preintegrated terms and states S i and S i+1, we adopt the definition of inertial residual r I i,i+1 from [61]

r I i,i+1 r Δ R i,i+1 r Δ v i,i+1 r Δ p i,i+1=[r Δ R i,i+1, r Δ v i,i+1, r Δ p i,i+1]=Log(Δ R T i,i+1 R T i R i+1)=R T i(v i+1−v i−g Δ t i,i+1)−Δ v i,i+1=R T i(p j−p i−v i Δ t i,i+1−1 2 gΔ t 2)−Δ p i,i+1(2)
View Source \begin{equation*} \begin{split} \mathbf {r}_{\mathcal {I}_{i,i+1}}& =[{\bf r}_{\Delta \text{R}_{i,i+1}}, {\bf r}_{\Delta \text{v}_{i,i+1}}, {\bf r}_{\Delta \text{p}_{i,i+1}}]\\ {\bf r}_{\Delta \text{R}_{i,i+1}}& = \text{Log}\left(\Delta {\bf R}_{i,i+1}^\text{T} {\bf R}_i^\text{T} {\bf R}_{i+1}\right)\\ {\bf r}_{\Delta \text{v}_{i,i+1}}& = {\bf R}_i^\text{T} \left({\bf v}_{i+1} - {\bf v}_i - {\bf g} \Delta t_{i,i+1}\right) - \Delta {\bf v}_{i,i+1} \\ {\bf r}_{\Delta \text{p}_{i,i+1}} & = {\bf R}_i^\text{T} \left({{\bf p}_j} - {{\bf p}}_i - {{\bf v}}_i \Delta t_{i,i+1} - \frac{1}{2}{\bf g} \Delta t^2\right) - \Delta {\bf p}_{i,i+1} \end{split} \tag{2} \end{equation*} where Log: SO(3)→R 3 maps from the Lie group to the vector space. Together with inertial residuals, we also use reprojection errors r ij between frame i and 3-D point j at position x j
r ij=u ij−Π(T CB T−1 i⊕x j)(3)
View Source \begin{equation*} \mathbf {r}_{ij} = \mathbf {u}_{ij}-\Pi \left(\mathbf {T}_{\text{CB}}\mathbf {T}_i^{-1} \oplus \mathbf {x}_j\right) \tag{3} \end{equation*} where Π: R 3→R n is the projection function for the corresponding camera model, u ij is the observation of point j at image i, having a covariance matrix Σ ij, T CB∈SE(3) stands for the rigid transformation from body-IMU to camera (left or right), known from calibration, and ⊕ is the transformation operation of SE(3) group over R 3 elements.

Combining inertial and visual residual terms, visual–inertial SLAM can be posed as a keyframe-based minimization problem [39]. Given a set of k+1 keyframes and its state S¯k≐{S 0…S k} and a set of l 3-D points and its state X≐{x 0…x l−1}, the visual–inertial optimization problem can be stated as follows:

min S¯k,X(∑i=1 k∥∥r I i−1,i∥∥2 Σ−1 I i,i+1+∑j=0 l−1∑i∈K j ρ Hub(∥r ij∥Σ−1 ij))(4)
View Source \begin{equation*} \underset{\bar{\mathcal {S}}_{k}, \mathcal {X}}{\min } \left(\sum _{i = 1}^{k} \left\Vert \mathbf {r}_{\mathcal {I}_{i-1,i}}\right\Vert _{\Sigma ^{-1}_{\mathcal {I}_{i,i+1}}}^2 + \sum _{j = 0}^{l-1} \sum _{i \in \mathcal {K}^{j}} \rho _{\text{Hub}}\left(\left\Vert \mathbf {r}_{ij}\right\Vert _{\Sigma ^{-1}_{ij}}\right) \right) \tag{4} \end{equation*} where K j is the set of keyframes observing 3-D point j. This optimization may be outlined as the factor-graph shown in Fig. 2(a). Note that for reprojection error, we use a robust Huber kernel ρ Hub to reduce the influence of spurious matchings, while for inertial residuals, it is not needed since miss-associations do not exist. This optimization needs to be adapted for efficiency during tracking and mapping, but, more importantly, it requires good initial seeds to converge to accurate solutions.

Fig. 2.

Factor graph representation for different optimizations along the system.

Show All

### B. IMU Initialization

The goal of this step is to obtain good initial values for the inertial variables: body velocities, gravity direction, and IMU biases. Some systems like VI-DSO [46] try to solve from scratch visual–inertial BA, sidestepping a specific initialization process, obtaining slow convergence for inertial parameters (up to 30 s).

In this work, we propose a fast and accurate initialization method based on the following three key insights.

1. Pure monocular SLAM can provide very accurate initial maps [2], whose main problem is that scale is unknown. Solving first the vision-only problem will enhance IMU initialization.

2. As shown in [56], scale converges much faster when it is explicitly represented as an optimization variable, instead of using the implicit representation of BA.

3. Ignoring sensor uncertainties during IMU initialization produces large unpredictable errors [64].

So, taking properly into account sensor uncertainties, we state the IMU initialization as a MAP estimation problem, split into the following three steps.

1. Vision-Only MAP Estimation: We initialize pure monocular SLAM [2] and run it during 2 s, inserting keyframes at 4 Hz. After this period, we have an up-to-scale map composed of k=10 camera poses and hundreds of points, which is optimized using visual-only BA [Fig. 2(b)]. These poses are transformed to body reference, obtaining the trajectory T¯0: k=[R, p¯]0: k where the bar denotes up-to-scale variables in the monocular case.

2. Inertial-Only MAP Estimation: In this step, we aim to obtain the optimal estimation of the inertial variables, in the sense of MAP estimation, using only T¯0: k and inertial measurements between these keyframes. These inertial variables may be stacked in the inertial-only state vector

   Y k={s, R wg, b, v¯0: k}(5)
   View Source \begin{equation*} \mathcal {Y}_k=\lbrace s, {\bf R}_{\text{w}g}, \mathbf {b}, \bar{\mathbf {v}}_{0:k} \rbrace \tag{5} \end{equation*} where s∈R+ is the scale factor of the vision-only solution; R wg∈SO(3) is a rotation matrix used to compute gravity vector g in the world reference as g=R wg g I, where g I=(0, 0, G)T and G is the gravity magnitude; b=(b a, b g)∈R 6 are the accelerometer and gyroscope biases assumed to be constant during initialization; and v¯0: k∈R 3 is the up-to-scale body velocities from first to last keyframe, initially estimated from T¯0: k. At this point, we are only considering the set of inertial measurements I 0: k≐{I 0,1…I k−1,k}. Thus, we can state an MAP estimation problem, where the posterior distribution to be maximized is
   p(Y k|I 0: k)∝p(I 0: k|Y k)p(Y k)(6)
   View Source \begin{equation*} p(\mathcal {Y}_k \vert \mathcal {I}_{0:k}) \propto p (\mathcal {I}_{0:k} \vert \mathcal {Y}_k) p(\mathcal {Y}_k) \tag{6} \end{equation*} where p(I 0: k|Y k) stands for likelihood and p(Y k) for prior. Considering independence of measurements, the inertial-only MAP estimation problem can be written as
   Y∗k=arg max Y k(p(Y k)∏i=1 k p(I i−1,i|s, R wg, b, v¯i−1, v¯i)).(7)
   View Source \begin{equation*} \mathcal {Y}_{k}^* = \underset{\mathcal {Y}_{k}}{\arg \max } \left(p(\mathcal {Y}_k) \prod _{i=1}^k p(\mathcal {I}_{i-1,i} \vert s, {\bf R}_{\text{w}g}, \mathbf {b}, \bar{\mathbf {v}}_{i-1}, \bar{\mathbf {v}}_{i}) \right). \tag{7} \end{equation*}

   Taking negative logarithm and assuming Gaussian error for IMU preintegration and prior distribution, this finally results in the optimization problem

   Y∗k=arg min Y k(∥b∥2 Σ−1 b+∑i=1 k∥r I i−1,i∥2 Σ−1 I i−1,i).(8)
   View Source \begin{equation*} \mathcal {Y}_k^* = \underset{\mathcal {Y}_{k}}{\arg \min } \left(\Vert \mathbf {b}\Vert _{\Sigma ^{-1}_b}^2 + \sum _{i=1}^{k} \Vert \mathbf {r}_{\mathcal {I}_{i-1,i}}\Vert _{\Sigma ^{-1}_{\mathcal {I}_{i-1,i}}}^2 \right) . \tag{8} \end{equation*}

   This optimization, represented in Fig. 2(c), differs from (4) in not including visual residuals, as the up-to-scale trajectory estimated by visual SLAM is taken as constant, and adding a prior residual that forces IMU biases to be close to zero. Covariance matrix Σ b represents prior knowledge about the range of values IMU biases may take. Details for preintegration of IMU covariance Σ I i−1,i can be found at [61].

   As we are optimizing in a manifold, we need to define a retraction [61] to update R wg during the optimization. Since rotation around gravity direction does not suppose a change in gravity, this update is parameterized with two angles (δ α g, δ β g)

   R new wg=R old wg Exp(δ α g, δ β g, 0)(9)
   View Source \begin{equation*} {\bf R}_{ \text{wg}}^{\text{new}} = {\bf R}_{ \text{wg}}^{\text{old}} \text{Exp}(\delta \alpha _{{\bf g}}, \delta \beta _{{\bf g}}, 0) \tag{9} \end{equation*} with Exp(.) being the exponential map from R 3 to SO(3). To guarantee that scale factor remains positive during optimization, we define its update as
   s new=s old exp(δ s).(10)
   View Source \begin{equation*} s^{\text{new}} = s^{\text{old}} \exp {(\delta s).} \tag{10} \end{equation*}

   Once the inertial-only optimization is finished, the frame poses and velocities and the 3-D map points are scaled with the estimated scale factor and rotated to align the z-axis with the estimated gravity direction. Biases are updated and IMU preintegration is repeated, aiming to reduce future linearization errors.

3. Visual–Inertial MAP Estimation: Once we have a good estimation for inertial and visual parameters, we can perform a joint visual–inertial optimization for further refining the solution. This optimization may be represented as Fig. 2(a) but having common biases for all keyframes and including the same prior information for biases than in the inertial-only step.

Our exhaustive initialization experiments on the EuRoC dataset [6] show that this initialization is very efficient, achieving 5% scale error with trajectories of 2 s. To improve the initial estimation, visual–inertial BA is performed 5 and 15 s after initialization, converging to 1% scale error as shown in Section VII. After these BAs, we say that the map is mature, meaning that scale, IMU parameters, and gravity directions are already accurately estimated.

Our initialization is much more accurate than joint initialization methods that solve a set of algebraic equations [62]–​[64] and much faster than the initialization used in ORB-SLAM-VI [4] that needed 15 s to get the first scale estimation or that used in VI-DSO [46] that starts with a huge scale error and requires 20–30 s to converge to 1% error. Comparisons between different initialization methods may be found at [6].

In some specific cases, when slow motion does not provide good observability of the inertial parameters, initialization may fail to converge to accurate solutions in just 15 s. To get robustness against this situation, we propose a novel scale refinement technique based on a modified inertial-only optimization, where all inserted keyframes are included, but scale and gravity directions are the only parameters to be estimated [Fig. 2(d)]. Note that, in that case, the assumption of constant biases would not be correct. Instead, we use the values estimated from mapping, and we fix them. This optimization, which is very computationally efficient, is performed in the local mapping thread every 10 s until the map has more than 100 keyframes or more than 75 s have passed since initialization.

Finally, we have easily extended our monocular-inertial initialization to stereo-inertial by fixing the scale factor to one and taking it out from the inertial-only optimization variables, enhancing its convergence.

### C. Tracking and Mapping

For tracking and mapping, we adopt the schemes proposed in [4]. Tracking solves a simplified visual–inertial optimization where only the states of the last two frames are optimized, while map points remain fixed.

For mapping, trying to solve the whole optimization from (4) would be intractable for large maps. We use as optimizable variables a sliding window of keyframes and their points, including also observations to these points from covisible keyframes but keeping their pose fixed.

### D. Robustness to Tracking Loss

In pure visual SLAM or VO systems, temporal camera occlusion and fast motions result in losing track of visual elements, getting the system lost. ORB-SLAM pioneered the use of fast relocalization techniques based on bag-of-words place recognition, but they proved insufficient to solve difficult sequences in the EuRoC dataset [3]. Our visual–inertial system enters into visually lost state when less than 15 point maps are tracked and achieves robustness in the following two stages:

1. Short-term lost: The current body state is estimated from IMU readings, and map points are projected in the estimated camera pose and searched for matches within a large image window. The resulting matches are included in visual–inertial optimization. In most cases, this allows to recover visual tracking. Otherwise, after 5 s, we pass to the next stage.

2. Long-term lost: A new visual–inertial map is initialized as explained above, and it becomes the active map.

If the system gets lost within 15 s after IMU initialization, the map is discarded. This prevents to accumulate inaccurate and meaningless maps.

SECTION VI.

## Map Merging and Loop Closing

Short-term and mid-term data associations between a frame and the active map are routinely found by the tracking and mapping threads by projecting map points into the estimated camera pose and searching for matches in an image window of just a few pixels. To achieve long-term data association for relocalization and loop detection, ORB-SLAM uses the DBoW2 bag-of-words place recognition system [9], [75]. This method has been also adopted by most recent VO and SLAM systems that implement loop closures (Table I).

Unlike tracking, place recognition does not start from an initial guess for camera pose. Instead, DBoW2 builds a database of keyframes with their bag-of-words vectors and, given a query image, is able to efficiently provide the most similar keyframes according to their bag-of-words. Using only the first candidate, raw DBoW2 queries achieve precision and recall in the order of 50%–80% [9]. To avoid false positives that would corrupt the map, DBoW2 implements temporal and geometric consistency checks, moving the working point to 100% precision and 30%–40% recall [9], [75]. Crucially, the temporal consistency check delays place recognition at least during three keyframes. When trying to use it in our Atlas system, we found that this delay and the low recall resulted too often in duplicated areas in the same or in different maps.

In this work, we propose a new place recognition algorithm with improved recall for long-term and multimap data association. Whenever the mapping thread creates a new keyframe, place recognition is launched trying to detect matches with any of the keyframes already in the Atlas. If the matching keyframe found belongs to the active map, a loop closure is performed. Otherwise, it is a multimap data association, and then the active and the matching maps are merged. As a second novelty in our approach, once the relative pose between the new keyframe and the matching map is estimated, we define a local window with the matching keyframe and its neighbors in the covisibility graph. In this window, we intensively search for mid-term data associations, improving the accuracy of loop closing and map merging. These two novelties explain the better accuracy obtained by ORB-SLAM3 compared with ORB-SLAM2 in the EuRoC experiments. The details of the different operations are explained next.

### A. Place Recognition

To achieve higher recall, for every new active keyframe, we query the DBoW2 database for several similar keyframes in the Atlas. To achieve 100% precision, each of these candidates goes through several steps of geometric verification. The elementary operation of all the geometrical verification steps consists in checking whether there is an ORB keypoint inside an image window whose descriptor matches the ORB descriptor of a map point, using a threshold for the Hamming distance between them. If there are several candidates in the search window, to discard ambiguous matches, we check the distance ratio to the second-closest match [76]. The steps of our place recognition algorithm are as follows.

1. DBoW2 candidate keyframes. We query the Atlas DBoW2 database with the active keyframe K a to retrieve the three most similar keyframes, excluding keyframes covisible with K a. We refer to each matching candidate for place recognition as K m.

2. Local window. For each K m, we define a local window that includes K m, its best covisible keyframes, and the map points observed by all of them. The DBoW2 direct index provides a set of putative matches between keypoints in K a and in the local window keyframes. For each of these 2-D–2-D matches, we have also the 3-D–3-D match available between their corresponding map points.

3. 3-D aligning transformation. We compute using RANSAC the transformation T am that better aligns the map points in K m local window with those of K a. In pure monocular, or in monocular-inertial when the map is still not mature, we compute T am∈Sim(3); otherwise, T am∈SE(3). In both cases, we use Horn algorithm [77] using a minimal set of three 3-D–3-D matches to find each hypothesis for T am. The putative matches that, after transforming the map point in K a by T am, achieve a reprojection error in K a below a threshold give a positive vote to the hypothesis. The hypothesis with more votes is selected, provided the number is over a threshold.

4. Guided matching refinement. All the map points in the local window are transformed with T am to find more matches with the keypoints in K a. The search is also reversed, finding matches for K a map points in all the keyframes of the local window. Using all the matchings found, T am is refined by nonlinear optimization, where the goal function is the bidirectional reprojection error, using Huber influence function to provide robustness to spurious matches. If the number of inliers after the optimization is over a threshold, a second iteration of guided matching and nonlinear refinement is launched, using a smaller image search window.

5. Verification in three covisible keyframes. To avoid false positives, DBoW2 waited for place recognition to fire in three consecutive keyframes, delaying or missing place recognition. Our crucial insight is that, most of the time, the information required for verification is already in the map. To verify place recognition, we search in the active part of the map two keyframes covisible with K a where the number of matches with points in the local window is over a threshold. If they are not found, the validation is further tried with the new incoming keyframes, without requiring the bag-of-words to fire again. The validation continues until three keyframes verify T am or two consecutive new keyframes fail to verify it.

6. VI gravity direction verification. In the visual–inertial case, if the active map is mature, we have estimated T am∈SE(3). We further check whether the pitch and roll angles are below a threshold to definitively accept the place recognition hypothesis.

### B. Visual Map Merging

When a successful place recognition produces multimap data association between keyframe K a in the active map M a, and a matching keyframe K m from a different map stored in the Atlas M m, with an aligning transformation T am, we launch a map merging operation. In the process, special care must be taken to ensure that the information in M m can be promptly reused by the tracking thread to avoid map duplication. For this, we propose to bring the M a map into M m reference. As M a may contain many elements and merging them might take a long time, merging is split into two steps. First, the merge is performed in a welding window defined by the neighbors of K a and K m in the covisibility graph, and in a second stage, the correction is propagated to the rest of the merged map by a PG optimization. The detailed steps of the merging algorithm are as follows.

1. Welding window assembly. The welding window includes K a and its covisible keyframes, K m and its covisible keyframes, and all the map points observed by them. Before their inclusion in the welding window, the keyframes and map points belonging to M a are transformed by T ma to align them with respect to M m.

2. Merging maps. Maps M a and M m are fused together to become the new active map. To remove duplicated points, matches are actively searched for M a points in the M m keyframes. For each match, the point from M a is removed, and the point in M m is kept accumulating all the observations of the removed point. The covisibility and essential graphs [2] are updated by the addition of edges connecting keyframes from M m and M a, thanks to the new mid-term point associations found.

3. Welding bundle adjustment. A local BA is performed optimizing all the keyframes from M a and M m in the welding window along with the map points which are observed by them [Fig. 3(a)]. To fix gauge freedom, the keyframes of M m not belonging to the welding window but observing any of the local map points are included in the BA with their poses fixed. Once the optimization finishes, all the keyframes included in the welding area can be used for camera tracking, achieving fast and accurate reuse of map M m.

4. Essential-graph optimization. A PG optimization is performed using the essential graph of the whole merged map, keeping fixed the keyframes in the welding area. This optimization propagates corrections from the welding window to the rest of the map.

Fig. 3.

Factor graph representation for the welding BA, with reprojection error terms (blue squares), IMU preintegration terms (yellow squares), and bias random walk (purple squares). (a) Visual welding BA. (b) Visual-Inertial welding BA.

Show All

### C. Visual–Inertial Map Merging

The visual–inertial merging algorithm follows similar steps than the pure visual case. Steps 1) and 3) are modified to better exploit the inertial information.

1. VI welding window assembly: If the active map is mature, we apply the available T ma∈SE(3) to map M a before its inclusion in the welding window. If the active map is not mature, we align M a using the available T ma∈Sim(3).

2. VI welding bundle adjustment: Poses, velocities, and biases of keyframes K a and K m and their five last temporal keyframes are included as optimizable. These variables are related by IMU preintegration terms, as shown in Fig. 3(b). For M m, the keyframe immediately before the local window is included but fixed, while, for M a, the similar keyframe is included, but its pose remains optimizable. All map points seen by the above-mentioned keyframes are optimized, together with poses from K m and K a covisible keyframes. All keyframes and points are related by means of reprojection error.

### D. Loop Closing

Loop closing correction algorithm is analogous to map merging but in a situation where both keyframes matched by place recognition belong to the active map. A welding window is assembled from the matched keyframes, and point duplicates are detected and fused creating new links in the covisibility and essential graphs. The next step is a PG optimization to propagate the loop correction to the rest of the map. The final step is a global BA to find the MAP estimate after considering the loop closure mid-term and long-term matches. In the visual–inertial case, the global BA is only performed if the number of keyframes is below a threshold to avoid a huge computational cost.

SECTION VII.

## Experimental Results

The evaluation of the whole system is split into the following.

1. Single session experiments in EuRoC [79]: Each of the 11 sequences is processed to produce a map, with the four sensor configurations: monocular, monocular–inertial, stereo, and stereo–inertial.

2. Performance of monocular and stereo visual–inertial SLAM with fisheye cameras, in the challenging TUM-VI benchmark [80].

3. Multisession experiments in both datasets.

As usual in the field, we measure accuracy with rms ATE [81], aligning the estimated trajectory with ground-truth using a Sim(3) transformation in the pure monocular case, and an SE(3) transformation in the rest of sensor configurations. Scale error is computed using s from Sim(3) alignment, as |1−s|. All experiments have been run on an Intel Core i7-7700 CPU, at 3.6 GHz, with 32 GB memory, using only CPU.

### A. Single-Session SLAM on EuRoC

Table II compares the performance of ORB-SLAM3 using its four sensor configurations with the most relevant systems in the state of the art. Our reported values are the median after 10 executions. As shown in the table, ORB-SLAM3 achieves in all sensor configurations more accurate result than the best systems available in the literature, in most cases by a wide margin.

TABLE II Performance Comparison in the EuRoC Dataset (RMS ATE in M., Scale Error in %). Except Where Noted, We Show Results Reported by the Authors of Each System, for All the Frames in the Trajectory, Comparing With the Processed GT

In monocular and stereo configurations, our system is more precise than ORB-SLAM2 due to the better place recognition algorithm that closes loops earlier and provides more mid-term matches. Interestingly, the next best results are obtained by DSM that also uses mid-term matches, even though it does not close loops.

In monocular-inertial configuration, ORB-SLAM3 is five to ten times more accurate than MCSKF, OKVIS, and ROVIO and more than double the accuracy of VI-DSO and VINS-Mono, showing again the advantages of mid-term and long-term data associations. Compared with ORB-SLAM VI, our novel fast IMU initialization allows ORB-SLAM3 to calibrate the inertial sensor in a few seconds and uses it from the very beginning, being able to complete all EuRoC sequences, obtaining better accuracy.

In stereo-inertial configuration, ORB-SLAM3 is three to four times more accurate than Kimera and VINS-Fusion. Its accuracy is only approached by the recent BASALT that, being a native stereo-inertial system, was not able to complete sequence V203, where some frames from one of the cameras are missing. Comparing our monocular-inertial and stereo-inertial systems, the latter performs better in most cases. Only for two Machine Hall (MH) sequences, a lower accuracy is obtained. We hypothesize that greater depth scene for MH sequences may lead to less accurate stereo triangulation and, hence, a less precise scale.

To summarize performance, we have presented the median of ten executions for each sensor configuration. For a robust system, the median represents accurately the behavior of the system. But a nonrobust system will show high variance in its results. This can be analyzed using Fig. 4 that shows with colors the error obtained in each of the ten executions. Comparison with the figures for DSO, ROVIO, and VI-DSO published in [46] confirms the superiority of our method.

Fig. 4.

Colored squares represent the rms ATE for ten different executions in each sequence of the EuRoC dataset.

Show All

In pure visual configurations, the multimap system adds some robustness to fast motions by creating a new map when tracking is lost, which is merged later with the global map. This can be seen in sequences V103 monocular and V203 stereo that could not be solved by ORB-SLAM2 and are successfully solved by our system in most executions. As expected, stereo is more robust than monocular, thanks to its faster feature initialization, with the additional advantage that the real scale is estimated.

However, the big leap in robustness is obtained by our novel visual–inertial SLAM system, both in monocular and stereo configurations. The stereo-inertial system has a very slight advantage over monocular-inertial, particularly in the most challenging V203 sequence.

We can conclude that inertial integration not only boosts accuracy, reducing the median ATE error compared to pure visual solutions, but it also endows the system with excellent robustness, having a much more stable performance.

### B. Visual–Inertial SLAM on TUM-VI Benchmark

The TUM-VI dataset [80] consists of 28 sequences in 6 different environments, recorded using a hand-held fisheye stereo-inertial rig. Ground-truth for the trajectory is only available at the beginning and at the end of the sequences, which, for most of them, represents a very small portion of the whole trajectory. Many sequences in the dataset do not contain loops. Even if the starting and ending points are in the same room, point of view directions are opposite and place recognition cannot detect any common region. Using this ground-truth for evaluation amounts to measuring the accumulated drift along the whole trajectory.

We extract 1500 ORB points per image in monocular-inertial setup and 1000 points per image in stereo-inertial, after applying CLAHE equalization to address under and over exposure found in the dataset. For outdoor sequences, our system struggles with very far points coming from the cloudy sky, which is very visible in fisheye cameras. These points may have slow motion that can introduce drift in the camera pose. For preventing this, we discard points further than 20 m from the current camera pose, only for outdoor sequences. A more sophisticated solution would be to use an image segmentation algorithm to detect and discard the sky.

The results obtained are compared with the most relevant systems in the literature in Table III, which clearly shows the superiority of ORB-SLAM3 both in monocular-inertial and stereo-inertial configurations. The closest systems are VINS-Mono and BASALT, which are essentially visual–inertial odometry systems with loop closures, and miss mid-term data associations.

TABLE III TUM-VI Benchmark [80]: RMS ATE (m) for Regions With Available Ground-Truth Data

Analyzing more in detail the performance of our system, it gets the lowest error in small and medium indoor environments, room and corridor sequences, with errors below 10 cm for most of them. In these trajectories, the system is continuously revisiting and reusing previously mapped regions, which is one of the main strengths of ORB-SLAM3. Also, tracked points are typically closer than 5 m, which makes it easier to estimate inertial parameters, preventing them from diverging.

In magistrale indoor sequences, which are up to 900 m long, most tracked points are relatively close, and ORB-SLAM3 obtains errors around 1 m except in one sequence that goes close to 5 m. In contrast, in some long outdoor sequences, the scarcity of close visual features may cause drift of the inertial parameters, notably scale and accelerometer bias, which leads to errors in the order of 10–70 m. However, ORB-SLAM3 is the best performing system in the outdoor sequences.

This dataset also contains three really challenging slides sequences, where the user descends though a dark tubular slide with almost total lack of visual features. In this situation, a pure visual system would be lost, but our visual–inertial system is able to process the whole sequence with competitive error, even if no loop-closures can be detected. Interestingly, VINS-Mono and BASALT, which track features using Lukas–Kanade, obtain, in some of these sequences, better accuracy than ORB-SLAM3, which matches ORB descriptors.

Finally, the room sequences can be representative of typical AR/VR applications, where the user moves with a hand-held or head-mounted device in a small environment. For these sequences, ground-truth is available for the entire trajectory. Table III shows that ORB-SLAM3 is significantly more accurate than competing approaches. The results obtained using our four sensor configurations are compared in Table IV. The better accuracy of pure monocular compared with stereo is only apparent: The monocular solution is up-to-scale and is aligned with ground-truth with 7 DoFs, while stereo provides the true scale and is aligned with 6 DoFs. Using monocular-inertial, we further reduce the average rms ATE error close to 1 cm, also obtaining the true scale. Finally, our stereo-inertial SLAM brings error below 1 cm, making it an excellent choice for AR/VR applications.

TABLE IV RMS ATE (m) Obtained by ORB-SLAM3 With Four Sensor Configurations in the Room Sequences, Representative of AR/VR Scenarios (Median of Three Executions)

### C. Multisession SLAM

EuRoC dataset contains several sessions for each of its three environments: 5 in MH, 3 in Vicon1, and 3 in Vicon2. To test the multisession performance of ORB-SLAM3, we process sequentially all the sessions corresponding to each environment. Each trajectory in the same environment has ground-truth with the same world reference, which allows to perform a single global alignment to compute ATE.

The first sequence in each room provides an initial map. Processing the following sequences starts with the creation of a new active map, which is quickly merged with the map of the previous sessions, and from that point on, ORB-SLAM3 profits from reusing the previous map.

Table V reports the global multisession rms ATE for the four sensor configurations in the three rooms, comparing with the two only published multisession results in EuRoC dataset: CCM-SLAM [71] that reports pure monocular results in MH01-MH03, and VINS-Mono [7] in the five MH sequences, using monocular-inertial. In both cases, ORB-SLAM3 more than doubles the accuracy of competing methods. In the case of VINS-Mono, ORB-SLAM3 obtains 2.6 better accuracy in single-session, and the advantage goes up to 3.2 times in multisession, showing the superiority of our map merging operations.

TABLE V Multisession RMS ATE (m) on the EuRoC Dataset. For CCM-SLAM and VINS, We Show Results Reported by the Authors of Each System. Our Values Are the Median of Five Executions, Aligning the Trajectories With the Processed GT

Comparing these multisession performances with the single-session results reported in Table II, the most notable difference is that multisession monocular and stereo SLAM can robustly process the difficult sequences V103 and V203, thanks to the exploitation of the previous map.

We have also performed some multisession experiments on the TUM-VI dataset. Fig. 5 shows the result after processing several sequences inside the TUM building.1 In this case, the small room sequence provides loop closures that were missing in the longer sequences, bringing all errors to centimeter level. Although ground-truth is not available outside the room, comparing the figure with the figures published in [82] clearly shows our point: Our multisession SLAM system obtains far better accuracy than existing visual–inertial odometry systems. This is further exemplified in Fig. 6. Although ORB-SLAM3 ranks higher in stereo inertial single-session processing of outdoors1, there is still a significant drift (≈60 m). In contrast, if outdoors1 is processed after magistrale2 in a multisession manner, this drift is significantly reduced, and the final map is much more accurate.

Fig. 5.

Multisession stereo-inertial result with several sequences from TUM-VI dataset (front, side, and top views).

Show All

Fig. 6.

Multisession stereo-inertial. In red, the trajectory estimated after single-session processing of outdoors1. In blue, multisession processing of magistrale2 first and then outdoors1.

Show All

### D. Computing Time

Table VI summarizes the running time of the main operations performed in the tracking and mapping threads, showing that our system is able to run in real time at 30–40 frames and at 3–6 keyframes per second. The inertial part takes negligible time during tracking and, in fact, can render the system more efficient as the frame rate could be safely reduced. In the mapping thread, the higher number of variables per keyframe has been compensated with a smaller number of keyframes in the inertial local BA, achieving better accuracy, with similar running time. As the tracking and mapping threads work always in the active map, multimapping does not introduce significant overhead.

TABLE VI Running Time of the Main Parts of Our Tracking and Mapping Threads Compared to ORB-SLAM2, on EuRoC V202 (Mean Time and Standard Deviation in Ms)

Table VII summarizes the running time of the main steps for loop closing and map merging. The novel place recognition method only takes 10 ms per keyframe. Times for merging and loop closing remain below 1 s, running only a PG optimization. For loop closing, performing a full BA may increase times up to a few seconds, depending on the size of the involved maps. In any case, as both operations are executed in a separate thread (Fig. 1), they do not interfere with the real-time performance of the rest of the system. The visual–inertial systems perform just two map merges to join three sequences, while visual systems perform some additional merges to recover from tracking losses. Thanks to their lower drift, visual–inertial systems also perform less loop closing operations compared with pure visual systems.

TABLE VII Running Time of the Main Operations for Loop Closing and Map Merging for a Multisession Experiment on Sequences V201, V202, and V203 From EuRoC Dataset (Mean Time and Standard Deviation in Ms)

Although it would be interesting, we do not compare running time against other systems since this would require a significant effort that is beyond the scope of this work.

SECTION VIII.

## Conclusion

Building on [2]–​[4], we presented ORB-SLAM3, the most complete open-source library for visual, visual–inertial, and multisession SLAM, with monocular, stereo, RGB-D, pin-hole, and fisheye cameras. Our main contributions, apart from the integrated library itself, are the fast and accurate IMU initialization technique and the multisession map-merging functions that rely on a new place recognition technique with improved recall.

Our experimental results showed that ORB-SLAM3 is the first visual and visual–inertial system capable of effectively exploiting short-term, mid-term, long-term, and multimap data associations, reaching an accuracy level that is beyond the reach of existing systems. Our results also suggested that, regarding accuracy, the capability of using all these types of data association overpowers other choices such as using direct methods instead of features or performing keyframe marginalization for local BA, instead of assuming an outer set of static keyframes as we do.

The main failure case of ORB-SLAM3 is low-texture environments. Direct methods are more robust to low-texture but are limited to short-term [27] and mid-term [31] data association. On the other hand, matching feature descriptors successfully solves long-term and multimap data association but seems to be less robust for tracking than Lucas–Kanade, which uses photometric information. An interesting line of research could be developing photometric techniques adequate for the four data association problems. We are currently exploring this idea for map building from endoscope images inside the human body.

About the four different sensor configurations, there is no question; stereo-inertial SLAM provides the most robust and accurate solution. Furthermore, the inertial sensor allows to estimate pose at IMU rate, which is orders of magnitude higher than frame rate, being a key feature for some use cases. For applications where a stereo camera is undesirable because of its higher bulk, cost, or processing requirements, you can use monocular-inertial without missing much in terms of robustness and accuracy. Only keep in mind that pure rotations during exploration would not allow to estimate depth.

In applications with slow motions, or without roll and pitch rotations, such as a car in a flat area, IMU sensors can be difficult to initialize. In those cases, if possible, use stereo SLAM. Otherwise, recent advances on depth estimation from a single image with convolutional neural networks (CNNs) offer good promise for reliable and true-scale monocular SLAM [83], at least in the same type of environments where the CNN has been trained.

More Like This
Long-range outdoor localization of a mobile robot using a binocular camera

IECON 2015 - 41st Annual Conference of the IEEE Industrial Electronics Society

Published: 2015

Integration of a 3D-TOF camera into an autonomous, mobile robot system

2009 IEEE Instrumentation and Measurement Technology Conference

Published: 2009

Show More

# References

1.
C. Cadena et al., "Past present and future of simultaneous localization and mapping: Toward the robust-perception age", IEEE Trans. Robot., vol. 32, no. 6, pp. 1309-1332, Dec. 2016.
Show in Context View Article
Google Scholar
2.
R. Mur-Artal, J. M. M. Montiel and J. D. Tardós, "ORB-SLAM: A versatile and accurate monocular SLAM system", IEEE Trans. Robot., vol. 31, no. 5, pp. 1147-1163, Oct. 2015.
Show in Context View Article
Google Scholar
3.
R. Mur-Artal and J. D. Tardós, "ORB-SLAM2: An open-source SLAM system for monocular stereo and RGB-D cameras", IEEE Trans. Robot., vol. 33, no. 5, pp. 1255-1262, Oct. 2017.
Show in Context View Article
Google Scholar
4.
R. Mur-Artal and J. D. Tardós, "Visual-inertial monocular SLAM with map reuse", IEEE Robot. Autom. Lett., vol. 2, no. 2, pp. 796-803, Apr. 2017.
Show in Context View Article
Google Scholar
5.
C. Campos, R. Elvira, J. J. Gómez Rodríguez, J. M. M. Montiel and J. D. Tardós, "ORB-SLAM3: An accurate open-source library for visual visual-inertial and multi-map SLAM", [online] Available: https://github.com/UZ-SLAMLab/ORB_SLAM3.
Show in Context Google Scholar
6.
C. Campos, J. M. M. Montiel and J. D. Tardós, "Inertial-only optimization for visual-inertial initialization", Proc. IEEE Int. Conf. Robot. Autom., pp. 51-57, 2020.
Show in Context View Article
Google Scholar
7.
T. Qin, P. Li and S. Shen, "VINS-Mono: A robust and versatile monocular visual-inertial state estimator", IEEE Trans. Robot., vol. 34, no. 4, pp. 1004-1020, Aug. 2018.
Show in Context View Article
Google Scholar
8.
A. Rosinol, M. Abate, Y. Chang and L. Carlone, "Kimera: An open-source library for real-time metric-semantic localization and mapping", Proc. IEEE Int. Conf. Robot. Autom., pp. 1689-1696, 2020.
Show in Context View Article
Google Scholar
9.
D. Gálvez-López and J. D. Tardós, "Bags of binary words for fast place recognition in image sequences", IEEE Trans. Robot., vol. 28, no. 5, pp. 1188-1197, Oct. 2012.
Show in Context View Article
Google Scholar
10.
R. Elvira, J. D. Tardós and J. M. M. Montiel, "ORBSLAM-Atlas: A robust and accurate multi-map system", Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., pp. 6253-6259, 2019.
Show in Context View Article
Google Scholar
11.
R. Tsai, "A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf TV cameras and lenses", IEEE J. Robot. Autom., vol. 3, no. 4, pp. 323-344, Aug. 1987.
Show in Context View Article
Google Scholar
12.
J. Kannala and S. S. Brandt, "A generic camera model and calibration method for conventional wide-angle and fish-eye lenses", IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 8, pp. 1335-1340, Aug. 2006.
Show in Context View Article
Google Scholar
13.
A. J. Davison, "Real-time simultaneous localisation and mapping with a single camera", Proc. IEEE Int. Conf. Comput. Vision, vol. 2, pp. 1403-1410, 2003.
Show in Context View Article
Google Scholar
14.
A. J. Davison, I. D. Reid, N. D. Molton and O. Stasse, "MonoSLAM: Real-time single camera SLAM", IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 1052-1067, Aug. 2007.
Show in Context View Article
Google Scholar
15.
H. Kim, "SceneLib2 - MonoSLAM open-source library", [online] Available: https://github.com/hanmekim/SceneLib2.
Google Scholar
16.
G. Klein and D. Murray, "Parallel tracking and mapping for small AR workspaces", Proc. IEEE ACM Int. Symp. Mixed Augmented Reality, pp. 225-234, 2007.
Show in Context View Article
Google Scholar
17.
G. Klein and D. Murray, "Improving the agility of keyframe-based SLAM", Proc. Eur. Conf. Comput. Vis., pp. 802-815, 2008.
Google Scholar
18.
G. Klein and D. Murray, "Parallel tracking and mapping on a camera phone", Proc. 8th IEEE Int. Symp. Mixed Augmented Reality, pp. 83-86, Oct. 2009.
View Article
Google Scholar
19.
G. Klein and D. Murray, 2013, [online] Available: https://github.com/Oxford-PTAM/PTAM-GPL.
20.
J. Engel, T. Schöps and D. Cremers, "LSD-SLAM: Large-scale direct monocular SLAM", Proc. Eur. Conf. Comput. Vis., pp. 834-849, 2014.
Show in Context Google Scholar
21.
J. Engel, J. Stueckler and D. Cremers, "Large-scale direct SLAM with stereo cameras", Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., pp. 141-148, 2015.
View Article
Google Scholar
22.
J. Engel, T. Schöps and D. Cremers, "LSD-SLAM: Large-scale direct monocular SLAM", [online] Available: https://github.com/tum-vision/lsd_slam.
Google Scholar
23.
C. Forster, M. Pizzoli and D. Scaramuzza, "SVO: Fast semi-direct monocular visual odometry", Proc. IEEE Int. Conf. Robot. Autom., pp. 15-22, 2014.
Show in Context View Article
Google Scholar
24.
C. Forster, Z. Zhang, M. Gassner, M. Werlberger and D. Scaramuzza, "SVO: Semidirect visual odometry for monocular and multicamera systems", IEEE Trans. Robot., vol. 33, no. 2, pp. 249-265, Apr. 2017.
Show in Context View Article
Google Scholar
25.
C. Forster, M. Pizzoli and D. Scaramuzza, 2014, [online] Available: https://github.com/uzh-rpg/rpg_svo.
26.
R. Mur-Artal, J. D. Tardós, J. M. M. Montiel and D. Gálvez-López, "ORB-SLAM2", 2016, [online] Available: https://github.com/raulmur/ORB_SLAM2.
Google Scholar
27.
J. Engel, V. Koltun and D. Cremers, "Direct sparse odometry", IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 3, pp. 611-625, Mar. 2018.
Show in Context View Article
Google Scholar
28.
H. Matsuki, L. Stumberg, V. Usenko, J. Stückler and D. Cremers, "Omnidirectional DSO: Direct sparse odometry with fisheye cameras", IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 3693-3700, Oct. 2018.
View Article
Google Scholar
29.
R. Wang, M. Schworer and D. Cremers, "Stereo DSO: Large-scale direct sparse visual odometry with stereo cameras" in Proc. IEEE Int. Conf. Comput. Vis., pp. 3903-3911, 2017.
Show in Context View Article
Google Scholar
30.
J. Engel, V. Koltun and D. Cremers, "DSO: Direct sparse odometry", 2018, [online] Available: https://github.com/JakobEngel/dso.
Google Scholar
31.
J. Zubizarreta, I. Aguinaga and J. M. M. Montiel, "Direct sparse mapping", IEEE Trans. Robot., vol. 36, no. 4, pp. 1363-1370, Aug. 2020.
Show in Context View Article
Google Scholar
32.
J. Zubizarreta, I. Aguinaga, J. D. Tardós and J. M. M. Montiel, "DSM: Direct sparse mapping", 2019, [online] Available: https://github.com/jzubizarreta/dsm.
Google Scholar
33.
A. I. Mourikis and S. I. Roumeliotis, "A multi-state constraint Kalman filter for vision-aided inertial navigation", Proc. IEEE Int. Conf. Robot. Autom., pp. 3565-3572, 2007.
Show in Context Google Scholar
34.
M. Li and A. I. Mourikis, "High-precision consistent EKF-based visual-inertial odometry", Int. J. Robot. Res., vol. 32, no. 6, pp. 690-711, 2013.
Show in Context CrossRef Google Scholar
35.
M. K. Paul, K. Wu, J. A. Hesch, E. D. Nerurkar and S. I. Roumeliotis, "A comparative analysis of tightly-coupled monocular binocular and stereo VINS", Proc. IEEE Int. Conf. Robot. Autom., pp. 165-172, 2017.
Show in Context View Article
Google Scholar
36.
M. K. Paul and S. I. Roumeliotis, "Alternating-stereo VINS: Observability analysis and performance evaluation", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4729-4737, 2018.
Show in Context View Article
Google Scholar
37.
K. Chaney, "Monocular MSCKF", 2018, [online] Available: https://github.com/daniilidis-group/msckf_mono.
Google Scholar
38.
S. Leutenegger, P. Furgale, V. Rabaud, M. Chli, K. Konolige and R. Siegwart, "Keyframe-based visual-inertial SLAM using nonlinear optimization", Proc. Robot. Sci. Syst., 2013.
Show in Context CrossRef Google Scholar
39.
S. Leutenegger, S. Lynen, M. Bosse, R. Siegwart and P. Furgale, "Keyframe-based visual-inertial odometry using nonlinear optimization", Int. J. Robot. Res., vol. 34, no. 3, pp. 314-334, 2015.
Show in Context CrossRef Google Scholar
40.
S. Leutenegger, A. Forster, P. Furgale, P. Gohl and S. Lynen, "OKVIS: Open keyframe-based visual-inertial SLAM (ROS Version", 2016, [online] Available: https://github.com/ethz-asl/okvis_ros.
Google Scholar
41.
M. Bloesch, S. Omari, M. Hutter and R. Siegwart, "Robust visual inertial odometry using a direct EKF-based approach", Proc. IEEE/RSJ Intell. Robots Syst., pp. 298-304, 2015.
Show in Context View Article
Google Scholar
42.
M. Bloesch, M. Burri, S. Omari, M. Hutter and R. Siegwart, "Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback", Int. J. Robot. Res., vol. 36, no. 10, pp. 1053-1072, 2017.
Show in Context CrossRef Google Scholar
43.
M. Bloesch, S. Omari, M. Hutter and R. Siegwart, 2015, [online] Available: https://github.com/ethz-asl/rovio.
44.
T. Qin, J. Pan, S. Cao and S. Shen, "A general optimization-based framework for local odometry estimation with multiple sensors", 2019.
Show in Context Google Scholar
45.
T. Qin, S. Cao, J. Pan, P. Li and S. Shen, "VINS-Fusion: An optimization-based multi-sensor state estimator", 2019, [online] Available: https://github.com/HKUST-Aerial-Robotics/VINS-Fusion.
Google Scholar
46.
L. Stumberg, V. Usenko and D. Cremers, "Direct sparse visual-inertial odometry using dynamic marginalization", Proc. IEEE Int. Conf. Robot. Autom., pp. 2510-2517, 2018.
Show in Context View Article
Google Scholar
47.
V. Usenko, N. Demmel, D. Schubert, J. Stückler and D. Cremers, "Visual-inertial mapping with non-linear factor recovery", IEEE Robot. Autom. Lett., vol. 5, no. 2, pp. 422-429, Apr. 2020.
Show in Context View Article
Google Scholar
48.
V. Usenko and N. Demmel, 2019, [online] Available: https://gitlab.com/VladyslavUsenko/basalt.
49.
A. Rosinol, M. Abate, Y. Chang, L. Carlone and Kimera, 2019, [online] Available: https://github.com/MIT-SPARK/Kimera.
50.
A. J. Davison, [online] Available: https://www.doc.ic.ac.uk/ ajd/Scene/index.html.
Show in Context
51.
I. S, "Roumeliotis and A. I. Mourikis “Vision-aided inertial navigation", U.S. Patent, Sep. 2017.
Show in Context Google Scholar
52.
J. Civera, A. J. Davison and J. M. M. Montiel, "Inverse depth parametrization for monocular SLAM", IEEE Trans. Robot., vol. 24, no. 5, pp. 932-945, Oct. 2008.
Show in Context View Article
Google Scholar
53.
L. Clemente, A. J. Davison, I. D. Reid, J. Neira and J. D. Tardós, "Mapping large loops with a single hand-held camera", Proc. Robot.: Sci. Syst., Jun. 2007.
Show in Context CrossRef Google Scholar
54.
J. Civera, O. G. Grasa, A. J. Davison and J. M. M. Montiel, "1-point RANSAC for extended Kalman filtering: Application to real-time structure from motion and visual odometry", J. Field Robot., vol. 27, no. 5, pp. 609-631, 2010.
Show in Context CrossRef Google Scholar
55.
H. Strasdat, J. M. M. Montiel and A. J. Davison, Image Vis. Comput., vol. 30, no. 2, pp. 65-77, 2012.
Show in Context CrossRef Google Scholar
56.
H. Strasdat, J. M. M. Montiel and A. J. Davison, "Scale drift-aware large scale monocular SLAM", Robot.: Sci. Syst. VI, vol. 2, 2010.
Show in Context CrossRef Google Scholar
57.
H. Strasdat, A. J. Davison, J. M. M. Montiel and K. Konolige, "Double window optimisation for constant time visual SLAM", Proc. IEEE Int. Conf. Comput. Vis., pp. 2352-2359, 2011.
Show in Context View Article
Google Scholar
58.
X. Gao, R. Wang, N. Demmel and D. Cremers, "LDSO: Direct sparse odometry with loop closure", Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., pp. 2198-2204, 2018.
Show in Context View Article
Google Scholar
59.
S. H. Lee and J. Civera, "Loosely-coupled semi-direct monocular SLAM", IEEE Robot. Autom. Lett., vol. 4, no. 2, pp. 399-406, Apr. 2018.
Show in Context View Article
Google Scholar
60.
T. Lupton and S. Sukkarieh, "Visual-inertial-aided navigation for high-dynamic motion in built environments without initial conditions", IEEE Trans. Robot., vol. 28, no. 1, pp. 61-76, Feb. 2012.
Show in Context View Article
Google Scholar
61.
C. Forster, L. Carlone, F. Dellaert and D. Scaramuzza, "On-manifold preintegration for real-time visual-inertial odometry", IEEE Trans. Robot., vol. 33, no. 1, pp. 1-21, Feb. 2017.
Show in Context View Article
Google Scholar
62.
A. Martinelli, "Closed-form solution of visual-inertial structure from motion", Int. J. Comput. Vis., vol. 106, no. 2, pp. 138-152, 2014.
Show in Context CrossRef Google Scholar
63.
J. Kaiser, A. Martinelli, F. Fontana and D. Scaramuzza, "Simultaneous state initialization and gyroscope bias calibration in visual inertial aided navigation", IEEE Robot. Autom. Lett., vol. 2, no. 1, pp. 18-25, Jan. 2017.
Show in Context View Article
Google Scholar
64.
C. Campos, J. M. M. Montiel and J. D. Tardós, "Fast and robust initialization for visual-inertial SLAM", Proc. IEEE Int. Conf. Robot. Autom., pp. 1288-1294, 2019.
Show in Context View Article
Google Scholar
65.
E. Eade and T. Drummond, "Unified loop closing and recovery for real time monocular SLAM", Proc. 19th Brit. Mach. Vis. Conf., Sep. 2008.
Show in Context CrossRef Google Scholar
66.
R. Castle, G. Klein and D. W. Murray, "Video-rate localization in multiple maps for wearable augmented reality", Proc. 12th IEEE Int. Symp. Wearable Comput., pp. 15-22, Sep. 2008.
Show in Context View Article
Google Scholar
67.
C. Forster, S. Lynen, L. Kneip and D. Scaramuzza, "Collaborative monocular SLAM with multiple micro aerial vehicles", Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., pp. 3962-3970, 2013.
Show in Context View Article
Google Scholar
68.
L. Riazuelo, J. Civera and J. M. M. Montiel, "C2TAM: A cloud framework for cooperative tracking and mapping", Robot. Auton. Syst., vol. 62, no. 4, pp. 401-413, 2014.
Show in Context CrossRef Google Scholar
69.
J. G. Morrison, D. Gálvez-López and G. Sibley, "MOARSLAM: Multiple operator augmented rslAM" in Proc. Distrib. Auton. Robot. Syst., pp. 119-132, 2016.
Show in Context Google Scholar
70.
P. Schmuck and M. Chli, "Multi-UAV collaborative monocular SLAM", Proc. IEEE Int. Conf. Robot. Autom., pp. 3863-3870, 2017.
Show in Context View Article
Google Scholar
71.
P. Schmuck and M. Chli, "CCM-SLAM: Robust and efficient centralized collaborative monocular simultaneous localization and mapping for robotic teams", J. Field Robot., vol. 36, no. 4, pp. 763-781, 2019.
Show in Context CrossRef Google Scholar
72.
H. A. Daoud, A. Q. M. Sabri, C. K. Loo and A. M. Mansoor, "SLAMM: Visual monocular SLAM with continuous mapping using multiple maps", PLoS One, vol. 13, no. 4, 2018.
Show in Context CrossRef Google Scholar
73.
V. Lepetit, F. Moreno-Noguer and P. Fua, "EPnP: An accurate O(n) solution to the PnP problem", Int. J. Comput. Vis., vol. 81, no. 2, pp. 155-166, 2009.
Show in Context CrossRef Google Scholar
74.
S. Urban, J. Leitloff and S. Hinz, "MLPnP—A real-time maximum likelihood solution to the perspective-N-Point problem" in ISPRS Annal. Photogramm. Remote Sens. Spatial Inform. Sci., pp. 131-138, 2016.
Show in Context Google Scholar
75.
R. Mur-Artal and J. D. Tardós, "Fast relocalisation and loop closing in keyframe-based SLAM", Proc. IEEE Int. Conf. Robot. Autom., pp. 846-853, 2014.
Show in Context View Article
Google Scholar
76.
D. G. Lowe, "Distinctive image features from scale-invariant keypoints", Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
Show in Context CrossRef Google Scholar
77.
B. K. Horn, "Closed-form solution of absolute orientation using unit quaternions", J. Opt. Soc. Amer. A, vol. 4, no. 4, pp. 629-642, 1987.
Show in Context CrossRef Google Scholar
78.
J. Delmerico and D. Scaramuzza, "A benchmark comparison of monocular visual-inertial odometry algorithms for flying robots", Proc. IEEE Int. Conf. Robot. Autom., pp. 2502-2509, 2018.
Show in Context View Article
Google Scholar
79.
M. Burri et al., "The EuRoC micro aerial vehicle datasets", Int. J. Robot. Res., vol. 35, no. 10, pp. 1157-1163, 2016.
Show in Context CrossRef Google Scholar
80.
D. Schubert, T. Goll, N. Demmel, V. Usenko, J. Stückler and D. Cremers, "The TUM VI benchmark for evaluating visual-inertial odometry", Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., pp. 1680-1687, 2018.
Show in Context View Article
Google Scholar
81.
J. Sturm, N. Engelhard, F. Endres, W. Burgard and D. Cremers, "A benchmark for the evaluation of RGB-D SLAM systems", Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., pp. 573-580, 2012.
Show in Context View Article
Google Scholar
82.
D. Schubert, T. Goll, N. Demmel, V. Usenko, J. Stückler and D. Cremers, "The TUM VI benchmark for evaluating visual-inertial odometry", Mar. 2020.
Show in Context Google Scholar
83.
N. Yang, L. v. Stumberg, R. Wang and D. Cremers, "D3VO: Deep depth deep pose and deep uncertainty for monocular visual odometry", Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., pp. 1281-1292, 2020.
Show in Context View Article
Google Scholar
×
拖拽到此处
图片将完成下载