kitti_metrics_eval
CLI tool to evaluate the KITTI odometry bechmark metrics to trajectory files
Links
README
mola_input_kitti_dataset
CLI tool to evaluate the KITTI odometry benchmark metrics to arbitrary trajectory files,
in both, kitti
and tum
formats. Better used together with evo.
Build and install
Refer to the root MOLA repository for compilation instructions.
To install from the ROS repositories:
sudo apt install ros-${ROS_DISTRO}-mola-metrics-eval
This program is largely based on the public Kitti dataset evaluation C++ code. Rewritten to use Eigen instead of GNU GPL’d code, and to add additional features. Original source code notice:
###########################################################################
# THE KITTI VISION BENCHMARK SUITE: VISUAL ODOMETRY / SLAM BENCHMARK #
# Andreas Geiger Philip Lenz Raquel Urtasun #
# Karlsruhe Institute of Technology #
# Toyota Technological Institute at Chicago #
# www.cvlibs.net #
###########################################################################
Documentation
Basically, this CLI program can be used for evaluating the KITTI metrics in two ways:
To estimated trajectories of the KITTI odometry sequences, stored in the TUM format (unlike the original KITTI dev kit, which uses the kitti format). In this case, the program reads the ground truth sequences and the calibration files from a user’s local copy of the KITTI dataset. Calibration data is used to first transform the user’s input trajectory to the
cam0
frame, in which the KITTI ground truth paths are given.To any other pair of trajectory files for other datasets, i.e. a ground truth and an estimated trajectory file.
USAGE:
kitti-metrics-eval [--no-figures] [--gt-tum-path <trajectory_gt.txt>]
[-s <01>] ... [--save-as-kitti <result.kitti>] -r
<result.txt> [-k <>] [--] [--version] [-h]
Where:
--no-figures
Skip generating the error figures
--gt-tum-path <trajectory_gt.txt>
If provided, the --sequence flag will be ignored and this particular
file in TUM format will be read and used as ground truth to compare
against the resulting odometry path.
-s <01>, --sequence <01> (accepted multiple times)
The sequence number of the path(s) file(s) to evaluate, used to find
out GT and calibration files for the Kitti dataset.
--save-as-kitti <result.kitti>
If given, will transform the input path from the LIDAR frame to the
cam0 frame and save the path to a TXT file in the format expected by
KITTI dev kit.
-r <result.txt>, --result-tum-path <result.txt>
(required) File to evaluate, in TUM format
-k <>, --kitti-basedir <>
Path to the kitti datasets. Overrides to the default, which is reading
the env var `KITTI_BASE_DIR`.
--, --ignore_rest
Ignores the rest of the labeled arguments following this flag.
--version
Displays version information and exits.
-h, --help
Displays usage information and exits.
Examples of usage
Evaluate the KITTI metrics on a solution by your SLAM method
Evaluate on all KITTI test sequences 00-10.
kitti-metrics-eval \
-r results/estim_%02i.txt \
-s 00 -s 01 -s 02 -s 03 -s 04 -s 05 -s 06 -s 07 -s 08 -s 09 -s 10 \
--kitti-basedir /path/to/your/local/kitti_dataset
Expected tree layout under the kitti_dataset
directory:
/path/to/your/local/kitti_dataset
├── poses
└── sequences
Evaluate the KITTI metrics on another dataset
Evaluate on another arbitrary dataset, with ground truth trajectory given in the TUM format:
kitti-metrics-eval \
-r results/my_solution.txt \
--gt-tum-path /path/to/ground_truth.txt
Transform a KITTI solution file in TUM format to KITTI
kitti-metrics-eval \
-r results/estim_%02i.txt \
-s 00 \
--save-as-kitti results/00.txt
License
This package is released under the BSD-3-clause license.