Vision-Based Navigation and Large-Scale Estimation for Spacecraft Swarms

Citation

Matsuka, Kai (2023) Vision-Based Navigation and Large-Scale Estimation for Spacecraft Swarms. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/spf8-8p84. https://resolver.caltech.edu/CaltechTHESIS:06022023-235550987

Abstract

There has been dramatic growth in the space industry over the past 20 years. Around the same time, robotics and autonomy research has advanced significantly, resulting in a plethora of new mission concepts employing autonomy, such as on-orbit inspection, mission extension, space structure assembly, and orbital debris removal becoming within the realm of possibility. Two of the key autonomous technologies that are critical to the success of these missions are (1) advanced coordination of multi-agent systems and (2) robust vision-based navigation for on-orbit servicing in close proximity. However, there are challenges to simply applying the existing technology to space systems. First, there are domain-specific challenges that are unique to space, such as orbital mechanics and harsh lighting conditions. Second, even at a theoretical level, previous works in the controls and robotics literature are limited when applied to large-scale, locally coupled systems such as spacecraft swarms. To this end, this thesis develops novel algorithms for addressing these gaps.

In the first part of the thesis, we present a decentralized, scalable algorithm for swarm localization, called the Decentralized Pose Estimation (DPE) algorithm. With the DPE algorithm, each spacecraft computes relative navigation estimates with respect to others in the swarm but achieves higher performance through the benefit of multi-agent coordination. The DPE algorithm considers both communication and relative sensing graphs and defines an observable local formation. Each spacecraft jointly localizes its local subset of spacecraft using direct and communicated measurements. Since the algorithm is local, the algorithm complexity does not grow with the number of spacecraft in the swarm. As part of the DPE, we present the Swarm Reference Frame Estimation (SRFE) algorithm, a distributed consensus algorithm to co-estimate a common Local-Vertical, Local-Horizontal frame. The DPE combined with the SRFE provides a scalable, fully-decentralized navigation solution that improves the estimation accuracy compared to when without multi-agent coordination. Numerical simulations and experiments using Caltech's robotic spacecraft simulators are presented to validate the effectiveness and scalability of the DPE algorithm. We show that DPE has much higher accuracy than the best possible estimate without any coordination, while simultaneously being scalable to an arbitrarily large number of agents.

In the second part of the thesis, we propose a novel computer vision algorithm to track the pose of an unknown and uncooperative target using multiple decentralized observers. Vision-based pose determination of an unknown target is challenging due to factors such as lack of cooperative visual markers and harsh lighting conditions of space, and the problem is even harder for distributed observers. To address this challenge, we develop the algorithm called the Multi-Spacecraft Simultaneous Estimation of Pose and Shape algorithm or MSEPS. Within MSEPS, a team of chaser spacecraft, each equipped with a monocular camera, exchange information over a local network to jointly estimate the relative kinematic state of the target and its sparse shape landmarks. In this approach, each spacecraft processes its images and extracts its own set of visual keypoints in parallel. Then, the team uses the local network to jointly estimate the target pose and shape in a distributed fashion by applying the consensus algorithm over the inter-spacecraft communication links. To the best of the authors' knowledge, this is the first cooperative vision-based algorithm for estimating the pose and shape of a space object by means of an arbitrary number of spacecraft. We validate our algorithm using simulations of relative orbits and observations captured by each chaser spacecraft and show the multiple observers successfully agree on a consistent estimate and track the target pose accurately.

In the third part of the thesis, we develop some new simulation tools that bridge the gap between robotics and space technology. When developing robotics algorithms for on-orbit systems such as DPE and MSEPS, we identified a need for new simulation tools that tightly integrate robotics algorithms with high-fidelity models of space environments such as astrodynamics effects and visual conditions. To this end, we first develop a ROS2-compatible software interface for Basilisk, the open-source astrodynamics simulation software. This tool allows running Basilisk in parallel with ROS2 in real-time and translates messages between Basilisk modules and ROS2 modules, such that control algorithms implemented in ROS2 can interact with the high-fidelity dynamics within Basilisk in a closed-loop fashion. Second, we develop a ROS2-compatible camera simulation module that uses the Neural Radiance Fields (NeRF) to rapidly generate novel images. These synthetic images are used as inputs to validate the vision-based navigation algorithm in a closed-loop fashion. To validate these simulation tools, we also developed a set of autonomous algorithms for on-orbit inspection and use the simulated measurements as inputs to the algorithm. The real-time numerical simulations demonstrate that our tools can be integrated with autonomy algorithms implemented in ROS2 in a closed-loop fashion to validate the feasibility of the mission.

In the process of addressing some lessons learned from DPE and MSEPS works, we identified that there is a gap in general frameworks for solving the optimal estimation problems for probabilistic inference of large-scale problems involving networked systems. This gap is not just applicable to spacecraft swarms, but also to a general class of large-scale, multi-agent problems in robotics and controls such as localization and mapping, wireless sensor networks, and electrical power grids. Therefore, in the fourth part of the thesis, we address this fundamental gap by developing novel algorithms for Distributed Factor Graph Optimization (DFGO) problems that arise in large-scale networked systems. We develop algorithms for both batch and real-time problems. First, for the batch DFGO problem, we derive a type of the Alternating Direction Method of Multipliers (ADMM) algorithm called the Local Consensus ADMM (LC-ADMM). LC-ADMM is fully localized; therefore, the computational effort, communication bandwidth, and memory for each agent scale like O(1) with respect to the network size. We establish two new theoretical results for LC-ADMM: (1) exponential convergence when the objective is strongly convex and has a Lipschitz continuous subdifferential, and (2) o(1/k) when the objective is convex and has a unique solution. We also show that LC-ADMM allows the use of non-quadratic loss functions, such as l1-norm and Huber loss. Second, we also develop the Incremental DFGO algorithm (iDFGO) for real-time problems by combining the ideas from LC-ADMM and the Bayes tree. To derive a time-scalable algorithm, we exploit the temporal sparsity of the real-time factor graph and the convergence of the augmented factors of LC-ADMM. The iDFGO algorithm incrementally recomputes estimates when new factors are added to the graph and is scalable with respect to both network size and time. We validate LC-ADMM and iDFGO in simulations with examples from multi-agent Simultaneous Localization and Mapping (SLAM) and power grids.

Thesis Files