J. Guo
Please Note
73 records found
1
Reinforcement learning has proven its power on various occasions. However, its performance is not always guaranteed when system dynamics change. Instead, it largely relies on users’ empirical experience. For reinforcement learning algorithms with an actor–critic structure, the critic neural network reflects the approximation and optimization process in the RL algorithm. Analyzing the performance of the critic neural network helps to understand the mechanism of the algorithm. To support systematic interpretation of such algorithms in dynamic control problems, this work proposes a critic match loss landscape visualization method for online reinforcement learning. The method constructs a loss landscape by projecting recorded critic parameter trajectories onto a low-dimensional linear subspace. The critic match loss is evaluated over the projected parameter grid using fixed reference state samples and temporal-difference targets. This yields a three-dimensional loss surface together with a two-dimensional optimization path that characterizes critic learning behavior. To extend analysis beyond visual inspection, quantitative landscape indices and a normalized system performance index are introduced, enabling structured comparison across different training outcomes. The approach is demonstrated using the Action-Dependent Heuristic Dynamic Programming algorithm on cart–pole and spacecraft attitude control tasks. Comparative analyses across projection methods and training stages reveal distinct landscape characteristics associated with stable convergence and unstable learning. The proposed framework enables both qualitative and quantitative interpretation of critic optimization behavior in online reinforcement learning.
The Earth Observation (EO) sector is rapidly evolving into Earth Action (EA), with onboard Artificial Intelligence (AI) processing emerging as a key enabler. This technology offers strategic advantages through 1) enabling autonomous and low-latency EO missions with adaptive data processing capabilities that overcome the limitations of ground-based post-processing in handling the vast data volumes produced by growing satellite constellations and 2) supporting the evolution toward AI-driven and distributed EO mission architectures. This review explores the key technological advancements, mission architectures, and emerging paradigms that are shaping the next-generation EO systems. Pioneering EO missions are presented to showcase current capabilities, while the commercial, technical, and operational implications are analyzed alongside key challenges and ongoing research efforts (January 2020 to March 2025) aimed at enabling Real-Time (RT) Earth system intelligence.
This paper introduces a motion planning method for capture of tumbling objects using a free-floating space robot. The proposed approach incorporates an improved Rapidly Exploring Random Tree Star (RRT*) algorithm enabling obstacle avoidance and generating desired trajectories for the robot's end-effectors. Additionally, a multi-layer optimization process and a greedy policy are proposed to achieve singularity avoidance, joint velocity, and acceleration optimization by leveraging the robot arm's joint energy distribution, torque, and manipulability. By adopting this motion planning strategy, the space robotic system demonstrates improved performance in obstacle and singularity avoidance, without the need for inverse Jacobian matrix calculations. Furthermore, the multi-layer optimization process enhances trajectory smoothness and reduces end-effector vibration through energy and torque optimization. This research contributes to advancing space robotic systems by enhancing the entire energy and torque consumption on motion planning to make the end-effector move smooth and reduce the vibration.
Inaccuracies in robotic arms can significantly hinder their performance in tasks where precision is critical. This paper focuses on the kinematic calibration and elasticity compensation of the six degrees of freedom robotic arm integrated into the Lunar Rover Mini, developed in collaboration with the Robotics and Mechatronics Institute of the German Aerospace Center (DLR), Wessling. The arm, constructed using 3D-printed components and driven by affordable RC servo motors, experiences notable inaccuracies in end-effector positioning due to joint flexibility and structural deformation, especially under load. A model-based calibration technique is proposed to compensate for elastic deformations and geometric misalignments, addressing the absence of feedback sensors. This cost-effective approach, which requires only 3D measurements of the end-effector’s position, has resulted in an approximately 80% reduction in the robotic arm’s average position error.
AltiCube+
A low-cost long fixed-baseline radar altimeter solution based on cubesats on-orbit assembly
Free Space Optical Communications with Multi-Beam Laser Terminals for Satellites
Design Insights and Applications
Traditional laser communication terminals are limited to point-to-point links, which constrains their scalability and flexibility for global networks that require simultaneous connections with multiple targets. While multiple single-beam terminals can expand capacity, this approach multiplies Size, Weight, Power, and Cost (SWaPC), limiting scalability. Multi-beam laser communication terminals offer a promising alternative, though the design of an effective beam steering system remains a key challenge. This paper explores the design process of such a system, providing an overview of multi-beam steering literature as well as an comparison and trade-off of existing space-borne multi-beam steering technologies. It also analyzes insights from related fields such as terrestrial laser communications, LiFi, single-beam laser communication, optical cross-connects, radio links, and multiple target tracking. Key system functions are identified and visualized in a function flow diagram, and various design options are evaluated, culminating in a design options tree which serves as a design recipe. Two application scenarios, involving high and low target densities, demonstrate that steering systems based on micro-mirror arrays and spatial light modulators present significant advantages over alternatives. This study offers a comprehensive framework for designing multi-beam steering systems for space-based laser communication terminals.
Robotics facilities have a long history in the development of space equipment, since they allow to perform tests on systems like guidance, navigation and control, visual-based navigation and docking mechanisms. Those facilities are based on two manipulators, one representing the a target satellite, the other the chaser satellite which perform a relative motion with respect to the first. This approach has been used in the past to perform tests on docking operations, visual-base navigation system to populate databases. Delft University of Technology recently developed its own robotics facility for GNC and multisatellite systems applications. It hosts two robots on a moving base, which work in synergy to extend the operational space. They operate in a dark environment, where there are lights to simulate the sun disturbance and a beamer that projects the Earth to have a representative background. The purpose of this paper is to describe the laboratory, along with the control architecture of the robots and provide some tests executed to assess the accuracy of them in tracking a given trajectory.
Steering multiple laser beams using spatial light modulators (SLMs) creates unwanted diffraction and reflections that are not modulated by the SLM, which can make beam tracking difficult. A novel, to the best of our knowledge, and simple beam steering methodology is proposed, which aims at reducing the influence of this clutter while maintaining tracking performance. The beam(s) are deliberately defocused before steering with a superposition of a phase ramp and Fresnel lens (PRFL) phase screen on the SLM. As a result, the non-modulated reflections and diffracted light are decreased in relative intensity to the steered beam, in turn allowing simple and standard peak intensity and center of gravity (CG) algorithms for tracking. Hardware demonstration shows tracking performance using the PRFL remained on-par with more complex filtering approaches while adding no additional hardware. This method has potential to improve the communication performance of multi-beam laser communication terminals.
AltiCube+
A Low-Cost Long Fixed-Baseline Radar Altimeter Solution Based On CubeSats On-Orbit Assembly
Radar interferoinetry can be used to obtain sub-kilometer resolution over a swath at the expense of additional transmit power and a sufficiently long baseline to accommodate at least two antennas. This paper reports an innovative concept called AltiCube+, a low-cost long fixed-baseline interferometric radar altimeter based on CubeSats on-orbit assembly. The AltiCube+ concept consists of multiple 16U CubeSats, After an early operation and commissioning phase, these CubeSats will perform autonomous rendezvous and docking with each other via deployable booms to establish a long fixed-baseline, and then deploy antennas for an interferometric altimeter configuration. The uniqueness of AltiCube+ is on the potential scientific opportunities brought by two left and right looking interferometric altimeters with around 6 meter baseline (total system length is more than 8 m) and the sustainability due to its significantly low cost and short development lifecycle. If budget allows, multiple AltiCube+ systems with same or different altimetry capabilities can form a constellation to dramatically reduce the revisit time and, therefore, provide much better spatiotemporal coverage.
We will illustrate some experiences and projects presented at the VITE I conference in the field of extended reality technologies for public engagement and skill development. Extended technologies have many benefits when applied to educational contexts, providing immersive and engaging learning spaces, enhancing the sense of perception and grasp to the students, and thus improving the process of learning and motivation to access to science. Many speakers at the conference underlined that the lack of interest towards scientific topics and the difficulty in understanding science were the main problems they commonly encountered at public events, especially on the part of the young public. The use of extended realities can help overcome these problems, and is also effective for professional training in risky situations providing “a training platform that can be used multiple times for training, without worrying about the cost, availability, risk, and complexity of the equipment or system” (Doolani et al. (2020)). Moreover, when innovation generated by research responds to the need of the production companies, especially SMEs, the knowledge transfer generates a virtuous path, that stimulate the developement of our Country.
Humans are embarking on a new era of space exploration with the plan of sending crewed spacecraft to the Moon, Mars, and beyond. Extravehicular activities (EVAs) will be an essential part of the scientific activities to be carried out in these missions, and they will involve extensive geological fieldwork. These EVAs entail many challenges as real-time support from ground control cannot be provided to astronauts. Hence, new human–machine interfaces are urgently needed to enhance mission autonomy for astronauts and reduce ground communication dependability for real-time operations. This study introduces an Augmented Reality (AR) Internet of Things tool for astronauts to carry out geological activities. It proposes a theoretically-informed user-centred design method supported by expert feedback and an evaluation method. The tool was assessed via questionnaires and semi-structured interviews with European Space Agency (ESA) astronauts and geological field activities experts. Content analysis of the interviews revealed that user satisfaction was the first most mentioned (32% of 139 quotes) usability aspect. Key design factors identified were: displaying solely important information in the field of view while adjusting it to the user's visual acuity, easy usage, extensibility, and simplicity. User interaction was the second most mentioned (24% of 139 quotes) usability aspect, with voice seen as the most intuitive input. Finally, this research highlights important factors determining the usability and operational feasibility of an AR tool for analogue training missions and provides a foundation for future design iterations and an eventual integration of AR into the spacesuit's visor.
Using robotic arms to capture space debris is a promising method in active debris removal. Since most space debris is uncooperative target, uncertainties exist in the inertial parameters of the combined spacecraft after target capture. In this paper, an attitude takeover control method based on adaptive dynamic programming is investigated for the postcapture combined spacecraft. The controller requires no inertial information. Firstly, the dynamic and kinematic model of combined spacecraft is established. Then, the learning-based control, action-dependent heuristic dynamic programming with actor-critic structure is introduced. Finally, the algorithm is tested with the benchmark cart-pole system and the combined spacecraft. Simulation results show that the inertia-free control method is sensitive to the learning parameters and initial weight of the actor and the critic, which will bring problems in practical use.
This article focuses on the validation of a classical PID controller scheme for flexible spacecraft with regards to the effect of parameter uncertainty on system stability and pointing precision. A high-fidelity simulation environment with external disturbances was built in Simulink using a control-oriented model of an Earth-observing satellite with a flexible appendage and on-board microvibration sources in orbit around the planet. Then, a PID control loop was designed with sensor dynamics, time delay behaviour, and a smooth trajectory generator. After declaring the natural frequencies, damping ratio, and rotation angle of the appendage, as well as the propellant tank mass to be uncertain, two worst-case scenarios were identified. Comparing the response of worst-case systems with nominal settings, only a minor drop has been found in the phase margins, with little to no difference in the pointing errors (smaller than ±2 arcsec for both roll and pitch).
The purpose of this research was to investigate the suitability of the Fused Filament Fabrication (FFF) process for low pressure/vacuum environment. This included investigating the ability of an FFF printer to function in a vacuum and evaluation of the dimensional accuracy and mechanical properties of the manufactured components. For this purpose, a commercially available FFF printer using polycarbonate as raw material was placed in a vacuum environment of 10 mbar. Test components were then fabricated in vacuum with a control group fabricated in a normal atmosphere (1 bar). Test components were evaluated for dimensional and mass accuracy, quality and presence of defects. Flexural, tensile and compressive testing was carried out according to ASTM D790, D638 and D695 respectively. Dimensional analysis of components showed equivalent small deviation for both environments. Components fabricated in the vacuum environment had 5.4% higher tensile yield strength and 59% higher extension at break compared to components printed in a normal atmosphere indicating an increased strength and ductility. Components tested in compression had approximately 11.2% higher compressive strength when printed in a vacuum environment. No differences were observed during the flexural test. In space, due to the vacuum environment, polymers and organic material are susceptible to release molecules via an outgassing process. Assessment of the molecular organic contamination generate during the printing process in vacuum is low and seems to mostly originated from the components of the printer. The results provided demonstrated the possibility to use the FFF process in a vacuum environment to fabricate dimensionally accurate, high-quality polycarbonate components with a variety of geometries without loss of mechanical performance. This work provides a proof of concept that FFF can be used to develop out-of-earth manufacturing technologies (in orbit/in space/on planet) allowing part production for new maintenance and repair strategy or to potentially manufacture entire structure more efficiently overpassing launch constrain by using only raw material brought from earth.
CubeSat Altimeter Constellation Systems
Performance Analysis and Methodology
Multiple CubeSat altimeters can work independently or corporately to form altimeter constellations. Different configurations of the constellations can acquire distinguished advantages: improved spatial/temporal sampling and high cross-track resolution, which will be helpful for observations of oceanic small-scale structures and weather forecasting. Compared to single conventional altimeters, CubeSat altimeter constellations may achieve better performances with lower costs. To fully understand these systems, this article focuses on the performance analysis and methodology for CubeSat altimeter constellations. Besides the typical analyses of the resolution, revisit, and absolute sea surface height (SSH) accuracy, the performance analysis was conducted by considering the characteristics of multiple measurements provided by CubeSat altimeter constellations. Local and global spatial sampling performances are investigated for various constellations and compared by sampling density and swath size. Moreover, relative SSH accuracy is introduced and evaluated based on the spatial structure functions of errors to effectively evaluate the measurement performance. Related system requirements on power, delta-v, etc., to achieve the performance are also discussed, which ensures that the analysis fits the boundary conditions of implementation. Finally, different concepts of the CubeSat altimeter constellations are compared, where their limitations and possible solutions are also discussed.
The tethered formation system has been widely studied due to its extensive use in aerospace engineering, such as Earth observation, orbital location, and deep space exploration. The deployment of such a multitethered system is a problem because of the oscillations and complex formation maintenance caused by the space tether's elasticity and flexibility. In this article, a triangle tethered formation system is modeled, and an exact stable condition for the system's maintaining is carefully analyzed, which is given as the desired trajectories; then, a new control scheme is designed for its spinning deployment and stable maintenance. In the proposed scheme, a novel second-order sliding mode controller is given with a designed nonsingular sliding-variable. Based on the theoretical proof, the addressed sliding variable from the arbitrary initial condition can converge to the manifold in finite time, and then sliding to the equilibrium in finite time as well. The simulation results show that compared with classic second sliding-mode control, the proposed scheme can speed up the convergence of the states and sliding variables.