One-Shot Generalization in Offline Reinforcement Learning with WSAC-N
M.D.I. Museur (TU Delft - Electrical Engineering, Mathematics and Computer Science)
M.R. Weltevrede – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
M.T.J. Spaan – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
E. Congeduti – Graduation committee member (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Recent work has shown that offline reinforcement learning (RL) does not generalize well to new environments compared to behavioral cloning (BC). We propose WSAC-N, an ensemble model of soft actor-critics with weights to de-emphasize actions with high variance. We compare the zero-shot generalization abilities of WSAC-N with the baseline BC in a four-room maze-like environment, testing on unseen tasks. Our findings indicate that WSAC-N has worse zero-shot generalization compared to BC, aligning with previous work. Additionally, we investigate the impact of dataset characteristics on generalization, finding that dataset size has a negligible impact, while the quality of trajectories generally has a positive effect. These results are consistent with prior research.