Fluorescence live-cell microscopy is one of the most frequently used techniques to study dynamic processes in organoids. However, it is often limited by laborious fluorescent reporter engineering, limited numbers of fluorescence channels, and adverse phototoxicity and protein overexpression effects. Label-free imaging is a promising alternative but not yet established for 3D cultures. Here, we introduce LabelFreeTracker, a label-free machine-learning-based method to visualize the nuclei and membranes in bright-field images of 3D mouse intestinal organoids. The approach uses U-Net neural networks trained on the bright-field transmitted light and fluorescence images of mouse intestinal organoids as obtained by standard confocal microscopy. LabelFreeTracker frees up fluorescence channels to study fluorescent reporters and allows (semi-)automated quantification of cell movement, cell shape and volume changes, proliferation, differentiation, and lineage trees. This method greatly simplifies live-cell imaging of tissue dynamics and will accelerate screening of patient-derived organoids, for which reporter engineering is not feasible.