The fractal spectrum of magnetic minibands (Hofstadter butterfly), induced by the moiré superlattice of graphene on a hexagonal crystal substrate, is known to exhibit gapped Dirac cones. We show that the gap can be closed by slightly misaligning the substrate, producing a hierarchy of conical singularities (Dirac points) in the band structure at rational values ?=(p/q)(h/e) of the magnetic flux per supercell. Each Dirac point signals a switch of the topological quantum number in the connected component of the quantum Hall phase diagram. Model calculations reveal the scale-invariant conductivity ?=2qe2/?h and Klein tunneling associated with massless Dirac fermions at these connectivity switches.