I needed to add transitions to the Animator component, or set the 'Animator Controller' field to null.
Animation tracks on the Timeline play Animation sequences on Animators in the scene. If the Animator has an Animator Controller asset selected, this normally overrides the state machine flow of the Animator component. However, when the Timeline pauses, control returns to the Animator. If the Animator doesn't include any transitions to the animation state that was playing when the Timeline paused (meaning the Animator on its own would never reach that state), it will revert to the default state.

In this example, there is no transition from Part 1 to Part 2 or Part 3. We can use this Animator on a Timeline, and play any of the three animations from the Timeline, but if the Timeline ever pauses, the Animator will immediately revert to "Part 1".
Note that if we have an Animator Controller selected in the Animator component, pausing the Timeline will return control to the Animator, which may cause the Animator Controller to advance to a new state and continue playing animations that you don't want it to play. If you don't want the Animator to do anything when the Timeline is paused, set the 'Animator Controller' to null. The Timeline can still play animations on the Animator when the 'Animator Controller' field is null.