J-head fine feed disassembly help

I am trying to do a basic external teardown on my J-head so I can splash on a bit of paint to match the rest of the machine. In a nutshell, not trying to do unnecessary work as the machine works fine and is quiet as-is. As you hopefully can see in the pics, Ive got it mostly apart. Unfortunately, Im a bit stuck with removing the handwheel clutch (113). Both the setscrew (6) and the screw and spring (116 & 174) opposite were easily removed but the 3/16" ball bearing (173) refuses to come out. Am I wrong to think the ball bearing should simply fall out without the screw and spring holding it in, thus allowing the handwheel clutch (113) to simply pull off?

Any/all help is greatly appreciated!

http://www.truetex.com/bridgeport-manual.pdf