If a ball bearing fell out of your fuel pump i suspect you would notice it, by the sound of the pump (if it would run at all).
However I would remove the tank and the fuel pump for inspection, just to be safe.
[edit]
It is not a complete bearing? With inner and outer races? Only a single ball?