Is the bike new? How long between oil changes? Rub the "shavings" between your fingers. Are they smaller particles stuck together with the magnet, or large ones?
If they were small, and you don't change the oil every ride, I wouldn't worry too much.
Agreed. I generally have a few fine particles that come out with every oil change. More so the first few rides, or if you abuse the clutch heavily and don't change the oil.