We show that in the standard model the parametrically leading (by a factor 1/alpha(s)) contribution to the inclusive CP asymmetry in (B) over bar -> X-s,X-d gamma decays arises from a long-distance effect in the interference of the electromagnetic dipole amplitude with the amplitude for an up-quark penguin transition accompanied by soft gluon emission. Using model estimates for the associated hadronic parameter (A) over tilde (u)(17), we predict a value in the range -0.6% < ASM(Xs gamma)(SM) < 2.8%. In view of current experimental data, a future precision measurement of the flavor-averaged CP asymmetry would signal the presence of new physics only if a value below -2% was found. A cleaner probe of new physics is offered by the difference of the CP asymmetries in charged versus neutral (B) over bar -> X-s gamma decays.