We study the asymptotic behavior of dispersing solutions to the Vlasov-Poisson system. Due to long interaction range, we do not expect linear scattering (Choi S-H and Ha S-Y 2011 SIAM J. Math. Anal. 43 2050-77). Instead, we prove a modified scattering result (or long range scattering result) of small and dispersing solutions. We find a quasi-free forward trajectory so that along the trajectory, the solution has an asymptotic limit. We extract the logarithmic growth part of the Duhamel term, and absorb it into the quasi-free trajectory, then the remaining part enjoys faster decay so as to obtain the asymptotic state