The effect of nanobump structure (NS) constructed with molybdenum oxide (MoO3) covering aerosol-derived Ag nanoparticles on organic solar cells is systematically explored by varying the MoO3 thickness. The amount of enhanced light absorption by the NS-induced scattering and plasmonic resonance effects increases with decreasing MoO3 thickness. Meanwhile, the oval-shape NS with fully enclosed Ag nanoparticles by MoO3 leads to significantly improved carrier extraction due to decreased ohmic loss, while the NS with partially covered Ag nanoparticles by thin MoO3 degrades device performance arising from increased recombination and leakage loss. Thus, the optimal MoO3 thickness occurs around 20 nm for 40 nm sized nanoparticles where the enhanced carrier generation and the improved carrier extraction by the NS effectively contribute to the improvement in photocurrent. Therefore, our analysis on carrier dynamics of plasmonic organic solar cells incorporating the NS would provide a clear guideline for optimizing device performance.