Ferromagnetic honeycomb systems are known to exhibit a magnonic topological phase under the existence of the next-nearest-neighbor Dzyaloshinskii-Moriya interaction (DMI). Motivated by the recent progress in the sublattice-specific control of magnetic anisotropy, we study the topological phase of magnon bands of a honeycomb ferromagnetic monolayer and a bilayer with sublattice symmetry breaking due to the different anisotropy energy in the presence of the DMI. We show that there is a topological phase transition between the topological magnon insulator and the topologically trivial magnon phase driven by the change of the relative size of the DMI and the anisotropy differences between the sublattices. The magnon thermal Hall conductivity is proposed as an experimental probe of the magnon topology.