Small transportable Bluetooth- and smart/AI speakers are becoming an increasing part of many households. Due to their small size, these speakers have limited output at low frequencies, and typically have a vented port or in some cases a passive radiator to improve the low-frequency performance. In this work, we show how a density based material optimization approach can be used to optimize the frequency response of the system. The passive radiator is placed at the top of a cylinder with a down-firing speaker, driven with a voltage source, which, via a lumped model, is connected to a multi-physics finite element framework. The speaker radiates sound into an unbounded domain, which is realized by using perfectly matched layers. The performance of the speaker is based on a numerical measurement 1 meter away from the speaker. Several optimization results are shown and these are compared with a more generic type of small speaker system.