We implemented a single-walled carbon nanotube (SWCNT)-based saturable absorber (SA) that evanescently interacts with light on a side-polished polarization-maintaining fiber (PMF) platform. A high-quality SWCNT/polymer composite was spin-coated onto a side-polished PMF embedded in a quartz block, where the slow optical birefringence axis of the PMF was perpendicular to the polished surface. The fabricated in-line SA had an insertion loss of -1.5 dB with a modulation depth of 0.5% for the polarization direction associated with the transvers magnetic mode. An all-PMF laser was built using the fabricated in-line SA to demonstrate self-starting, turn-key operation of Er-doped, all-fiber lasers delivering stable scalar soliton pulses. The measured 3-dB spectral bandwidth and pulse duration of the laser output were 5.1 nm and 510 fs, respectively, resulting in a time-bandwidth product of 0.320. We examined the polarization state of the soliton fiber laser output and found that the laser stably generated linearly polarized ultrashort pulses with a polarization extinction ratio of -18.4 dB.