Sound field control using multiple loudspeakers requires prior information on the loudspeaker layout and user locations. If such information can be automatically identified loudspeaker itself, more flexible control of sound fields would be realizable in many practical systems. This work deals with the development of a wireless audio system with self-configuration and user-sensing abilities. It is assumed that each loudspeaker includes single microphone and are connected via asynchronous wireless network. Under this circumstance, the problem of identifying the loudspeaker layout is tackled using high frequency probing signals embedded in the multichannel audio tracks. Once the synchronization and layout identification is made, the proposed system attains the ability to track user activities by utilizing acoustic scattering waves from a human body. A prototype system has been built on a wired loudspeaker network with simulated time mismatch, which shows that the intelligent loudspeaker system with real-time sensing and tracking functionality is practically realizable.