This paper proposes a novel cross-layer packet prioritization scheme that overcomes the limitations of conventional video prioritization schemes for IPTV system. In the conventional schemes, the Extend profile or frame unit is used for prioritization and these incur limited uses of the video prioritization schemes or unreliable transmission of the P-frame and B-frame headers. In order to overcome these limitations, the proposed scheme uses the MPEG-2 system information and the H.264/AVC NAL header information to prioritize the important video packets including the P-frame and B-frame headers. The experimental results using a testbed demonstrated that the proposed scheme has a significant performance enhancement over the conventional video prioritization schemes in terms of packet utilization ratio for video streaming, direct frame loss ratio, and PSNR.