In wireless communication, many resource allocation and management problems are related with optimization problems. In this dissertation, we consider nonconvex power and rate optimization problems in wireless networks such as DS-CDMA, OFDMA, cognitive network using convex relaxation. This dissertation deals with efficient numerical solutions for nonconvex optimization problems for a class of optimization problems in wireless communications systems. By using convex relaxation, we can approximate the solution of nonconvex optimization problems in wireless communication systems. In this dissertation, we considers a distributed utility maximization power control scheme in up-link DS-CDMA systems. Maximizing a utility function is solved by Lagrangian dual decomposition. Numerical result shows that the proposed algorithm achieves around 95% of the optimal utility maximization with a modest computational burden. Also, we considers a sum-rate maximizing power allocation problem under Gaussian cognitive multiple-access channel (MAC) environment, where primary and secondary users may communicate under mutual interference. Formulating the problem as a standard nonconvex quadratically constrained quadratic problem (QCQP) provides a simple method to find a solution using semidefinite relaxation (SDR). Numerical results show that the solution achieves the similar performance of the exhaustive search, within polynomial time. In addition, we considers a novel auction algorithm for subchannel allocation using the difference of throughput among subchannels to allow users to compete through bidding in OFDMA system. The algorithm we proposed can achieve a competitive fair subchannel allocation through auction mechanism. Numerical result shows that the proposed algorithm has the similar performance with sum rate maximization and guarantees fairness by minimizing user``s potential throughput loss.