The $\mu$-bases of rational curves/surfaces are newly developed tools which play an important role in connecting parametric forms and implicit forms of the rational curves/surfaces. They provide efficient algorithms to implicitize rational \linebreak curves/surfaces as well as algorithms to compute singular points of rational curves and to reparametrize rational ruled surfaces. In this paper, we present an efficient algorithm to compute the $\mu$-basis of a rational curve/surface by using polynomial matrix factorization followed by a technique similar to Gaussian elimination. The algorithm is shown superior than previous algorithms to compute the $\mu$-basis of a rational curve, and it is by far the only known algorithm that can rigorously compute the $\mu$-basis of a general rational surface. We present some examples to illustrate the algorithm.