Yes, an AT-LP5 is everything you need to get going. Has a built in phone stage for MMs and the supplied cart is more than OK. (Not that I'd know as i broke mine before it touched a groove.)
Im using an AT33PGTii, a budget LOMC and a pair of Hashimoto SUTs so have to bypass the intermal phonostage (a switch). The arm can take cart upgrades if you decide to go down that route.
My £50 Nagaoka mp100 (or the supplied AT mm) is more than capable of producing very competent sounds. If i didnt already have the 33pgt i would still be using the Nag.