Leslie is grappling (sorry) with this question, as am I. (See her blog post here.)
My sense is most guys measure progress by looking at who they tap to and who taps them. It's easy, I think, to evaluate readiness to be promoted by looking at abilities to dominate others of your own belt level and ability to compete with the next rank up.
If you're female and not privileged to be training somewhere with lots of other chicks, you probably don't use these metrics. Or do you? I don't, usually, because then I'd be depressed. Even whitebelts with 3 months of training can at least stalemate me if they're male. I used to use Leila as one of my metrics, but it's tough because she's improving faster than I am, so I'm pretty much perpetually frustrated rolling with her. I can only hope I'm occasionally stymieing her too.
Sometimes I measure success by single positions. Am I getting armbarred in guard, passing scissor, remembering sweeps? Sometimes I measure it by length of time (emotional time!) before I tap. What do I mean by emotional time? I'm obviously not really counting the seconds before a tap, nor am I trying to hold out without tapping to a well-set sub. I just mean I can get a good feel for others' difficulty level-- is this someone I will be happy to defend against and call it a victory if they can't sub me till the end of a 6 min round? will I be happy if I get through half? a minute?
Sometimes it's my comfort level with bigger picture perspectives-- am I chaining attacks? flowing? how's my hip movement? Sometimes I measure my progress by the kinds of moves my opponent pulls-- are they taking it really easy on me? less easy? whoa, did they just grunt? Sometimes if they grunt, sweat, or breathe hard, that's a sign I'm doing well. But ultimately I prefer quantifiable measures, things that are numeric or objective vs. subjective and "warm fuzzy." And unfortunately that means sometimes I am left with the "how many taps" metric.
I don't mean to crow about tapping people. It's just my insecurity talking. It's not "ooh, I'm so good.." It's "Ooh, normally I'm so bad, and here I was a little less bad."