They had this really clean high-res, high-poly look and nice colours compared to the more blocky and blurry looking fighting games on the PS2.
I've been messing around with DOA3 and DOA2U on the XEMU emulator and the resolution here is upscaled 3x so what you're seeing is definitely a lot sharper than it would have been on original hardware, but still. Blown up like this it you could tell me this was a PS3 or Xbox 360 game and I'd believe you.
As much as I like the series for its gameplay and as nice as I think the Xbox games look I've always thought that the weakest part of the DOA games has always been their incredibly bland character design and artstyle. Even though they draw from the same pool of martial arts archetypes and stereotypes as every other fighting game series in existence pretty much every character is much more lame and feels less interesting than their equivalents from Tekken, Street Fighter, etc. Kazuya Mishima has more character in his eyebrows than the entire roster of the DOA franchise.
The female cast is supposed to be the draw 
but on balance the designs aren't that much more hornier than most other Japanese fighting game franchises and the characters in those tend to have some personality to go with the fanservice.
Yes, DOA3 was clearly a massive step up from DOA2 on the DC/PS2 in both graphics and scale. I didn't get an X-Box, and that was one of the games I really wanted to try.
Interestingly, DOA4 on the Xbox 360 didn't end up being that big of a leap over DOA3 and DOA2U. It was in widescreen, had bloom lighting and cool materials and shaders but it otherwise felt like an upgraded version of the DOA3 engine.
DOA5 was a complete makeover for the entire franchise. They shifted the artstyle to be slightly more realistic and gave every character more defined and unique facial features, something especially welcome on the female cast who had all suffered very badly from same face syndrome. Then they proceeded to walk all those attempts at realism back on the DLC characters who were the most generic anime waifus the franchise had ever seen. It's my most played game in the series by far but I think the graphics kinda suffered from the seventh gen brown realism trend going on at the time. I honestly prefer the more saturated colours in the Xbox games.