Can someone explain the advantage of increasing the native resolution vs upscaling after rendering?
For something like a photograph, you want as high a 'native resolution' as possible, so that there is as much information as possible in the original image. But for old video games, the assets and textures are still the same. Is this a bit like taking a high megapixel photo of a low quality print? Or is my understanding wrong?
I'm sure things like AA work better in the native renderer. Are there other advantages?
The N64's "GPU" is the Reality Display Processor (RDP) chip which is specialized hardware built by Silicon Graphics specifically for the N64 console. The RDP runs a custom program that is part of each game that tells it how to take high-level commands from the game code and render them. For this accurate N64 emulation, it is being emulated at a lower level than before.
Previously, N64 emulators just used the commands sent to the RDP to tell the host system GPU what to do with something higher-level like OpenGL or DirectX (of course, this meant a lot of game-specific "hacks" in the emulator), rather than emulating the RDP itself by sending lower-level commands directly from the RDP to the host GPU with something like Vulkan. This is so-called High-Level Emulation (HLE), and it's a massive shortcut to emulating the whole RDP -- which meant that N64 could even be "emulated" on a PC from 1999.
Lower-level emulation of the RDP itself has been recently made possible, and now it can also now be "up-scaled" to arbitrary resolutions -- instead of just sticking to high level emulation and telling OpenGL or DirectX to render to a larger resolution -- or even worse, scaling the rasterized rendered output frames by treating them as images.
In practice, Mario 64 was just telling the RDP to render triangles anyways so it mapped nicely to OpenGL for the HLE case, but for more "accurate" emulation (this is like getting from 95% to 99%), the RDP itself needs to be emulated as well for things like the Perfect Dark drone camera mentioned elsewhere.
To be pedantic the RDP was the rasterizer (fixed function) which was fed by the RSP processor. The RSP ran the display-list commands from the cpu/memory (very similar to a Vulkan command list) and did vertex transforms/clipping before feeding its output to the RDP.
Most games used one of a few Nintendo provided RSP programs although later in the machine's lifetime they opened up the RSP compiler and tools to developers.
For 2d stuff you'd be correct - but with 3d you can scale the polygons infinitely just like vector art. This works especially well on the N64 because it has a lot of games that make heavy use of flat shading (see Super Mario 64, but Perfect Dark isn't one of these - it uses a lot of pretty low res textures).
I'm a (sort of) purist who has an emulation (hence the sort of) PC hooked up to a 240p CRT TV for older games, but N64 running at higher res does look pretty nice in some games, and anything looks better than native 240p output with blurry bilinear upscaling on an LCD.
The 'p' in '240p' doesn't stand for pixels, it stands for progressive scan. Also 240p/480i/480p are the standard accepted terms for these low resolution video signals[0], nitpicking technical details as a 'gotcha' when people use standard terminology isn't helpful.
Phosphors may not be pixels, but 240p doesn't say anything about pixels. The number tells us how many lines, and the p tells us that each screenful of lines covers the whole picture (the p is for progressive, vs i for interlaced). The whole phrase 240p CRT TV tells us it's a normalish NTSC tv, not a hi-res tv with fancier electronics to work with digital tv and which would likely have more processing delays.
Interestingly the '240p' signal sent out by video game consoles of that era is really a hack, as 240p wasn't a standard signal supported by TVs of the time.
It's actually a 480i signal with the timing fiddled with so that the alternate lines still strike the same part of the screen (this is why games from that era had such noticeable scanlines - the CRT beam is only lighting up alternate horizontal lines).
This also means that a lot of more modern TVs (and even some upscalers marketed for retro gaming) do an extra terrible job of upscaling 240p signals because they run the same logic that they would if it was normal 480i, resulting in unnecessary flickering or dropped frames.
The Analog TV doesn't have a framebuffer, and neither do most consoles, until you get into the 3d era.
My understanding is that the timing of the vblank signalling that comes between fields determines weather the next field is an even field or an odd field. If the vblank signalling comes in the middle of the last scanline, the next field is an even field; if the vblank comes aligned with the end of the last scanline, the next field is an odd field.
If you always start vblank signalling in the middle of a scanline, you get all even fields, if you always start vblank signalling at the end of a scanline, you get all odd fields.
There's also two halves to this, sure the TV itself might not be made up of clean square pixels like an LCD. But the source image that's being sent to it absolutely does have a discrete horizontal and vertical resolution in square/rectangular pixels.
PlayStation 1 games suffer from a similar problem, and rendering at higher resolution in an emulator also often reveals a surprisingly high polygon count for eg. character models that look like badly-drawn sprites at native resolution.
> I'm sure things like AA work better in the native renderer.
Rendering at higher resolution is the highest-quality form of AA possible.
There are some projects that swap out textures for higher res versions.
But I think the bigger advantage is for distant objects - if you zoom in the goldeneye screenshot, you can see the face texture of the guy at the other end of the hall. If the regular native resolution had been used and upscaled afterwards, you wouldn't see the face at all, just a face-coloured blur
With ML the textures can be upscaled too. Though, no one has attempted to do this as an emulator plugin yet as far as I know.
Image superscaling is a well known problem with real world solutions so, it's only a matter of time and interest.
ML will draw in details that never existed to begin with. It's quite amazing, but most versions of image superscaling that exists today is trained on drawn art, like from Deviant Art, so the ML makes the images look subtly pastel. It's great if you want a 4k desktop background.
Heck, I could make one if I cared enough. A friend of mine makes a popular emulator. Maybe she'd appreciate the functionality.
The difference is especially noticeable on distant objects in games where you need to see ahead: e.g. in Gran Turismo you can't tell which way chevron turn signs are pointing unless you have higher internal resolution.
(Not suggesting that one should drive by signs in GT, but they're still fine as a cue.)
Generally details will still be better the closer to native display resolution you can render rather than rendering to native source resolution and then up scaling.
Even if the textures are blurry (they can be swapped too), the polygon details will be easier to see.
I wonder if someone has already thought of using one of the available pixel art upscalers to improve the texture resolutions. If not, it's probably only a matter of time.
Rendering at a higher resolution makes a giant difference, even for low poly games. Old games were not only low res, but didn't have a lot of antialiasing either. On an emulator you can have resolution, antialiasing and better texture filtering.
I don't know the details of this project but I have seen emulators use processes more complex than a basic image upscale to get really nice results out of low resolution textures. I think its similar to those anime upscailing tools where its trained on a dataset of textures and is able to redraw them at higher resolutions.
You'd be surprised at what image processing algorithms are fast enough to work in realtime when they have optimised GPU implementations. Anime4K is one of those CNN-based anime upscaling tools, and it can run in realtime as a shader in mpv. (I'm not a fan of those upscaling tools myself, but I don't see why they couldn't be used in emulators, for the people who like them.)
For something like a photograph, you want as high a 'native resolution' as possible, so that there is as much information as possible in the original image. But for old video games, the assets and textures are still the same. Is this a bit like taking a high megapixel photo of a low quality print? Or is my understanding wrong?
I'm sure things like AA work better in the native renderer. Are there other advantages?