 Xbox 360 Compression, Efficient Storage
Published: June 9, 2005, 6:39 AM CST
By chairmansteve
Since Xbox 360 will be using dual-layer DVD (9GB) for games, some people are probably wondering if that's enough for next generation games. Let's look at what will be stored and how it will be stored.
Two techniques will save tons of space: Compression and Procedural Synthesis. Compression involves taking existing data and squeezing it into a smaller space either with a lossy or lossless algorithm. Procedural Synthesis involves creating new data (geometry, textures, etc.) at runtime.
How much can 9GB store with compressed data?
WMV 720p 6Mbps - 3 Hours
WMV 480p 2Mbps - 9 Hours
WMA 128Kbps - 156 Hours
WMA 96Kbps - 209 Hours
WMA 64Kbps - 312 Hours
WMA 32Kbps - 625 Hours
Texture 1MB - 9,000 Files
Model 256KB - 35,000 Files
Level 20MB - 450 Files
Obviously, WMV 720p is high definition (HD) video, while WMV 480p is standard definition. WMA 96-128Kbps would probably be used for music, while WMA 32-64Kbps should be enough for most sound effect samples.
1MB texture is enough for 32-bit 1024x1024 resolution with 4:1 compression rate. The ATI GPU has compression formats for color maps and normal maps, so all textures can be compressed with excellent quality.
I've estimated average sizes for model and level files. Models are objects and characters. Levels contain the geometry and scripts for a game map or section.
Where might procedural synthesis be applied?
We should see procedural synthesis most commonly with geometry. In some cases, entire complex objects can be created in real-time from a tiny dataset. In other cases, extra geometric detail can be added for smoother curves (i.e. tessellation).
Procedural textures are also useful. Instead of taking a texture file from disc, the GPU (or CPU) can just render the texture to memory and apply that to polygon surfaces. Procedural textures could be created in real-time or at load-time.
Some games may try to get fancy with real-time generated levels or procedural audio.
How much of each file type might be used in a game?
Depending on the game and procedural synthesis applied, the balance of file types will vary. But we can estimate sizes for an example imaginary action/adventure game.
Levels - 2GB
Textures - 2.5GB
Models - 512MB
Video - 512MB
Audio - 384MB
Other - 128MB
Total - 6GB
2GB of levels, 2.5GB of textures, and 512MB of models would make a huge game. 512MB of FMV (full motion video) is enough for either 12 minutes of 720p HD or 35 minutes of 480p. 384MB audio can store 419 minutes of WMA 128Kbps or 839 minutes of WMA 64Kbps.
What do you think? Will DVD be enough for next generation games? The answer should be, "Yes!" If needed in some rare cases, multiple DVDs could easily be used to multiply the total capacity. Those rare cases would be linear games that include tons of 720p movies. | | | Comments | June 9, 2005, 7:39 AM by eb
I'm already on record as saying 'no'.
Not that I wish to disagree just for the sake...the fact is games have always expanded in size, even some PS2 games use over 4.7gb already!
As games become increasingly complicated, with more detailed graphics and bigger 'movie style' budgets I expect within a couple of years to be seeing X360 games on 2 DVDs (I can't imagine they'd drop the quality rather than use a 2nd disk which would cost a few pence/cents).
Just look at the history of games, it shows clearly an big increase per generation.
Just my opinions, I'm sure there's plenty who think otherwise :) | June 9, 2005, 7:51 AM by da_deadly_sniper to chairmansteve
2GB of levels, 2.5GB of textures, and 512MB of models would make a huge game.
Can you elaborate on this please? What would you classify as a huge game? GTA:SA? And yes, I think 9GB is enough for most next gen games, we might see a few games using more than 1 Disc. | June 9, 2005, 3:54 PM by chairmansteve to eb
You've been mislead by your opinions.
Current generation games, especially on PS2, waste disc space with duplicate files to improve access times. Plus, to save processing time, compression algorithms are not very advanced.
For next generation games, compression and procedural techniques can be improved for the much faster processors. And duplicate files are not needed when we have faster drives (12x DVD) and a hard disk drive (or part of 512MB RAM) for caching.
Current Generation
- Duplicate Files
- Minimal Compression
- Minimal or No Procedural Synthesis
- Wasteful FMV
Next Generation
- Minimal or No Duplicate Files
- Advanced Compression
- Advanced Procedural Synthesis
- Real Time Cutscenes | June 9, 2005, 4:15 PM by Quaid to eb
As technology and development techniques increase, there should be less of a requirement for disc space, not more. With better realtime graphics, there should be less need for FMV.
Procedural synthesis can be used for just about any type of texture as well - grass, concrete, bricks, skin, hair/fur, etc.
Of course, better processing and better programming leads to higher compression as well. 9GB is more than enough for next-gen games. | June 9, 2005, 7:05 PM by hop1hop2
And remember now, that they now have twice the space on DVD's then they used to, an you remember when there has actually been a 2x improvment in games. I'm pretty sure it'l take up about thew same space as last time (unless developers get lazy).
P.S. I'm sick of Sony Zombies | June 10, 2005, 6:05 AM by eb to chairmansteve
I would only argue that compression must cost somewhere - be it in the time it takes to decompress, or the quality loss (compression always comes at some cost).
I'd rather have it uncompressed, but I suppose only time will tell! :) | June 10, 2005, 6:08 AM by chairmansteve to eb
Apparently, you have not much experience with compression techniques. Some can take more processing but provide better quality, smaller sizes, and greater bandwidth efficiency. Since you lack adequate knowledge of compression, I'll assume that procedural synthesis is way out of your league.
I hope you realize that Xbox 360 has a very powerful customized CPU, much more capable for tasks like compression and procedural synthesis than the Intel 733MHz CPU in the original Xbox. Likewise, the X360 ATI GPU is far more capable at procedural rendering (and normal map compression) than the X1 NVIDIA GPU.
Uncompressed? Let's look at one example of why uncompressed is not ideal.
1080p Uncompressed Video at 24 FPS
Storage Per Hour = 537GB
Bandwidth Required = 150MB/sec (1.2Gbps)
Add a little extra for uncompressed audio.
For that uncompressed video, we'd need a disc capacity much larger than DVD, HD-DVD, or Blu-ray. And we'd need the transfer rate of 33x Blu-ray or 108x DVD. With advanced compression, the space and bandwidth problems can both be solved, and multichannel digital audio can be included. | June 10, 2005, 8:29 AM by eb
Compression will mean a drop in quality - that was my point...and as far as I'm concerned compression = quality drop, I can't think of any examples otherwise.
You are right about "Procedural Synthesis" though, I know nothing about this. lol | June 10, 2005, 10:26 AM by Quaid to eb
Compression will mean a drop in quality - that was my point...and as far as I'm concerned compression = quality drop,
You're wrong. Obviously you don't know the difference between lossy and lossless compression, and you obviously don't know how compression works at all. Take this simple example: Xbox and GameCube have texture and z-buffer compression, while PS2 does not. Would you try and argue that PS2 games look better because they don't have compression? That would be ridiculous. The advantages of compression have already shown themselves this generation, and next generation the compression will be even better. | June 10, 2005, 10:55 AM by eb to Quaid
ah, it's a different type of compression.
If I compress a picture, I WILL lose something. Like DVD is compressed from the original etc.
However, your example has made it clear(er) to me we're talking about different things.
I never meant an 'uncompressed' PS2 texture would look better than a XB texture (obviously), but could you answer me this...would a compressed XB texture look as good, and not suffer any problems from decompression? (which was my initial point).
I'm just trying to understand, don't slate for wanting to understand. | June 10, 2005, 11:17 AM by Quaid to eb
THere's 2 types of compression - lossy and lossless.
JPG compression for pictures is a lossy form of compression. Hoever, even with lossy compression, more processing power can be used to encode and decode the compression and there will be less loss. MP3 and MPEG are also lossy compressions that can be imporved with greater processing power.
ZIP and RAR are forms of lossless compression. You can also compress a picture with ZIP, and decompress it with no quality loss whatsoever. The downside is that the file sizes cannot be compressed as much as a lossy form. But with greater processing power, better programming can be done to increase the compression (decrease the file size) while keeping the compression lossless.
So, the programmers have more options, what type of compression to use, how much loss they find acceptible, how much processing power they want to use, and how much data storage space and memory bandwidth they want to use.
So to answer your question - would a compressed XB texture look as good, and not suffer any problems from decompression? - it depends on what the developers want. The types of compression Steve is talking about for X360 are generally lossless types of compression, designed to improve throughput on the system and imrpove the amount of data processed. | June 10, 2005, 6:10 PM by chairmansteve to eb
You use JPEG when compressing a picture, yes? That's a bad example. JPEG provides very high compression rates (for example: 16:1) and is good mainly for photographs. It's an excellent way to save space for digital photos. For perfect quality to store an original content, either uncompressed data or lossless compression should be used.
DXTC (DirectX Texture Compression) is designed for 4:1 compression of color (RGBA) maps. 3Dc is also designed for 4:1 compression but for normal maps. Both are lossy formats, but you need to zoom up close in Photoshop to notice the difference (if any). Textures are filtered (to decrease aliasing) in real-time in games anyway, so the images in game will look different (compressed or not) from the image file opened in a photo application.
Compression brings two advantages: space and speed. Not only does a compressed texture take 1/4 the space in RAM and Disk, it also only needs 1/4 the bandwidth. With 4:1 compression, developers can add 4x as much detail to graphics without requiring more memory space or memory bandwidth.
It doesn't even matter if the DVD has the room for uncompressed textures, because the textures are better off compressed when used in real-time in the games. So they might as well be already compressed (with the same format) on the DVD too. Plus, compressed files will load faster from DVD. A lossless compression via software might be added to require even less DVD space, and it could be unpacked to the native hardware compressed texture formats when stored in RAM.
Original Vs Compressed Textures
http://www.ati.com/developer/images/compresonator_1_image.JPG# (color)
http://www.ati.com/developer/images/compresonator_2_image.JPG# (color)
http://www.ati.com/developer/samples/dx9/NormalCompression.jpg# (normal)
ATI 3Dc Whitepaper
http://www.ati.com/products/radeonx800/3DcWhitePaper.pdf | June 12, 2005, 2:26 PM by VRXJudge
Croteam's demonstration of 3Dc compression in Serious Sam 2 shows that compression can increase quality. Take a normal map, compress it, adda bit more detail, and it'll look better/operate faster.
The red/ATi labeled guy is the compressed one.
http://www.seriouszone.com/images/gallery/ss2/27052004_12.jpg
http://www.seriouszone.com/images/gallery/ss2/27052004_11.jpg
http://www.seriouszone.com/images/gallery/ss2/ss2_3dc2.sized.jpg | June 12, 2005, 5:36 PM by David_South to chairmansteve
There isat least one subject being missed, 5.1 Discrete Audio.
Even if a game doesn't use (or uses very little) Full Motion Video , by instead opting for realtime rendering. Audio will require a lot of space.
I think 9GB is still a lot because of the procederal geometry techiquies anf other wayof reducing things. But textures must trully be considered as expanding is size significantly. Normal Maps are a great asset. Yes they can be compressed, but really, why bother? Better quality comes from less compression.
If the source files look great, why should developers have to do the extra work of compressing data?
One other thing, I thought the games used Triple Layer (15GB) discs? | June 12, 2005, 5:46 PM by guicho80 to David_South
Normal Maps are a great asset. Yes they can be compressed, but really, why bother?
Why bother? The benefits of requiring only 1/4 the storage and 1/4 the bandwidth of an uncompressed normal map should be readily apparent. You sir, are suffering from a lack of imagination. | |
|
| |
