D3 RoE - extracting pk4's in order to improve FPS

[CrAcKo]

I extracted the PK4 files in the d3xp folder, hoping it would improve FPS, but it doesn't seem to work like it does for Doom3. There's no difference before and after I extracted em.

I'm I using the wrong method on this one, or isn't it just possible?

- CrAcKo

[Morglum007]

If package is enough huge and have compressed files inside, decompressing them will improve game experience, but if package havent so compressed files ( just like a "tar" file), then uncompressing them will not succeed, even being a very huge file. It is a developer strategy for saving space, or to cover most market computers. Files are compressed into another files (pk4, wad, pak, whatever) only to avoid PC to slowdown searching sources when playing game. Making them to be together in same memory area (ram or swap) will make game to execute faster. Thats a very important development when creating a videogame. If that made D3 to be "slow" in a lot of systems, they have solved this already in newer expansion. U will need to find another workarounds to execute faster...sorry.

Just a how to.

Good luck!

[CrAcKo]

The method I described was meant to work for all the D3 based games. I did the same thing for Quake4... it just won't work for RoE. It does make load times longer but improves FPS alot, since during the loading, everything is pretty much preloaded. RoE is just an expansion, so I don't understand...

[Joe Forster/STA]

Guys, .PK4 (and .PK3 and many "big files" in other games, for that matter) files are actually plain ZIP archives. There are three possibilities to deal with lots of game data files:

1. Have each file as a separate file in the file system. Very stupid idea as it eats up a lot of space (cluster slack, even with small cluster sizes, as with NTFS) and opening files takes relatively lot of time because the OS has to find the file in the file system. And this is especially true today when many games have their data in thousands of files.

2. Have all files put into a single, large, uncompressed "big file". (This "big file" can also be a ZIP archive, created with the "store" - that is, don't compress - method.) This is much faster as there's only one file for the OS and, when scanning the big file upon initialization, the game can build an internal list for all the files inside the "big file". This means that opening a file only means seeking inside the "big file".

3. Have all files put into a single, large, compressed archive. This gives you the advantage of the previous implementation and more... If you're using ZIP, you can use Info-ZIP's free implementation. And ZIP gives you a relatively good compression ratio at an extremely high decompression speed. Actually, extracting files from ZIP archives is not significantly slower than simply reading (or copying) files as the bottleneck is rather the hard disk reading (and writing) data, not the CPU running the decompression algorithm.

So, if you ask me, the best way is keeping game data inside a ZIP archive. As for preloading data, the way how files are stored doesn't matter, if you have enough RAM: all data will be loaded when the level is loaded so data doesn't need to be loaded while you're playing! (And RAM is cheap as hell anyway...!) So, I think, CrAcKo is experiencing some kind of optical illusion when he says uncompressing the .PK4 files improves anything at all...!

[CrAcKo]

I got a amd xp 2500+, 512 mb ram, with a 9600 xt radeon.... and even though it eats lots of space on your HD, extracting the pk4 helps for me to gain more FPS... Loading time takes longer, but it doesnt matter as long as there won't be no lag anymore... Because it is compressed, loading time goes faster... with the exception it keeps loading when the game has finished loading. Low end system users may want to try this themselves.

Example, after a door opens, my HD starts loading like mad, just to load what was actually behind that door (a monster...). With the files uncompressed, I wait longer to load, to gain alot more FPS.

I'm not experiencing any illusions, I'm looking at facts.

[Joe Forster/STA]

@CrAcKo: Okay, you turned solution #3 from my list into solution #1. Now, if you have some time for testing around, try this... For each original .PK4 file, uncompress it into an empty directory and then recompress it, with Info-ZIP (http://www.info-zip.org ), recursively, without compression (!) into a ZIP archive and overwrite the .PK4 file with the ZIP archive. (Make sure to backup your original game files, of course. ) The proper command line for this is zip -0r whatever.pk4 *.*. This is solution #2. How much does game play change with this? I'm really interested in your results. Happy gaming anyway!