Simple Tutorial of Compressing GTAV

[Razor12911]

My PM box is flooded by member asking questions as to how some people managed to repacked Grand Theft Auto V to the 30-40GB region. Well this is how.

I saw the idea of reflate back in 2014, I left it because I didn't know how to make it work, but thanks to GOZARCK by making it a common compressor here on fileforums, it now has a purpose.

Ok, all you need is a deflate decompressor (reflate), there are couple of threads and examples in this forum as to how to use reflate created by GOZARCK, after reflate, use srep on the result and then LZMA (LZMA can be swapped with something better, your choice). You can also try adding precomp to the party but your funeral.

My GTAV is 60.5GB (Steam) (1.0.350.1), I say steam because on different version of GTA, I noticed a file like x64b.rpf is different, don't remember the size but all I know is that the one in steam is bigger than the one in that version.

Grand Theft Auto V has about 43.7GB of data (game files + dlcs) excluding the audio and other files. Reflate can make that 43.7GB become 87.6GB (about 160 000 streams will be found), don't have srep information but the final size after both srep and LZMA is around 18-19GB, can be less with LZTurbo or something better from that point.

Then you can just use copy/store compression for the audio files and executable files which are equal to 16.7GB. Do the math 18GB + 16.7GB ~ 34.7GB, else if you hard headed, you can use compression on the audio files, just use 7-Zip this time with its LZMA2 compression or use LZTurbo (I don't think msc works on GTAV Audio files).
You can have final result of 34GB more or less without modifying or deleting anything.
Warning: When using reflate on the whole game or at least that 43.7GB of data, you will have problems using srep because it will fail because it won't be able to allocate compression memory to process a huge file like that, you must use srep64 so that it can be allocated and have about 8GB or more RAM installed to proceed with compression, Srep64 will need about 5.5GB memory, if you don't have enough RAM then tough luck, but you can still try compressing with fingers crossed hoping that srep won't fail. Um, it is also possible to make conversion of this, already calculated and put solid compression into the equation, the result is 9xDVD5s or 5xDVD9s (can be pushed to 4xDVD9s). All that you should know is that compressing this game takes less than half a day, about 4 hours if your CPU has 8 threads, have 16GB RAM Installed and using SSD (At your own risk). With reflate, at least processing speed can be adjusted, if you set very high values, it slows down PC, but don't set low values though your PC can handle them under pressure, ran a couple of tests, reflate beats precomp with compression and decompression speed, the ratio differs though, sometimes precomp wins and sometime reflate wins.

Before using reflate, just disable Windows Offender. Downclock CPU or leave it on Stock, reflate does have a tendency of producing Bluescreens (Well at least on my PC).

My Grand Theft Auto V compression history (by file):

x64a.rpf (46.4MB)   x64a.ref (147MB)   x64a.srep (94.5MB)  x64a.arc (35.2MB)
x64b.rpf (191MB)    x64b.ref (359MB)   x64b.srep (253MB)   x64b.arc (162MB)
x64c.rpf (2.04GB)   x64c.ref (3.93GB)  x64c.srep (2.16GB)  x64c.arc (1.22GB)
x64d.rpf (1.68GB)   x64d.ref (1.85GB)  x64d.srep (1.76GB)  x64d.arc (1.60GB)
x64e.rpf (1.96GB)   x64e.ref (5.51GB)  x64e.srep (3.77GB)  x64e.arc (1.11GB)
x64f.rpf (981MB)    x64f.ref (2.07GB)  x64f.srep (1.13GB)  x64f.arc (456MB)
x64g.rpf (2.37GB)   x64g.ref (4.46GB)  x64g.srep (1.81GB)  x64g.arc (949MB)
x64h.rpf (1.58GB)   x64h.ref (3.73GB)  x64h.srep (1.66GB)  x64h.arc (696MB)
x64i.rpf (1.66GB)   x64i.ref (3.79GB)  x64i.srep (1.86GB)  x64i.arc (762MB)
x64j.rpf (1.96GB)   x64j.ref (3.90GB)  x64j.srep (1.85GB)  x64j.arc (860MB)
x64k.rpf (2.20GB)   x64k.ref (4.40GB)  x64k.srep (2.01GB)  x64k.arc (887MB)
x64l.rpf (2.40GB)   x64l.ref (5.07GB)  x64l.srep (2.66GB)  x64l.arc (1.11GB)
x64m.rpf (1.68GB)   x64m.ref (3.51GB)  x64m.srep (1.92GB)  x64m.arc (850MB)
x64n.rpf (1.52GB)   x64n.ref (2.80GB)  x64n.srep (1.43GB)  x64n.arc (727MB)
x64o.rpf (1.63GB)   x64o.ref (3.04GB)  x64o.srep (1.41GB)  x64o.arc (699MB)
x64p.rpf (1.33GB)   x64p.ref (2.42GB)  x64p.srep (1.38GB)  x64p.arc (707MB)
x64q.rpf (2.62GB)   x64q.ref (4.61GB)  x64q.srep (2.22GB)  x64q.arc (1.14GB)
x64r.rpf (1.59GB)   x64r.ref (2.86GB)  x64r.srep (1.50GB)  x64r.arc (776MB)
x64s.rpf (1.53GB)   x64s.ref (2.61GB)  x64s.srep (1.13GB)  x64s.arc (619MB)
x64t.rpf (1.79GB)   x64t.ref (3.27GB)  x64t.srep (1.58GB)  x64t.arc (816MB)
x64u.rpf (1.26GB)   x64u.ref (2.15GB)  x64u.srep (949MB)   x64u.arc (517MB)
x64v.rpf (1.80GB)   x64v.ref (5.18GB)  x64v.srep (2.77GB)  x64v.arc (1.06GB)
x64w.rpf (893MB)    x64w.ref (1.98GB)  x64w.srep (1.02GB)  x64w.arc (516MB)

============================================================================
Input (36.7GB)      Reflate (73.6GB)   Srep ~ (35.6GB)     LZMA ~ (18.0GB)

//These Srep and LZMA results are false can be improved by compressing everything at once.

And here are other reflate results:

For GTAIV using reflate:
Input = 15.0GB > Reflate = 23.0GB > Srep = 15.2GB > LZMA = 9.20GB 
Nothing removed, all languages kepts and bik not resampled, Base release (2008) + 1.0.4.0 update

For PES2016 using reflate:
Input = 6.31 GB > Reflate = 13.1 GB > Srep = 6.68 GB > LZMA = 3.22 GB
Version 1.01, only English language kept and no data packs.

Compressor comparision:

File        :     UI.sb from Need For Speed The Run
Original          :    82.3 MB
WinRAR 5.0        :    14.8 MB
7-Zip LZMA2       :    13.3 MB
Srep, LZMA        :    13.2 MB
Freearc Ultra     :    13.2 MB
PCF, Srep, LZMA   :    9.46 MB
REF, Srep, LZMA   :    8.73 MB
REF, Srep, LZT    :    8.09 MB 
// If only LZTurbo's stdin, stdout handles worked just as the creator said.

Speed and Size comparison:

Precomp version used was: 0.43 (intense)

Input information
Application: Pro Evolution Soccer 2016
Filename: dt35_win.cpk

Comparision results
Precomp input = 78.8 MB
Reflate input = 78.8 MB > same file

Precomp compression speed = 0.46 MB/s
Reflate compression speed = 1.43 MB/s > 3.11x faster

Precomp decompression speed = 3.56 MB/s
Reflate decompression speed = 5.64 MB/s > 1.58x faster

Precomp output = 7.08 MB
Reflate output = 7.08 MB > 4kb bigger
_____________________________________________________
Input information
Application: Pro Evolution Soccer 2016
Filename: dt13_win.cpk

Comparision results
Precomp input = 19.5 MB
Reflate input = 19.5 MB > same file

Precomp compression speed = 0.21 MB/s
Reflate compression speed = 0.52 MB/s > 2.48x faster

Precomp decompression speed = 1.41 MB/s
Reflate decompression speed = 1.43 MB/s > 1.58x faster

Precomp output = 14.0 MB
Reflate output = 14.0 MB > 7kb smaller
_____________________________________________________
Input information
Application: Grand Theft Auto V
Filename: x64a.rpf

Comparision results
Precomp input = 46.4 MB
Reflate input = 46.4 MB > same file

Precomp compression speed = unknown
Reflate compression speed = 0.42 MB/s > undefinedx faster

Precomp decompression speed = unknown
Reflate decompression speed = 0.82 MB/s > undefinedx faster

Precomp output = failed
Reflate output = 35.2 MB > undefined smaller/bigger

But with all this information, lets not turn our heads away from what we had in the first place,

either way, reflate is still in development.

[panker1992]

Well written article you really are a maestro of complexity :P

with that said regarding your CPU because you said that you take BSODs

share model frequency and if overclocked

[felice2011]

Downclock CPU or leave it on Stock, reflate does have a tendency of producing Bluescreens (Well at least on my PC)

@Razor, the teacher is you, but from my long experience of technical hardware and overclockers, the blue screens always occur for to problems related to hardware, or by a wrong configuration of bios, if you talk about overclocking.
I think reflate all ago, except create blue screens...logically on standard PC, or at least when it is to overclocking in rock solid.

[Razor12911]

Originally Posted by panker1992

Well written article you really are a maestro of complexity :P

with that said regarding your CPU because you said that you take BSODs

share model frequency and if overclocked

Well on stock, it does none of that, I'm just giving extra information, just incase it is overclocked, imagine waiting for 3-4 hours then PC switches off. (Talking from experience, it happened yesterday). I recently downgraded to this old processor. Not much from 3.0GHz to 3.5GHz.

Originally Posted by felice2011

Downclock CPU or leave it on Stock,  reflate does have a tendency of producing Bluescreens (Well at least on  my PC)

@Razor, the teacher is you, but from my long experience of technical hardware and overclockers, the blue screens always occur for to problems related to hardware, or by a wrong configuration of bios, if you talk about overclocking.
I think reflate all ago, except create blue screens...logically on standard PC, or at least when it is to overclocking in rock solid.

Haha, I never get bluescreens from playing GTAV, but reflate is just another thing. I checked online for tips of overclocking this cpu, they said something about voltage and that it reduces cpu life and blah blah blah, I haven't adjusted voltage after overclocking, my brother thinks that's the reason for bluescreens. nevertheless It was just giving additional information, CPU Usage stays at 100 percent on my side when reflate is running, I make it execute a lot of raw2hifs at the same time, I compared to just running one at a time but the speed is rather poor, a friend of mine tested reflate on a Xeon Processor, he first doubted when we were trying to compress GTAV, he said "This is the latest Xeon Processor, there is no way reflate can make it reach 100% usage", and it did, it was the first time he saw it from a compressor.

[panker1992]

your overclock isnt stable enough unless you play with voltage !!!

i was running a 200.000 reflate batch yesterday :P when i BSODed so i went from
4.7 to ~ 4.3 ghertz

now i know it was reflate :P

here it is while i play some games,
this was good to now that reflate can become very very aggresive on CPU; precomp on the other hand is only taking 1 core and is kinda old

[felice2011]

Originally Posted by Razor12911

Haha, I never get bluescreens from playing GTAV, but reflate is just another thing. I checked online for tips of overclocking this cpu, they said something about voltage and that it reduces cpu life and blah blah blah, I haven't adjusted voltage after overclocking, my brother thinks that's the reason for bluescreens. nevertheless It was just giving additional information, CPU Usage stays at 100 percent on my side when reflate is running, I make it execute a lot of raw2hifs at the same time, I compared to just running one at a time but the speed is rather poor, a friend of mine tested reflate on a Xeon Processor, he first doubted when we were trying to compress GTAV, he said "This is the latest Xeon Processor, there is no way reflate can make it reach 100% usage", and it did, it was the first time he saw it from a compressor.

Woow, you're telling me that the parallel use of reflate stresses the CPU more than LinX for Benchmark, I think it's difficult, but it could be. You not never stop learning, at this point I could create a utility interface with executions in parallel on "raw2hifs" to test the overclocking on the CPU, using the same concept as the console application "x264 HD Benchmark" by graysky.

[1234567890123]

can you post reflate program there are too many post which reflate i should use

[ChronoCross]

Blue screen is not about cpu, win tell you he can�0…7t fill again the RAM because is fragmented or the first RAM slot is full or fragmented with high temperature. Win always try to use the first slot of RAM. Blue screen is Ram if cpu cant access to ram you get the blue screen. If cpu fail the pc shutdown.

[gozarck]

How the new guys dont see this beautiful info. Thanx Razor12911.

[chum]

Originally Posted by Razor12911

CPU Usage stays at 100 percent on my side when reflate is running, I make it execute a lot of raw2hifs at the same time, I compared to just running one at a time but the speed is rather poor, a friend of mine tested reflate on a Xeon Processor, he first doubted when we were trying to compress GTAV, he said "This is the latest Xeon Processor, there is no way reflate can make it reach 100% usage", and it did, it was the first time he saw it from a compressor.

Razor can you share the reflate version you used to compress GTA V? Gozarck version dont even put my cpu usage above 25%, its only disk intensive.

[Razor12911]

that's still under development, development is very slow. I'll be uploading a test very soon though.

[panker1992]

if Razor keeps popping tools with this rate, i will be testing with saint Peter in the afterlife :P

[buttignol]

Razor12911 could you please post the files that you used to compress srep64, arc.ini and pack.bat

I am not able to compress files up to 2GB
x64l.rpf
x64q.rpf
x64g.rpf

I am using Reflate_1400.7z
pc 8g

thanks

[Amsal]

Razor, is your new reflate compatible with is done is good and effective for this GTA v conversion??

[Pratikofficial]

@Razor i am using your reflate (which have 2GB limit).but everytime it gives greater size than your test.could you post the reflate that you used to compress