pZLib v3 (Final release)

[Razor12911]

Here's a Christmas gift people, I wonder what vultures have for me though.

Anyways nothing more I can do to this project, it reached its limits in terms of optimisation and offering more speed while producing more or less the same output as precomp, reflate, antiz and etc.

Project was started in July 17, 2016, finished today December 21, 2016. No longer working on this anymore, no more potential updates because everything works, haven't tested on DOOM but it should work to roughly give you 25-27GB output as final size if you compressed it right while giving good speed. pZLib v2 bugs have been fixed as well, tested on PES2017, Mafia 3 and some other games that gave it troubles.

Deal still stands, don't use program if you aren't going to give credits, took 5 months and 4 days to make this.

[Razor12911]

You can place libraries near pzlib to use reflating functions to maximum precompression.

Use '-x' to get the best ratio, that's the ultra setting, use '-r#' to set level default value is 0 which is set by program for best output, 1-9 to set manually.

[kenzo34]

thank you
beautiful, as always

[felice2011]

At Christmas it is all better people, and gifts will make even those who do not deserve it, great Razor ...:)

Quick Test ...


-↓- [ CMD Bench.Test.Info v0.6.9c ] -↓- Compressed Archive Completed At -↓- 21/12/2016 08:16:12
================================================== ================================================== =========================================
○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

pzlib_v3 981,1°MB 2,08°GB 212,01% 1°File 00:01:14:45 e -m2 -st50p -t8 -ct8 -c32m -b512k -v -o
================================================== ================================================== =========================================

-↓- [ CMD Bench.Test.Info v0.6.9c ] -↓- Compressed Archive Completed At -↓- 21/12/2016 08:20:00
================================================== ================================================== =========================================
○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

pzlib_v2 981,1°MB 2,07°GB 210,99% 1°File 00:01:12:93 e -m2 -st50p -t8 -ct8 -c32m -b512k -v -o
================================================== ================================================== =========================================


Is perfect ...;)

[Razor12911]

can you do me a favour, do a comparison between precomp045, reflate by shelwien and antiz, use 1 thread. I want to see difference.

[Gupta]

can u add this precomp too, compiled from latest commit
only uses 2 temp files per stream and lzma and xz compression in-built and also some other performance related tweaks

[felice2011]

-↓- [ CMD Bench.Test.Info v0.6.9c ] -↓- Compressed Archive Completed At -↓- 21/12/2016 10:51:41
================================================== ================================================== =========================================
○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

reflate 122,1°MB 240,4°MB 196,89% 1°File 00:01:58:992 -process -c6 -t1
================================================== ================================================== =========================================

-↓- [ CMD Bench.Test.Info v0.6.9c ] -↓- Compressed Archive Completed At -↓- 21/12/2016 10:52:40
================================================== ================================================== =========================================
○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

antiz 122,1°MB 241,0°MB 197,38% 1°File 00:00:18:920 --brute-window -i -o
================================================== ================================================== =========================================

-↓- [ CMD Bench.Test.Info v0.6.9c ] -↓- Compressed Archive Completed At -↓- 21/12/2016 10:55:59
================================================== ================================================== =========================================
○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

precomp045 122,1°MB 241,4°MB 197,71% 1°File 00:01:56:911 -intense -cn -v
================================================== ================================================== =========================================

-↓- [ CMD Bench.Test.Info v0.6.9c ] -↓- Compressed Archive Completed At -↓- 21/12/2016 10:57:51
================================================== ================================================== =========================================
○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

pzlib_v3 122,1°MB 240,9°MB 197,30% 1°File 00:00:09:943 e -m2 -st50p -t1 -ct1 -c32m -b128k -v -o
================================================== ================================================== =========================================

-↓- [ CMD Bench.Test.Info v0.6.9c ] -↓- Compressed Archive Completed At -↓- 21/12/2016 11:00:04
================================================== ================================================== =========================================
○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

pzlib_v3 122,1°MB 240,9°MB 197,30% 1°File 00:00:10:49 e -m2 -st50p -t1 -ct1 -c16m -b64k -v -o
================================================== ================================================== =========================================

-↓- [ CMD Bench.Test.Info v0.6.9c ] -↓- Compressed Archive Completed At -↓- 21/12/2016 11:06:43
================================================== ================================================== =========================================
○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

pzlib_v3 122,1°MB 240,9°MB 197,30% 1°File 00:00:10:54 e -m1 -st50p -t1 -ct1 -c16m -b64k -v -o
================================================== ================================================== =========================================


if you want to test with the FA support let me know ....

[unarc 125]

Ok first of all thanks Razor.
Now clear me some things... Where I wrong clear me.
First I used bf3 cas_10.cas file I used this with Freearc method used -m2 -f -t8 then I Begin
Input was 411mb output 714 mb took 47.53 second all done good
When I unpack it it's not even moving from zero℅ ok closed it
I go to arc.ini removed-f (I thought it may be an CRC error)
Again same output but extraction failure again(not actually failed it didn't moved to a bit after 7min)
Used in other game files too compression goes well decompression stucks.
Why?
Now reflate (used x64)
I put x64 version libraries (forgot to put -x in arc.ini)
Input 411 mb output 720 mb took 49 second ok good
But when I remembered and put -x in arc.ini it gives output less 411 mb input 714 mb output .
Why??

[Joe Forster/STA]

At Christmas it is all better people, and gifts will make even those who do not deserve it, great Razor ...:)

A saying that I heard recently: heroes are people who died in vain for people who lived in vain. :)

[Razor12911]

you used -f with m2, you increased the chances of getting crc, -f breaks crcs sometimes but its mostly broken when you use m2 than m1, so remove -f

[unarc 125]

Thanks for reply
I completely removed -f and precompressed the file again and then extraction stucks again .
Only using -m2 -t8
I changed things many time. And then I used version x86 and I got an arc crash.
And about reflate why output changes if I put-x in arc.ini
If I do not put it than ratio is more good.???

[Razor12911]

i don't understand when you say, extration is stuck, try encoding and decoding without freearc.

[unarc 125]

I mean after precompression went successfull I decode/unpack it it stucks/hangs

I used without freearc same happens precompression went successfull decoding stucks/hangs
Status : restoration data
Data: 0.25MB > 0.00MB <0℅>
Speed: 0.00mb/s <read> 0.00mb/s <write>
Two minutes went nothing proceeds :(

[Razor12911]

that bug must go to hell, ain't got time for it right now.

[FitGirl]

Many thanks, mate, for this Christmas gift, you're awesome! Will continue to credit you in my future repacks, as I already do with PZlib2.

[Mini]

big thanks! I can't deal with unpacking, 0% and all hang :(

[felice2011]

A saying that I heard recently: heroes are people who died in vain for people who lived in vain. :)

The saying is interpreted by those people who throughout history have tried to be heroes but they were not able to be, the greatest heroes in the history still have a name in the most famous books, your thinking, like your sentence was written by those people who have never been able to distinguish themselves in the crowd, or to incise your name in the memories of mankind.;)

[-XCX-]

Many thanks my dear brother razor for this

[Mini]

how to unpack? hangs for 0%! help

[-XCX-]

Bro, How to Unpack Bin files

[felice2011]

Yes :o ERROR: CRC failed .... File is broken with FA support :(

[aswadd]

thnx razor but may i ask you why you release it by the version 3.7 why that specific number ? Why you always release your tools not according to the ordinary order !!

[LuzhinE]

felice2011

In what does these parameters help, to understanding

-st50p -ct8 -c32m -b512k

[felice2011]

felice2011

In what does these parameters help, to understanding

-st50p -ct8 -c32m -b512k


-st50p

I already see -t60p set, in order for that to be realistic, you must have a CPU with 25 threads, pzlib will use 15 threads which 60%. If you got 8 threads, pzlib will override your 60% order and opt for 50% because you literary can't use 4 and 8-tenth threads so 8 threads * 60% = 4.8, when this value is rounded off, it's 5. therefore 5/8 = 63%.

-st#: number of scanning threads

-ct8

-ct#: number of precompressing threads

-c32m

-c#: chunk size (default 16m)

-b512k

-b#: buffer size (default 256k)

If you'd used my "Cmd_Info" program you can have the detailed information directly in the info window.;)

Last thing, when it relates to the chunk size, happened several times that I have had requested in private on the issue of progress in block and not continues of the progress bar during installation through Inno-setup, with compressors that use the chunk size option, you should use a low memory block, or do not overdo in the order of 16m 32, 64 or 128m, so as to obtain during decompression a minor block for writing of the files.:)

[Toximan69]

Razor12911 thanx!!!

But help pleace guys, same as user Mini, not unpacked "hangs for 0%"

use
[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -c256m -t8 - -o - <stdin> <stdout>
unpackcmd = pZLib d -t8 - -o - <stdin> <stdout>

before version working normal on this parameters

may be need another parameters?

[LuzhinE]

-st50p -ct8

Uses 50% CPU to compress no?

This -t8 parameter uses the entire cpu to compress

or am I wrong?

[Razor12911]

Hotfix uploaded

Take note, no idea if the decompression bug is fixed because that's not what I experience from my end so can someone test this.

[Mini]

game files - 320 mb
after pzlib+srep+lzma - 258 mb

unpacking:
Extracting 1 file, 336,093,110 bytes. Processed 100%

game files - 1.5 gb also continues to increase, unpacking the file over and over again

hotfix didn't help, unfortunately

[felice2011]

Hotfix uploaded

Take note, no idea if the decompression bug is fixed because that's not what I experience from my end so can someone test this.

OK ... successful decompression:D

[LuzhinE]

Mini What game are you trying to accomplish

[Mini]

DOOM. mp_gameresources_003.patch

[LuzhinE]

You are trying it with the fix?

[Mini]

yes!

[aswadd]

Hey Guys don't forget the 4 days XD Man Many thnx , Hope you'll make a TUTs in the next few centuries

[LuzhinE]

Try with the normal version

Now Battlefield 3 already works with the fix

[Razor12911]

Ok so it works and it doesn't work?

[FitGirl]

Ok so it works and it doesn't work?
For me initial V3 gave error (forever unpacking, actually), hotfix works. On tests sets, will test later on big files.

Thanks again and for hotfix ofc!

[Razor12911]

Cool

Here are some tips guys:

-The increment of -c# does give better output, but uses more memory, pzlib will reach 40GB ram usage if you set it up like an idiot, 16MB is the default value, there are very less inputs that will need a higher value like for example, Mad Max will need something like 256MB-320MB, GTAV with 128MB is fine, and DOOM needs something like 600-700MB to catch the most biggest stream I've ever seen.

-If you still want to get all the streams that a game has but you want to lower down the RAM usage as I explained with "-c#" then you have to adjust -st# scanning threads assigns the number of threads to scan data with, more threads = more speed = more ram usage, size is maintained whichever number you set here but speed and memory usage isn't.

-the fast verification option, this does break CRC sometimes, it quickly determines ways of precompressing data, it has a 95% accuracy, the other 5% is when it breaks CRC, you should use this when you use -m1 as method, with m2, accuracy goes down to 70%.

-decompression memory, well pzlib is actually not memory hungry, it all depends on what settings you used when encoding so, you might wonder if pzlib used 12GB memory when encoding, how much memory is it going to need for decompression. Well this is how you calculate, you must find out what ratio was the data inflated to, let's say GTAV for example, you used -c128mb and pzlib ended up using 8GB memory for encoding, pzlib encoded 47.6GB and made it become 95.7GB for example, the ratio is 201%. to calculate it's chunk size (-c128MB) multiplied by 2.01 which is 256MB, that's roughly the memory which will be used, things like IO buffer and processing memory will make it roughly 300-350MB as a result.

-this last version utilises raw2hif_dll.dll and hif2raw_dll.dll, these are reflate functions, the moment you place them near pzlib.exe, pzlib utilizes them and switches off internal functions, I ran many benchmarks, the usage of these libraries make pzlib slower than when it's using internal functions, however, with DOOM, you really don't have a choice but to use them because internal function cannot process the streams that game has, this is why precomp and pzlib v2 never works on that game, however I blended the internal functions with reflate to give the same size when reflate is used alone but more speed, meaning the best method to use when using pzlib is pzlib -m2 -x...., "-x" gives a better output than pzlib alone.

-for DOOM and some games that pzlib fails to process, remember I created ZlibChecker, you can first use this program to see what size should you really get and pzlib must give that output. for example, zlibchecker might report that 20GB became 37GB, but when you use pzlib, pzlib gives 21GB as inflated size, actually this is what happens when you use pzlib on DOOM, then you have to use "-s" for those special streams because clearly the game has special deflate streams. For DOOM, the best method you can use is -m2 -c640m -x -s -r9....

[-XCX-]

Thanks Razor Bro For hotfix ofc!
decompression works great bro

[LuzhinE]

The normal version never decompresses.

Tested with x64 fix

packcmd = pZLib e -m2 -t8 - -o - <stdin> <stdout>

http://i.imgur.com/WxAu7Vq.jpg

packcmd = pZLib e -m1 -t8 - -o - <stdin> <stdout>

http://i.imgur.com/aFnNVlC.jpg

packcmd = pZLib e -m1 -t8 -f - -o - <stdin> <stdout>

http://i.imgur.com/CTSuJtX.jpg

[Amsal]

The normal version never decompresses.

Tested with x64 fix

packcmd = pZLib e -m2 -t8 - -o - <stdin> <stdout>

http://i.imgur.com/WxAu7Vq.jpg

packcmd = pZLib e -m1 -t8 - -o - <stdin> <stdout>

http://i.imgur.com/aFnNVlC.jpg

packcmd = pZLib e -m1 -t8 -f - -o - <stdin> <stdout>

http://i.imgur.com/CTSuJtX.jpg

You should remove -f if you get error while decompression. Remember, the -f switch ia for FASTER verification and not for BETTER verification

[LuzhinE]

Delete it, but still CRC error

packcmd = pZLib e -m1 -t8 - -o - <stdin> <stdout>
http://i.imgur.com/aFnNVlC.jpg

[felice2011]

LuzhinE....I do not understand why you are using the files *.CAS with pzlib, files *.CAS no contain stream, tested a sample of files with the game "Mirror's Edge Catalyst"...:confused:

[FitGirl]

LuzhinE....I do not understand why you are using the files *.CAS with pzlib, files *.CAS no contain stream, tested a sample of files with the game "Mirror's Edge Catalyst"...:confused:
MEC uses LZ4 compression, while earlier games, like BF3 use zlib.

[felice2011]

Ahh ok... BF3 I had not tried it yet:)
MEC I'm testing with oodle with different codecs.

[FitGirl]

Ahh ok... BF3 I had not tried it yet:)
MEC I'm testing with oodle with different codecs.
Oodle is meh.

[LuzhinE]

Tested on DOOM - Update 5
Ripeados all languages except Spanish, credits and folder mp and palettes

58.8 GB

Use pzlib+srep+lzma

packcmd = pZLib e -m2 -c640m -x -s -r9 -t8 - -o - <stdin> <stdout>

Final size 41 GB

Razor you said it ends in 25-27 GB

[Mini]

+1

[Simorq]

Tested on DOOM - Update 5
Ripeados all languages except Spanish, credits and folder mp and palettes

58.8 GB

Use pzlib+srep+lzma

packcmd = pZLib e -m2 -c640m -x -s -r9 -t8 - -o - <stdin> <stdout>

Final size 41 GB

Razor you said it ends in 25-27 GB

DOOM Update 5 Only ENG (Remove MP , SnapMap , mega.pages)
Srep+Lzma2 : 34GB
pzlib Only Work DOOM File
base\gameresources.resources
base\gameresources.patch
pZLib+Srep+Lzma2: 30 GB Unpack Crash (I think the lowlead system)

[ChronoCross]

Pzlib V3H works really good!. Very clever to include reflate libs.
pack and unpack without errors.

[Gupta]

If i recompress something without std input output, everything works awesomely, amazingly fast, delivers best ratio and most amazingly restoration always works
but whenever the std input output is used with arc, restoration never works it always gonna struck at 0.0%, i don't get a single file on which it works but without std in-out every thing works and yes crc matches always

Tested on Witcher 3, Grand theft auto 5, Skyrim Legendery edition.

[LuzhinE]

Pro Evolution Soccer 2017 with Argentina commentary

Game Size = 5.54 GB
Rpack Size = 3.55 GB

Installation time 8 minutes

pzlib+srep+lzma2

[shazzla]

Hi guys !
i hope someone can help me,because im in trouble with pzlib3 hotfix.
I cant decompress the archive using cls_pzlib.dll. It says :

ERROR: archive data corrupted (decompression fails)

The facts :
PZLib 3 hotfix
FreeArc 0.67 (December 12 2012)

@Compression :

Arc.ini :
[External compressor:pzlib]
packcmd = pzlib e - -o - <stdin> <stdout>
unpackcmd = pzlib d - -o - <stdin> <stdout>

Command:
d:\Trashcan\ISDone_Reflate\pack>arc.exe a -mpzlib d:\Trashcan\ISDone_Reflate\pack\1.arc d:\Trashcan\ISDone_Reflate\pack\autotest.tmd
FreeArc 0.67 (December 12 2012) updating archive: d:\Trashcan\ISDone_Reflate\pack\1.arc
Compressed 1 file, 93,613 => 446,170 bytes. Ratio 476.6%
Compression time: real 0.10 secs. Speed 955 kB/s
All OK

@Decompression:

Arc.ini:
[External compressor:pzlib]
unpackcmd = pzlib d - -o - <stdin> <stdout>

Arc.ini and CLS_pzlib.dll are in the decompression folder.

d:\Trashcan\ISDone_Reflate\unpack>unarc x 1.arc
FreeArc 0.67 unpacker. Extracting archive: 1.arc
Extracting Trashcan\ISDone_Reflate\pack\autotest.tmd (93613 bytes)

ERROR: archive data corrupted (decompression fails)

BUT if i remove the CLS and copy pzlib.exe and its DLL's to the decompression folder... It unpacks !

Am i miss something ? :(

[zouzouni]

Hi guys !
i hope someone can help me,because im in trouble with pzlib3 hotfix.
I cant decompress the archive using cls_pzlib.dll. It says :

ERROR: archive data corrupted (decompression fails)

The facts :
PZLib 3 hotfix
FreeArc 0.67 (December 12 2012)

@Compression :

Arc.ini :
[External compressor:pzlib]
packcmd = pzlib e - -o - <stdin> <stdout>
unpackcmd = pzlib d - -o - <stdin> <stdout>

Command:
d:\Trashcan\ISDone_Reflate\pack>arc.exe a -mpzlib d:\Trashcan\ISDone_Reflate\pack\1.arc d:\Trashcan\ISDone_Reflate\pack\autotest.tmd
FreeArc 0.67 (December 12 2012) updating archive: d:\Trashcan\ISDone_Reflate\pack\1.arc
Compressed 1 file, 93,613 => 446,170 bytes. Ratio 476.6%
Compression time: real 0.10 secs. Speed 955 kB/s
All OK

@Decompression:

Arc.ini:
[External compressor:pzlib]
unpackcmd = pzlib d - -o - <stdin> <stdout>

Arc.ini and CLS_pzlib.dll are in the decompression folder.

d:\Trashcan\ISDone_Reflate\unpack>unarc x 1.arc
FreeArc 0.67 unpacker. Extracting archive: 1.arc
Extracting Trashcan\ISDone_Reflate\pack\autotest.tmd (93613 bytes)

ERROR: archive data corrupted (decompression fails)

BUT if i remove the CLS and copy pzlib.exe and its DLL's to the decompression folder... It unpacks !

Am i miss something ? :(

This pzlib v3 (so is V2) doesn't work with cls pzlib Dll ....not meant to work with , Razor 12911 didn't program it to use cls Dll... ; )

Hey Razor , Thx again for that impressive tool , awesome work adding reflate functionnality for special zlib headers (optionnal Dll) ... wow ,very fast and great compression ratio

.... but main thing is that I can't unpack at all , always give crc checksum error at some point!

I tried on Shadow Warrior 2 bin files . Pzlib could reflate 12.9 to 22 Gb and then applying srep +lzma I could get a nice 8.4 Gb but decomp ( using ASC v0.0065 ) will always get stuck no matter how I do ... (same with pzlib v2)

I tried both V3 and hotfix, with and without optionnal files, I tried
parameters like :
-m2 -x -r9 -t4 (out of 8 threads for my xeon 3440) with optionnal Dlls
-m2 -c128m -t4
-m2 -c32m -st50
.....all giving roughly same ratio but still unpacking errors (ASC just stops decomp but no error)

Then i tried changing Input/Output settings in arc.in
from: - -0 - <stdin> <stdout>
to - $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp (PrinceGupta recommendation)
....still errors at decomp , ASC stops

Then I tried to remove files on which ASC seems to hang... it always hangs on a new file, eventhough those files could be decomp previously (before I start to remove them)

Does this has something to do with ASC ....???

Then I finally tried to change Srep settings playing between m3f/m5f and even adding -hash=md5 crc checksum.... same results, ASC stops.

I am talking about errors because I am pretty sure that if I use inno script it will send me crc checksum error ....this already happenned to me using pzlib v2 with Skyrim SE and MGSV TPP (ASC stops and Inno gives a crc error)

....and I ve just finished DarkSiders Warmastered Edition .... It wasn't decompressing either (-m2 -c400m -t4, seems to be the best setting to catch all zlibs) ....I tried Srep m5f setting and it now decompress ok with ASC ..... awesome!


BIG thk you anyway for this great christmas Gift !!! we have a new toy to play with!!

[shazzla]

I had no information about PZlib V3 was not supported. Thanks,but in this case another problem :
ISDONE :

[External compressor:pzlib]
header = 0
packcmd = pzlib e -v - -o - <stdin> <stdout>
unpackcmd = pzlib d - -o - <stdin> <stdout>

pzlib3hotfix.exe ,both dlls and arc.ini included...
error : unsupported compression method pzlib
unarc.dll returned an error code : -2

i dont understand. :(
and where can i get new information about PZlib ? Is this the only place ?

[zouzouni]

I had no information about PZlib V3 was not supported. Thanks,but in this case another problem :
ISDONE :

[External compressor:pzlib]
header = 0
packcmd = pzlib e -v - -o - <stdin> <stdout>
unpackcmd = pzlib d - -o - <stdin> <stdout>

pzlib3hotfix.exe ,both dlls and arc.ini included...
error : unsupported compression method pzlib
unarc.dll returned an error code : -2

i dont understand. :(
and where can i get new information about PZlib ? Is this the only place ?



Did you encode using pzlib v3 along with the 2 optionnal Dlls ... if so you need then in decompression folder and you need to add exctraction commands in INNO....

PZLIB+DLLS at comp and decomp ...otherwise it will give you an error!

your arc.ini is ok for decomp, don't change it ; ) ... but -v at comp is from previous pzlib version ...

you need something like that : packcmd = pZLib e -m2 -x -r9 -t4 - -o - <stdin> <stdout> to use pzlib v3 with dlls
-t4 = 4 cores
-x = ultra setting
-r9 = processing level ( from 1 to 9 ; 0 for automatic)

and for unpacking: unpackcmd = pZLib d -t8 - -o - <stdin> <stdout> (it will use up to 8 cores for decomp if available)

[shazzla]

So... then it cant be extracted by the ISArcExtract function ? :o

Is the -f worked ?

[zouzouni]

yes it can be extracted through ISArcExtract.... just add lines in your inno script to extract needed files before isarcextract starts!

eg: ExtractTemporaryFile('pZLib.exe');
ExtractTemporaryFile('raw2hif_dll.dll');
ExtractTemporaryFile('hif2raw_dll.dll');

-f is for fast verification of files, don't use it with -m2 option ; should only be used with -m1 option

[shazzla]

yes it can be extracted through ISArcExtract.... just add lines in your inno script to extract needed files before isarcextract starts!

eg: ExtractTemporaryFile('pZLib.exe');
ExtractTemporaryFile('raw2hiff_dll.dll');
ExtractTemporaryFile('hif2raw_dll.dll');



Those files are included and extracted after running the "setup.exe" ! Arc.ini too -->

unsupported compression pzlib. :D

It cant decompress the archive :
if not ISArcExtract ( 0, 0, ExpandConstant('{src}\c1.dat'), ExpandConstant('{app}\'), '', false, '', '', ExpandConstant('{app}'), notPCFonFLY {PCFonFLY}) then break;

WTH ?!


Edit :

if not ISExec(0, 0, 0, ExpandConstant('{tmp}\arc.exe'), ExpandConstant('x -o+ -cfg"{tmp}\arc.ini" -w"{app}" -dp"{app}" "{src}\try.arc"'), ExpandConstant('{tmp}'), '...', False) then break;

Extract successul !

But ISArcExtract would be better....

[zouzouni]

a picture to illustrate crc checksum error in inno ( ASC stops decompression but doesn't give this error)

I had to setup a test inno installer with existing script of mine for Skyrim SE....

it shows install dir for Skyrim SE but files are from Shadow Warrior 2 ...bin files as described in my previous post !

If I recompress "pc_arch_common.bin" alone it works ok at decomp but as soon as I compress several files together (big and small files mixed together with no sorting option in freearc) .... I have this CRC ERROR

I tried both inno and ASC and same results.... this bug might comes from pzlib
and was already present on V2 .... with a few games type of files! (MGSTPP, Skyrim SE and previous versions, also had this bug on DarkSiders 2 files)

I tried on xeon 3440 + 8Gb Ram, and i5 3570 + 16 Gb ram , fully stable machines ....same results

[zouzouni]

Those files are included and extracted after running the "setup.exe" ! Arc.ini too -->

unsupported compression pzlib. :D

It cant decompress the archive :
if not ISArcExtract ( 0, 0, ExpandConstant('{src}\c1.dat'), ExpandConstant('{app}\'), '', false, '', '', ExpandConstant('{app}'), notPCFonFLY {PCFonFLY}) then break;

WTH ?!


Edit :

if not ISExec(0, 0, 0, ExpandConstant('{tmp}\arc.exe'), ExpandConstant('x -o+ -cfg"{tmp}\arc.ini" -w"{app}" -dp"{app}" "{src}\try.arc"'), ExpandConstant('{tmp}'), '...', False) then break;

Extract successul !

But ISArcExtract would be better....



try this:

if not ISArcExtract ( 0, 100, ExpandConstant('{src}\data.bin'), ExpandConstant('{app}'), '', false, '', ExpandConstant('{tmp}\arc.ini'), ExpandConstant('{app}'), False) then break;

[shazzla]

try this:

if not ISArcExtract ( 0, 100, ExpandConstant('{src}\data.bin'), ExpandConstant('{app}'), '', false, '', ExpandConstant('{tmp}\arc.ini'), ExpandConstant('{app}'), False) then break;

unarc.dll returned an error code : -2
error : unsupported compression method pzlib+srep+delta+lzma....

[LuzhinE]

shazzla What game is trying to compress?

[shazzla]

crysis 1

[LuzhinE]

Your system is 64 or 32 bits

[shazzla]

x64.

but as i wrote before :
I can unpack the archives using isdone with ISExec.

IsArcExtract doesnt work. And i dont know why..... :/

[LuzhinE]

X64 version
You do not need to add anything else
You can open pZLib.exe directly to read its documentation

https://drive.google.com/open?id=0B6SkrA_4COzgTldRTWxDSW9uQ2c

I hope you understand, if you get CRC error try changing to -m1 or use reflate for some games

[shazzla]

everytings clear as it was before. btw thanks ! :)

[zouzouni]

OK... I finally succeeded in compressing Shadow Warrior 2 bin files :cool:

after numerous attempts.. I found "THE SETTING" : ) for this game and no more crc checksum error at decomp ....installation is ok in both ASC and INNO

for that used pzlib x86 (instead of x64) v3H, I had to split games files in 4 parts and I had to get rid of -c# setting and apply :
packcmd = pZLib e -m2 -st50p -ct8 -b1024k - -o - <stdin> <stdout>

I had to "sacrify" size (from 8.4Gb to 8.53Gb) but still I am happy with it !!

Thx again Razor for sharing this fantastic tool , I will never forget to give you the credits for it! ...brilliant work :D

[ZAZA4EVER]

Very Thanks razor For V3 .. but I want Help in install wpi
how add pzlip unpack to wpi setup
i make just cause 3 from 54.5gb to 20.4gb by pzlip+msc+serp+lzma
eng & arabic lang - commnt ... and uppack perfect by the attachment cmd
and check md5 files perfect well ..
i want to add pzlip and command lines to wpi setup
i try it but fail not progress with me installing and not progress
please help and happy new year

[ZAZA4EVER]

i try to add pzlib.exe to wpi but when install progress ~ 2% and stop not make any progress please help ... where the problem in unarc.dll or arc.ini or what? ..
i make again repack make it 16.6gb not 20.4gb ... as well unpack by attachment i uploaded it in previous post and not unpack by wpi

[oltjon]

try to write so ( pZLib.exe ) in the script :)

[pakrat2k2]

also check your arc.ini

[External compressor:pzlib]
header = 0
unpackcmd = pzlib d -t8 - -o - <stdin> <stdout>


post#71 has the following & used x86 version NOT x64 versions.


packcmd = pZLib e -m2 -st50p -ct8 -b1024k - -o - <stdin> <stdout>


simple copy / paste into your arc.ini to see if it helps any, try using pZLib instead of pzlib
don't think it matters any but try as suggested.

[-XCX-]

i try to add pzlib.exe to wpi but when install progress ~ 2% and stop not make any progress please help ... where the problem in unarc.dll or arc.ini or what? ..
i make again repack make it 16.6gb not 20.4gb ... as well unpack by attachment i uploaded it in previous post and not unpack by wpi
add raw2hif_dll.dll , hif2raw_dll.dll and pzlib.exe in your script

[ZAZA4EVER]

try to write so ( pZLib.exe ) in the script :)

Not Solve , Thanks For Reply And Help

also check your arc.ini

[External compressor:pzlib]
header = 0
unpackcmd = pzlib d -t8 - -o - <stdin> <stdout>


Not Solve , Thanks For Reply And Help

post#71 has the following & used x86 version NOT x64 versions.


packcmd = pZLib e -m2 -st50p -ct8 -b1024k - -o - <stdin> <stdout>


simple copy / paste into your arc.ini to see if it helps any, try using pZLib instead of pzlib
don't think it matters any but try as suggested.

add raw2hif_dll.dll , hif2raw_dll.dll and pzlib.exe in your script

Thanks Very Much ... Extract Perfect ... Finally

[Razor12911]

People make yourselves useful. ;)

I was away for 2 weeks and the only thing razor gets is people telling me the program does not work, CRC errors and etc.

I'm not a repacker guys, I'm just a programmer, I create programs for you guys, and the least you guys can do is upload the file that gives pzlib problems, nothing has been uploaded so far. Firstly I'm on capped net your attachments must be small, can't expect me to download some game so that I can figure out the bug myself yet I don't even use pzlib at all, I simply buy games or just keep them "DP" Direct play, I really don't have time for repacking at all, the effort of repacking can easily be converted to buying another hard drive, anyways. What I'm trying to say is, none of my games do raise any CRC errors, and I'm giving you guys a job to fix it.

Now it must be really difficult to figure out which small file can have bugs well it is simple like it is difficult for me to figure out a bug in several gigabytes of data.

For example. PES2017, game is like 6GB. When I first used pzlib v2, it was buggy and gave CRC errors everytime it comes across dt15_win.cpk, but when you use pzlib on that file alone, the error isn't there. pzlib works with chunks and it keeps its chunks constant so what I did was flush the bug out. Firstly, the game is 6.2GB or so, don't really remember. (Game was crap so I deleted it and only kept the buggy part of it)

Ok so from 6.2GB, I know that the problem lies in dt15_win.cpk, so all I did was remove all files that came after that file and kept everything else and I remained with 1.22 GB because everything after dt15_win.cpk was close to 5GB meaning I'm trapping the bug and making it easy to spot, also making it easy for you guys to upload.

From that 1.22 GB, there was dt00_win.cpk, dt10_win.cpk, dt11_win.cpk, dt12_win.cpk, dt13_win.cpk, all those files were 964 MB, now from what I remember, in the past I created dummy file creator, just make dummy files of those files meaning you'll still have 1.22GB but now the data will be compressible for upload for me to investigate. I ran pzlib on the data with the dummy files, since the bug only lies in dt15_win.cpk, the bug was kept which is good because now the bug is even more trapped.

Remember pzlib works with chunk sizes, by default it's 16mb, so to trap the bug even more, you can start by deleting data from position 0 of the data, keep removing 16mb until the bug is gone, I ended up with 16.8mb of data from 964MB, and the bug was gone, I then readded 16mb to make sure the bug is still there so that I can investigate and find out what was wrong, the final size was 32.8 MB. So from 6.2GB worth of data, I ended up with 32.8MB but the bug was maintained, I then figured out what the problem was, fixed via programming and then added a few touches then made pzlib v3 having thought all the bugs were gone but they weren't.

So in short, you guys should do something roughly like this for the bug to be fixed.

Anyways, I uploaded an example. In the attachment. This file is buggy when used in pzlib v2, but it works in v3 because I did have the game that made pzlib to be buggy and fixed it, I don't have game that makes pzlib v3 buggy which is why it is buggy.

[felice2011]

I do not understand, the file you uploaded works with both the v2 that with v3, compression and decompression OK

[Razor12911]

try default settings.

[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -t2 - -o - <stdin> <stdout>
unpackcmd = pZLib d -t2 - -o - <stdin> <stdout>

[felice2011]

Ok Ok wait...bug found

[Razor12911]

hmm... you don't say...

[felice2011]

pzlibV2

pzlib e -m3 -st50p -t8 -ct8 -c16m -b64k ( OK Comp ... OK Decomp )

pzlib e -m2 -st50p -t8 -ct8 -c16m -b64k ( OK Comp ... Decomp = infinite loop, remains so... )

https://i.imgsafe.org/edd13bfc4f.png

[Razor12911]

Hehe, -t# controls -st# and -ct#, if you set -t7, st and ct become -st7 and -ct7 automatically. Anyways, it's not an infinite loop. pzlib keeps giving freearc incomplete data or rather damaged data, Freearc wants 32.8MB and pzlib keeps giving it 31.9MB, just for that, Freearc keeps calling pzlib, just check process hacker or task manager, you'll see pzlib is run and terminated over and over again, and when you check output directory, output is there but it keeps increasing endlessly, if there was a file after that dt15_win.cpk, you'll get CRC, but since it's just one file, CRC error is not called which happens to look like an endless loop.

m2 and m3 chunk data differently which is why m3 works and m2 doesn't, however with m3 working, it doesn't mean the problem is solved, m3 has problems of its own.

[felice2011]

woo if I cancel the infinite loop the file becomes more than 1GB...:D
OK then return to the starting point...:rolleyes:
Set in default ini ... we see

[External compressorzlib]
header = 0
packcmd = pZLib e -m2 -t2 - -o - <stdin> <stdout>
unpackcmd = pZLib d -t2 - -o - <stdin> <stdout>

Result.

https://i.imgsafe.org/ee303460b5.png

"dt15_win.cpk" will be your nightmare tonight...:rolleyes::o

[Razor12911]

Not really my nightmare mate, I don't use program plus bug is fixed in pzlib v3, but some bugs are still there, dt15_win.cpk is just a demonstration of what I had to do to fix the bug caused by it, you guys should do other ones, should be easy enough.

http://fileforums.com/showpost.php?p=455033&postcount=77

[Sebazz]

Hi. I try to repack gta v version 1.36, and my question is what is the best method shelwien reflate or pzlib v3? And the recommended parameters ? Thanks

[ZAZA4EVER]

not Use Reflate Level
R6 ... pack output very big size and load on your hdd
R9 ... Has Problems in unpack

[Razor12911]

not Use Reflate Level
R6 ... pack output very big size and load on your hdd
R9 ... Has Problems in unpack
this is false.

[ImmortalArc]

1.Best way to compress bf3 game I am ending with 9.30 gb while other people's doing it in 8.80 gb how????
2.what is the best thing after precompsiom of a file like lzma but I want more ratio good. Is there any other??? Is lzma2 is good??
3.how to rebuild games archives for better results?? Like fitgirl?

[-XCX-]

1.Best way to compress bf3 game I am ending with 9.30 gb while other people's doing it in 8.80 gb how????
2.what is the best thing after precompsiom of a file like lzma but I want more ratio good. Is there any other??? Is lzma2 is good??
3.how to rebuild games archives for better results?? Like fitgirl?
Extract the files of bf3 and reimport them then compress them with srep+lzma you may get the good ratio

[ImmortalArc]

Extract the files of bf3 and reimport them then compress them with srep+lzma you may get the good ratio
What I see in the mechanics temp folder is precomp zlib1 files and they are reaching 8.80gb
If I pzlibv3+srep+lzma then I getting 7.5 gb only in cas.cas files update is 232 mb and 1.47 gb of win32folder And other files 150 mb I am having problems In en.sb ru.sb files???

[doofoo24]

great tool and great precompmt
i have a question is there a way to get constant speed in decompression when using mpzlib+srep+lzma:a1:mfbt4:d185m:fb273:mc10000:lc8.
i montor the disk usage is about 20 mb/s is there a better way to get 50 or 60 mb/s with the built in lzma not using fastzip or external xz.
i used 4x4:lzma work great it decompress gta 5 in 29 min using I7 6700K 4.7GHZ ON HDD WD BLACK but 4x4:lzma use half of cpu and i limit pzlib to 50%,
i disabled hyper threading to see how it work on I5 and agian try on 2 core 4 thread to see the different...
i try to use zstd with pzlib but i found the raito not great compare to lzma or 4x4:lzma...
so agian is there a middle ground improve lzma speed without using 4x4 only lzma.
thank

editing message
no need to answer i already found it.
i wish you best of luck in your study
yours truly

[Razor12911]

If you're experiencing crc errors with pZLib. Send me a private message in attempt to fix it via Team Viewer.

[rinaldo]

razor precomp046 It was released by schneider, if you .. you can use with pzlib

[Razor12911]

razor precomp046 It was released by schneider, if you .. you can use with pzlib
Why would I want to use it with pzlib? :confused:

[rinaldo]

Why would I want to use it with pzlib? :confused:
because it is not possible :confused:

[Razor12911]

because it is not possible :confused:
Well pzlib was created as an alternate of precomp. It's not really innovative to create an alternative while still indirectly using the old product, it's hustling backwards.

http://krinkels.org/threads/pzlib_v3-versus-precompmt.3333/#post-29919

Just look at the times yourself.

[doofoo24]

precomp 0.46 ?
i thought precomp 0.46 still in development still needed work like remove temp file which make <stdin> <stdout> work ?
i check on github but nothing only 0.44 and 0.45

[doofoo24]

just a tip after using pzlib+srep+lzma if decompress hang or give an crc try to reinstall visual studio all ver form 2008 to 2015 and see other program use it like intel management engine interface ?
and see in windows features remote differential compression if enable or disable ?
most of the time 8/10 pzlib work like charm.
another way try to use cls-precompmt only with 0.45 ver i didn't encounter any error...

[rinaldo]

doofoo24
8 to 10 no good :eek:
Razor12911
thanks! very good wok

[doofoo24]

i said 8/10 it mean 80% with m2 i did't try m1
the setting

[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -c16m -b256k -t1 - -o - <stdin> <stdout>

agian great tool
thank's razor

[Mini]

Dishonored 2 http://www28.zippyshare.com/v/2OssNi5I/file.html
dont work with pZLib e -m2 -t8 -x -s -r9 - -o - <stdin> <stdout>

reflate work good:(
Razor, any help?

[Nizar3003]

sometimes one or two games could not use pzlib, I tried extract no man sky and could not but other game work fine.

[Dansoft]

PLEASE ANYONE. Can someone please help create for me a compressor having pzlib+srep+lzma or something better? please

[felice2011]

PLEASE ANYONE. Can someone please help create for me a compressor having pzlib+srep+lzma or something better? please

If you do a search in forum, you don't need no help;)

[rinaldo]

:rolleyes:

[rinaldo]

:rolleyes:

x unpacked

[Dansoft]

If you do a search in forum, you don't need no help;)

ive tried, could you point me somewhere?

[felice2011]

PLEASE ANYONE. Can someone please help create for me a compressor having pzlib+srep+lzma or something better? please

If you do a search in forum, you don't need no help;)

ive tried, could you point me somewhere?

:eek:

Rinaldo has posted a compressor for your request...:cool:

Here you find another more complete...http://fileforums.com/showthread.php?t=97611

Here you find another more complete of the M&S Kompressor http://fileforums.com/showthread.php?t=98709

But if you want a complete index just go here..http://www.fileforums.com/showthread.php?t=96782....;)

[Dansoft]

:rolleyes:

while compressing, it just closes. have no idea why

[doofoo24]

on windows 8.1 when i try to use 4x4:lzma with pzlib+srep i get this error many time ?
on windows 10 work great...
any idea...
thank for help
:confused:

[TWOELV]

is pzlib better than the masked data compressor ?

[rinaldo]

is pzlib better than the masked data compressor ?
no

[Chayan Manna]

Realy the best. It is Very fast but same ratio as reflate, most of times better than reflate.
I Used it in "Crysis 2 Maximum Edition" and got better ratio than reflate without any CRC error.
Where Precomp failed and reflate gave CRC error.
Compressed it From 9.19 GB to 6.66 GB with pzlib+srep+delta+lzma.
with reflate decompression time = 32 mins (but at94.7% CRC Check failed)
with pzlibv3 decompression time = 13 mins
Other Precompressor makers must learn from you.
All your projects are best and most usefull.
Keep up the good work.

[doofoo24]

Pzlib v3 does not work on cod black ops 3 / infinite warfare / modern warfare remastered ???
i tried plz4 it work but not the same ratio that i see on the internet some mange to get more ratio ?

[Gupta]

>>Pzlib v3 does not work on cod black ops 3 / infinite warfare / modern warfare remastered ???

because they got lz4 and pzlib is for deflate

>>i tried plz4 it work but not the same ratio that i see on the internet some mange to get more ratio ?
fuc* of* videos
reduce their quailities

[doofoo24]

>>Pzlib v3 does not work on cod black ops 3 / infinite warfare / modern warfare remastered ???

because they got lz4 and pzlib is for deflate

>>i tried plz4 it work but not the same ratio that i see on the internet some mange to get more ratio ?
fuc* of* videos
reduce their quailities

it seems that fitgirl use modified cls-precomp with code of lz4 and use xdelta3....
there is no difference in between this repack and my steam version of cod infinite warfare...

[Gupta]

>> it seems that fitgirl use modified cls-precomp with code of lz4 and use xdelta3....'
May be he made a universal cls for freearc may be working same as Razor's precompMt

[doofoo24]

LZ4io.c - LZ4 File/Stream Interface
https://github.com/mldbai/mldb/blob/master/ext/lz4/lz4io.c

[MCWRX]

Hey there. First, thanks Razor for pZlib.

I compressed Mad Max, and when I install it my CPU kinda gets fully loaded for some seconds and setup progress goes on, then CPU load is between 0-1% and setup progress just freezes, and so on. So, I am getting a very bad decompression time. How can I cure this?

[COPyCAT]

This pzlib v3 (so is V2) doesn't work with cls pzlib Dll ....not meant to work with , Razor 12911 didn't program it to use cls Dll... ; )

Hey Razor , Thx again for that impressive tool , awesome work adding reflate functionnality for special zlib headers (optionnal Dll) ... wow ,very fast and great compression ratio

.... but main thing is that I can't unpack at all , always give crc checksum error at some point!

I tried on Shadow Warrior 2 bin files . Pzlib could reflate 12.9 to 22 Gb and then applying srep +lzma I could get a nice 8.4 Gb but decomp ( using ASC v0.0065 ) will always get stuck no matter how I do ... (same with pzlib v2)

I tried both V3 and hotfix, with and without optionnal files, I tried
parameters like :
-m2 -x -r9 -t4 (out of 8 threads for my xeon 3440) with optionnal Dlls
-m2 -c128m -t4
-m2 -c32m -st50
.....all giving roughly same ratio but still unpacking errors (ASC just stops decomp but no error)

Then i tried changing Input/Output settings in arc.in
from: - -0 - <stdin> <stdout>
to - $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp (PrinceGupta recommendation)
....still errors at decomp , ASC stops

Then I tried to remove files on which ASC seems to hang... it always hangs on a new file, eventhough those files could be decomp previously (before I start to remove them)

Does this has something to do with ASC ....???

Then I finally tried to change Srep settings playing between m3f/m5f and even adding -hash=md5 crc checksum.... same results, ASC stops.

I am talking about errors because I am pretty sure that if I use inno script it will send me crc checksum error ....this already happenned to me using pzlib v2 with Skyrim SE and MGSV TPP (ASC stops and Inno gives a crc error)

....and I ve just finished DarkSiders Warmastered Edition .... It wasn't decompressing either (-m2 -c400m -t4, seems to be the best setting to catch all zlibs) ....I tried Srep m5f setting and it now decompress ok with ASC ..... awesome!


BIG thk you anyway for this great christmas Gift !!! we have a new toy to play with!!

@zouzouni, Shadow Warrior 2 has UNIQUE special stream on 1 BiG file, wherebout 6 GB single file (pc_arch_common.bin). Those reflate method will not passed the unique special stream and result in error while decompressing. Located wherebout 45% of file streams, try to use batch command reflate & combine with precomp (if precomp fail, then that's the unique stream) to see the address of exact location.

By the way, pZlib v3 was AWESOME!!! Thanks A Lot [RAZOR12911]
Working great on MAD MAX. It gives 12.95% compression ratio and 30 minutes of installation with no checksum error.
OS: Win7 SP1 x64
Pro: AMD FX-8370 (8-core) 4.01 GHz
RAM: 16 GB
HDD: TOSHIBA 500 GB (7200 RPM)
Compresstion time took: 10 hours 27 minutes
Restoration time took: 30 minutes
Condition: game running smooth
17614
17613
Thank Q. [Razor12911] for Your Time craft/forging the pZlib3 Compressor. for all of us @this forum. SALUTE :) & God Bless You.

[ramazan19833]

help

MAD MAX. It gives 12.86% compression ratio and 24 minutes

[ZakirAhmad]

Any one please help...
I tried pzlib v4 +precomp042 on a file of crysis warhead... compression went well but decompression is giving me error at the end of extraction... (archive data corrupted)... please help...

[danswano]

Hello,
I'm unable to unpack an arc file using pzlib+srep+lzma using ISDone although i've included these in my inno project:
[Files]
Source: Include\pZLib.exe; DestDir: {tmp}; Flags: dontcopy
Source: Include\pZLib64.exe; DestDir: {tmp}; Flags: dontcopy
Source: Include\hif2raw_dll.dll; DestDir: {tmp}; Flags: dontcopy
Source: Include\raw2hif_dll.dll; DestDir: {tmp}; Flags: dontcopy
Source: Include\hif2raw_dll64.dll; DestDir: {tmp}; Flags: dontcopy
Source: Include\raw2hif_dll64.dll; DestDir: {tmp}; Flags: dontcopy
Source: include\arc.ini; DestDir: {app}; Flags: dontcopy;
ExtractTemporaryFile('pzlib.exe');
ExtractTemporaryFile('pzlib64.exe');
ExtractTemporaryFile('raw2hif_dll.dll');
ExtractTemporaryFile('raw2hif_dll64.dll');
ExtractTemporaryFile('hif2raw_dll.dll');
ExtractTemporaryFile('hif2raw_dll64.dll');

arc.ini:
[External compressor:pzlib]
header = 0
unpackcmd = pZLib d -t8 - -o - <stdin> <stdout>

What am i missing here?
Thanks.

[ZakirAhmad]

I tried pzlibv3 64 on .iwd files of cod blackops but while installing, installer says 2hours to extract... please help...

[hasandgn34]

Hello,
I'm unable to unpack an arc file using pzlib+srep+lzma using ISDone although i've included these in my inno project:




arc.ini:


What am i missing here?
Thanks.

change like this;


ExtractTemporaryFile('arc.ini'); // your script probably doesnt have this line
ExtractTemporaryFile('pzlib.exe');
ExtractTemporaryFile('pzlib64.exe');
ExtractTemporaryFile('raw2hif_dll.dll');
ExtractTemporaryFile('raw2hif_dll64.dll');
ExtractTemporaryFile('hif2raw_dll.dll');
ExtractTemporaryFile('hif2raw_dll64.dll');

[danswano]

change like this;


ExtractTemporaryFile('arc.ini'); // your script probably doesnt have this line
ExtractTemporaryFile('pzlib.exe');
ExtractTemporaryFile('pzlib64.exe');
ExtractTemporaryFile('raw2hif_dll.dll');
ExtractTemporaryFile('raw2hif_dll64.dll');
ExtractTemporaryFile('hif2raw_dll.dll');
ExtractTemporaryFile('hif2raw_dll64.dll');

Ah stupid me, totally forgot about it.
Thank you and big thanks to razor for his hard working efforts.

[-XCX-]

What is the best method of pzlib for gta 5?
Can anyone tell me please?

[danswano]

A quick question please:
Although i've read all the pages and replies but i couldn't figure out the purpose of those 4 versions of pZlib
the version refers to the project it self advancing forward or it refers that every single version handle different files using older or newer version of zlib headers?
I mean if the game uses zlib 2 headers can i use pzlib v4 to precompress it?
Thanks.

[shazzla]

Pzlib v4 ? 0_o Are you sure ?!

[Razor12911]

What is the best method of pzlib for gta 5?
Can anyone tell me please?

-m2 -x -c256m

A quick question please:
Although i've read all the pages and replies but i couldn't figure out the purpose of those 4 versions of pZlib
the version refers to the project it self advancing forward or it refers that every single version handle different files using older or newer version of zlib headers?
I mean if the game uses zlib 2 headers can i use pzlib v4 to precompress it?
Thanks.

there are 3 versions of pzlib, the versions of pzlib are not affiliated zlib versions at all, they are just titled to pzlib only.

[ramazan19833]

Razor12911
Hello there

pzlib . v3

Is this the best? Adjustment

GTA 5 GAME BEST SETTING WHAT?

HELP......

[External compressor:pzlib]
header = 0
packcmd = pzlib e -m2 -d900m -b4m -t75p -t8 -ct8 -c512m -b512k $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd = pzlib d -t8 - -o - <stdin> <stdout>[/QUOTE]

[Razor12911]

Razor12911
Hello there

pzlib . v3

Is this the best? Adjustment

GTA 5 GAME BEST SETTING WHAT?

HELP......

[External compressor:pzlib]
header = 0
packcmd = pzlib e -m2 -d900m -b4m -t75p -t8 -ct8 -c512m -b512k $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd = pzlib d -t8 - -o - <stdin> <stdout>[/QUOTE]

http://fileforums.com/showpost.php?p=457116&postcount=131

the f*ck is -d900m doing there when encoding.

[ramazan19833]

Razor12911
Hello there

srep64.V

RAM : 32 GB
CPU : Intel® Core™ i7-5820K 6Core 3.30 GHz
SSD: 250 GB

Is this the best? Adjustment

GTA 5 GAME BEST SETTING WHAT?

HELP......

[External compressor:srep64]
header = 0
packcmd = srep64 {options} -m5f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep64 -d -s - - <stdin> <stdout

[felice2011]

I would cut the thumb down to this user, but then it will be worse it will use the middle finger...:p:)

[L0v3craft]

Hi Razor12911 and thank you for this awesome precompressor.

I have a simple question. At post #39 you wrote :

"-The increment of -c# does give better output... Mad Max will need something like 256MB-320MB, GTAV with 128MB is fine, and DOOM needs something like 600-700MB"

good but how we can understand how much are bigs the streams of the files of a game ? Which tool do you used to understand that ? Thanks.

[ZakirAhmad]

@L0v3craft:
Use ds4pzlib by razor...
just drop file on it and wait for result...
try to use switches like force detection headerless streams and chunksize.....
use chunksize to find chunksize u r going to use in pzlib...
additional info...
u can also see possible memory usage during preprocess by pzlib...

[Razor12911]

I would cut the thumb down to this user, but then it will be worse it will use the middle finger...:p:)

Haha, I don't know why he always does that, and I thought I was the only one who noticed.

[-XCX-]

I repacked 35.6gb on Gta5+all dlcs.
Thanks a lot Razor12911 Bro for your library

[Sebazz]

Hello guys, can you please what is the pzlib best methods for phantom pain, rise of tomb raider and hitman 2016 ? Thanks

[-XCX-]

It took too much time to compress
then I got this fu**k**g error while unpacking any solution please
Method Used:
packcmd = pzlib e -m2 -x -c256m - -o - <stdin> <stdout>

[Sebazz]

i think you need specificate decoding/encoding streams option (-t#), try this config.

[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -x -c256m -t75p $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd = pZLib d -t75p - -o - <stdin> <stdout>

[-XCX-]

i think you need specificate decoding/encoding streams option (-t#), try this config.

[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -x -c256m -t75p $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd = pZLib d -t75p - -o - <stdin> <stdout>

did you try any time

[ramazan19833]

Hello there

Help Please..

http://i.hizliresim.com/gqopnR.jpg (http://hizliresim.com/gqopnR)
GTA 5 ORIGINAL SIZE :[/B] 75 GB
GTA 5 COMPRESS : 37 GB
METHOD : -mpzlib+srep+lzma2
http://i.hizliresim.com/gqopkR.jpg (http://hizliresim.com/gqopkR)

Arc.ini

[External compressor:pzlib]
header = 0
packcmd = pZLib e -m2 -x -c256m -t100p - -o - <stdin> <stdout>
unpackcmd = pZLib d -t100p - -o - <stdin> <stdout>

[External compressor:srep]
header = 0
packcmd = srep64 {options} -m3f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep64 {options} -d -s - - <stdin> <stdout>

[External compressor:Lzma2]
header = 0
packcmd = 7z a -txz -an -mcrc=0 -m1=lzma2:d25:fb=273:mf=bt4:mc=1000000:lc=4:lp=0 -mmt=4 -mx9 -si -so <stdin> <stdout>
unpackcmd = 7z x -txz -an -y -si -so <stdin> <stdout>

pZLib+srep+Lzma2 38GB GTA V

[ImmortalArc]

Have you tried with wpi and asc??

[L0v3craft]

[External compressor:srep]
header = 0
packcmd = srep64 {options} -m3f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep64 {options} -d -s - - <stdin> <stdout>

Try to change that with this:


[External compressor:srep64]
header = 0
packcmd = srep64 -m3f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep64 -d -s -- <stdin> <stdout>


and change the command: pZLib+srep+Lzma2

with this: pZLib+srep64+Lzma2

[chum]

Pzlib compress MP3 too? Thats odd because Precomp045 cause issues trying to compress some MP3 files, when i disabled Precomp045 from compressing MP3 then the compression and decompression works perfect.

Pzlib crashes when try to compress same audio files than precomp045 crashes.

[danswano]

Hello,
I' using these parameters on Mafia III
;srepx64
[External compressor:srep]
header = 0
packcmd = srep64 {options} -m3f -d1g -a2 -v0 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:pzlib]
header = 0
packcmd = pZLib {options} e -m2 -c256m -t100p $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
hif2raw_dll.dll and raw2hif_dll.dll are next to pzlib.exe in the same folder as well.

But getting data checksum error when decopressing, any idea why?

[Gupta]

>> But getting data checksum error when decopressing, any idea why?
don't use stdio/stdin while compressing, you will never get crc error
Tested 1000 times

[danswano]

Can you modify my code please, not sure what you mean where i must remove them.
You mean completely remove it from both srep and pzlib or just pzlib?
Or you mean remove it from unapckcmd?

[danswano]

Any help is appreciated, tried many pzlib and srep settings and still getting so many freaking crc errors.
also tried splitting the files across multiple archives didn't work for all of them.

[008black]

What is the best method to compress GTA v audio

[Nizar3003]

What is the best method to compress GTA v audio

Try to use PZLIB and MSC.
You can get better compresssion...

[008black]

Try to use PZLIB and MSC.
You can get better compresssion...

Thanks i will try

[008black]

I used Pzlib -m2 -x c256m +srep +Lzma2 to compress GTA 5 and also used msc for audio (compressed to 15.3 gb intial size 16.6 gb ) and final compressed size is of 39 gb version 1.36 but there are repacks on net like fitgirl 36 gb approx can anyone tell how to achieve that size

[ShadowEagle]

Compressing Mafia III worked fine, but there are several problems to extract. Esspecially on some sds texture files there is a endless loop.

Is there any way to catch these files before compressing? I mean it`s very ennoying to compress the game 20 times to catch all files which won`t extract correctly.

[romulus_ut3]

I think we'll have to be patient with pZlib, as it's still very much a work in progress. Till Razor12911 can put some more of his valuable time into this, all we can do is go through the process of trial and error; provide him with much detailed information of the problems we encounter as possible.

[Razor12911]

reflate_0c4 libraries added to main post.

Speed has been improved

[doofoo24]

great test it on doom gameresources.resources and patch mange to compress 7.90gb to 4.40gb using pzlib+srep+lzma
for pzlib i used pzlib e -m2 -t4 -c640m -x -s -r9,
decompress 9 minutes on i7 6700k
i set pzlib to d -t100p...
great any new news on plz4

[008black]

Hello razor what's the progress of zarc

[Razor12911]

Hello razor what's the progress of zarc

Currently paused, will be resumed after my exams.

[fifazalata]

hi guys I have a quisition
does the order of the External compressor matters?
I mean for example in the arc.ini file:

[External compressor:srep]
[External compressor:precomp]
[External compressor:pzlib]
[External compressor:lzma64]

If I changed the order would it matter with compresion?

[Bulat]

order in arc.ini doesn't affect anything

[romulus_ut3]

I have made an observation, which appeared very strange to me.

If your compression method looks something like this:

pzlib64+srep64+lzma2
or
pzlib+srep64+lzma2

You've to rename cls-srep.dll to cls-srep64.dll in order for it to decompress, or else, it'll get stuck after decompressing a few files every time. This is applicable to WPI, btw. I don't know how to work with any other scripts so I can't speak for the others.

[Bulat]

cls-XXX.dl decompress method XXX, so just look into compressed data. may be method is encoded as "srep64" - in this case cls-srep.dll isn't used at all and you probably stuck with arc.ini-specified srep.exe in stdin-to-stdout mode, which isn't very reliable in freearc

[Razor12911]

cls-XXX.dl decompress method XXX, so just look into compressed data. may be method is encoded as "srep64" - in this case cls-srep.dll isn't used at all and you probably stuck with arc.ini-specified srep.exe in stdin-to-stdout mode, which isn't very reliable in freearc

Is this always the case or stdio isn't properly implemented in FA?

[Bulat]

yes, stdio-to-stdout mode in freearc has bugs and i don't know how to implement it correct

[felice2011]

@Bulat..Why not open a new thread with the beta version 0.20 of FANext compiled.
So you can discuss, test it, and test pros and cons.
Honestly I tested the 0.11 release, but I have had various difficulties in its operation, continuing to love only FreeArc.:)

[Bulat]

i will do that when it will be released

here i answered on the question about old arc.exe

[felice2011]

Oh Yepp the context I had understood of the old FA ....:), more or less when is estimated the FANext release with a new update?

[ShadowEagle]

Stupid question: What`s the difference between 32 and 64bit version?

[Sebazz]

Conversion speed, x64 is more faster.

[KaktoR]

Thanks for this razor.

First i was dumb, i got everytime errors in compression, then - after some hours of try&error - i checked arc.ini... there was no packcmd for pzlib... Now i got it :D

I think my first conversion using pzlib is just behind the mountain.

PS: Is there any pzlib checker? Can't find it on INDEX.

[-XCX-]

zlib checker
http://fileforums.com/showthread.php?t=97657
http://fileforums.com/showthread.php?t=98198

[romulus_ut3]

Razor, what would you recommended settings be for a PC equipped with 16 GBs of RAM and four cores/four threads processor when using pZlibV3?

I am getting CRC error during decompression while compressing Witcher 3, BTW.

My current settings:

[External compressor:pzlib64]
header = 0
packcmd = pzlib64 e -m2 -x -r9 -t4 $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd = pzlib64 d -t8 - -o - <stdin> <stdout>

[shazzla]

I had CRCs too. Define number of threads like this : -t75p and it will be fine (in pack/unpack section)

[romulus_ut3]

Thank you!

[Simorq]

More time for games with the big size pZLib comes problem for extracting

But these settings will fix the problem.

[External compressor:pZLib]
header = 0
packcmd = pZLib e -m2 -t125p - -o - <stdin> <stdout>
unpackcmd = pZLib d -t100p - -o - <stdin> <stdout>

[danswano]

More time for games with the big size pZLib comes problem for extracting

But these settings will fix the problem.

[External compressor:pZLib]
header = 0
packcmd = pZLib e -m2 -t125p - -o - <stdin> <stdout>
unpackcmd = pZLib d -t100p - -o - <stdin> <stdout>

What games and errors like what?

[Simorq]

What games and errors like what?

BERSERK And The Band Of The Hawk
crc error

[poker2332]

Hello, I'm trying to compress the bf3 multiplayer.
But I'm having problems when it comes to unpacking it.
One of the most common problems is CRC error
Could you say that I am doing wrong?

https://k60.kn3.net/F/8/E/7/6/C/177.jpg
https://k60.kn3.net/F/5/1/6/6/7/7FB.jpg
https://k61.kn3.net/0/F/E/1/6/8/D24.jpg
https://k61.kn3.net/2/1/9/1/8/C/392.jpg

ARC.INI
[External compressor:msc]
header = 0
packcmd = msc c -v -f -wav=1 -raw=1 -bmp=1 -ddsraw=1 -ddsdxt=1 -mp3=1 -bmf=9s -tak=9+ -dxt=1 -lzma=hc4,lc8,lp2,pb2,fb273,mc1000000 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:pzlib]
header = 0
packcmd = pzlib e -m2 -st75p -t4 -ct8 -c32m -b512k $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd = pzlib d -t4 - -o - <stdin> <stdout>


[External compressor:srep64]
header = 0
packcmd = srep64 -m5f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep64 -d -s -- <stdin> <stdout>

[External compressor:xz,lz77]
header = 0
packcmd = 7z a -txz -an -mcrc=0 -m1=lzma2:d120m:fb=273:mf=bt4:mc=1000000:lc=4:lp=0 -mmt=4 -mx9 -si -so <stdin> <stdout>
unpackcmd = dec x -txz -an -y -si -so <stdin> <stdout>

Repack.bat
Arc a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"D:\TMP" "E:\BF4 repack\Setup-1a.bin" -mpzlib+msc+srep64+delta+exe2+lz77 "E:\Program Files (x86)\Split_Kompress_v1.3d\Temp Folders\Battlefield 3_1\*"

I am using this version of arc.exe and unarc.dll

http://fileforums.com/attachment.php?attachmentid=17386&d=1488018486

[Razor12911]

pzlib is not perfect, it does have bugs.

[Simorq]

pzlib is not perfect, it does have bugs.

But I do every time I use the single-core (t25p), there is no problem and error:D

Sample
Just Cause 3 pZLib e -t100p Crc Error | -t25p No Crc Error
BERSERK And The Band Of The Hawk pZLib e -t100p Crc Error | -t25p No Crc Error
Final Fantasy X X-2 HD Remaster pZLib e -t100p Crc Error | -t25p No Crc Error

[KaktoR]

Someone on the previous pages telling not to use <stdin> - <stdout>. Then you should use all settings without crc error. But honestly, i don't know how to use this.

[Razor12911]

But I do every time I use the single-core (t25p), there is no problem and error:D

Sample
Just Cause 3 pZLib e -t100p Crc Error | -t25p No Crc Error
BERSERK And The Band Of The Hawk pZLib e -t100p Crc Error | -t25p No Crc Error
Final Fantasy X X-2 HD Remaster pZLib e -t100p Crc Error | -t25p No Crc Error

single thread has a different code in pzlib, no point in using multi threading technique for 1 thread, result will just be slower.

[Simorq]

single thread has a different code in pzlib, no point in using multi threading technique for 1 thread, result will just be slower.

I wonder the same thing
However, for me this is it.

I've tested many games with large file

For example
BERSERK And The Band Of The Hawk
Just Cause 3
pzlib:m2:x:s:r9:t100p:st100p:ct100p Crc Error
pzlib:m1:x:s:r9:t100p:st100p:ct100p Crc Error
pzlib:m2:t100p:st100p:ct100p Crc Error
pzlib:m2:c250m:t100p Crc Error
pzlib:m2:t100p Crc Error
pzlib:m2:t75p Crc Error
pzlib:m2:t50p Crc Error

pzlib:m2:t25p No Crc Error

Use External compressor

[External compressor:PZLib]
header = 0
;packcmd = pZLib e -m2 - -o - $$arcdatafile$$.tmp -o - <stdout>
;packcmd = pzlib e {options} - -o - <stdin> <stdout>
packcmd = pZLib e -m2 -t25p - -o - <stdin> <stdout>
unpackcmd = pzlib d -t100p - -o - <stdin> <stdout>

[Razor12911]

CLS uploaded

Cls decodes only, might fix bugs.

@Simorq

-t#, automatically sets -st# and -ct# to be the same value.
Try,
pzlib:m2:t100p:st1

[Nizar3003]

CLS uploaded

Cls decodes only, might fix bugs.

@Simorq

-t#, automatically sets -st# and -ct# to be the same value.
Try,
pzlib:m2:t100p:st1

CLS can decompress without pzlib.exe or not ?
btw, thanks still update for PZLIB.

update :
got some error in cls, always close....

[Razor12911]

CLS can decompress without pzlib.exe or not ?
btw, thanks still update for PZLIB.

update :
got some error in cls, always close....

It just closes?

[Nizar3003]

It just closes?

yes, always close. i try in other archive and same, still closing while extracting.
Always close in 54% - 57%...


Before, i compress use pzlib.exe for 64-bit but in extacting i use pzlib.exe 32-bit and working fine.

[romulus_ut3]

The CLS works wonderfully, thank you Razor!

Does not require pzlib executable (I assume that's the purpose of cls to begin with?)

Tip: If you used pzlib64 to compress your game, rename the cls-pzlib.dll to cls-pzlib64.dll and it'll work just fine.

[Simorq]

CLS uploaded

Cls decodes only, might fix bugs.

@Simorq

-t#, automatically sets -st# and -ct# to be the same value.
Try,
pzlib:m2:t100p:st1

Thank you very much
But the problem is not solved.

Styx Shards of Darkness
pzlib:m2:t100p:st1 Crc Error
pzlib:m2:t25p No Error

Mad Max
pzlib:m2:t100p:st1:c256m Crc Error
pzlib:m2:t25p:c256m No Error

I think it is a bug in -t.

[chum]

With most games i get ok decompression with CLS+Freearc GUI, but with one game specifically i get errors during decompression like others.

Apparently CLS-pzllib.dll is not respecting the argument -t#p or -t# for decompression, even if i set -t50p it always using total of 7 threads (-t75p). I have 8 threads CPU.


Edit: Ok i find the cause of decompression errors. If i compress that specific game with -t75p then decompression will cause error. But compressing with -t50p give correct decompression without any error. Anyway decompression always using -t75p for cls-pzlib.dll.

[Razor12911]

Thank you very much
But the problem is not solved.

Styx Shards of Darkness
pzlib:m2:t100p:st1 Crc Error
pzlib:m2:t25p No Error

Mad Max
pzlib:m2:t100p:st1:c256m Crc Error
pzlib:m2:t25p:c256m No Error

I think it is a bug in -t.

Like I said before, -t# sets -st# and -ct# to be the same value, so if st1 gives errors, you might wanna try -st2

[shazzla]

Try to set t75p @comp/decomp.
It worked for me flawless!
Previously i had many CRCs,too.
Let me know the results.
And before i forget,do not use pzlib's stdin/out !!!
Specify output/temp files,like in precomp ! :)

[Simorq]

Like I said before, -t# sets -st# and -ct# to be the same value, so if st1 gives errors, you might wanna try -st2

Thank
I'm testing and I'm Report
But it it does not matter
It is important for decompression using t100p:D there is no problem

[FitGirl]

Razor, thanks for cls, great update! One question: is it locked to 50p or 75p of available cores? And can it be adjusted like you did it in config for cls-precomp?

[Simorq]

Mad Max
pZLibv2:c32m+srep+Lzma2 4.36 GB

pZLibv3:c256m+srep+Lzma2 6.4 GB

pZLibv3:x:c32m+srep+Lzma2 4.26 GB

[Razor12911]

Mad Max
pZLibv2:c32m+srep+Lzma2 4.36 GB

pZLibv3:c256m+srep+Lzma2 6.4 GB

pZLibv3:x:c32m+srep+Lzma2 4.26 GB

That's not surprising.

Mad Mad is compressed using level 1, by default pzlib is level 6, or rather reflate, the moment you place raw2hif_dll.dll, you are disabling internal recompression, so this is what you will get.

ZLibv3:c256m+srep+Lzma2 6.4 GB
because level 6 was used, you didn't set reflating level, which is the -r# option.
Secondly, the option -x, enables both internal recompression to work with reflate. which is why 4.26GB is the best size because unlike reflate, internal recompression works like precomp, trial and error, it will try all levels and pick one that is best.

@Everyone
pzlib does work without raw2hif_dll.dll, only place the library if you know what it does, or how it works or how it is used else you'll just end up with negative ratios.

[Razor12911]

CLS updated

-Added confirguration
-There a few noobs out there, so hif2raw_dll.dll is embedded in cls, it's loaded automatically, no need to place near it near the CLS, at least this could rule out a few errors.

[FitGirl]

Tested new updated CLS - it works good, but the bug from pzlib.exe still exists - it's impossible to cancel unpacking. Before it was possible to kill pzlib.exe process, now the whole setup must be shut down.

[FitGirl]

A friend of mine did more tests of cls vs generic pzlib. Decoding speed.

| t1 | t2 | t4 | t7 |
----------*--------------------------*--------------------------*--------------------------*---------------------------*
pzlib_x86 | 192 sec. / 2613 kB/sec. | 118 sec. / 4249 kB/sec. | 78.3 sec. / 6419 kB/sec. | 61.7 sec. / 8144 kB/sec. |
pzlib_x64 | 141 sec. / 3556 kB/sec. | 87.5 sec. / 5745 kB/sec. | 57.9 sec. / 8680 kB/sec. | 44.1 sec. / 11386 kB/sec. |
cls-pzlib | 198 sec. / 2537 kB/sec. | 119 sec. / 4223 kB/sec. | 78.3 sec. / 6422 kB/sec. | 61.5 sec. / 8169 kB/sec. |


enc opts: pzlib:m2

win7_sp1_x64
i7-4700MQ / 8 GB RAM
RAM-Disk

[Razor12911]

CLS updated

-Cancel bug fixed
-Added multi threading user control example

[Razor12911]

A friend of mine did more tests of cls vs generic pzlib. Decoding speed.

| t1 | t2 | t4 | t7 |
----------*--------------------------*--------------------------*--------------------------*---------------------------*
pzlib_x86 | 192 sec. / 2613 kB/sec. | 118 sec. / 4249 kB/sec. | 78.3 sec. / 6419 kB/sec. | 61.7 sec. / 8144 kB/sec. |
pzlib_x64 | 141 sec. / 3556 kB/sec. | 87.5 sec. / 5745 kB/sec. | 57.9 sec. / 8680 kB/sec. | 44.1 sec. / 11386 kB/sec. |
cls-pzlib | 198 sec. / 2537 kB/sec. | 119 sec. / 4223 kB/sec. | 78.3 sec. / 6422 kB/sec. | 61.5 sec. / 8169 kB/sec. |


enc opts: pzlib:m2

win7_sp1_x64
i7-4700MQ / 8 GB RAM
RAM-Disk

Well it's not surprising, cls is actually x86, it should produce roughly the same speed as pzlib x86.

[danswano]

Mad Max
pZLibv2:c32m+srep+Lzma2 4.36 GB

pZLibv3:c256m+srep+Lzma2 6.4 GB

pZLibv3:x:c32m+srep+Lzma2 4.26 GB

What are your srep/lzma2 parameters you got 4.36GB with?
Please provide .bat and arc.ini parameters.
Thanks.

[Simorq]

What are your srep/lzma2 parameters you got 4.36GB with?
Please provide .bat and arc.ini parameters.
Thanks.


pzlib+srep+lzma2 4.26 GB
Arc.ini
[External compressor:pzlib3]
header = 0
packcmd = pZLib e -m2 -x -c32m -t25p - -o - <stdin> <stdout>
unpackcmd = pZLib d -t100p - -o - <stdin> <stdout>

[External compressor:pzlib2]
header = 0
packcmd = pZLib e -m2 -c32m -t25p - -o - <stdin> <stdout>
unpackcmd = pZLib d -t100p - -o - <stdin> <stdout>


[External compressor:srep]
header = 0
;packcmd = srep {options} -a1 -m5f -l256 -t8 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
packcmd = srep {options} -m5f $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[External compressor:lzma2, Lzma2, LZMA2, XZ, xz]
header = 0
packcmd = 7z a -txz -an -mcrc=0 -m1=lzma2:d25:fb=273:mf=bt4:mc=1000000:lc=4:lp=0 -mmt=on -mx9 -si -so <stdin> <stdout>
unpackcmd = 7z x -txz -mmt=on -an -y -si -so <stdin> <stdout>

Test pzlib2
http://s8.picofile.com/file/8294951042/Mad_Max_pZLib.png

[FitGirl]

Razor, thank you for the latest update - now the installer closes properly!
You're a gift to this community!

[pakrat2k2]

Someone on the previous pages telling not to use <stdin> - <stdout>. Then you should use all settings without crc error. But honestly, i don't know how to use this.

change arc.ini to use the commands listed ( parts without crc error )

[LuzhinE]

With t25p and t75p works well until now free of CRC, before there was that problem to cancel and did not, do not comment because Razor12911 is busy in classes, but now if it works well.

But waiting for an improvement so that the CPU can be used to maximum when compressing t8.

Thank you very much

[Razor12911]

Version 3.2 now available

Changes

-Tried to fix crc errors, no idea if they are fixed or not.
-Removed several options because there are just to many for some people to understand what they do.
-Tried to fix decoding termination, in case you want to cancel decoding process, it must now work without killing process, not sure either.
-method setting removed, from now it just searches for deflate streams which is m2 by default.
-by default reflate and zlib are blended.
-special stream option "s" is removed, found a way to process them.

[Simorq]

Deflate Streams Better.

Test (Act Of Aggression Reboot Edition Deflate Streams)
pZLib version 2 and 3 Not Support
pZLib2017 Support

[felice2011]

More than an Update seems to be a Downgrade, I enjoy working on pzlib, great Razor.

[Mini]

thx.

unpack with CLS x64 gives an error message:
ERROR: unsupported compression method pzlib

x86 - OK

[Razor12911]

More than an Update seems to be a Downgrade, I enjoy working on pzlib, great Razor.

Downgrade in what way? I just removed options that you guys have no knowledge of how they are used then complain afterwards. For example, I've seen most people using 's' when I specified that the option is only for the game DOOM. Otherwise it messes up ratio. I've seen people setting things like b4096m or even c4096m. Pzlib will obviously crash. Maximum chunk size of program is 1792mb. And even that is very high because there are no streams in games that big. The changes now allow you to use 400mb chunk size for DOOM rather than 600mb or 700mb to capture all streams. So the only thing I did was make program decide by itself for all the options I removed.

[Razor12911]

thx.

unpack with CLS x64 gives an error message:
ERROR: unsupported compression method pzlib

x86 - OK
X64 DLLs can only be imported by x64 executables -_- CLS x64 can only work with x64 arc.exe and unarc.DLL x64

This is where this comes in http://fileforums.com/showthread.php?t=99263

@everyone
If CRC errors are gone, then bug was caused by multi threaded scanning and/or processing. Threads tend to disturb each other since they are using the same data and info to function. Problem isn't related to I/O

[felice2011]

Downgrade in what way? I just removed options that you guys have no knowledge of how they are used then complain afterwards. For example, I've seen most people using 's' when I specified that the option is only for the game DOOM. Otherwise it messes up ratio. I've seen people setting things like b4096m or even c4096m. Pzlib will obviously crash. Maximum chunk size of program is 1792mb. And even that is very high because there are no streams in games that big. The changes now allow you to use 400mb chunk size for DOOM rather than 600mb or 700mb to capture all streams. So the only thing I did was make program decide by itself for all the options I removed.

It was a sarcastic joke, obviously for the options you've taken away, and not added.;)

[LuzhinE]

It works like a charm

[shazzla]

Is any chance for a pzlib like preprocessor which unpacks ZSTD streams ?

[Gupta]

Is any chance for a pzlib like preprocessor which unpacks ZSTD streams ?

I assure you a lot of names have tried
And "also you have to be religious and ask your god if u need it even" - Deadsec

[shazzla]

I didnt know that.
Thanks!

[doofoo24]

can't set number of thread for the new cls-pzlib.dll ?
using pzlib with arc.ini set t100p is much faster...
is there cls-pzlib.ini option ???

[MCWRX]

Hi everyone! Thanks Razor again for all the work done.

I've got a question regarding the CLS. I've taken a 3GB data and compressed/decompressed via .exe and .dll. So, CLS decompresses data 2 times slower.


I've used x64 executable, x64 cls and x64 lib. dlls for compression.
During decompression I am using x86 cls.


Using x64 cls gives me CRC error at start and afaik it is not possible to use it with x86 unarc. What suggestions do you have to get nearly (not same) decompression time as in case with .exe?

And am I using the correct files for decompression process?

[Simorq]

Hi Razor12911
How do we know what is best chunk size of each game?

!
Mad Max pZLib:c32+srep+lzma 1hour Extract
Mad Max pZLib:c256+srep+lzma 35Min Extract

[harshlohaan]

Not Solve , Thanks For Reply And Help





Thanks Very Much ... Extract Perfect ... Finally

Can you please provide the code & resource files you used in your script that you used to decompress the pzlib based archive. I'm suffering from the same issue, my installer can't extract. I'm just asking for the required code and resource files that instruct the installer to extract the pzlib archives.
Please..:(

[ZakirAhmad]

What is arc.ini setting for ztool posted by razor in another thread.
Please help me.

[EzzEldin16]

don't you feel like the man who made this tool is gone because it was stolen and you are asking for an arc.ini setting give the man some respect

[ZakirAhmad]

I am not that selfish. I already thanked him. BTW he posted it in anpther thread, what is the point of posting it, if he is not going to tell anyone how to make it work.

[EzzEldin16]

okay i didn't say that you are selfish but someone in his place for example me would do the same as he did maybe more

[linggar17x]

why error disk full when i am compressing?

[darkwolves]

What is arc.ini setting for ztool posted by razor in another thread.
Please help me.

Ztool is no longer under production anyways better off using pzlib or precomp :rolleyes:
i am not going to play innocent and claim i have not tried it thats how i know..
it won't work the way you want it to it is an abandoned project by razor wich
also led to his retirement due to it being leaked and he worked very hard on that tool..
it was not supposed to be public

[felice2011]

Ztool is no longer under production anyways better off using pzlib or precomp :rolleyes:
i am not going to play innocent and claim i have not tried it thats how i know..
it won't work the way you want it to it is an abandoned project by razor wich
also led to his retirement due to it being leaked and he worked very hard on that tool..
it was not supposed to be public

Razor will not stop updating his instruments, he is in his DNA is stronger than he, an updated and improved version of ztool surely already exists, logically not to the public service...;)

[78372]

The stable(v1) version was just not leaked, but exists. Unstable version was leaked. And yep it's not updated anymore with this name.

[darkwolves]

The stable(v1) version was just not leaked, but exists. Unstable version was leaked. And yep it's not updated anymore with this name.

i don't need Ztool lol i can compress things just fine without it lol:D

[darkwolves]

Razor will not stop updating his instruments, he is in his DNA is stronger than he, an updated and improved version of ztool surely already exists, logically not to the public service...;)

given all the free he has given us it's his right to keep things private
if he wants to he never had to make any of the free tools i don't need it i compress just fine without it lol

[GBT repack]

How is the program used? I hope someone will provide a way to thank everyone

[GBT repack]

help me, thanks.....

[darkwolves]

How is the program used? I hope someone will provide a way to thank everyone

as i said its pointless to use it its not supported anymore and doesn't work
except for two functions on certain games pzlib3 does the same thing

[oltjon]

as i said its pointless to use it its not supported anymore and doesn't work
except for two functions on certain games pzlib3 does the same thing

is used by all repackers in the world and you say that is not good. :D
It is what works according to you??
With the words we are good all but with the facts are in a few and one of those who made so many facts is razor
let me see what you are good at doing..

[Razor12911]

Razor will not stop updating his instruments, he is in his DNA is stronger than he, an updated and improved version of ztool surely already exists, logically not to the public service...;)

I program far less than you think now, slowly depleting in DNA and no, no improved version of ztool exists, all that I created after that are just scattered unfinished projects like pUnity and pOodle which were just created and put away :(

is used by all repackers in the world and you say that is not good. :D
It is what works according to you??
With the words we are good all but with the facts are in a few and one of those who made so many facts is razor
let me see what you are good at doing..
It's actually true, pzlib in ztool is terrible, even sometimes using pzlib v3 is better due to the fact that the person who leaked it saved me a lot of trouble by leaking the most unstable release, the release that came before 0.0.19.8 is even better so jokes on him.
plz4 only works on Frostbite 3 engine, nothing much to offer there since lz4 is a weak algorithm, you gain 1gb diff at most.
plzo only works on anvil engine games but not all
pzstd works on Frostbite 3 engine games which happens to be only Mass Effect Andromeda and doesn't work on FIFA 17 while nor in any upcoming games because game devs decided to switch to Oodle for FIFA 18 and any other upcoming game probably so he's right by saying it's not good, the leaker leaked tool before it was worth it :)

[darkwolves]

is used by all repackers in the world and you say that is not good. :D
It is what works according to you??
With the words we are good all but with the facts are in a few and one of those who made so many facts is razor
let me see what you are good at doing..

if that is so they are modfiying it themselves doesn't change the fact it was stolen

[KaktoR]

[...]because game devs decided to switch to Oodle for FIFA 18 and any other upcoming game[...]

How you know that? Serious question.:D

[78372]

Project Cars 2 uses oodle, fifa 18 uses oodle, so maybe in that way he knows that future games will be oodle compressed :p or something else?

[Razor12911]

because of this, might be incorrect but I just follow a trend to see what might happen.

[KaktoR]

Just found out that oodle is commercial software, so no try here :D

[darkwolves]

because of this, might be incorrect but I just follow a trend to see what might happen.

need for speed the run i repacked that using plz4 so for that one it worked
on the frostbite but i did it again with pzlib3 and it worked even better

[DiCaPrIo]

need for speed the run i repacked that using plz4 so for that one it worked
on the frostbite but i did it again with pzlib3 and it worked even better

For nfstr has zlib streams not plz4

[darkwolves]

For nfstr has zlib streams not plz4

it's a frostbite2 engine game and plz4 worked on it

[78372]

Frostbite 2 engine uses zlib, not lz4, that's why you got better ratio by pzlib. Frostbite 3 uses lz4/zstd/oodle and maye more compressors. So when you got a game using FB2 engine, use pzlib.

[darkwolves]

Frostbite 2 engine uses zlib, not lz4, that's why you got better ratio by pzlib. Frostbite 3 uses lz4/zstd/oodle and maye more compressors. So when you got a game using FB2 engine, use pzlib.

yes i know but my point is it still inflates it

[Razor12911]

Just found out that oodle is commercial software, so no try here :D

:)

[Simorq]

Hi Razor12911
Do you use Repack by REALiTY of oodle?
http://uupload.ir/files/v5_2017-10-07_135912.png