FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   PZlib (Alternative to Precomp) (https://fileforums.com/showthread.php?t=97888)

ChronoCross 23-05-2016 23:56

Don't worry mini. Is beta test. Test pzlib only. I was tested a game 68gb with pzlib+lzma and my hdd now is full and the test never end. But the application not crash. :)

Razor12911 24-05-2016 12:41

Quote:

Originally Posted by Mini (Post 449191)
Dark Souls 3
Data5.bdt - 11.4 gb

pzlib v1.0 (max level) + lzma = 9.95 gb
pzlib v1.1 + lzma = 11 gb (http://i.imgur.com/Jt9r2qn.png)

wtf

is restoration possible for version 1.0?

Quote:

Originally Posted by ChronoCross (Post 449200)
Don't worry mini. Is beta test. Test pzlib only. I was tested a game 68gb with pzlib+lzma and my hdd now is full and the test never end. But the application not crash. :)

I try to avoid errors.

Update available

Changes


Threads inversely proportional to speed for stability (new maximum for threads is 4 instead of 8 )
-Speed for multi threading improved
-ms conversion to readable time added as requested
-encode/decode can be shorted to e/d

Mini 25-05-2016 08:54

Quote:

is restoration possible for version 1.0?
dont know

reflate result - 9.96 gb

I want to receive the same result with the help pzlib :)

felice2011 04-06-2016 09:55

Hi Razor i have a problem in the data decompression using "PZlib 1110" I get this error.

http://i63.tinypic.com/2s991dt.png

Code:

unpackcmd = pzlib d - -o - <stdin> <stdout>
:confused:

Razor12911 04-06-2016 12:46

that's weird. will check.

Prince4 04-06-2016 22:40

Can we integrate this with ISDone?

felice2011 25-06-2016 10:42

Hi Razor, news of some stable update. I was willing to use it but I always problems on decompression.

Razor12911 05-07-2016 10:44

Quote:

Originally Posted by felice2011 (Post 449708)
Hi Razor, news of some stable update. I was willing to use it but I always problems on decompression.

What problems?

Carldric Clement 05-07-2016 11:34

Quote:

Originally Posted by felice2011 (Post 449708)
Hi Razor, news of some stable update. I was willing to use it but I always problems on decompression.

If you problem about your Compressor GUI, like my Mini Compressor, use this if helpful:
Code:

[External compressor:pzlib]
header = 0
unpackcmd = pzlib d $$arcpackedfile$$.tmp -o $$arcdatafile$$.tmp


felice2011 05-07-2016 15:16

Quote:

Originally Posted by Carldric Clement (Post 449871)
If you problem about your Compressor GUI, like my Mini Compressor, use this if helpful:
Code:

[External compressor:pzlib]
header = 0
unpackcmd = pzlib d $$arcpackedfile$$.tmp -o $$arcdatafile$$.tmp


I do not think that's what the problem was too obvious, already I tried to decompress with both lines:
Code:

unpackcmd = pzlib d $$ arcpackedfile $$. tmp -o $$ arcdatafile $$. tmp
&
Code:

unpackcmd = pzlib -d - -o - <stdin> <stdout>
In fact out of curiosity I wanted to try your utility, the same error.

http://i68.tinypic.com/sdzrqc.png
Quote:

Originally Posted by Razor12911 (Post 449867)
What problems?

The problem is always the same, already posted ...

http://www.fileforums.com/showthread...7888&page=3#45

Using the line "unpackcmd = pzlib -d - -o - <stdin> <stdout>"

&

http://i67.tinypic.com/9hszlz.png

Using the line "unpackcmd = pzlib d $$arcpackedfile$$.tmp -o $$arcdatafile$$.tmp"

Personally I think this is one of the best compression projects developed. On my part I have much interest in the excellent level of compression, with a high speed of execution of the packing, compared to the classic precompressor.
My intention was also to integrate the MSK in the latest version, but if errors are generated during decompression of the archive created, I cannot guarantee the complete functionality.
I examine of more to see if there any file or type of files that can generate the error during decompression.

Razor12911 05-07-2016 16:01

add -v option during comprssion if you have CRC errors during decompression

[External compressor:pzlib]
header = 0
packcmd = pzlib e - -o - <stdin> <stdout>
unpackcmd = pzlib d - -o - <stdin> <stdout>

felice2011 06-07-2016 03:11

OK I solved used PZlib v1100 (24/05/16), with this line everything works perfectly in Compr & Decomp..;)

Code:

[External compressor:pzlib]
header = 0
packcmd = pzlib e -v - -o - <stdin> <stdout>
unpackcmd = pzlib d - -o - <stdin> <stdout>

:p

LuzhinE 06-07-2016 17:35

He tried in PES 2016, is good PZlib

Razor12911 07-07-2016 04:11

Quote:

Originally Posted by felice2011 (Post 449878)
OK I solved used PZlib v1100 (24/05/16), with this line everything works perfectly in Compr & Decomp..;)

Code:

[External compressor:pzlib]
header = 0
packcmd = pzlib e -v - -o - <stdin> <stdout>
unpackcmd = pzlib d - -o - <stdin> <stdout>

:p

I even mentioned beginning of topic, if you encounter CRC errors or just to make sure CRC stays the same, add -v

Code:

Options:
-t#: Number of threads to use (maximum = 8)
-t#p: (default = 50) Percentage of threads to use
-v: Verification so that restoration keeps CRC the same
-ai: Prints out additional information


LuzhinE 07-07-2016 23:02

.arc not compress files

felice2011 07-07-2016 23:32

Quote:

Originally Posted by LuzhinE (Post 449905)
.arc not compress files

:confused:

felice2011 08-07-2016 05:05

Sorry Razor but two out of three times I get crash of the arc.exe at the beginning of compression ...

https://goo.gl/ylQXJL

I don't understand what the problem...:o

Razor12911 08-07-2016 05:31

It happens, freearc bug.

felice2011 08-07-2016 06:02

Quote:

Originally Posted by Razor12911 (Post 449918)
It happens, freearc bug.

:eek:

Tests performed on 33 different compressors, combined with various methods and masks, never any crashes with arc.exe, the thing is quite strange.:confused:

EDIT:

Probably I solved the problem lies in the line of options, run other tests of safety and I post the right configuration.

Final Test PZlib V1100 recommended options if you are experiencing app-crash in arc.exe

Comp;

Code:

[External compressor:pzlib]
header = 0
packcmd = pzlib e -v $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp


Decomp;


Code:

[External compressor:pzlib]
header = 0
unpackcmd = pzlib d - -o - <stdin> <stdout>


Razor12911 09-07-2016 04:09

It's now an ordinary bug on my side, compressors with stdin and stdout during compression, and if you're compressing a large number of files, this happens. Pretty much annoying.

felice2011 09-07-2016 04:50

Quote:

Originally Posted by Razor12911 (Post 449931)
Pretty much annoying.

:cool:

;)The important that the problem is solved, even with a large number of files...:p

Razor12911 09-07-2016 05:22

:p but still, when everything is piped, overall speed is better.

felice2011 10-07-2016 11:10

Razor I need of the command for the console help info of PZlib and Reflate, if you have implemented.
I tried it with the command "pzlib.exe -h" or "pzlib.exe -help" but it does not work.

Razor12911 11-07-2016 03:15

well unfortunately, i haven't added that.

felice2011 11-07-2016 03:36

Quote:

Originally Posted by Razor12911 (Post 449976)
well unfortunately, i haven't added that.

Ah..Ok.
I am entering a new source on the help info in MSK on 34 different compressors, when I will have finished to compile the ContextMenuStrip I send you a PM to complete the info on some compressors.
Thanks Bro..

LuzhinE 14-07-2016 14:30

and error because?

http://i.imgur.com/gEl4I5o.png

felice2011 14-07-2016 23:39

Check set..http://fileforums.com/showthread.php?t=97888&page=4#60

Razor12911 18-08-2016 10:56

Update for PZLib available

Changes

Only added x64 support.

Comparison between x86 and x64

Code:

File = UI.sb
Size = 82.3MB > 348MB

Recompression times with verification off:
x86: 4.81 Seconds (17.93MB/s)
x64: 4.20 Seconds (20.57MB/s)

Recompression times with verification on:
x86: 38.27 Seconds (2.26MB/s)
x64: 26.65 Seconds (3.24MB/s)

Restoration times using 4 threads:
x86: 11.08 Seconds (7.79MB/s)
x64: 7.95 Seconds (10.87MB/s)


Prince4 18-08-2016 23:15

Quote:

Originally Posted by Razor12911 (Post 450864)
Update for PZLib available

Changes

Only added x64 support.

Comparison between x86 and x64

Code:

File = UI.sb
Size = 82.3MB > 348MB

Recompression times with verification off:
x86: 4.81 Seconds (17.93MB/s)
x64: 4.20 Seconds (20.57MB/s)

Recompression times with verification on:
x86: 38.27 Seconds (2.26MB/s)
x64: 26.65 Seconds (3.24MB/s)

Restoration times using 4 threads:
x86: 11.08 Seconds (7.79MB/s)
x64: 7.95 Seconds (10.87MB/s)


Where's the download link? :P

mausschieber 19-08-2016 01:10

Quote:

Originally Posted by Prince4 (Post 450890)
Where's the download link? :P

look at first post

LuzhinE 19-08-2016 14:43

It does not open how it works?

Razor12911 19-08-2016 15:03

Quote:

Originally Posted by LuzhinE (Post 450913)
It does not open how it works?

-_-, this works with Freearc man...

RamiroCruzo 19-08-2016 15:10

Quote:

Originally Posted by Razor12911 (Post 450915)
-_-, this works with Freearc man...

hahaha...

Razor12911 11-09-2016 11:08

Update for PZLib available

Changes

Updated multi threading

chum 18-09-2016 18:31

Hey bro, you said to use -v for CRC problems, but using -v just blocked any compressing gains, the compressed file just become out same size of input file. Without -v decompressing is just failing with CRC Always.

I like Pzlib for compresisng since it is much faster than Precomp and give nearly same final size but decompressing problem is just avoiding to it be a standard usage for me.

If you just remove the multithreading for decompressing dont it fix the problem since decompressing is known to work properly in single-thread..

Razor12911 18-09-2016 18:40

Quote:

Originally Posted by chum (Post 451809)
Hey bro, you said to use -v for CRC problems, but using -v just blocked any compressing gains, the compressed file just become out same size of input file. Without -v decompressing is just failing with CRC Always.

That isn't the case most of the time.

Quote:

I like Pzlib for compresisng since it is much faster than Precomp and give nearly same final size but decompressing problem is just avoiding to it be a standard usage for me.

If you just remove the multithreading for decompressing dont it fix the problem since decompressing is known to work properly in single-thread..
I don't understand this part here.

aswadd 18-09-2016 22:01

I have a problem
CRC failed in "data\corecharacter.arch05". File is broken.
http://i.imgur.com/uwGpfaJ.png
it happened with any file type :\
I also turned off threads but still this error / tried -v option but it didn't even make an output file :(

aswadd 18-09-2016 22:16

Quote:

Originally Posted by Razor12911 (Post 451810)
That isn't the case most of the time.


I don't understand this part here.

the problem goes with -v parameter , the way you add it in arc.ini :D
I finally got it work comp/decomp was success

Just add -v at the end of the line & not after pzlib ex:
packcmd = pzlib e - -o - <stdin> <stdout> -v

don't put it like this or even in the middle ... etc
packcmd = pzlib -v e - -o - <stdin> <stdout>

same problem happens with antiz Just add -r at the end :D

in the end :D thnx razor & Panker for this great work :)

Razor12911 19-09-2016 14:25

Guys, I said this in the beginning of the thread, you must add -v if you're getting CRC errors.

-v is just verification, it just makes sure that the streams that were found match CRC when being restored, it permutes some streams which is why you get a better size without verification than when it's turned on.

aswadd 19-09-2016 15:47

I got another problem Arc crash when trying pzlib on
Need for speed MW 2012 :( while other pre-compressors didn't make arc crash
http://i.imgur.com/bgaNOkF.png


All times are GMT -7. The time now is 19:09.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com