FileForums - LZO precompressor (Side project)

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - LZO precompressor (Side project) (https://fileforums.com/showthread.php?t=102615)

Razor12911

13-02-2019 17:05

LZO precompressor (Side project)

2 Attachment(s)

Hi again guys, here's another procompressor for another algorithm.

Things you should know:
* This thing does not detect every single stream in the game, like specific precompressors like DELZORec when dealing with game titles like Far Cry 3 and 4.
* It's slow af from my opinion but it's first release, probably a few releases down the line, it can be made faster.
* It can be buggy, had problems setting it up, it wasn't like oodle and zstd, for now. I have no idea what causes certain bugs because firstly, I lack samples which produce bugs, I may need help in this department when you come across bugs, do report with helpful information
* For now, I have added support for lzo1x_999 (1-9)
* Max decompressed stream size should be no more than 4MB for it to be detected.

Ran a test on a portion of lzo streams I bundled together and these were results:

Code:

Compressed 1 file, 33,987,678 => 62,617,798 bytes. Ratio 184.24%

Compression time: cpu 0.06 sec/real 19.78 sec = 0%. Speed 1.72 mB/s



Extracted 1 file, 62,617,798 => 33,987,678 bytes. Ratio 184.24%

Extraction time: cpu 0.13 sec/real 3.46 sec = 4%. Speed 9.82 mB/s

Before I forget, I wonder who this guy is :rolleyes:
Attachment 24115

JustFun

13-02-2019 17:34

Quote:

Originally Posted by Razor12911 (Post 479515)

Before I forget, I wonder who this guy is :rolleyes:
Attachment 24115

Laughed soo much when I saw it, he even thanked you in this thread :D:D

darkwolves

13-02-2019 19:36

Quote:

Originally Posted by JustFun (Post 479517)

Laughed soo much when I saw it, he even thanked you in this thread :D:D

imposter lol

KaktoR

14-02-2019 01:23

Quote:

Originally Posted by Razor12911 (Post 479515)

Before I forget, I wonder who this guy is :rolleyes:
Attachment 24115

Maybe it's you from an other dimension?:D

oltjon

14-02-2019 11:52

1 Attachment(s)

I wanted to compress far cry 3

ZAZA4EVER

14-02-2019 13:01

Far Cry New Dawn

Far Cry New Dawn - patch_english.dat

Code:



D:\ZAZA\Repacks\Far Cry New Dawn\LZO Test>arc.exe a -ep1 -r -ed -s; -w.\temp -ml

zorec data.arc "pack\*"

FreeArc 0.67 (March 15 2014) creating archive: data.arc

Compressed 1 file, 122,241 => 122,253 bytes. Ratio 100.01%

Compression time: cpu 0.00 sec/real 0.31 sec = 0%. Speed 0.39 mB/s

All OK

Razor12911

14-02-2019 14:14

Quote:

Originally Posted by oltjon (Post 479526)

I wanted to compress far cry 3

could you find out what file pops up that error?

Quote:

Originally Posted by ZAZA4EVER (Post 479527)

Far Cry New Dawn - patch_english.dat

Code:



D:\ZAZA\Repacks\Far Cry New Dawn\LZO Test>arc.exe a -ep1 -r -ed -s; -w.\temp -ml

zorec data.arc "pack\*"

FreeArc 0.67 (March 15 2014) creating archive: data.arc

Compressed 1 file, 122,241 => 122,253 bytes. Ratio 100.01%

Compression time: cpu 0.00 sec/real 0.31 sec = 0%. Speed 0.39 mB/s

All OK

is the game lzo compressed?

ZAZA4EVER

14-02-2019 15:46

Quote:

Originally Posted by Razor12911 (Post 479529)

could you find out what file pops up that error?

is the game lzo compressed?

yep

FitGirl

14-02-2019 16:16

Quote:

Originally Posted by ZAZA4EVER (Post 479531)

yep

Nope

elit	14-02-2019 17:52

I was thinking about modifying QuickBMS to make it universal "precomp". It already support -O option which mean you can inflate almost any data into a single file, just like precomp or your tools do. Of course you need bms script and skill but it would remain universal forever.

Its main problem currently seem to be that it doesn't "reimport" from that single file back, it is still looking for the whole extracted directory. And of course, QuickBMS do not try to maintain 1:1 CRC so that would have to be dealt with as well. But facilities are already there. I dont know when/if I can ever get time for that but... damn. Such potential.

ZAZA4EVER

14-02-2019 22:56

1 Attachment(s)

Quote:

Originally Posted by FitGirl (Post 479532)

Nope

game Use lz4 stream ... there is differance betwwen lzo & lz4?
sorry for my mistake

KaktoR

15-02-2019 01:36

^This scanner is crap, even razor say this. Use GFS instead.

Razor12911

15-02-2019 01:37

Quote:

Originally Posted by ZAZA4EVER (Post 479535)

there is differance betwwen lzo & lz4?

I'm sorry but I can't help it

https://i.gifer.com/Uiv.gif

FitGirl

15-02-2019 02:01

Quote:

Originally Posted by ZAZA4EVER (Post 479535)

game Use lz4 stream ... there is differance betwwen lzo & lz4?
sorry for my mistake

lzo, lzma, lzham, lolz, lzma2, lz4, lz5, rar, zip are all the same. Devs are simply having fun renaming them here and there.

ZAZA4EVER

15-02-2019 02:52

Quote:

Originally Posted by FitGirl (Post 479538)

lzo, lzma, lzham, lolz, lzma2, lz4, lz5, rar, zip are all the same. Devs are simply having fun renaming them here and there.

Thanks Fitgirl for your replay and respect

ZAZA4EVER

15-02-2019 03:49

Quote:

Originally Posted by Razor12911 (Post 479537)

I'm sorry but I can't help it

https://i.gifer.com/Uiv.gif

Your replay is not interested in it .. because every day I learn

doofoo24

15-02-2019 04:07

Quote:

Originally Posted by ZAZA4EVER (Post 479540)

every day I learn

you don't seems to get it :o:rolleyes:

ZAZA4EVER

15-02-2019 04:31

Quote:

Originally Posted by doofoo24 (Post 479542)

you don't seems to get it :o:rolleyes:

Ok man as you see

elit	15-02-2019 12:53

ZAZA, both lzo and lz4 are aimed at same thing - (de)compression speed and both are LZ variants. However they are made by different people and are internally different, thus incompatible. Maybe you misread lzo as lz0 and thought lz4 is just higher and backward compatible version? Thats not the case, its lzo not lz0 and they are different libraries with their own LZ variant algo implementation. Hope it help.

oltjon

15-02-2019 13:35

Quote:

Originally Posted by Razor12911 (Post 479529)

could you find out what file pops up that

the file are multicommon_vistas.dat, common.dat, patch.dat,
Sound.dat

elit	17-02-2019 04:44

Quote:

Originally Posted by elit (Post 479533)

I was thinking about modifying QuickBMS to make it universal "precomp".

Well, maybe the most versatile, ultimate, multi-format, multi-cmp-method "precomp" is on its way after all :rolleyes: :
https://zenhax.com/viewtopic.php?f=13&t=9504

Simorq

19-02-2019 12:04

Fc3

fc3_main.dat 3.25 GB

Code:

Creating archive: Test.Bin using rep+lzorec

Compressed 1 file, 3,496,061,837 => 4,593,335,651 bytes. Ratio 131.39%     

Compression time: cpu 13.44 sec/real 3423.74 sec = 0%. Speed 1.02 mB/s

All OK



Testing archive: Test.Bin

Tested 1 file, 4,593,335,651 => 3,496,061,837 bytes. Ratio 131.39%         

Testing time: cpu 2.47 sec/real 98.14 sec = 3%. Speed 35.62 mB/s

All OK

C:( D:D

yasitha

22-02-2019 03:41

Quote:

Originally Posted by KaktoR (Post 479519)

Maybe it's you from an other dimension?:D

No. No i think That's his Ex-Wife :D
Messing around..

otherwise who would do that kind of stupid thing :D
It's a zero *0* not O lol ��

All times are GMT -7. The time now is 15:02.