FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Oodle precompressor (Side project) (https://fileforums.com/showthread.php?t=102453)

KaktoR 13-06-2020 12:39

This game use kraken?

Edit: https://fileforums.com/showpost.php?...23&postcount=7 + https://fileforums.com/showpost.php?...21&postcount=5

Masquerade 13-06-2020 13:02

Quote:

Originally Posted by KaktoR (Post 486348)

https://i.imgur.com/8Imx1Yi.png

If I am incorrect, I will try a different tool. I saw that panker pointed to poodle including an oo2core_4_win64.dll, will this not cause incompatibility with the oo2core_6 used in Sekiro?

I read that you recommended oo2reck on another thread: https://www.fileforums.com/showpost....18&postcount=2

As well as another poster: https://www.fileforums.com/showpost....44&postcount=8

KaktoR 13-06-2020 13:07

Only because I totally forgot about ztool oodle :D

I would say just test on a single file and compare :p

Masquerade 13-06-2020 13:13

Quote:

Originally Posted by KaktoR (Post 486351)
Only because I totally forgot about ztool oodle :D

I would say just test on a single file and compare :p

Thank you, I will use ZTool. What is the purpose of oo2ext dlls? I hope they aren't required since I do not have one for version 6, only a core.

Edit: ZTool doesn't appear to have a oodle option, perhaps I am missing something?

Edit 2: I just figured out Poodle will work with different cores. I think I am sorted. Thank you again sir.

Edit 3: poodle has 100% ratio on a 516mb sample.

Edit 4: kraken has 100% ratio too.

I am slightly clueless.

Masquerade 13-06-2020 13:42

Update: I also tried XTool Oodle but still no luck. 100% ratio every time.

KaktoR 13-06-2020 13:46

XTool oodle is broken (still). Hopefully razor12911 sort this problem somewhere in future :p

I am download this game right now again and will test something.

Masquerade 13-06-2020 13:57

Quote:

Originally Posted by KaktoR (Post 486354)
XTool oodle is broken (still). Hopefully razor12911 sort this problem somewhere in future :p

I am download this game right now again and will test something.

I can send you sample, save you some time and bandwidth if you wish.

KaktoR 13-06-2020 14:26

Already finished :p

KaktoR 13-06-2020 14:34

Data2.bdt
Code:

Compressed 1 file, 387,579,088 => 717,672,363 bytes. Ratio 185.17%
Compression time: cpu 0.36 sec/real 39.50 sec = 1%. Speed 9.81 mB/s
All OK

Extracted 1 file, 717,672,363 => 387,579,088 bytes. Ratio 185.17%
Extraction time: cpu 0.23 sec/real 46.70 sec = 1%. Speed 8.30 mB/s
All OK


Arc.ini
Code:

[External compressor:zOodle,zoodle]
header = 0
packcmd  = "_Compressors\zoodle\zOodle_x64" e:poodle:t75p:cm0 - - <stdin> <stdout>
unpackcmd = "_Compressors\zoodle\zOodle_x64" d:poodle:t75p:cm0 - - <stdin> <stdout>

oo2core_4_win64.dll, oo2ext_4_win64.dll

Masquerade 13-06-2020 14:39

Quote:

Originally Posted by KaktoR (Post 486357)
Data2.bdt
Code:

Compressed 1 file, 387,579,088 => 717,672,363 bytes. Ratio 185.17%
Compression time: cpu 0.36 sec/real 39.50 sec = 1%. Speed 9.81 mB/s
All OK

Extracted 1 file, 717,672,363 => 387,579,088 bytes. Ratio 185.17%
Extraction time: cpu 0.23 sec/real 46.70 sec = 1%. Speed 8.30 mB/s
All OK


Arc.ini
Code:

[External compressor:zOodle,zoodle]
header = 0
packcmd  = "_Compressors\zoodle\zOodle_x64" e:poodle:t75p:cm0 - - <stdin> <stdout>
unpackcmd = "_Compressors\zoodle\zOodle_x64" d:poodle:t75p:cm0 - - <stdin> <stdout>

oo2core_4_win64.dll, oo2ext_4_win64.dll

Wow! So if it can be precompressed with core 4, why does the game include core 6 (or are they interchangeable)?

Where can I finf a copy of ZTool since my copy doesn't have an Oodle option in the list of codecs? I currently assume UltraArc will have such a copy?

KaktoR 13-06-2020 14:43

If I remember correctly oodle codec for ztool was standalone (seperated from ztool). I think it was a leaked version. Use the files from previous linked post.

And I don't know why it won't work with the o6 library. Just use the o4 library and all is good :p

Masquerade 14-06-2020 01:05

I went back today and did some digging, it turns out you are correct in saying that core 4 works, because I removed the core 6 from oodle rec and replaced with core + ext 4 from poodle and I got better results than poodle:

https://i.imgur.com/Qk1g1AM.png

Here they are side by side. "kraken" is oo2reck while poodle is poodle.

Tested on Data4.bdt (516mb).

Update: Precompression has worked, here is result:

https://i.imgur.com/lxQGJC5.png

panker1992 14-06-2020 17:35

Quote:

Originally Posted by Masquerade (Post 486362)
I went back today and did some digging, it turns out you are correct in saying that core 4 works, because I removed the core 6 from oodle rec and replaced with core + ext 4 from poodle and I got better results than poodle:

https://i.imgur.com/Qk1g1AM.png

Here they are side by side. "kraken" is oo2reck while poodle is poodle.

Tested on Data4.bdt (516mb).

Update: Precompression has worked, here is result:

https://i.imgur.com/lxQGJC5.png


"That" Game you talk about is my favorite game :D

I have made it 7.1GB in total patch 1.04

use my way if you want and here are the blocks if you want to go sfk style.

offset (hex)

Block 1
17f332aa to 1b898ca9

Block 2
28c02dba to 53ae4699

Block 3
55e2a35a to 59987939

Block 4
5d37301a to 622e4cf9

Block 5
647e7c6a to 66b999a9

Block 6
6acdb9ba to 754c22d9

Block 7
79eb250a to 7c00e649

Remember to pack sounds together except the language ones :D

Masquerade 15-06-2020 01:21

Hello panker1992, I hope my mess of oodlerec hasn't offended you or your favourite game :o

I do not know what you mean by "go sfk style" - I saw in your Resident Evil 3 Remake conversion that is some kind of patching tool. I hope you don't mind if I ask which files contain which blocks?

Will this result in a CRC perfect installation (when compared with retail build)?

Thank you.

panker1992 15-06-2020 02:11

Quote:

Originally Posted by Masquerade (Post 486377)
Hello panker1992, I hope my mess of oodlerec hasn't offended you or your favourite game :o

I do not know what you mean by "go sfk style" - I saw in your Resident Evil 3 Remake conversion that is some kind of patching tool. I hope you don't mind if I ask which files contain which blocks?

Will this result in a CRC perfect installation (when compared with retail build)?

Thank you.


Let me answer your questions one by one.

Fitgirl went sfk style first and she manages to keep crc the same.
it Separates data in large archives and it makes lolz ridiculously fast and efficient. in some options it completely removes data... foreign languages you dont need.


the above blocks i removed result in all languages being removed and you get to keep what you want from folder "sound" of the game.

Data1.bdh has the entire folder named sound compressed in it, by removing the data you wont miss anything but a bad crc. you can even repatch it after to get it perfect crc match.

another example: DMC5 has a 32GB file which contains 10GB of movies and sounds.

if you sfk remove the 10GB of high entropy low compressability of stuff you will feed lolzi with pure textures and get high compressed file and very good speed.



In order to remove the data from the blocks above you need hex editor or convert them in decimal and use sfk, then zero them out and patch them back into position as shown in re3.
its only 7 blocks so use hex editor it will fast :D

Using a xdelta or zstd patch or bsdiff is a bad idea since you know what blocks to remove/inject.

Masquerade 15-06-2020 02:39

Thank you for the explanation.

I'm following slightly, here's what I understand:

Data1.bdh contains the sound folder, so you can use sfk to remove it (effectively removing the duplicated data) then after installing you can patch the sound folder back in by using sfk again.

Quote:

In order to remove the data from the blocks above you need hex editor or convert them in decimal and use sfk, then zero them out and patch them back into position as shown in re3.
So from this I gather that in the hex editor, it will show address "17f332aa" and then you go to address "1b898ca9" and replace all of the data between these 2 addresses with 0s so that it essentially becomes empty - leading to a deduplicated file with more efficient compression - yes?

Quote:

its only 7 blocks so use hex editor it will fast :D
Are all of these blocks in Data1.bdh or are they scattered over the 5 data files?

Quote:

Using a xdelta or zstd patch or bsdiff is a bad idea since you know what blocks to remove/inject.
So If you didn't know what parts of a file to remove and reinject, how would xdelta help? I thought xdelta can only create a patch based on an old file and a new file (by creating a patch with changed data betweeen the 2) - this part confuses me, because how will you be able to know which part of the file to remove?

I see the method would be create an xdelta patch using the modified, smaller file, and the original file then apply it after installing.

panker1992 17-06-2020 08:47

you select the block of the data then right click fill block with zeroes :D

i think i posted a how to on the forum. and i will likely pop a video at some point but i am bad at videos!!

even if you pack the game itslef entirely it will cost you 8 GB with all languages.

bunti_o4u 11-07-2020 11:07

Quote:

Originally Posted by Razor12911 (Post 481936)
then add more threads :)
also good to know there is something that uses leviathan, should be easy to add

Hi Razor
do you have any plan to add leviathan and hydra?

Razor12911 11-07-2020 15:38

yeah but it will be part of xtool, currently busy with the project so you're gonna have to wait.

prince55 18-07-2020 18:38

Death Stranding
 
this game seems use oodle but i tested some files and there is no expand
Is it encrypted or i miss something:confused::confused:

Razor12911 18-07-2020 18:44

It's encrypted, I took a look at it and tried to make decryption tool

prince55 19-07-2020 03:21

Quote:

Originally Posted by Razor12911 (Post 486917)
It's encrypted, I took a look at it and tried to make decryption tool

will you continue to make this decryption tool ?

It will be great !

Razor12911 19-07-2020 15:19

Lel you'd have to wait for these guys
https://forum.xentax.com/viewtopic.p...21829&start=45

I don't have this game, only have a few samples and I have never dealt with the encryption used in the game so wait for the people in that forum to come up with ideas and I may use their findings to make decryption tool for freearc.

prince55 20-07-2020 00:28

Sure, we all will wait for them and you too :)
Thanks

L33THAK0R 29-05-2021 07:42

Sorry to revive an old topic with a rather silly lot of questions, but what exactly should the arc.ini and resource folder structure look like, when attempting to use more than one of the available Oodle pre-compressors? The title I'm attempting to apply these pre-compressors too is the GOG release of "Wolfenstein II (2): The New Colossus", the Oodle stream scanner tool indicates both Oodle Mermaid & Selkie are used.

I've only ever compressed titles using the Oodle Kraken precompressor included with XTool 2020 and have never used this project before (out of both a lack of understanding and a lack of need). My arc.ini currently looks like this, with the string of compressors to use (Tiny Compressor), looking like this.

Two areas of confusion that I have is how exactly does oo2rec(k/m/s).exe know where to look for the oo2core..._win64.dll, and where should this .dll be located relative to the compressor's .exe? The current structure I have looks like this (oo2core_5_win64.dll is the same .dll present in the "Wolfenstein II: The New Colossus" root directory.).

I've gone through the included arc.ini in the attached tool on this topic as well as the pack/unpack.bat files but I still can't seem to understand how to operate these tools. If anyone is willing to help me out and requires any additional information I am happy to do so. Thank you for reading through the entirety of this rather lengthy post, any help anyone can offer will be greatly appreciated!

Masquerade 29-05-2021 07:51

Your arc.ini is incorrect.

Code:

[External compressor:oo2reck,oo2recm,oo2recs]
header    = 0
packcmd  = {compressor} e <stdin> <stdout>
unpackcmd = {compressor} d <stdin> <stdout>

This is the packcmd given. {compressor} translates the name of the method into the exe file used for the method.

Put the oodle core in the same folder as the oodle precompressor. It will automatically use it.

Then your method: oo2recm+oo2recs+srep+lolz

L33THAK0R 29-05-2021 19:13

Quote:

Originally Posted by Masquerade (Post 492492)
Your arc.ini is incorrect.

Code:

[External compressor:oo2reck,oo2recm,oo2recs]
header    = 0
packcmd  = {compressor} e <stdin> <stdout>
unpackcmd = {compressor} d <stdin> <stdout>

This is the packcmd given. {compressor} translates the name of the method into the exe file used for the method.

Put the oodle core in the same folder as the oodle precompressor. It will automatically use it.

Then your method: oo2recm+oo2recs+srep+lolz

Thank you very much for your speedy response! Unfortunately I'm still unable to compress the files of "Wolfenstein II (2): The New Colossus". My arc.ini now looks like the following, and the folder structure for the Oodle pre-compressor tool now looks like the following. When attempting to compress the files, however, I recieve this error. The same error is received regardless of the order of the Mermaid & Selkie Oodle pre-compressors used. I'm honestly rather confused why this tool refuses to play nicely.

:( Sad8669 29-05-2021 22:14

Brother, I have mentioned it before, that you have to use <stdin> <stdout>.

Here, for your satisfaction i have tested it on your favourite tool.

$$arcdatafile$$.tmp $$arcpackedfile$$.tmp :
Code:

FreeArc 0.67 (March 15 2014) updating archive: Data.arc
Compressing 1 file, 5,656,576 bytes. Processed  0%
Compressing 5,656,576 bytes with "Resources\OodleRec\oo2reck" e $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

Errorlevel=0

ERROR: general (de)compression error in oo2reck

<stdin> <stdout> :
Code:

FreeArc 0.67 (March 15 2014) updating archive: Data.arc
Compressed 1 file, 5,656,576 => 6,925,113 bytes. Ratio 122.43%
Compression time: cpu 0.02 sec/real 1.18 sec = 1%. Speed 4.78 mB/s
All OK

Arc.ini :
Code:

[External compressor:oo2reck,oo2recs,oo2recm]
header    = 0
packcmd  = "Resources\OodleRec\{compressor}" e <stdin> <stdout>

Note : All test are performed using Tiny Compressor v1.4 and always use <stdin> <stdout> when using constants.

L33THAK0R 29-05-2021 23:46

Quote:

Originally Posted by :( Sad8669 (Post 492494)
Brother, I have mentioned it before, that you have to use <stdin> <stdout>.

Ah, I assumed that
Code:

<stdin> <stdout>
and
Code:

$$arcdatafile$$.tmp $$arcpackedfile$$.tmp
could be used interchangeably, thank you for clearing this up!

Masquerade 29-05-2021 23:58

Quote:

Originally Posted by L33THAK0R (Post 492495)
Ah, I assumed that
Code:

<stdin> <stdout>
and
Code:

$$arcdatafile$$.tmp $$arcpackedfile$$.tmp
could be used interchangeably, thank you for clearing this up!

This is true only in some cases. E.g. XTool and SREP can either run in stdio or $$arc$$ mode. It's best not to deviate from whatever is given to you in the packing example unless you are prepared ro sit and test various methods.

L33THAK0R 30-05-2021 01:57

Quote:

Originally Posted by Masquerade (Post 492496)
This is true only in some cases. E.g. XTool and SREP can either run in stdio or $$arc$$ mode. It's best not to deviate from whatever is given to you in the packing example unless you are prepared to sit and test various methods.

I'll be sure to do this in the future! Thank you both for your very useful help!

yasitha 12-05-2022 21:42

hey guys, where to find latest DLL files on below list?

DLLs:
oo2core_4_win64.dll
oo2core_5_win64.dll
oo2core_6_win64.dll
oo2core_7_win64.dll

elit 17-07-2022 04:08

1 Attachment(s)
Quote:

Originally Posted by yasitha (Post 496790)
hey guys, where to find latest DLL files

This have all dll's v4-9:
Attachment 32205

On the side note, I found it interesting that oo2reck can deal with complex unreal structure, like in upk files in XCOM2 for example, but xtool with -mue3+kraken does not. And I tried both ue3:m1 and ue3:m2.

dixen 17-07-2022 11:23

Quote:

Originally Posted by elit (Post 497652)
This have all dll's v4-9:
Attachment 32205

On the side note, I found it interesting that oo2reck can deal with complex unreal structure, like in upk files in XCOM2 for example, but xtool with -mue3+kraken does not. And I tried both ue3:m1 and ue3:m2.

XCOM2 - only uelr worked and xtool (zlib+kraken)

elit 17-07-2022 17:58

Quote:

Originally Posted by dixen (Post 497659)
XCOM2 - only uelr worked and xtool (zlib+kraken)

In XCOM2, there are 2 "xcomgame" directories containing upk and tfc files: One in main dir == uelr.
The other inside xcom2-warofthechosen == oodle[kraken].
Finally, in both there are still additional DLC/day0 dirs, and those have upk's that are uncompressed but full of png's and xtool:mpng:reflate:d1 will eat those for breakfast.
Thats it. There is no "zlib+kraken" or anything like that. But my point is that without specialized plugin, xtool won't work on those oodle compressed files, but oo2reck does out of the box.

KaktoR 18-07-2022 06:30

Quote:

Originally Posted by elit (Post 497661)
In XCOM2, there are 2 "xcomgame" directories containing upk and tfc files: One in main dir == uelr.
The other inside xcom2-warofthechosen == oodle[kraken].
Finally, in both there are still additional DLC/day0 dirs, and those have upk's that are uncompressed but full of png's and xtool:mpng:reflate:d1 will eat those for breakfast.
Thats it. There is no "zlib+kraken" or anything like that. But my point is that without specialized plugin, xtool won't work on those oodle compressed files, but oo2reck does out of the box.

dixen is right.

For *.upk, *.tfc you should use like this
Code:

uelr+xtool:zlib+oo2reck
With exclusion of *_XPACK*.upk and *_XPACK*.tfc
For this you can just use oo2reck

dixen 18-07-2022 06:40

Quote:

Originally Posted by elit (Post 497661)
Thats it. There is no "zlib+kraken" or anything like that.

Really?
https://fileforums.com/showpost.php?...postcount=1580

elit 19-07-2022 04:23

I was testing it with lzscan, so I am fairly sure my info is correct.
Again, using all 3 together(uelr+oo2reck+xtool) can only be handy if you want to pass through whole game in 1 go and don't want to be bothered to separate. But warofthechosen does NOT use uelr and base xcomgame does NOT use oodle, and NEITHER contain any significant zlibs inside.
Only full of png's are upk's in Day0 DLC 2 dirs and for those you need -mpng method, NOT zlib. My version is the same as latest fitgirl version.

Go on and test it individually if you don't believe me. I did. My final repack is of same size as fitgirl's, minus ~5gb, because I recoded bink videos and saved about that much.

dixen 19-07-2022 09:23

Quote:

Originally Posted by elit (Post 497688)
Go on and test it individually if you don't believe me.

Dude, I don't care about your problems and I'm not going to test anything, Sorry

elit 19-07-2022 13:04

Quote:

Originally Posted by dixen (Post 497693)
Dude, I don't care about your problems and I'm not going to test anything, Sorry

There is no "problem", I am telling you you are simply wrong. Feel free to ignore it I really couldn't care less.


All times are GMT -7. The time now is 01:44.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com