FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   new tool (https://fileforums.com/showthread.php?t=99953)

KaktoR 16-02-2018 08:17

Quote:

Originally Posted by Simorq (Post 466742)
Outlast 2
Get Even
BioShock Infinite
Mortal Kombat X (Uelr:x)

Mass Effect 1+2

3 is using pzlib

elit 16-02-2018 13:00

Quote:

Originally Posted by vollachr (Post 466732)
Any ideas what the problem might be?

As the output says, stream does not contain LZO compressed blocks(that uelr recognize). You need to apply this to only known Unreal LZO files that uelr can surely process.

elit 16-02-2018 14:38

1 Attachment(s)
Game Finding Paradise contain .ogg files that crash OGGRE, either already during compression or later in decompression. This one for example:
Attachment 21035
crash during decompression test. So tool is still buggy, use with caution.
EDIT: Here is the cmd output:
Code:

D:\Downloads\lz_scan>OGGRE_enc.exe 008-System08.ogg 008-System08.ogg.cmp
OGGRE v0.1.1
ogg_dedup.duplicates_size = 0kb
ogg_dedup.dedup_buf_size = 0kb
frames_dedup.duplicates_size = 0kb
frames_dedup.dedup_buf_size = 0kb
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
residue val overflow!
number of used codebooks: 14


vollachr 18-02-2018 00:48

1 Attachment(s)
Quote:

Originally Posted by elit (Post 466757)
As the output says, stream does not contain LZO compressed blocks(that uelr recognize). You need to apply this to only known Unreal LZO files that uelr can surely process.

Already got the answer but thanks anyway.

....

Also, now that I tested it correctly it seems to work fine.

Another question though, regarding lolz...

is there any way to prevent it from output all those numbers on the CMD console while it compressing? (See Attached Image)

I'm trying to have a clean compression script that only display minimal output and display progress in title bar.

My Command is something like that:

Code:

Arc.exe a -ep1 -r -ed -s; -w"PathTotemp" -dp"PathToGameFolder" -mlolz "H:\Temp\Test.bin" "*" >nul
if I Remove the >nul at the end it display all the lolz information and appears to be showing normal progress instead of those numbers but I need the >nul to be there.

My lolz Arc.ini entry:

Code:

[External compressor:lolz]
header = 0
packcmd = lolz\lolz_x64.exe {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

I don't really understand the different options in lolz, so any ideas will be appreciated.

doofoo24 25-02-2018 04:57

use lolz a7.
best setting for lolz with pc 16gb ram and 8 thread cpu...
memory during compression 12.5gb...
for decompression it recommended 2 thread with pzlib...
with uler or afr 4 thread for lolz...

vollachr 25-02-2018 05:03

Quote:

Originally Posted by doofoo24 (Post 468412)
use lolz a7

Tried that since, same result, it seems to need some kind of patch that I have no idea how to make.

doofoo24 25-02-2018 19:49

setting for oggre thank to Mickey1s from krinkels...

usage: oggre_enc.exe [-sX] [-dd] [-df] [-ds] infile outfile
-sX - set books statistic buffer size, X = [0;3].
-dd - disable ogg deduplication
-df - disable setup frame deduplication
-ds - disable solid mode

Sergey3695 26-02-2018 00:09

http://i100.fastpic.ru/big/2018/0226...5e488ca7a8.jpg
:)

ZakirAhmad 01-03-2018 01:12

how much compression ratio u lose when using multithreading in lolz

78372 08-03-2018 01:44

Quote:

Originally Posted by vollachr (Post 468413)
Tried that since, same result, it seems to need some kind of patch that I have no idea how to make.

Alternatively, using this cls for compression will make it show nothing in the console window
http://fileforums.com/showpost.php?p...4&postcount=72

vollachr 08-03-2018 04:50

Quote:

Originally Posted by 78372 (Post 468711)
Alternatively, using this cls for compression will make it show nothing in the console window
http://fileforums.com/showpost.php?p...4&postcount=72

I'll try that

UPDATE: It actually works, finally a way to use lolz while keeping my cmd window clean and clear of clutter.

KaktoR 08-03-2018 12:15

Any of them supports stdin/out?

KaktoR 09-03-2018 01:33

2 Attachment(s)
It seems bpk has a problem with wildcards, so you can't just write *.bk2. BPK will find nothing.

List all bk2 in directory instead and make freearc reads from list file.

Edit:
Ok, even read from list file doesn't work

Edit 2:
Same with bik files.

KaktoR 10-03-2018 06:49

I've found out what's the problem with bpk..

arc.exe has to be next to bink_pack.exe and cls-bpk.dll (same folder)

elit 10-03-2018 13:41

Ok I can confirm problem with bpk cls, use exe's, some types of bk2 give error on packing, however exe's without cls/dll work. Adding another reason to my list of cls's dislike... :)
EDIT: And also, some bk2's are not packed remaining at same size + 1 byte. But exe's can at least process them without error.

KaktoR 10-03-2018 14:16

ProFrager has wrote about this on krinkels.org. It's a rare bug if i understand google translator correctly (russian -> english) :D

darkwolves 20-03-2018 09:47

my lolz will not show more than one thread when i multithread but it does read higher anyone have the cls maker i've seen pictures of on here?

vollachr 21-03-2018 01:50

How do I unpack archive created with freearc and bpk?

I've tried with command line and also with FreeArc GUI, in both cases the cls-bpk_u.dll is near the arc.exe/unarc.exe file, also the bink_unpack.exe and an arc.ini file with the bpk method added:

Code:

[External compressor:bpk]
header = 0
unpackcmd = bink_unpack.exe $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

With or without the arc.ini edit I keep getting either either ERROR: unsupported compression method bpk in command line or WARNING: invalid compression method or parameters in bpk with the FreeArc GUI.

the method I used was only BPK, no other compressors, I used it on a file list that include full paths and file names of the bik files (13 files).

my compression command was this:

Code:

arc a -r -ed -s; -w"h:\temp" -dp"i:\Games\Borderlands The Pre-Sequel" -x"unins*" -mbpk "h:\temp\BTPS\Data1a.bin" "[email protected]"
Compression end successfully with no errors, but I can't unpack the file.

And if I can't even unpack it manually, how can I expect CIUv2 installer to unpack it?!

Please help.

EDIT: Nevermind, figured it out, forgot to remove the "_u" from the cls dll file, once I did that it worked.

KaktoR 21-03-2018 06:23

Yep you have to rename the cls to "cls-bpk.dll". Just remove "_u" from filename (i was on the same boat a few weeks ago^^)

darkwolves 21-03-2018 06:29

Quote:

Originally Posted by KaktoR (Post 469391)
Yep you have to rename the cls to "cls-bpk.dll". Just remove "_u" from filename (i was on the same boat a few weeks ago^^)

yep me too i don't have any arc commands either

elit 27-03-2018 14:45

I figured out at why default lolz CLS did not worked. Both in cls.ini and arc.ini must be > Lolz not lolz! This is hard-coded, other names don't work.

Webster17 31-03-2018 21:25

Can anybody help me speed up lolz? It takes very long time and I think it does not use the full power of my system. My PC has 4gb ram 3.1 ghz quad core specs. Is there any way to speed up lolz without losing the compression ratio but using the system's full potential?

mikey26 31-03-2018 22:24

Quote:

Originally Posted by Webster17 (Post 469796)
Can anybody help me speed up lolz? It takes very long time and I think it does not use the full power of my system. My PC has 4gb ram 3.1 ghz quad core specs. Is there any way to speed up lolz without losing the compression ratio but using the system's full potential?

tbh nothing much you can do with those specs.you need a upgrade.

78372 31-03-2018 23:14

I prefer using DLZ if you have 4GB RAM only. Problem is, it's ST. I wonder if someone can write a wrapper/cls for making it MT with customisable chunks+threads.

Webster17 01-04-2018 02:44

Quote:

Originally Posted by mikey26 (Post 469797)
tbh nothing much you can do with those specs.you need a upgrade.

I think you are right but i've seen it not use much of my ram, only around 2.50gb and processor about 30%

Webster17 01-04-2018 02:45

Quote:

Originally Posted by 78372 (Post 469798)
I prefer using DLZ if you have 4GB RAM only. Problem is, it's ST. I wonder if someone can write a wrapper/cls for making it MT with customisable chunks+threads.

Some one better make one

Webster17 01-04-2018 02:50

anyways what's the very best compression method for dlz?

Mini 01-04-2018 03:10

@Webster17 dlz:d256m:mc1024:pc0:pl1:dc0:em0


pakrat2k2> if inputting methods-disable smilies in text
checkbox just below text box on LHS

elit 01-04-2018 11:14

First, to the guy above who asked: -mt4 -mtt1 -mc8 -tt1 -dto0 -dm00
This will speedup lolz at least 2-3x and almost on par with lzma(with big dictionary and block size):
Code:

xnlz -s128m -B512m -m64 -M32                          = ~5-6M/s
lolz -mt4 -mtt1 -mtb256 -d256 -mc8 -tt1 -dto0 -dm00  = ~ 4-5M/s

(with ~7-14% gain against lzma and only slightly worse than default)

I compressed 75GB game and they both lasted about ~6 or so hours. Lolz resulted in about ~2GB less size(41GB vs 43GB). So yes, lolz can still be totally worth it, or you can use it selectively on certain files. EDIT: and for comparison, on normal settings with everything enabled I got ~40GB, but it took ~20h.


Second and this is important, I just figured out(for real this time) why I had problems with cls(thanks to profrager). Scratch my last message. Lolz cannot cooperate between its pack exe and unpack cls dll/exe's in same directory! In fact, this may be the issue with most cls* I had. I am trying to figure right now if there is way to force FA to use arc.ini and lolz_x64.exe for compression, even if cls* are present in same /bin FA dir. If anyone knows pls let me know. I want to use FA GUI not custom setups/installers.

elit 01-04-2018 11:21

And one more thing, in general for game files aka textures(mainly) you are less likely going to benefit from high 'mc', which is one of key slowdowns. Seeing these numbers like 1000+ in some of you guys setups is simply ridiculous, I can almost guarantee you will not see much diff from 16/32 up. Secondly, lc param of lzma is worse than lp param and pb param for games from my most recent tests, if you use lzma2 try lc0 pb4 lp4 on game files, or lc8 lp4 pb4 with lzma1. I am considering to make pb4 from default 2 in xnlz in future(still want to see more tests).

doofoo24 04-04-2018 06:34

Quote:

Originally Posted by elit (Post 469813)
lolz -mt4 -mtt1 -mtb256 -d256 -mc8 -tt1 -dto0 -dm00 = ~ 4-5M/s



:confused::rolleyes:

NOT RECOMMENDED ;)

elit 05-04-2018 05:36

Quote:

Originally Posted by doofoo24 (Post 469883)
:confused::rolleyes:

NOT RECOMMENDED ;)

Why?

doofoo24 18-04-2018 05:13

UELR FROM Edison007...
 
i yet test it :o

ZakirAhmad 18-04-2018 06:29

Bro thanks for posting this.

BTW do u know how can i decompress .sfar files of mass effect 3.
please post the decompressor if u have it.

Illume 19-04-2018 05:22

Quote:

do u know how can i decompress .sfar files of mass effect 3
krinkels.org/threads/mass-effect-3.3450/#post-31053
also you need to split them to 2GB (or less) size

Spexxos 19-04-2018 08:06

Can anyone tell what the parameter is to unpack lolz archives. I compressed my testfiles succsessfully but I am unable to find the right parameter for unpacking? Using lolz21a7 with a batchfile

KaktoR 19-04-2018 08:14

There is no

All you need is either cls-lolz.exe or for batch, cls-lolz_x64.exe + CLS.ini with arc.exe

Example
Code:

arc.exe x -w.\ -dunpack\ data.arc

Mini 19-04-2018 08:15

Quote:

Originally Posted by doofoo24 (Post 470342)
i yet test it :o

Type 8

Tested 1 file, 72,747,307 => 50,003,968 bytes. Ratio 145.48%
UELR v0.1.1b x64
Testing time: cpu 0.19 sec/real 74.81 sec = 0%. Speed 0.67 mB/s
Testing time: cpu 0.17 sec/real 74.11 sec = 0%. Speed 0.67 mB/s
Testing time: cpu 0.17 sec/real 75.22 sec = 0%. Speed 0.66 mB/s
UELR v0.1.1b x32
Testing time: cpu 0.20 sec/real 95.23 sec = 0%. Speed 0.53 mB/s
Testing time: cpu 0.25 sec/real 92.51 sec = 0%. Speed 0.54 mB/s
Testing time: cpu 0.30 sec/real 92.92 sec = 0%. Speed 0.54 mB/s
UELR v0.1.0
Testing time: cpu 116.64 sec/real 116.93 sec = 100%. Speed 0.43 mB/s
Testing time: cpu 116.58 sec/real 116.92 sec = 100%. Speed 0.43 mB/s
Testing time: cpu 116.56 sec/real 116.79 sec = 100%. Speed 0.43 mB/s

Tested 1 file, 15,465,068 => 7,550,845 bytes. Ratio 204.81%
UELR v0.1.1b x64
Testing time: cpu 0.06 sec/real 34.84 sec = 0%. Speed 0.22 mB/s
Testing time: cpu 0.06 sec/real 34.80 sec = 0%. Speed 0.22 mB/s
Testing time: cpu 0.11 sec/real 34.80 sec = 0%. Speed 0.22 mB/s
UELR v0.1.1b x32
Testing time: cpu 0.14 sec/real 43.99 sec = 0%. Speed 0.17 mB/s
Testing time: cpu 0.17 sec/real 44.06 sec = 0%. Speed 0.17 mB/s
Testing time: cpu 0.09 sec/real 44.12 sec = 0%. Speed 0.17 mB/s
UELR v0.1.0
Testing time: cpu 57.52 sec/real 57.50 sec = 100%. Speed 0.13 mB/s
Testing time: cpu 57.33 sec/real 57.47 sec = 100%. Speed 0.13 mB/s
Testing time: cpu 57.22 sec/real 57.47 sec = 100%. Speed 0.13 mB/s

not Type 8

uelr+srep:m5f:l2048:a64:s11gb+lolz_21a7:mtt1:mt2:m tb256:d256:tt8:gm20
Tested 1,699 files, 1,893,350,232 => 10,156,721,450 bytes. Ratio 18.64%
UELR v0.1.1b x64
Testing time: cpu 17.75 sec/real 111.12 sec = 16%. Speed 91.40 mB/s
Testing time: cpu 17.35 sec/real 109.90 sec = 16%. Speed 92.42 mB/s
Testing time: cpu 17.21 sec/real 110.64 sec = 16%. Speed 91.80 mB/s
UELR v0.1.0
Testing time: cpu 113.13 sec/real 295.03 sec = 38%. Speed 34.43 mB/s
Testing time: cpu 112.82 sec/real 289.87 sec = 39%. Speed 35.04 mB/s
Testing time: cpu 112.35 sec/real 295.81 sec = 38%. Speed 34.34 mB/s

uelr:uv+srep:m5f:l2048:a64:s2gb+lolz_21a7:mtt1:mt1 :mtb256:d256:tt8
Tested 3 files, 756,000,615 => 1,364,351,031 bytes. Ratio 55.41%
UELR v0.1.1b x64
Testing time: cpu 3.51 sec/real 55.52 sec = 6%. Speed 24.57 mB/s
Testing time: cpu 3.07 sec/real 55.36 sec = 6%. Speed 24.65 mB/s
Testing time: cpu 3.10 sec/real 55.20 sec = 6%. Speed 24.71 mB/s
UELR v0.1.0
Testing time: cpu 3.82 sec/real 54.96 sec = 7%. Speed 24.83 mB/s
Testing time: cpu 3.92 sec/real 55.35 sec = 7%. Speed 24.65 mB/s
Testing time: cpu 3.71 sec/real 56.61 sec = 7%. Speed 24.10 mB/s

Spexxos 19-04-2018 08:29

Quote:

Originally Posted by KaktoR (Post 470394)
There is no

All you need is either cls-lolz.exe or for batch, cls-lolz_x64.exe + CLS.ini with arc.exe

Example
Code:

arc.exe x -w.\ -dunpack\ data.arc

Like so?

cls-lolz_x64.exe "input" "output"

Its seems to be working. Thank you

doofoo24 05-05-2018 02:31

uelr ver0.1.1c
 
pack and unpack improved speed...
AS ALWAYS THANK TO Edison007..


All times are GMT -7. The time now is 18:57.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com