Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #1  
Old 29-12-2018, 08:06
doofoo24 doofoo24 is offline
Registered User
 
Join Date: Nov 2016
Location: canada
Posts: 408
Thanks: 138
Thanked 468 Times in 227 Posts
doofoo24 is on a distinguished road
Talking New lolz

BIG THANK TO ProFrager from krinkels
updated lolz test22c4b
Attached Files
File Type: 7z lolz22c4b.7z (427.2 KB, 1192 views)

Last edited by doofoo24; 30-12-2018 at 07:44.
Reply With Quote
The Following 20 Users Say Thank You to doofoo24 For This Useful Post:
78372 (28-01-2019), ADMIRAL (11-06-2024), allgravy (24-03-2023), COPyCAT (12-01-2022), darkwolves (29-12-2018), elit (04-01-2019), FoRMaT-2007 (16-11-2021), Harsh ojha (15-11-2021), JRD! (15-01-2019), JustFun (29-12-2018), KaktoR (29-12-2018), kj911 (25-05-2024), knife16 (29-12-2018), mdaak001 (18-02-2022), oltjon (29-12-2018), ozerune (26-09-2024), shazzla (29-12-2018), yasserdivar (15-10-2023), ZAZA4EVER (29-12-2018), zirkhaki (30-12-2018)
Sponsored Links
  #2  
Old 29-12-2018, 08:10
shazzla shazzla is offline
Registered User
 
Join Date: Nov 2010
Location: Hunnia
Posts: 273
Thanks: 501
Thanked 95 Times in 72 Posts
shazzla is on a distinguished road
do u have any changelog ?

nevermind. got it :

implemented ldmf (long distance match finder), which looks for matches outside the range of the main matchfinder dictionary. It was developed as an alternative to srep, with its pluses and minuses. Disable / Enabled with the option -ldmf [0..1]. The ratio of the compression ratio to the required memory for decompression is set with the option -ldc [0..9] at 0, the decompression memory control is disabled, maximizing the compression ratio. The size of the minimum search length is specified by the option -ldl [5..12]. This option affects compression (the smaller, the better compression), the used memory for compression (the smaller, the more memory according to the formula ldmf_compr_mem = src_size / ((2 ^ [ldl]) / 16), i.e. ldl5 will add another half of the size of the input file to the memory already used by lolz, the used memory for unpacking (the smaller, the more memory will be needed) and the compression speed (the smaller, the slower). Added the option -lde [0..2] specifying the level of parsing ldmf matches. 2nd mode is the slowest, 0 is the fastest, but there is a minimum difference in compression. 1st mode is almost as fast as 2nd mode, but compression is usually worse than even the 0th mode. Therefore, at the moment I recommend using either -lde0 or -lde2. The default is -lde0;
added for anpak console and cls.ini parameters -ldmfTempPath and -ldmfMaxMemoryUsage, specifying the path to the ldmf swap files and the size of the memory for ldmf, respectively. The ldmfMaxMemoryUsage parameter when calculating memory includes the size of the main dictionary and even a penny on the model (but the latter is considered by eye, from the balda), for example, if -ldmfMaxMemoryUsage = 64m, one stream and the size of the 16 MB dictionary will be allocated for ldmf memory 16MB, the rest will be discarded in swap. The parameter -ldmfDeleteTmp [0..1] disables / enables deletion of the swap file after unpacking. Disabling is only necessary for debugging purposes and controlling the size of this file. Default = 1;
remade the synchronization function of the thread of the optimal parser and matchfighters. The efficiency of the use of cores in compression with detection has increased, respectively, the overall compression rate has increased;
knocked out the use of large pages, no difference anyway;
added the option -ac [0..1], which includes transferring pricing coding of matches in the matchfinder thread, which speeds up compression with -tt is greater than 4, with less -tt the speed is either the same or worse. Compression may differ slightly from the -ac0 mode, both in plus and minus. Also in this mode, you need a little more memory for compression, which depends on the -tt parameter (add_mem = 200k + 450k * tt);
reduced the size of the statistics of the basic models by limiting some parameters -b * that do not change in practice;
changed the coding model dxt color idx and alpha idx. Now, better compression and faster unpacking of dxt textures;
fixed small jambs and added additional contexts for the raw graphics compression model. The size of the statistics of the graphical model increased from 700kb to 2mb, but compression also improved (in a 24-bit picture from 5'544'705 to 5'406'975).
repaired the -fbb option that was inoperable from some point, but at least the sense of it;
Reply With Quote
The Following User Says Thank You to shazzla For This Useful Post:
devil777 (17-01-2020)
  #3  
Old 29-12-2018, 08:19
doofoo24 doofoo24 is offline
Registered User
 
Join Date: Nov 2016
Location: canada
Posts: 408
Thanks: 138
Thanked 468 Times in 227 Posts
doofoo24 is on a distinguished road
lolz v22c4 [Dec 29 2018 17:22:52]

Settings:
----------------------------- detection settings -----------------------------
detection is enabled(-dt)
use probs propagate(-dtp[0..1]): 1 use brute detect(-dtb[0..1]): 0
o1 detect(-dto[0..1]): 1
raw graphic mm detect(-dtm[0..1]): 1 width detect algo(-dtw[0..1]): 1
dxt detect(-dtd[0..1]): 1
--------------------------- multithreading settings --------------------------
mt type(-mtt[0..1]): 0
mt threads(-mt[1..16]): 1 mt block size MB(-mtb[2..512]): 8m
---------------------------- lz & parser settings ----------------------------
dict size in MB(-d[0..2047]): 16m num parser arrivals(-tt[1..256]): 4
parser hard lim(-oh[8..14]): 12 parser soft lim(-os[0..hard_lim]): 8
fb type A(-fba[0..4096,0=off]): 256 fb type B(-fbb[0..4096,0=off]): 0
calc all literals(-al[0..1]): 1 slow parser mode (mlr)(-x[0..2]): 0
move some of calculations to lz matchfinder thread(-ac[0..1]): 0
---------------------------- matchfinder settings ----------------------------
use only lz matchfinder (-rt[0..2]: 0)
lz mf cycles(-mc[2..1023]): 128
use long distance mf(-ldmf[0..1]): 1
ldmf min len exp(-ldl[5..12]): 8 ldmf compr/mem coeff (-ldc[0..9]): 1
ldmf efficient lvl (-lde[0..2]): 0
------------------------------- models settings ------------------------------
general models :
use simple cm(-cm[0..1]): 1
context mixer o1 ctx (-bc[0..6]): 4
use only o1 nibble literal model (-lm[0..4]): 0
literal hi o1 ctx (-blo[0..8]): 8 literal lo o1 ctx (-bll[0..8]): 8
literal hi rep0lit ctx (-blr[0..6]): 4
match flag rep0lit ctx (-bm[0..6]): 4
all -p* settings are ignored due to using detect
dxt models :
dxt color(X) and alpha(Y) model (-dmXY[X[0..3], Y[0..4]]: 34)
raw graphic models :
color mdl options (-gmXY[X[0..2], Y[0..1]]): 00
Reply With Quote
The Following 5 Users Say Thank You to doofoo24 For This Useful Post:
elit (04-01-2019), kokousic (19-10-2020), oltjon (29-12-2018), shazzla (29-12-2018), ZAZA4EVER (29-12-2018)
  #4  
Old 29-12-2018, 11:55
doofoo24 doofoo24 is offline
Registered User
 
Join Date: Nov 2016
Location: canada
Posts: 408
Thanks: 138
Thanked 468 Times in 227 Posts
doofoo24 is on a distinguished road
*note
ldmf does not work with -mtt1.
ldmf option more like srep just use it for testing , very unlikely to help improve ratio
setting for ldmf -ldl -ldc -lde.
ac1 help speed up compression when tt larger than 4...
also fbb option work...

Last edited by doofoo24; 30-12-2018 at 09:45.
Reply With Quote
The Following 2 Users Say Thank You to doofoo24 For This Useful Post:
elit (04-01-2019), shazzla (29-12-2018)
  #5  
Old 29-12-2018, 14:35
felice2011's Avatar
felice2011 felice2011 is offline
Registered User
 
Join Date: Feb 2011
Location: italy
Posts: 836
Thanks: 354
Thanked 1,152 Times in 388 Posts
felice2011 is on a distinguished road
@doofoo24 ... I try some quick tests to realize the real news.

Test On Enwik9: 1GB

Quote:
Old Lolz_x64

Mthd Default :tt1

Comp/Time = 14m20s

Ratio = 21.83%

Used CPU % = (20% - 23%)

Used RAM = (733MB) Max Peak
Quote:
New Lolz_x64

Mthd Default :tt1

Comp/Time = 15m25s

Ratio = 21.72%

Used CPU(%) = (20% - 27%)

Used RAM(MB) = (798MB) Max Peak
Improves the compression ratio, but with about 1 min more time.

Quote:
Extraction Time: cpu 3.25 sec/real 10.05 sec = 32%. Speed 99.50 mB/s
I Try :

Quote:
-mlolz_x64:ldmf1:ldl5:mt6:mc1023

-mlolz_x64:ldmf0:ldl12:mt6:mc1023
I did not see any difference same time and compression ratio...any tip or fix?
__________________
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« I Mediocri Imitano, I Geni Copiano, Dio Crea & Distrugge » (Io Ridefinisco & Perfeziono le Loro Opere Rendendole Uniche)
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« Mediocrities Imitate, Genius Copy, God Creates & Destroys » (I Reconsider & Improve Their Works, Rending Them One And Only)
Reply With Quote
The Following 2 Users Say Thank You to felice2011 For This Useful Post:
knife16 (29-12-2018), shazzla (29-12-2018)
  #6  
Old 29-12-2018, 16:24
doofoo24 doofoo24 is offline
Registered User
 
Join Date: Nov 2016
Location: canada
Posts: 408
Thanks: 138
Thanked 468 Times in 227 Posts
doofoo24 is on a distinguished road
"mlolz_x64:ldmf0:ldl12:mt6:mc1023"

why ldl with ldmf set to 0
ldl -ldc -lde options when ldmf are enabled (set to 1)

Last edited by doofoo24; 29-12-2018 at 16:30.
Reply With Quote
  #7  
Old 29-12-2018, 17:01
felice2011's Avatar
felice2011 felice2011 is offline
Registered User
 
Join Date: Feb 2011
Location: italy
Posts: 836
Thanks: 354
Thanked 1,152 Times in 388 Posts
felice2011 is on a distinguished road
Quote:
Originally Posted by doofoo24 View Post
"mlolz_x64:ldmf0:ldl12:mt6:mc1023"

why ldl with ldmf set to 0
logically I'm trying to understand the difference in performance, speed and compression with the switches enabled or without.

Quote:
Originally Posted by doofoo24 View Post
ldl -ldc -lde options when ldmf are enabled (set to 1)
ok I'll try with different options.
__________________
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« I Mediocri Imitano, I Geni Copiano, Dio Crea & Distrugge » (Io Ridefinisco & Perfeziono le Loro Opere Rendendole Uniche)
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« Mediocrities Imitate, Genius Copy, God Creates & Destroys » (I Reconsider & Improve Their Works, Rending Them One And Only)
Reply With Quote
  #8  
Old 30-12-2018, 03:28
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 4,413
Thanks: 1,078
Thanked 7,026 Times in 2,659 Posts
KaktoR is on a distinguished road
Can't get unpack to work.

"Archive data corrupted"
__________________
Haters gonna hate
Reply With Quote
  #9  
Old 30-12-2018, 04:57
felice2011's Avatar
felice2011 felice2011 is offline
Registered User
 
Join Date: Feb 2011
Location: italy
Posts: 836
Thanks: 354
Thanked 1,152 Times in 388 Posts
felice2011 is on a distinguished road
yes you need new *.dll files in unpacked.
__________________
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« I Mediocri Imitano, I Geni Copiano, Dio Crea & Distrugge » (Io Ridefinisco & Perfeziono le Loro Opere Rendendole Uniche)
≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈ ≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈≈
« Mediocrities Imitate, Genius Copy, God Creates & Destroys » (I Reconsider & Improve Their Works, Rending Them One And Only)
Reply With Quote
  #10  
Old 30-12-2018, 05:35
KaktoR's Avatar
KaktoR KaktoR is offline
Lame User
 
Join Date: Jan 2012
Location: From outer space
Posts: 4,413
Thanks: 1,078
Thanked 7,026 Times in 2,659 Posts
KaktoR is on a distinguished road
I have.

It seems that archives created with previous lolz version are not compatible anymore (I have compressed a game with previous lolz version).

I will test this with a small file and report back.
__________________
Haters gonna hate
Reply With Quote
  #11  
Old 30-12-2018, 07:41
doofoo24 doofoo24 is offline
Registered User
 
Join Date: Nov 2016
Location: canada
Posts: 408
Thanks: 138
Thanked 468 Times in 227 Posts
doofoo24 is on a distinguished road
Arrow new lolz test22c4b

fix for unpacking error or something
Attached Files
File Type: 7z lolz22c4b.7z (427.2 KB, 249 views)
Reply With Quote
The Following 3 Users Say Thank You to doofoo24 For This Useful Post:
Jiva newstone (30-12-2018), KaktoR (30-12-2018), oltjon (30-12-2018)
  #12  
Old 30-12-2018, 13:25
yasitha's Avatar
yasitha yasitha is offline
Registered User
 
Join Date: Apr 2013
Location: Krypton
Posts: 297
Thanks: 203
Thanked 48 Times in 33 Posts
yasitha is on a distinguished road
I can help just tell me what to do
__________________
Keep Up The Good Works!

Last edited by yasitha; 31-12-2018 at 06:25.
Reply With Quote
  #13  
Old 30-12-2018, 23:43
oltjon's Avatar
oltjon oltjon is offline
Registered User
 
Join Date: Dec 2013
Location: UNITED STATES OF ALBANIA
Posts: 218
Thanks: 1,132
Thanked 167 Times in 79 Posts
oltjon is on a distinguished road
file size 219.585.545 srep+lolz 135.655.003

file size 219.585.545 lolz:ldmf1 135.602.872

file size 219.585.545 srep+lolz:ldmf1 135.658.543
__________________
....................always rains in the wet.....................
Reply With Quote
The Following 2 Users Say Thank You to oltjon For This Useful Post:
doofoo24 (31-12-2018), Simorq (31-12-2018)
  #14  
Old 31-12-2018, 05:56
doofoo24 doofoo24 is offline
Registered User
 
Join Date: Nov 2016
Location: canada
Posts: 408
Thanks: 138
Thanked 468 Times in 227 Posts
doofoo24 is on a distinguished road
the issue for me using ldmf ram usage on large games in decompression
you need to limit idmf in cls.ini

[lolz]
Bufsize=512k
transfer_ReadBufSize=512k
transfer_WriteBufSize=512k
MaxThreadsUsage=50%
MaxMemoryUsage=50%
ldmfTempPath={app}
ldmfMaxMemoryUsage=1024m
ldmfDeleteTmp=0
*ldmfMaxMemoryUsage you need to specify ram, percentage option does not work..

Last edited by doofoo24; 31-12-2018 at 05:58.
Reply With Quote
The Following 2 Users Say Thank You to doofoo24 For This Useful Post:
oltjon (31-12-2018), Simorq (31-12-2018)
  #15  
Old 31-12-2018, 06:07
doofoo24 doofoo24 is offline
Registered User
 
Join Date: Nov 2016
Location: canada
Posts: 408
Thanks: 138
Thanked 468 Times in 227 Posts
doofoo24 is on a distinguished road
Quote:
Originally Posted by oltjon View Post
file size 219.585.545 srep+lolz 135.655.003

file size 219.585.545 lolz:ldmf1 135.602.872

file size 219.585.545 srep+lolz:ldmf1 135.658.543
@oltjon

can you test ldmf1 with ldl5 and ldc0
Reply With Quote
The Following 2 Users Say Thank You to doofoo24 For This Useful Post:
oltjon (31-12-2018), yasitha (31-12-2018)
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Best Compression Methods for 'Specific' Games INDEX JustFun Conversion Tutorials 46 02-12-2024 21:20
Bench Test (LOLZ vs RAZOR vs MCM vs LZMA2) felice2011 Conversion Tutorials 5 19-10-2020 08:40
LZMA vs LOLZ & Scan Compress Method yasitha Conversion Tutorials 58 11-01-2019 10:01
problem with lolz Kitsune1982 Conversion Tutorials 6 11-06-2018 14:04
Cost vs return - aka worthiness debate elit Conversion Tutorials 44 15-01-2018 16:45



All times are GMT -7. The time now is 01:54.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com