FileForums
Page 4 of 5 « First 23 4 5
Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   PMT - Parallel Multithreaded Encoder/Decoder (https://fileforums.com/showthread.php?t=103230)

78372 26-03-2020 02:47
Quote:
Originally Posted by shazzla (Post 484882)
In some rare case,it happens with me,too.. (for example,Quantum break ,1xx GB version)
Maybe a bug in Razor ?!
With Razor's patch, razor now supports stdio (What a pun :D )
Try using that one to remove temps and probably that will fix some stuff.

shazzla 26-03-2020 03:33
I used the STDIO patched version ,without PMT ! (RZ V1.03.7)

KaktoR 27-03-2020 02:16
I have further tested with full games, it works like a charm and pretty fast :D

Code:
1:36:00 - Starting Archive Creation.
 2:54:33 - Archive Creation Complete.
 2:54:33 - Sorting Storage Directories Started.
 2:54:34 - Sorting Storage Directories Complete.
 2:54:34 - Starting Archive Creation.
 4:11:32 - Archive Creation Complete.
 4:11:32 - Sorting Storage Directories Started.
 4:11:32 - Sorting Storage Directories Complete.
 4:11:32 - Starting Archive Creation.
 9:56:22 - Archive Creation Complete.
 9:56:22 - Sorting Storage Directories Started.
 9:56:22 - Sorting Storage Directories Complete.

10:28:36 - The Witcher Enhanced Edition Size: 13.67 GB.
10:28:36 - The Witcher 2 Size: 17.21 GB.
10:28:37 - The Witcher 3 Size: 42.99 GB.
10:28:37 - Full Input Size: 73.89 GB.
10:28:37 - Output Size: 45.63 GB.
Decode only possible with t2 or t1 for me, I don't know why :(

Tested with batch and CIU installer.

I used razor

78372 27-03-2020 07:20
Seems like some internal issues with razor. Did you test with other compressors? How do they behave? Probably the fast speed of razor is causing some I/O bottleneck or something with multiple threads. Single thread is enough for razor to be frank with such lightning decompression speed.

KaktoR 30-03-2020 10:19
Not yet, but it's ok neverthless because rz is lightning fast in decompression :)

What other compressors do you suggest? What about zstd?

78372 30-03-2020 11:13
ZSTD has mt and way too fast to use with PMT.
Try with slow tools.
BCM, precomp, rzm etc. can work better.
We have.alternatives for them though.

Cesar82 30-03-2020 15:52
Has anyone done any tests using PMT + pZLib3?

If yes please share the values for PMT.ini and Arc.ini.
Thanks!

78372 30-03-2020 22:38
Quote:
Originally Posted by Cesar82 (Post 484988)
Has anyone done any tests using PMT + pZLib3?

If yes please share the values for PMT.ini and Arc.ini.
Thanks!
What's the point? pZlib is already MT and fast. Moreover, you have xtool now.

Cesar82 30-03-2020 23:15
@78372
Thanks for the information. I know little about compression methods.
So of the compressors used in CIU only RAZOR has benefits if used with PMT.

CIU supports these compressors in addition to some RAZOR tools for specific games.
SREP, ZTool, XTool(not v2019), pZLib3, UELR, AFR, OodleRec [4,5,6,7,8], MSC, MPZ, OGGRE, BPK, 7z, LOLZ, ZSTD, RAZOR

Does any of this list other than razor benefit if used with PMT?

78372 30-03-2020 23:55
Quote:
Originally Posted by Cesar82 (Post 484991)
@78372
Thanks for the information. I know little about compression methods.
So of the compressors used in CIU only RAZOR has benefits if used with PMT.

CIU supports these compressors in addition to some RAZOR tools for specific games.
SREP, ZTool, XTool(not v2019), pZLib3, UELR, AFR, OodleRec [4,5,6,7,8], MSC, MPZ, OGGRE, BPK, 7z, LOLZ, ZSTD, RAZOR

Does any of this list other than razor benefit if used with PMT?
mpz only I guess, at least for now.

KaktoR 31-03-2020 02:32
Code:
mpz with PMT
Compressed 64 files, 209,434,083 => 180,728,551 bytes. Ratio 86.29%
Compression time: cpu 0.20 sec/real 38.46 sec = 1%. Speed 5.45 mB/s

Tested 64 files, 180,728,551 => 209,434,083 bytes. Ratio 86.29%
Testing time: cpu 0.19 sec/real 36.54 sec = 1%. Speed 5.73 mB/s

mpz only
Compressed 64 files, 209,434,083 => 180,670,089 bytes. Ratio 86.27%
Compression time: cpu 0.30 sec/real 103.93 sec = 0%. Speed 2.02 mB/s

Tested 64 files, 180,670,089 => 209,434,083 bytes. Ratio 86.27%
Testing time: cpu 0.36 sec/real 95.90 sec = 0%. Speed 2.18 mB/s
Tested with settings -b64m -t100p for encode/decode with a Ryzen 5 2600 (12 threads).

I just wonder why ratio is a little bit better without pmt :D

78372 31-03-2020 20:57
The same reason for everything, chunking.

DiCaPrIo 08-06-2020 02:15
Quote:
Originally Posted by L0v3craft (Post 486217)
Hi. Is possible to add bpk compatibility? Thanks.
see at first page

L0v3craft 02-07-2020 20:15
Is possible to add this?

Code:
https://github.com/encode84/bcm
it is BCM v1.5. If is not possible someone has a cls dll for BCM to make it multithreaded? Thanks.

78372 02-07-2020 22:47
Quote:
Originally Posted by L0v3craft (Post 486638)
Is possible to add this?

Code:
https://github.com/encode84/bcm
it is BCM v1.5. If is not possible someone has a cls dll for BCM to make it multithreaded? Thanks.
BCM 1.4 works with this. I will make a stdio patch for 1.5 when I have time.


All times are GMT -7. The time now is 06:01.
Page 4 of 5 « First 23 4 5
Show 40 post(s) from this thread on one page

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com