MTX - The Universal Accelerator

[BLACKFIRE69]

1. Created MTX2023 from scratch and optimized for speed and efficiency.
- As a result, it isn't compatible with any previous versions.

2. The bugs experienced in previous versions have been addressed and fixed.

3. MTX can handle scenarios where the #userThreads > #compressionThreads.

4. Simultaneous reading and writing of data takes place.
- Previous versions required extra waiting time that's no longer necessary in this version.
- Make sure to specify the '-ds' parameter with the 'arc.exe a'. (arc.exe a -ds ...)

Remark:

1. '--TempPath=' parameter has been removed.
- Tmp files will now be created in the freearc temp dir instead.

2. '%' sign has been removed from threads (100%).
3. Support for "MTX32.ini" and "MTX64.ini" files has been discontinued.
4. There won't be a 32-bit version of MTX.


1. Recommended setting (stdio mode) but no info will be displayed.

packcmd = MTX64.exe a -mNz64 -c32m {options} - - <stdin> <stdout>
unpackcmd = MTX64.exe x -mNz64 {options} - - <stdin> <stdout>


2. Info will only be displayed for packing.

packcmd = MTX64.exe a -mNz64 -c32m {options} - $$arcpackedfile$$.tmp <stdin>
unpackcmd = MTX64.exe x -mNz64 {options} - - <stdin> <stdout>


3. Info will only be displayed for unpacking.

packcmd = MTX64.exe a -mNz64 -c32m {options} - - <stdin> <stdout>
unpackcmd = MTX64.exe x -mNz64 {options} - $$arcdatafile$$.tmp <stdin>


4. Info will be displayed for both packing and unpacking.

packcmd = MTX64.exe a -mNz64 -c32m {options} - $$arcpackedfile$$.tmp <stdin>
unpackcmd = MTX64.exe x -mNz64 {options} - $$arcdatafile$$.tmp <stdin>


.

[BLACKFIRE69]

If anyone wants an example of "Inno setup" for MTX.exe, just let me know. ;)

[Balaji007]

MTX console version works perfectly. Also I was tested MCM works perfectly.

[BLACKFIRE69]

MTX.exe supports MASKS (Arc.Groups)

example:

[Compression Methods]
MASK = srep+razorx/$zlib=precompx+srep+bcmx/$other=srep+razorx/$lyrfile=precompx+srep+bcmx/$bmp=srep+razorx

[Balaji007]

MTX.exe supports MASKS (Arc.Groups)
Can you explain about masks?

[Masquerade]

Can you explain about masks?

Masks allow you to use more than one method for different files in an archive

Different methods are specified with groups, which you can write yourself into the file "arc.groups" next to your arc.exe for compression.

Write your group as example:

$nameofgroup
*.filename
*.otherfile

Then we can use this in our compression method:

srep+lolz/$nameofgroup=rzx

So here, all of the files defined in the $nameofgroup list will be compressed using rzx, where all other files will be compressed with srep+lolz.

[Balaji007]

Masks allow you to use more than one method for different files in an archive

Different methods are specified with groups, which you can write yourself into the file "arc.groups" next to your arc.exe for compression.

Write your group as example:

$nameofgroup
*.filename
*.otherfile

Then we can use this in our compression method:

srep+lolz/$nameofgroup=rzx

So here, all of the files defined in the $nameofgroup list will be compressed using rzx, where all other files will be compressed with srep+lolz.

Ok thanks for your information friend

[BLACKFIRE69]

MTX.exe Feature update v.3.0.0.0



What's New:


Some improvements.


A new key (MainPath) has been added to MTX.ini.
-- this allows you to set the root directory manually.
-- MTX.exe can be executed within a sub-folder.


There is no need to rename "mtx.ini" (as before).
-- before: MTX64.exe ---> MTX64.ini --> [MTX64]
-- now : MTX64.exe ---> MTX.ini --> [MTX]


Added an example for XLolz.
Added an example for MASKS.




Additional Options For both Compression and the Decompression:


-----------------------------------------------------------------------------
[MTX]
MainPath=..\..\
Config=arc.ini
//TmpPath=C:\Users\Rexton\Desktop\temptest\
-----------------------------------------------------------------------------



// config := default is arc.ini (current directory).

// config = arc.ini <--- MainPath\arc.ini
// config = cfg\config0.ini <--- MainPath\cfg\config0.ini



// TmpPath= full path. ( with or without quotes (") )




_

[Nezha]

Oh, thank you!
Can I have an example or small guide to adapt Inno/ASIS to this please? I'm probably doing something wrong with my frankenstein script lol

Edit/question: Is Xlolz working multithreading via MTX? It doesn't seem for me.
Edit2: It works with fixed thread count and not % like default (?)

[BLACKFIRE69]

Oh, thank you!
Can I have an example or small guide to adapt Inno/ASIS to this please? I'm probably doing something wrong with my frankenstein script lol

Edit/question: Is Xlolz working multithreading via MTX? It doesn't seem for me.
Edit2: It works with fixed thread count and not % like default (?)


You can't use "XLolz" via "MTX" (because both are accelerators). but it supports "MTX + Lolz".

ok, I'll give you some examples for both INNO Setup and Lolz with MTX.

[Nezha]

You can't use "XLolz" via "MTX" (because both are accelerators). but it supports "MTX + Lolz".

ok, I'll give you some examples for both INNO Setup and Lolz with MTX.

Oh, yea, my bad: I mean the one inside the default "arc.ini" with MTX, for reference this one:
[External compressor:XLolz, LolzMTX, xlolz]
header = 0
packcmd = bin\MTX\MTX.exe a:lolz -c:64m -t:100p - - <stdin> <stdout>
unpackcmd = bin\MTX\MTX.exe x:lolz -t:100p - - <stdin> <stdout>

___

So.. Something works but not well

Using unpack.bat
Extracted 6,874 files, 10,021,997,180 => 16,726,226,492 bytes. Ratio 59.92%
Extraction time: cpu 30.00 sec/real 469.39 sec = 6%. Speed 35.63 mB/s
All OK

Using my modified InnoSetup scripts with MTX gives "UnArc.dll -11 error: archive damaged" at ~80% but it actually extracts something

[BLACKFIRE69]

Oh, yea, my bad: I mean the one inside the default "arc.ini" with MTX, for reference this one:


___

So.. Something works but not well

Using unpack.bat


Using my modified InnoSetup scripts with MTX gives "UnArc.dll -11 error: archive damaged" at ~80% but it actually extracts something

ok, just wait for the example.

[BLACKFIRE69]

MTX v.5.0.0.0 - October Update





MTX.ini
Example:=

exefile: MTX64.exe
ini: supports both MTX.ini or MTX64.ini (next to the exe file)
section: [MTX] or [MTX64]

exefile: MTX32.exe
ini: supports both MTX.ini or MTX32.ini (next to the exe file)
section: [MTX] or [MTX32]


Percentage
Example:=

packcmd = "MTX.exe" a:bcm -c:64m -t:100p - - <stdin> <stdout>

or

packcmd = "MTX.exe" a:bcm -c:64m -t:100% - - <stdin> <stdout>


Supports Custom Input/Output
Example:=

[External compressor:bcm, bcm32]
header = 0
default = -9

packcmd = "Bcm\{compressor}" {options} MyInput.in MyOutput.out
unpackcmd = "Bcm\{compressor}" -d MyOutput.out MyInput.in
datafile = MyInput.in
packedfile = MyOutput.out


New Options


1. --help (Show long help)
2. --logs (Generate a log file)
3. --mainpath=< main path >
4. --config=< config file >
5. --tmppath=< temp path >


Example:= (When not using MTX.ini.)


[External compressor:BscX, BscMTX, bscx]
header = 0
default = --logs --mainpath=".\..\" --config="%MainPath%\confis\arc.ini" --tmppath="%ExePath%\TEMP"
packcmd = MTX\MTX.exe a:bsc32 -c:64m -t:100% {options} - - <stdin> <stdout>
unpackcmd = MTX\MTX.exe x:bsc32 -t:100p {options} - - <stdin> <stdout>


MTX.ini Configuration

MTX Variables:
1. %MainPath%
-- default is the directory that containing MTX.exe.
ex: C:\Testing\mtxTest\Res\MTX\MTX.exe then %MainPath% = C:\Testing\mtxTest\Res\MTX\

-- Users can change manually.

2. %ExePath%
-- the directory that containing MTX.exe.
ex: C:\Testing\mtxTest\Res\MTX\MTX.exe then %ExePath% = C:\Testing\mtxTest\Res\MTX\


Other
1. MainPath
-- default is %ExePath%

-- Users can change manually.

-- MainPath = (full path)
ex: MainPath=C:\Testing\mtxTest\Res\

-- MainPath = (short path)
ex: MainPath=.\..\
> MainPath is centered on ExePath. (for short paths)
> If ExePath = C:\Testing\mtxTest\Res\MTX\ then MainPath = C:\Testing\mtxTest\Res\ (MainPath=.\..\)

-- MainPath = (alternatively)
ex: MainPath=%ExePath%


2. Config
-- default is %ExePath%\Arc.ini

-- Users can change manually.

-- Config = (full path)
ex: Config=C:\Testing\mtxTest\Res\Arc.ini

-- Config = (short path)
ex: Config=.\..\Arc.ini
> Config is centered on MainPath. (for short paths)
> If MainPath = C:\Testing\mtxTest\Res\MTX\ then Config = C:\Testing\mtxTest\Res\Arc.ini (Config=.\..\Arc.ini)

-- Config = (alternatively)
ex: Config=%MainPath%\Arc.ini
ex: Config=%ExePath%\Arc.ini


3. TmpPath
-- default is %ExePath%

-- Users can change manually.

-- TmpPath = (full path)
ex: TmpPath=C:\Testing\mtxTest\Res\TEMP\

-- TmpPath = (short path)
ex: TmpPath=.\..\TEMP\
> TmpPath is centered on MainPath. (for short paths)
> If MainPath = C:\Testing\mtxTest\Res\MTX\ then Config = C:\Testing\mtxTest\Res\TEMP\ (TmpPath=.\..\TEMP\)

-- TmpPath = (alternatively)
ex: TmpPath=%MainPath%\TEMP\
ex: TmpPath=%ExePath%\TEMP\




Will show some errors in nanozip decompression with MTX.exe. therefore it is better to use cls-nzx.dll (https://fileforums.com/showthread.php?t=103732) instead of MTX.exe. ;)


guys, I don't have enough space to upload it here. so use this link....


MTX v5.0.0.0 + Full Examples.rar (https://app.box.com/s/bi0zbzz3weg36ksiub3235ve3fydkgom)




_

[Cesar82]

@BLACKFIRE69, very good the new key default options in example 5.
It will be very useful for the need to use MRX.ini.
Thanks for the great job.

If you have both configurations, which one will be used?
I mean if you have MTX.ini configured in one way and use the key default with --mainpath, etc. in another way, which one will be used?

[BLACKFIRE69]

@BLACKFIRE69, very good the new key default options in example 5.
It will be very useful for the need to use MRX.ini.
Thanks for the great job.

If you have both configurations, which one will be used?
I mean if you have MTX.ini configured in one way and use the key default with --mainpath, etc. in another way, which one will be used?

first configured file (MTX.ini) then checks the default keys (--mainpath , ...).

means,

If MTX.ini is not configured, it uses the default keys.

[Cesar82]

first configured file (MTX.ini) then checks the default keys (--mainpath , ...).

means,

If MTX.ini is not configured, it uses the default keys.
Just out of curiosity, if I use it like this:

DiskSpan_x64.ini:

[External compressor:mpz,MPZ,mpzapi,MPZAPI]
header = 0
packcmd = "MEDIA\MPZ\mpzapi.exe" c <stdin> <stdout>
unpackcmd = "MEDIA\MPZ\mpzapi.exe" d <stdin> <stdout>

[External compressor:mpz_mtx,MPZ_MTX,mpzapi_mtx,MPZAPI_MTX]
header = 0
default = -c:64m -t:100p --mainpath=".\..\..\..\" --config="%MainPath%\DiskSpan_x64.ini" --tmppath="%MainPath%\..\TEMP"
packcmd = "OTHERS\MTX\Win64\MTX.exe" a:mpz {options} - - <stdin> <stdout>
unpackcmd = "OTHERS\MTX\Win64\MTX.exe" x:mpz {options} - - <stdin> <stdout>

And add MTX.ini file next to MTX.exe with the config:

MTX.ini
[MTX]
MainPath=.\..\..\..\
Config=%MainPath%\DiskSpan_x64.ini
TmpPath=%MainPath%\..\TEMP
Question is as follows:
Will the "-c:64m -t:100p" settings be applied to MTX.exe, or will they be ignored?

Thanks!

[BLACKFIRE69]

Just out of curiosity, if I use it like this:

DiskSpan_x64.ini:

[External compressor:mpz,MPZ,mpzapi,MPZAPI]
header = 0
packcmd = "MEDIA\MPZ\mpzapi.exe" c <stdin> <stdout>
unpackcmd = "MEDIA\MPZ\mpzapi.exe" d <stdin> <stdout>

[External compressor:mpz_mtx,MPZ_MTX,mpzapi_mtx,MPZAPI_MTX]
header = 0
default = -c:64m -t:100p --mainpath=".\..\..\..\" --config="%MainPath%\DiskSpan_x64.ini" --tmppath="%MainPath%\..\TEMP"
packcmd = "OTHERS\MTX\Win64\MTX.exe" a:mpz {options} - - <stdin> <stdout>
unpackcmd = "OTHERS\MTX\Win64\MTX.exe" x:mpz {options} - - <stdin> <stdout>

And add MTX.ini file next to MTX.exe with the config:

MTX.ini
[MTX]
MainPath=.\..\..\..\
Config=%MainPath%\DiskSpan_x64.ini
TmpPath=%MainPath%\..\TEMP
Question is as follows:
Will the "-c:64m -t:100p" settings be applied to MTX.exe, or will they be ignored?

Thanks!

those settings are not ignored ( -c:64m -t:100p ).


ok cesar,

we use MTX.ini to define only "MainPath", "ConfigFile" and "TempPath".


let us consider,
if MTX.exe doesn't find any MTX.ini near MTX.exe, it will look for MTX.exe options ( --mainpath, --config and --tmppath). if those options are also not found, it uses its default values.

in your case, there is MTX.ini next to MTX.exe. therefore, the "--mainpath", "--config" and "--tmppath" options will be ignored.

but the only place to set values for "-t:" and "-c:" is the command line. so nothing can ignore those values.


finally, MTX.exe will ignore some options as the code below because you have used MTX.ini. (Marked in red).

[External compressor:mpz_mtx,MPZ_MTX,mpzapi_mtx,MPZAPI_MTX]
header = 0
default = -c:64m -t:100p --mainpath=".\..\..\..\" --config="%MainPath%\DiskSpan_x64.ini" --tmppath="%MainPath%\..\TEMP"
packcmd = "OTHERS\MTX\Win64\MTX.exe" a:mpz {options} - - <stdin> <stdout>
unpackcmd = "OTHERS\MTX\Win64\MTX.exe" x:mpz {options} - - <stdin> <stdout>



Extra: (not related to the question)

during decompression, MTX.exe doesn't retrieve values for "ChunkSize" from users. therefore it is better to use as follows. :)

[External compressor:mpz_mtx,MPZ_MTX,mpzapi_mtx,MPZAPI_MTX]
header = 0
default = -t:100p
packcmd = "OTHERS\MTX\Win64\MTX.exe" a:mpz -c:64m {options} - - <stdin> <stdout>
unpackcmd = "OTHERS\MTX\Win64\MTX.exe" x:mpz {options} - - <stdin> <stdout>

[Snapppr6]

first thing thanks to you for this tool and other cls
i need IS example to unpack compressed data by mtx and thanks

[BLACKFIRE69]

first thing thanks to you for this tool and other cls
i need IS example to unpack compressed data by mtx and thanks


here is the basic inno example for MTX v5.0.0.0.

note that the two arc.ini files in the "Compressor" folder and the "IS_Example" folder are different. ;)

MTX v5.0.0.0 Inno Example.rar (https://app.box.com/s/jng8rpggwn6tywfgaa0lg4qd01y95ssj)

[BLACKFIRE69]

MTX v5.1.0.0 New Update Test

it's time for a new update. :D you guys, can you do some testing for me?

TESTS:

MTX + oo2reck
MTX + oo2recm
MTX + oo2recs



Arc.ini
; //================== FreeArc =================\\
[External compressor:oo2reck,oo2recm,oo2recs]
header = 0
packcmd = {compressor} e <stdin> <stdout>
unpackcmd = {compressor} d <stdin> <stdout>


; //==================== MTX ====================\\
[External compressor:MTX_oo2reck, oo2reckMTX, oo2reckX]
header = 0
default = -t:100% --logs
packcmd = MTX.exe a:oo2reck -c:64m {options} - - <stdin> <stdout>
unpackcmd = MTX.exe x:oo2reck {options} - - <stdin> <stdout>

[External compressor:MTX_oo2recm, oo2recmMTX, oo2recmX]
header = 0
default = -t:100% --logs
packcmd = MTX.exe a:oo2recm -c:64m {options} - - <stdin> <stdout>
unpackcmd = MTX.exe x:oo2recm {options} - - <stdin> <stdout>

[External compressor:MTX_oo2recs, oo2recsMTX, oo2recsX]
header = 0
default = -t:100% --logs
packcmd = MTX.exe a:oo2recs -c:64m {options} - - <stdin> <stdout>
unpackcmd = MTX.exe x:oo2recs {options} - - <stdin> <stdout>


i'll attach the MTX v5.1.0.0 test files.

thanks @dixen for the idea.

_

[KaktoR]

MTX
Compressed 1 file, 119,898,862 => 168,186,700 bytes. Ratio 140.27%
Compression time: cpu 0.13 sec/real 8.59 sec = 1%. Speed 13.96 mB/s
All OK

Extracted 1 file, 168,186,700 => 119,898,862 bytes. Ratio 140.27%
Extraction time: cpu 0.19 sec/real 3.90 sec = 5%. Speed 30.74 mB/s
All OK


No MTX
Compressed 1 file, 119,898,862 => 168,186,552 bytes. Ratio 140.27%
Compression time: cpu 0.14 sec/real 10.16 sec = 1%. Speed 11.80 mB/s
All OK

Extracted 1 file, 168,186,552 => 119,898,862 bytes. Ratio 140.27%
Extraction time: cpu 0.17 sec/real 3.84 sec = 4%. Speed 31.26 mB/s
All OK

[dixen]

Thanks to BLACKFIRE69 for update but...tests..Not particularly impressive

patchpack002\dlc.rpf

MTX+OO2RECK (pack)

Compressed 1 file, 1,060,477,032 => 1,820,126,829 bytes. Ratio 171.63%
Compression time: cpu 1.28 sec/real 186.69 sec = 1%. Speed 5.68 mB/s
All OK

MTX+OO2RECK (Unpack)

Extracted 1 file, 1,820,126,829 => 1,060,477,032 bytes. Ratio 171.63%
Extraction time: cpu 2.11 sec/real 132.35 sec = 2%. Speed 8.01 mB/s
All OK

Razor's oo2reck (Pack)

Compressed 1 file, 1,060,477,032 => 1,820,656,947 bytes. Ratio 171.68%
Compression time: cpu 1.19 sec/real 216.05 sec = 1%. Speed 4.91 mB/s
All OK

Razor's oo2reck (unpack)

Extracted 1 file, 1,820,656,947 => 1,060,477,032 bytes. Ratio 171.68%
Extraction time: cpu 1.77 sec/real 125.85 sec = 1%. Speed 8.43 mB/s
All OK

I'll test this on DOOM Eternal & Horizon Zero Dawn..

[BLACKFIRE69]

Guys, try with more chunk sizes.

-c:128mb
-c:256mb
-c:512mb

[dixen]

Same file, chunk size 256

Unpack

Extracted 1 file, 1,820,527,500 => 1,060,477,032 bytes. Ratio 171.67%
Extraction time: cpu 2.06 sec/real 117.64 sec = 2%. Speed 9.01 mB/s
All OK

chunk size 512

Extracted 1 file, 1,820,527,500 => 1,060,477,032 bytes. Ratio 171.67%
Extraction time: cpu 2.09 sec/real 123.59 sec = 2%. Speed 8.58 mB/s
All OK

[BLACKFIRE69]

Same file, chunk size 256

Unpack



chunk size 512


It seems to be MTX is 0.58 mb/s faster than the original version right ? ( for chunk = 256mb )

[dixen]

It seems to be MTX is 0.58 mb/s faster than the original version right ? ( for chunk = 256mb )

Yeah it looks like the idea failed

[BLACKFIRE69]

MTX + Lolz may work well in high chunk sizes.

[Razor12911]

Isn't the oodle precompressor multi threaded already? :confused:

I made the tool in such a way that there no streams being cut off when chunking takes place because the program knows what it is doing and how it should make cuts in the chunks, if for example there is a stream that can be cut off at a certain position, the program self adjusts the position and your chunking method makes no such considerations and gets in the way of how the program should work in the first place which makes it prone to crashes or a drop in ratio. :o

Multi threading a multi threaded program does not make it faster especially if the main program uses all of the cpu, furthermore your MTX does not prepare chunks in advance which could be the reason it doesn't work very well.

Compare the speeds of MTX vs Freearc's 4x4

I'll also leave this here
28232

I can't believe this community has been reduced to this, I don't know if I should find this funny or be sad that there are no new inventions will be made. :(

I would totally understand if the community gets excited every time EA Sports releases FIFA every year :D

[BLACKFIRE69]

no more developments. it doesn't seem to be useful.

@mods, plz delete this thread.

[Razor12911]

I wonder what makes you cease the development of your project, I just pointed out that using MTX on already multi threaded programs may have such an issue of being slower than when run natively. I wouldn't have said anything if you had figured out a smarter way of making the precompressor making sure it stays at 100% because there are times when it doesn't fully utilize the cpu furthermore there is also overutilization because the precompressor would be using all of the cpu while MTX is also making several instances of the program and now you have the problem of system being unresponsive. Your project has its uses just not handling multi threaded programs, you'd need to come up with a better solution for this.

[78372]

What people thinks development is? Remaking old projects with more bugs and making non sense project is not development. If you can make something new that's development, even if you remake and make something better than before that's development. Posting same thing over and over again by changing a bit of information is spamming, not development.

[Ele]

Sorry for bothering mr.BLACKFIRE, This may be a useful thread but we can't find the download section here. so please reupload it.
Thank you.

[github]

Hi,

I tried to use the settings in the snapshot, which should be using precomp048.exe

but when I test a packing, method called is

MTXprecomp048
-> calls method precomp048
-> calls precomp048.exe

however it is reporting error with precomp038.exe in the same directory.

It is MTX 5.0.0.0

https://i.ibb.co/qxF1j5m/2021-02-07-031238.png

[Ele]

Hi,

I tried to use the settings in the snapshot, which should be using precomp048.exe

but when I test a packing, method called is

MTXprecomp048
-> calls method precomp048
-> calls precomp048.exe

however it is reporting error with precomp038.exe in the same directory.

It is MTX 5.0.0.0

https://i.ibb.co/qxF1j5m/2021-02-07-031238.png

Precomp v0.38 requires "packjpg_dll.dll" and "packjpg_dll1.dll" next to the precomp038.exe.

[github]

Thanks, that could explain part of it.
But why it would invoke precomp038.exe?
Is MTX parsing every external compressor defined in arc.ini as well as try to load them once to build some kind of lookup table into memory for later use?

[github]

; ========== for MTX testing ================

[External compressor:rz, razor, rz64]
header = 0
default = -d 128m
; packcmd = "Rz\RZ.exe" a {options} $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
; unpackcmd = "Rz\RZ.exe" e -y $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
; or

packcmd = "Rz\RZ.exe" a {options} MyOutput.out MyInput.in
unpackcmd = "Rz\RZ.exe" e -y MyOutput.out MyInput.in
datafile = MyInput.in
packedfile = MyOutput.out

[External compressor:bcm, bcm32]
header = 0
default = -9
; packcmd = "Bcm\{compressor}" {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
; unpackcmd = "Bcm\{compressor}" -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
; or

packcmd = "Bcm\{compressor}" {options} MyInput.in MyOutput.out
unpackcmd = "Bcm\{compressor}" -d MyOutput.out MyInput.in
datafile = MyInput.in
packedfile = MyOutput.out

[External compressor:lolz, lolz64]
header = 0
default = -dt -dtb1 -dtw1 -dtm1 -dto1 -dtd1 -mtt1 -mt4 -d64m -fba4096
; packcmd = Lolz\lolz_x64.exe {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
; unpackcmd = Lolz\lolz_x64_u.exe $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
; or

packcmd = Lolz\lolz_x64.exe {options} MyInput.in MyOutput.out
unpackcmd = Lolz\lolz_x64_u.exe MyOutput.out MyInput.in
datafile = MyInput.in
packedfile = MyOutput.out

[External compressor:RazorX, RazorMTX, razorx]
header = 0
default = -c:64m -t:100% --logs
packcmd = "MTX.exe" a:rz {options} - - <stdin> <stdout>
unpackcmd = "MTX.exe" x:rz -t:100p --logs - - <stdin> <stdout>

[External compressor:BcmX, BcmMTX, bcmx]
header = 0
default = -c:64m -t:100% --logs
packcmd = "MTX.exe" a:bcm {options} - - <stdin> <stdout>
unpackcmd = "MTX.exe" x:bcm -t:100p --logs - - <stdin> <stdout>

[External compressor:XLolz, LolzMTX, xlolz]
header = 0
default = -c:64m -t:100% --logs
packcmd = MTX.exe a:lolz {options} - - <stdin> <stdout>
unpackcmd = MTX.exe x:lolz -t:100p --logs - - <stdin> <stdout>

[External compressor:precomp]
header = 0
default = -cn
packcmd = "Precomp\{compressor}" {options} - - <stdin> <stdout>
unpackcmd = "Precomp\{compressor}" -r -o$$arcdatafile$$.tmp $$arcpackedfile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp
; or

;packcmd = "Precomp\{compressor}" {options} -oMyOutput.out MyInput.in
;unpackcmd = "Precomp\{compressor}" -r -oMyInput.in MyOutput.out
;datafile = MyInput.in
;packedfile = MyOutput.out

[External compressor:PrecompX, PrecompMTX, precompx]
header = 0
default = -c:64m -t:50% --logs
packcmd = MTX.exe a:precomp {options} - - <stdin> <stdout>
unpackcmd = MTX.exe x:precomp -t:100p --logs - - <stdin> <stdout>



I gave the default example and some random files a go:

precomp048+srep+lolz64 works
precomp048+srep+LolzMTX ERROR: general (de)compression error in LolzMTX

precomp048+srep+rz64 works
precomp048+srep+RazorMTX ERROR: general (de)compression error in RazorMTX

precomp048+srep+bcm32 ERROR: general (de)compression error in bcm32
precomp048+srep+BcmMTX works

It is a bit strange that, if not using MTX it works, using MTX will make it not working.
If not using MTX doesn't work, using MTX will make it work?
Is it because of the input output in the settings?

Another test for
PrecompMTX+srep+ any normal or MTX final compressor

none works.

The console / log doesn't say why at all.
It seems as soon as MTX invokes precomp, it exits/crashes.
The temp folder for precomp has been created, only the 0 folder has 512KB file, all others are 0

[BLACKFIRE69]

MTX New Update v5.0.0.1


What's new: MTX v5.0.0.1

Some improvements.
Minor bugs fixed.
Exception handling.
MTX.ini is no longer supported by default.
{compressor} keyword is no longer supported.
A few more changes:

%exePath% ==> $exePath$
%basePath% ==> $basePath$



Example contains:

Bcm
Bsc
Lolz
Mcm
Mpz
Nanozip
Precomp
Razor
Uharc
Zcm
Zstd

[ffmla]

^^
Thanks for the MTX update.
But in freearc alpha build struggled with Antivirus:(.
https://www.virustotal.com/gui/file/84f957d07a1604fccc7bf09bbf455157f873e56069ba1ea37f d67fc5c16d5ac8/detection
If we turnoff all Ok:rolleyes:

[Ele]

^^
Thanks for the MTX update.
But in freearc alpha build struggled with Antivirus:(.
https://www.virustotal.com/gui/file/84f957d07a1604fccc7bf09bbf455157f873e56069ba1ea37f d67fc5c16d5ac8/detection
If we turnoff all Ok:rolleyes:

Maybe because of UPX. He has compressed all exe files with UPX?

[BLACKFIRE69]

^^
Thanks for the MTX update.
But in freearc alpha build struggled with Antivirus:(.
https://www.virustotal.com/gui/file/84f957d07a1604fccc7bf09bbf455157f873e56069ba1ea37f d67fc5c16d5ac8/detection
If we turnoff all Ok:rolleyes:

it's common to see false positives due to upx. I didn't include any trojans with them. :p

[Masquerade]

@Blackfire69

I've hbeen having issues with both MTX.exe and RazorX trying to compress a few files, leading to an FreeArc error at 99.1% in both tools (bad file descriptor error).

I use srep before running. Will this cause issues during decompression? I've never encountered that before. I'm currently testing that myself. Arc says decompression error in razorx/mtx though, and not srep

[BLACKFIRE69]

@Blackfire69

I've hbeen having issues with both MTX.exe and RazorX trying to compress a few files, leading to an FreeArc error at 99.1% in both tools (bad file descriptor error).

I use srep before running. Will this cause issues during decompression? I've never encountered that before. I'm currently testing that myself. Arc says decompression error in razorx/mtx though, and not srep

ok, more info please.

how many threads did you use? (for the compression and the decompression)
did you run this test on the same cpu?


in general, MTX returns errors because the number of threads doesn't match.
I mean for example, MTX can't control this situation if you use 8 threads for compression and 16 threads for decompression. this's because the number of threads for decompression should always be less than or equal to the number of threads you used in compression.

[BLACKFIRE69]

how do I fix this? (MSVCP140.dll)

solution: Install VC++ 2015-2017-2019 package (both x86 and x64)

latest Visual C++ downloads (https://support.microsoft.com/en-us/topic/the-latest-supported-visual-c-downloads-2647da03-1eea-4433-9aff-95f26a218cc0)

[Masquerade]

I'd argue this is a better solution :D
https://github.com/abbodi1406/vcredist/releases

Anyways, I solved my issues, it appears the files being extracted were simply too small (1-2kb) and something just went wrong.

I played around a bit and managed to reach a correct extraction:

FreeArc 0.67 (March 15 2014) extracting archive: MASQUERADE-Data_01.MSQ
Extracting 41 files, 5,071,054,255 bytes. Processed 2.6%
Extracted 41 files, 1,955,615,101 => 5,071,054,255 bytes. Ratio 38.56%
Extraction time: cpu 28.59 sec/real 158.83 sec = 18%. Speed 31.93 mB/s
All OK

[BLACKFIRE69]

MTX v5.0.0.2 New Update


What's new:
1. Fixed config file parsing bug. (Thanks to @Cesar82)

2. Added support of MTX.ini.
-- When you use MTX.ini next to the EXE file,
the settings "--basePath", "--cfgfile" and "--tmpPath" are overridden.
This simply means that if you use MTX.ini, you no longer need the "--basePath",
"--cfgfile" and "--tmpPath" setting in the commandline.

-- If you use both MTX.ini and the commandline settings, the MTX will ignore
the commandline settings ("--basePath", "--cfgfile" and "--tmpPath").

3. Compatible with the previous version (v5.0.0.1).

MTX.ini
[MTX64]
MainPath=".\"
Config="$basePath$\arc.ini"
TmpPath="$basePath$\temp64"Section name
Filename>> MTX32.exe then [MTX32]
Filename>> MTX64.exe then [MTX64]
Filename>> MTX.exe then [MTX]
Filename>> FileName.exe then [FileName]

MainPath
MainPath="$exePath$" or
MainPath=".\..\..\" or
MainPath="C:\test"

Config
Config="$basePath$\arc.ini" or
Config="$exePath$\arc.ini" or
Config=".\..\..\arc.ini" or
Config="C:\test\arc.ini"

TmpPath
TmpPath="$basePath$\temp" or
TmpPath="$exePath$\temp" or
TmpPath=".\..\..\temp" or
TmpPath="C:\test\temp"

[BLACKFIRE69]

MTX v5.0.0.2 Improved version

What's new:


Fixed a few minor bugs.
Improved exception handling.
Returns an error code for each situation.



* Check the first post for more info & downloads.


.

[Masquerade]

@BLACKFIRE69

MTX crashes on when decompressing on systems with strange numbers of cores/threads (e.g. 6c/6t and 6c/12t)

Are you able to fix this? (issues in both EXE and CLS versions)

[KaktoR]

I can only decompress with mtx if t100p is set in packcmd in arc.ini.

If I set t75p (or any other number other than 100) freearc can't decompress the archive.

Is this a bug or am I missing something?

Edit: encode and decode have to be equal. I missed it. If you used t100p in encode you have to use t100p in decode too. Sorry :D

[Cesar82]

BLACKFIRE69, you could remove the ":" from the parameters in your MTX.
Example, how can I send 4 threads (-t4) as a parameter from pack.bat. It is not allowed to send "t:4" because the ":" are already FreeArc parameter separators.

[BLACKFIRE69]

BLACKFIRE69, you could remove the ":" from the parameters in your MTX.
Example, how can I send 4 threads (-t4) as a parameter from pack.bat. It is not allowed to send "t:4" because the ":" are already FreeArc parameter separators.

sure,
[External compressor:precompx,mtx_precomp]
header = 0
default = -t100p --logs --basePath=".\..\.." --cfgfile="$basePath$\arc.ini" --tmpPath="$exePath$\temp"

packcmd = MTX\Win64\MTX.exe a -mprecomp048 -c256m {option} - - <stdin> <stdout>
unpackcmd = MTX\Win64\MTX.exe x -mprecomp048 {option} - - <stdin> <stdout>

[BLACKFIRE69]

I can only decompress with mtx if t100p is set in packcmd in arc.ini.

If I set t75p (or any other number other than 100) freearc can't decompress the archive.

Is this a bug or am I missing something?

Edit: encode and decode have to be equal. I missed it. If you used t100p in encode you have to use t100p in decode too. Sorry :D

an equal or lesser value is also valid.
ex:
encode: 16 (Threads)
decode: 16, 8, 6, 4, 2, ...

if any user has entered an invalid value for the decoding threads, i'll improve MTX so that it can be corrected by MTX itself in a future update.
(since i'm a bit busy, give me some time for that ;))

in the meantime you can test out the beta version of new RazorX. i've used 50p for encoding and 100p for decoding. although this is usually invalid, it's automatically corrected by RazorX.

[External compressor:rzmt]
header = 0
packcmd = RazorX.exe a -c:64m -t:50p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = RazorX.exe x -t:100p - - <stdin> <stdout> (read Important.txt before use)
.

[Cesar82]

an equal or lesser value is also valid.
ex:
encode: 16 (Threads)
decode: 16, 8, 6, 4, 2, ...

if any user has entered an invalid value for the decoding threads, i'll improve MTX so that it can be corrected by MTX itself in a future update.
(since i'm a bit busy, give me some time for that ;))

in the meantime you can test out the beta version of new RazorX. i've used 50p for encoding and 100p for decoding. although this is usually invalid, it's automatically corrected by RazorX.

[External compressor:rzmt]
header = 0
packcmd = RazorX.exe a -c:64m -t:50p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = RazorX.exe x -t:100p - - <stdin> <stdout> (read Important.txt before use)
.

For you to better understand what happened.
Me and KaktoR were doing tests with RAZOR MTX.

The configuration used was:
packcmd = MTX.exe a -m:rz {option} -c:64m -t:100p - - <stdin> <stdout>
unpackcmd = MTX.exe x -m:rz {option} -t:100p - - <stdin> <stdout>

As his processor is 6/12 and my processor is 12/24
If it compressed and sent me the file I couldn't extract it because it would have been compressed using 12 threads and I would extract using 24 threads.
So if you make a game backup and upgrade the CPU to one with more threads, there will be an extraction error.
I know I could set it to t2 for extraction, but that loses the meaning of being multi threaded (use only 2 of 24).

The workaround is to set it to 100p use send thread number (get from system) as method parameter and when extract use {option} to set -t parameter.

Thanks for answering!

[BLACKFIRE69]

As his processor is 6/12 and my processor is 12/24
If it compressed and sent me the file I couldn't extract it because it would have been compressed using 12 threads and I would extract using 24 threads.
So if you make a game backup and upgrade the CPU to one with more threads, there will be an extraction error.



that won't happen in the next update of MTX. checkout the RazorX beta above,
the following config works without any problems.

[External compressor:rzmt]
header = 0
packcmd = RazorX.exe a -c:64m -t:12 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = RazorX.exe x -t:24 - - <stdin> <stdout>OR

KaktoR can use it to encode as,
packcmd = RazorX.exe a -c:64m -t:100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmpalso, cesar you can use the same to decode, -t:100p

unpackcmd = RazorX.exe x -t:100p - - <stdin> <stdout>

[BLACKFIRE69]

Update!


USAGE:

[External compressor:zstdmt,zstdx,mtx_zstd]
header = 0
default = -t100p --basePath=".\..\..\" --cfgfile="$basePath$\arc.ini"

packcmd = MTX\Win64\MTX64.exe a -mzstd64 -c128m {option} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = MTX\Win64\MTX64.exe x -mzstd64 {option} $$arcpackedfile$$.tmp $$arcdatafile$$.tmp


[External compressor:zstdmt,zstdx,mtx_zstd]
header = 0
default = --basePath=".\..\..\" --cfgfile="$basePath$\arc.ini"

packcmd = MTX\Win64\MTX64.exe a -mzstd64 -c128m -t8 {option} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = MTX\Win64\MTX64.exe x -mzstd64 -t8 {option} $$arcpackedfile$$.tmp $$arcdatafile$$.tmp


[External compressor:zstdmt,zstdx,mtx_zstd]
header = 0
default = --basePath=".\..\..\" --cfgfile="$basePath$\arc.ini"

packcmd = MTX\Win64\MTX64.exe a -mzstd64 -c128m -t4 {option} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = MTX\Win64\MTX64.exe x -mzstd64 -t8 {option} $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[BLACKFIRE69]

MTX v5.0.0.3 beta Update 1

-- Fixed a minor issue.

Thanks @Cesar82 for pointing this out.


MTX v5.0.0.3_beta Update 01 _ 64-Bit.rar (https://app.box.com/s/lcgjgpw27snldh08wwywk271vq2a55bk)

[BLACKFIRE69]

1. Created MTX2023 from scratch and optimized for speed and efficiency.
- As a result, it isn't compatible with any previous versions.

2. The bugs experienced in previous versions have been addressed and fixed.

3. MTX can handle scenarios where the #userThreads > #compressionThreads.

4. Simultaneous reading and writing of data takes place.
- Previous versions required extra waiting time that's no longer necessary in this version.
- Make sure to specify the '-ds' parameter with the 'arc.exe a'. (arc.exe a -ds ...)

Remark:

1. '--TempPath=' parameter has been removed.
- Tmp files will now be created in the freearc temp dir instead.

2. '%' sign has been removed from threads (100%).
3. Support for "MTX32.ini" and "MTX64.ini" files has been discontinued.
4. There won't be a 32-bit version of MTX.


1. Recommended setting (stdio mode) but no info will be displayed.

packcmd = MTX64.exe a -mNz64 -c32m {options} - - <stdin> <stdout>
unpackcmd = MTX64.exe x -mNz64 {options} - - <stdin> <stdout>


2. Info will only be displayed for packing.

packcmd = MTX64.exe a -mNz64 -c32m {options} - $$arcpackedfile$$.tmp <stdin>
unpackcmd = MTX64.exe x -mNz64 {options} - - <stdin> <stdout>


3. Info will only be displayed for unpacking.

packcmd = MTX64.exe a -mNz64 -c32m {options} - - <stdin> <stdout>
unpackcmd = MTX64.exe x -mNz64 {options} - $$arcdatafile$$.tmp <stdin>


4. Info will be displayed for both packing and unpacking.

packcmd = MTX64.exe a -mNz64 -c32m {options} - $$arcpackedfile$$.tmp <stdin>
unpackcmd = MTX64.exe x -mNz64 {options} - $$arcdatafile$$.tmp <stdin>


.

[L0v3craft]

Tested MTX 2023 with razor. It is a disaster, freezes my computer. Never happened with the accelerator razorx.

Looks like it has created 190k files in the temp folder of freearc, when I have compressed one single file of 2.77GB.

[Cesar82]

MTX v5.0.0.3 beta Update 1

-- Fixed a minor issue.

Thanks @Cesar82 for pointing this out.


MTX v5.0.0.3_beta Update 01 _ 64-Bit.rar (https://app.box.com/s/lcgjgpw27snldh08wwywk271vq2a55bk)

Could you get me the 32-bit release on UPX of this version 5.0.0.3?

[BLACKFIRE69]

Could you get me the 32-bit release on UPX of this version 5.0.0.3?


The MTX project has been rebooted, and it is now starting from version 0.1. The new version of MTX is compatible with both cls-diskspan.dll and DiskSpan-GUI.



The first post has been updated.


.

[Cesar82]

The MTX project has been rebooted, and it is now starting from version 0.1. The new version of MTX is compatible with both cls-diskspan.dll and DiskSpan-GUI.



The first post has been updated.


.
1) The --tmpPath= parameter and the $tmpPath$ variable no longer exist?
2) There is no longer support for external configurations in MTX.ini?

[BLACKFIRE69]

1) The --tmpPath= parameter and the $tmpPath$ variable no longer exist?
2) There is no longer support for external configurations in MTX.ini?



no.

[Cesar82]

Does it still require the "-ds" parameter in FreeArc?

[luxuse]

can you repost a link for the new versions of MTX ?