Useful programs

[BLACKFIRE69]

Originally Posted by BLACKFIRE69

Ext Listing Of All Files In A Folder

Usage:
 ListExt.exe <Dir>

I've improved ListExt a bit to be able to recommend a compression method for that folder if needed. The purpose of this was to simplify some of the things I do on a daily basis, and I hope it will be useful for you too.

This is based on the "arc.groups" file. So place the first "arc.groups" file next to the "ListExt.exe" file.
The next step is to set up the "ListExt.ini" file. The aim is to introduce compression methods for each group of files.

It is done as follows,

ListExt.ini

; Freearc Compression Methods according to the 'arc.groups' file.

$default = srep+nz

$text = dict:p+lzp+ppmd:10:512mb+nz
$binary = precomp+rep:512mb+lzma:1gb:200:ht4:mc128
$exe = bcj2+rep:512mb+delta+exe+rz
$iso = ecm+precomp+rep:512mb+bcj2+delta+lzma:1gb:273:ht4:mc256
$obj = precomp+srep+nz
$bmp = bmp+nz
$wav = tta+nz
$mod = rep:512mb+lzma:1gb:200:ht4:mc128
$precomp = precomp+rep:512mb+delta+rz
$jpg = jpg+nz
$jpgsolid = precomp+rep:32+nz
$compressed = rep:64m+lzma:128m:128:ht4:mc16



; Other ListExt Settings
OverwriteLogs = 1

We're all set now. All that is left is to drag a folder through "ListExt.exe".
At the end of the process it'll recommend an appropriate compression method for your respective folder.
The file groups and the recommended compression method can be viewed in the ListExt_log.log file created at the end of the process.

.

[Barbarossa178]

Originally Posted by BLACKFIRE69

I've improved ListExt a bit to be able to recommend a compression method for that folder if needed. The purpose of this was to simplify some of the things I do on a daily basis, and I hope it will be useful for you too.

This is based on the "arc.groups" file. So place the first "arc.groups" file next to the "ListExt.exe" file.
The next step is to set up the "ListExt.ini" file. The aim is to introduce compression methods for each group of files.

It is done as follows,

ListExt.ini

; Freearc Compression Methods according to the 'arc.groups' file.

$default = srep+nz

$text = dict:p+lzp+ppmd:10:512mb+nz
$binary = precomp+rep:512mb+lzma:1gb:200:ht4:mc128
$exe = bcj2+rep:512mb+delta+exe+rz
$iso = ecm+precomp+rep:512mb+bcj2+delta+lzma:1gb:273:ht4:mc256
$obj = precomp+srep+nz
$bmp = bmp+nz
$wav = tta+nz
$mod = rep:512mb+lzma:1gb:200:ht4:mc128
$precomp = precomp+rep:512mb+delta+rz
$jpg = jpg+nz
$jpgsolid = precomp+rep:32+nz
$compressed = rep:64m+lzma:128m:128:ht4:mc16



; Other ListExt Settings
OverwriteLogs = 1

We're all set now. All that is left is to drag a folder through "ListExt.exe".
At the end of the process it'll recommend an appropriate compression method for your respective folder.
The file groups and the recommended compression method can be viewed in the ListExt_log.log file created at the end of the process.

.

What kind of dict;p? Thank you

[BLACKFIRE69]

Originally Posted by Barbarossa178

What kind of dict;p? Thank you

dict = dictionary replacements for texts

,

[Barbarossa178]

Originally Posted by BLACKFIRE69

dict = dictionary replacements for texts

,

Where do I get this ppmd compressor?

[Masquerade]

Originally Posted by Barbarossa178

Where do I get this ppmd compressor?

It's built into FreeArc.

[Masquerade]

Originally Posted by BLACKFIRE69

ListExt v1.0.0.2 - Update

Hey BLACKFIRE69,

Please may you build a version of listext that orders the output of file sizes from largest to smallest?

E.g.

Source:  .
Files :  123456789  [1337 GB]

 EXTENSION      FILE COUNT      PERCENTAGE      TOTAL SIZE
----------------------------------------------------------
.OGG                ?             XXX%           3GB
----------------------------------------------------------
.ZIP                ?             XXX%           2GB
----------------------------------------------------------
.EXE                ?             XXX%           1GB
----------------------------------------------------------
.DLL                ?             XXX%           100MB
----------------------------------------------------------
.TXT                ?             XXX%           50MB

Thanks!

[BLACKFIRE69]

Originally Posted by Masquerade

Hey BLACKFIRE69,

Please may you build a version of listext that orders the output of file sizes from largest to smallest?

E.g.

Source:  .
Files :  123456789  [1337 GB]

 EXTENSION      FILE COUNT      PERCENTAGE      TOTAL SIZE
----------------------------------------------------------
.OGG                ?             XXX%           3GB
----------------------------------------------------------
.ZIP                ?             XXX%           2GB
----------------------------------------------------------
.EXE                ?             XXX%           1GB
----------------------------------------------------------
.DLL                ?             XXX%           100MB
----------------------------------------------------------
.TXT                ?             XXX%           50MB

Thanks!

ok!


ListExt - Update

USAGE:
  ListExt.exe  <Directory>  [Order_Type]
  (or just drag and drop a folder over the ListExt.exe.)

ORDER_TYPE:
  /SA = Sort Alphabetically.
  /SS = Sort by Size.
  /SP = Sort by Percentage.
  Default is /SS (Sort by Size).

EXAMPLE:
  ListExt.exe "C:\Test\Example1"  /SA

.

[brispuss]

Originally Posted by BLACKFIRE69

ok!


ListExt - Update

USAGE:
  ListExt.exe  <Directory>  [Order_Type]
  (or just drag and drop a folder over the ListExt.exe.)

ORDER_TYPE:
  /SA = Sort Alphabetically.
  /SS = Sort by Size.
  /SP = Sort by Percentage.
  Default is /SS (Sort by Size).

EXAMPLE:
  ListExt.exe "C:\Test\Example1"  /SA

.

Thanks for this very useful program!

Would it be possible to create a file listing based on reading file headers instead of reading file extensions? Reasons being that file extensions are not always valid, and some files don't have extensions at all!

And if possible, export file types to appropriately named folders for further processing. And maybe import processed files back to their original locations?

Yes, I know this would be a big task, but possible?

Thank you.

[BLACKFIRE69]

Originally Posted by brispuss

Thanks for this very useful program!

Would it be possible to create a file listing based on reading file headers instead of reading file extensions? Reasons being that file extensions are not always valid, and some files don't have extensions at all!

And if possible, export file types to appropriately named folders for further processing. And maybe import processed files back to their original locations?

Yes, I know this would be a big task, but possible?

Thank you.

• yes, it would be rather useful if it also exported the full path for each file.
  
  
• i just modified ListExt.exe, and now it creates TXT files that include the full paths of the corresponding files for each file extension. (if there's no file extension, the file will be named 'Unknown-#').
  
  
• by default, this feature is disabled. use the config file (ListExt.ini) to enable it as follows:

ListExt.ini

...

; Save the categorized file list in TXT files.
ExportFileList     = 1
ExportRelativeName = 0
ExportFileListPath = ".\MyOutput"

.

[brispuss]

Thanks for the prompt reply and update to this program!

A definite improvement over the last version of listext, especially with regard to file locations!

In my case, I have many thousands of extensionless files. By reading the file headers, there are a variety of file types, mostly audio, imagery with some video as well.

Ultimately, I would like to see the option of being able to read extensionless files to determine their file types and then determine the appropriate compression methods to be used.

[Masquerade]

^^
Mesa by Shegorat would be ideal for this but you will have to contribute to krinkels.org in order to obtain that.

[azuravian]

Originally Posted by brispuss

Thanks for the prompt reply and update to this program!

A definite improvement over the last version of listext, especially with regard to file locations!

In my case, I have many thousands of extensionless files. By reading the file headers, there are a variety of file types, mostly audio, imagery with some video as well.

Ultimately, I would like to see the option of being able to read extensionless files to determine their file types and then determine the appropriate compression methods to be used.

I'm fairly new, but I use a free application built by the British National Archives called DROID (Digital Record Object IDentification).

It's a very simple tool that uses signature files to detect file formats. It's not perfect (for example, I have quite a few mp4s it won't detect and have recently submitted a request for them to update their mp4 signature), but it's able to return data for most of my files and it's always accurate (in my experience) when it specifies the format.

I threw a bunch of files in a folder and scanned them so you can see the output.



As you can see, it found most files and even looks inside a non-password protected archive to give info on contained files as well.

DROID can be downloaded directly from the UK National Archives site:
https://www.nationalarchives.gov.uk/...records/droid/

Another application called TrID. It uses an algorithm to find similarities between files in order to determine filetype as opposed to looking for specific hard-coded headers. It's available as a command-line application:
https://mark0.net/soft-trid-e.html

or with a GUI:
https://mark0.net/soft-tridnet-e.html

I hope these are helpful for you.

[felice2011]

Originally Posted by BLACKFIRE69

• yes, it would be rather useful if it also exported the full path for each file.
  
  
• i just modified ListExt.exe, and now it creates TXT files that include the full paths of the corresponding files for each file extension. (if there's no file extension, the file will be named 'Unknown-#').
  
  
• by default, this feature is disabled. use the config file (ListExt.ini) to enable it as follows:

ListExt.ini

...

; Save the categorized file list in TXT files.
ExportFileList     = 1
ExportRelativeName = 0
ExportFileListPath = ".\MyOutput"

.

Nice idea, but it's all old stuff, here we've always tried to obtain and create something new, something that helps the user compress a game from start to finish, in the simplest way possible, I'll post something new too 'me, but it is very difficult if not impossible to create something intelligent in this field that can give you with certainty the exact method of compression of all existing file extensions without having to use additional external programs.