FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Question about automating generation of start & end file offsets for stored archives (https://fileforums.com/showthread.php?t=104609)

L33THAK0R 26-08-2021 02:27

Question about automating generation of start & end file offsets for stored archives
 
Hi all,

While the subject matter doesn't really relate to this sub-forum, its applications do. I'm asking it here, rather than somewhere like "Stack Overflow", since there are individuals here who have some experience with the manual component of this process, and I don't have anything to contribute, as partial work on a given subject matter is required/strongly encouraged within the submitted inquiry.

Some archives like the ".PCK" sound containers (which can be found in titles such as "A Plague Tale: Innocence) store their containing files uncompressed, and sequentially (based off of initial comparison of the archives hex-code and extracted files, extracted from the container), as well as "Metro 2033" & "Metro: Last Light"s, ".vfs0" archives (using the same methodology).

Currently, while extraction is supported, to my knowledge, tools for repackaging these file-types are not publicly available. While it is possible to manually calculate these offsets, such a process would be incredibly time consuming, as some of these archives have upwards of 1000+ files (ranging from as small as a few hundred kilobytes to a few megabytes).

While this is a long shot, does anyone know of any tool suite capable of being fed a series of files as an input, grabbing the first and last 20 or so offsets (for example), then search through a secondary input (the "main" archive) for the location (if present) of the start and end offsets, then record this to a separate file, such as a simple text document?

If such a tool suite doesn't exist, I'll likely start looking into developing my own tool for it, as I have a strong feeling I'll be dealing with this type of modification a fair bit in my future releases.

panker1992 26-08-2021 07:55

1 Attachment(s)
Quote:

Originally Posted by L33THAK0R (Post 493763)
Hi all,

While the subject matter doesn't really relate to this sub-forum, its applications do. I'm asking it here, rather than somewhere like "Stack Overflow", since there are individuals here who have some experience with the manual component of this process, and I don't have anything to contribute, as partial work on a given subject matter is required/strongly encouraged within the submitted inquiry.

Some archives like the ".PCK" sound containers (which can be found in titles such as "A Plague Tale: Innocence) store their containing files uncompressed, and sequentially (based off of initial comparison of the archives hex-code and extracted files, extracted from the container), as well as "Metro 2033" & "Metro: Last Light"s, ".vfs0" archives (using the same methodology).

Currently, while extraction is supported, to my knowledge, tools for repackaging these file-types are not publicly available. While it is possible to manually calculate these offsets, such a process would be incredibly time consuming, as some of these archives have upwards of 1000+ files (ranging from as small as a few hundred kilobytes to a few megabytes).

While this is a long shot, does anyone know of any tool suite capable of being fed a series of files as an input, grabbing the first and last 20 or so offsets (for example), then search through a secondary input (the "main" archive) for the location (if present) of the start and end offsets, then record this to a separate file, such as a simple text document?

If such a tool suite doesn't exist, I'll likely start looking into developing my own tool for it, as I have a strong feeling I'll be dealing with this type of modification a fair bit in my future releases.

Here is how to extract PCK files


quickbms -d -Y pck_to_wem.bms data.pck

quickbms -w -r pck_to_wem.bms data.pck data.pck_extracted

Second one is how to import back in

L33THAK0R 26-08-2021 09:06

Quote:

Originally Posted by panker1992 (Post 493765)
Here is how to extract PCK files


quickbms -d -Y pck_to_wem.bms data.pck

quickbms -w -r pck_to_wem.bms data.pck data.pck_extracted

Second one is how to import back in

Ah sick, cheers for this, I'll get to testing this with "A Plague Tale: Innocence" right away! Should testing be successful, this'll make separating localised audio as an optional offering to the end-user possible!

I'm still quite interested in the inquiry to the existence of a tool/suite, mentioned in the original post, as should one not exist I'll get straight to work developing one!

panker1992 30-08-2021 15:21

My repack size of the game which is one of my favourites of course :D

9.15GB final size :D

Dragonis40 31-10-2022 01:47

Quote:

Originally Posted by panker1992 (Post 493765)
Here is how to extract PCK files


quickbms -d -Y pck_to_wem.bms data.pck

quickbms -w -r pck_to_wem.bms data.pck data.pck_extracted

Second one is how to import back in

Good morning, how can I use this instructions for videogame repacking? I mean, to reimport files into .pck file, I need the original one. I think it's better to create a new .pck file without the original one, otherwise it's useless in terms of installation videogame.
So, starting from the extracted files from .pck file, how can I "reduild" a new .pck file without the original one?

Masquerade 31-10-2022 02:14

Quote:

Originally Posted by Dragonis40 (Post 498946)
Good morning, how can I use this instructions for videogame repacking? I mean, to reimport files into .pck file, I need the original one. I think it's better to create a new .pck file without the original one, otherwise it's useless in terms of installation videogame.
So, starting from the extracted files from .pck file, how can I "reduild" a new .pck file without the original one?

PCK archives don't use compression, so what you can do is the following:

- use Bincat by Aluigi to concatenate the output wems into one large file

- use hdiffz to make a patch between the original PCK and the concatenated data

Then, in your repack installation, concatenate the wems with bincat and apply the patch you made with hpatchz to restore the original PCK.

KaktoR 31-10-2022 02:33

Another solution:

Use xtool erase and decode functions to restore original files back into modified pck archive.

Dragonis40 01-11-2022 23:18

Quote:

Originally Posted by Masquerade (Post 498948)
PCK archives don't use compression, so what you can do is the following:

- use Bincat by Aluigi to concatenate the output wems into one large file

- use hdiffz to make a patch between the original PCK and the concatenated data

Then, in your repack installation, concatenate the wems with bincat and apply the patch you made with hpatchz to restore the original PCK.

Thank you for your reply! Once all .pck files have been decompressed with quickbms, the output files are: .wav, .bnk, .dat files and no .wem files.
So how can I correctly use Bincat and hdiffz procedure?

Masquerade 02-11-2022 00:06

Quote:

Originally Posted by Dragonis40 (Post 498973)
the output files are: .wav, .bnk, .dat files and no .wem files.
So how can I correctly use Bincat and hdiffz procedure?

No need to worry about the different file types, just make sure all of the extracted files are in the same folder. Then, in cmd do:

Code:

bincat extracted_data\* new.pck
hdiffz-x64 new.pck old.pck patch.diffz

To apply the patch in your repack installation, do:

Code:

bincat extracted_data\* new.pck
hpatchz-x64 new.pck patch.diffz old.pck


Dragonis40 03-11-2022 13:00

Quote:

Originally Posted by Masquerade (Post 498974)
No need to worry about the different file types, just make sure all of the extracted files are in the same folder. Then, in cmd do:

Code:

bincat extracted_data\* new.pck
hdiffz-x64 new.pck old.pck patch.diffz

To apply the patch in your repack installation, do:

Code:

bincat extracted_data\* new.pck
hpatchz-x64 new.pck patch.diffz old.pck


Hello! I've done what you suggested me, but I can see no improvement at all in terms of file compression. The resulting files from hpatchz have exactly the same size of original .pck file. So, what' the advantage?

Dragonis40 06-11-2022 02:23

Quote:

Originally Posted by Dragonis40 (Post 498994)
Hello! I've done what you suggested me, but I can see no improvement at all in terms of file compression. The resulting files from hpatchz have exactly the same size of original .pck file. So, what' the advantage?

Hello everybody! what I don't understand is how to correctly compress these pck files. I'm talking about "A Plague Tale: Requiem". There's a file called "VO_MEDIA.PC.PCK"; using WWise.bms and quickbms, I can extract all files inside and they are, of course, .wem files. Then, I've tried to compress them using WemTool.exe, but the resulting compression is very poor. So, I've used bincat.exe to create a concatenated file with all .wem files and hdiffz-x64.exe to create a patch between the original PCK and the concatenated data.
Which file do I have to use in my repack? Only patch.diffz file? I don't think so, because, in order to create the original "VO_MEDIA.PC.PCK" file, I need patch.diffz file and .wem file or the file create by bincat.exe. So I increase the final .bin file, instead of reducing it. So, how can I do?

KaktoR 06-11-2022 03:11

You have to include modified PCK file + the patch file obviously to restore back to original file.


All times are GMT -7. The time now is 16:40.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com