Changing header of executable

[ptsz]

Hi,

I was wondering if someone knew how I could modify the header of an executable to NOT open a console window? After the executable has been compiled. I know you usually do that during compiling but.

It is my current understanding that you can modify the header of the exe file to not open a console window behind it. I hope that correct.

Hopefully someone can help me?


Thanks.

[TippeX]

change the os subtype setting in the pe header from win cui to win gui... simple patch

[ptsz]

Thanks for the response.

Could you also tell me how to do that, which tools and stuff to use plus well some notes on how to do it.

I have PE explorer, don't know if thats the one to use. Obviously I have a hex editor.

Using PE explorer. When I open it up, the header.

Subsystem    0003h     Win32 Console

This is what you mean I guess? Close I hope at least.


I am new to it, but happy to learn how to do it.

[TippeX]

yup thats it. .
if i remember right gui is 02 so change that, job done

[ptsz]

OK. And how do I change it? Because with PE explorer I can't as it seems. And with neo hex editor I didn't find the 003h (because I couldn't search for an "h").

Do you know another software I should be using?

And can I do this with an app without a GUI as well?


EDIT:

I found it. Thank you very much for the help. So glad I could FINALLY get it to work.

[TippeX]

h = hex, can also be 0x
its convention...

changing the subsystem isnt always a good idea, especially if it outputs to the console

[Joe Forster/STA]

Yup, if the software expects to be running as console/GUI - and what software wouldn't?! - then changing its subsystem will make it go completely crazy. I just tried it with one of my own programs, originally console and hacked into GUI: after processing its input and sending the results into its output (redirected into a file so that I can see it really did something), it keeps running as if it is unable to realize that its input has ended.

(For hacking executables, Hacker's View is probably your best choice.)

[ptsz]

Yea I wouldn't expect user input to work anymore probably.

But for some that don't use or need it, it would be great. Plus executables compiled from other languages that may not have a console remover in the compiler as option.

[TippeX]

not true..if its cui..its there for a reason..
example.. try looking at some dlls in system32

[ptsz]

OK, which one in particular?

I ran the app though process explorer but it uses a lot, so I didn't know which in particular you meant.

PS: Also on another note, where should/can I ask about if anyone knows anything about LZSS compression? Which forum topic would that be?

[TippeX]

quite a few of the system dlls are compiled as cui... have a look
im guessing this question is driven from something like gta iv or maybe la noire that pops up a useless cui window?

but just changing it can cause problems as some compilers / languages have subtle differences to the winmain / dllmain / main depending on how it was compiled

whats the compression question?

[ptsz]

Oh no, it doesn't derive from game exe's. More from other languages which would not have the option to compile it without a console. Or just exe I would like to change or make, but lets say not having the option to remove the console.

You can also do it by running a winapi program calling the designated app by using "execlp("notepad.exe", 0);" but than you would of course need a second app to run the initial one without a console.
But it would have the same effect. In general the apps would be C/C++ coded (except if in another language).

I noticed some dlls compiled as cui yes.

The compression question would be. I have a game which uses LZSS compression for most of its content and I would like to hack it. It has already been hacked so I have the AES encryption key and you can extract it. The problem is that every file above 200k is corrupt, completely. So you can't use them.
That is the problem. I use QuickBMS to extract it. The problem is with the quickbms extracting script but I am not qualified enough to fix it properly. I was hoping someone else would.
I asked the author if he would help but he didn't want to.

I have the files, 1 win_000.nfs file 8mb and 1 win_000.full 4gb ready and uploaded (including both pack/unpack scripts). But I need someone who can help. Basically someone who knows something about LZSS compression to get it right.

[Joe Forster/STA]

You can run a console program without a console using start /b <program>. But that works from the command line interface - %COMSPEC% - only as it's an internal command. It won't work from a GUI program which is why I wrote two versions of the "Minimize2Tray" programs - console and GUI -, to be released some time...

[TippeX]

yeh, i always wondered why a hybrid type didnt exist.
ie: run from within console then console mode, otherwise gui...would be nice

also only realised "hackers view" as hiew (what i usually call it) - excellent program..i even got work to license a copy...author is a nice guy too

[Joe Forster/STA]

Originally Posted by TippeX

i always wondered why a hybrid type didnt exist.

Hmmm, interesting, never thought of that. Actually, the program, compiled as GUI, can find out what type its parent process is and if it's a console then open a new console or attach itself to the parent's console; there are simple API calls for that. But I know already one problem: launching a GUI program from the console will return at once without waiting for the program to initiailize/send some signal/exit (start /wait <program> helps, though) and there can be more...