Fixing the download mirrors @ GCW...
 

As everyone has noticed by now: we are having download mirror problems for about the past week

We know that the mirrors are down so please do not mail is that they are ;)

The problem has been fixed for now but it will surely happen again as the amount of visitors is getting bigger, thus creating so much traffic (> 30Mbit/s). This is quite hard to maintain without real financial backing... (donations are still VERY welcome).

As we do NOT want to be a members only systems where members will have to pay a monthly fee to be able to download, we are seeking other means to keep GCW "alive" and free.

One of the idea's is to use eDonkey for sharing the small files but we think this will mean that most of the files will not be available anymore after about 3 months or so. But this is a start.
For this to work we need code in perl or php to generate the ed2k hash which is needed to create the ed2k link. If anyone has this code let us know!

Another one is using BitTottent but this will die so much sooner as it too much centralized to a single tracker

If anyone has a better idea or solution let us know! And do not be scared if you think that your solution is not good enough to tell!

Sounds like a great idea to me.

As for the problem with old files not being available, how about enabling HTTP-download of files when they become unavailable on the ed2k-network?
I see several ways to do this:

1. A pure timelimit. After a fixed time (like a couple of months) the file will be made available on the HTTP-mirrors wether it is still available on ed2k-network or not.

2. Some reporting-system where people can click a button to report that the file is no longer available on ed2k-network (possibly another button for "yes, it's available)
and when enough people have clicked "not available" it will be made available by HTTP.

3. The most advanced way would be to have some script do a search for complete sources of the files on ed2k-network, record how long ago it saw a complete source
and if it passes a preset timer (like a week) it will enable HTTP-download.

I'd definately go for number three if someone could implement it.
Unfortunately I am not a coder, so I can't help with implementation, just thought I'd supply some suggestions...

Edit: Just wanna stress the point of complete sources if someone decides to go for method #3.
I think we all know how annoying it is that jigle only reports "a source", but that source may have just 2 percent of the file.
That does not really help anyone...

Hi guys, if I may say so, I'd insist that there should still be a way to download stuff via HTTP or FTP. At my workplace, we have a _very_ strict firewall: I'm unable to use ICQ, Napster, Kazaa, eDonkey, eMule and any other peer-to-peer program... Even having problems with FTP sites, too...! :( Thanks,

Joe

You could insist this but there is just not enough bandwidth to only support http/ftp... We do this for the fun of it and the, not to get rich... Banners and other advertising just do not pay for the required bandwidth. It used to but the last 2 years it only seems to cost more than it pays... This is not a healthy solution...

Have you thought about Bittorrent? Its a little bit different from Edonkey.

The users have to download only a torrent which isn't greater then a txt-file.

Servers still host the crack but there are other servers (routet by trackers) who also have the cracks prepared for downloads.

The speed for new cracks is really fast, about 50 in the middle.

Old cracks could be asked, and then upped (seeded) by your servers or by us powerforumusers.

Hmm, just got another idea.
To help people like "Joe Forster/STA" there could be a delayed queueing system for HTTP-downloads to complement the ed2k-downloads.

This would mean that the prefered method would be to go through P2P, but if you can't you can request HTTP-download by clicking on a link that puts you in a queue for the file.
That queue would have a timer in it so that the download will not begin until a preset waiting time has been reached.
The waiting-time should of course be more than what is to be expected from ed2k-downloads...

This system should encourage using ed2k as a primary download-location, but still enables those who can't use that to get the files albeit with a slight delay.
Also, to pay for the download there could be a rotating banner in place on the queueing-page so that people downloading by HTTP will effectively pay for their download by advertisement revenues.

Most people still think that just showing banners get's someone paid (someone gets paid for sure but it ain't us).

BANNER FACTS:

o CPM banners (get paid to show a banner) are virtually extinct (or are only used for popups, which we do not want to use)

o CPC banners (get paid when a user clicks the banner) are paying about $0.01 per click making them almost useless as people just do not click banners as much as they did before (2 years ago).

o CPA banners (get paid when someone buys or signs-up for something from the link/banner they clicked) are the only ones that are preferred by advertisers as they only have to pay when they make a sale. But you need very targetted banners, like DVDXCOPY, to make them profitable... so this is the best deal for the advertiser and the worst for the webpublisher!

Average we get, for the same amount of visitors, about 50-100x less then we got about 2 years back... So you can see the predicament we (and everyone else running a website) are now in


@ b9AcE

Most of the mirrors we are given do not support scripting that is why GCW is a static website without any scripting. Still it would be possible but it would take some programming (anyone interested ;))

Like I said before, I'm no coder, so I can't really implement it, but i have a suggestion for anyone who wanna give it a try:

A nifty javascript+cookie+referer thingie.
Javascript is executed on the client, so it does not require anything specific by the server.
The first page would be a timer-page with a javascript that counts down the preset "discouragement-time".
When 0 is reached a cookie is set and you get sent to the real download page.
This page has another javascript which checks for the cookie (to make sure that the timer was not bypassed) and checks for a valid referer (to prevent external linking).
Since it is all executed on the client it can of course be bypassed, but most people wouldn't put the effort in I think?

Somewhere in there you could also set a cookie if a banner is clicked and thus require X amount of banners to be clicked.
I'm not even sure if javascript can do those things (cookie- and refererchecks) but I'm sure someone else around here knows?

I know I would sure dislike downloading that way, and so I'd go for ed2k-downloading instead... ;)

they did have a e-donkey search thing which was coded here: http://www.analogx.com/contents/news.htm but I cant seem to find it.

How about getting in touch with the guy that is running jigle to see if he could customize his software to only include full sources?
GCW being such a well known and respected site, I'd assume he might do it.
His email is on the page linked to below (don't wanna cause him spam...).

If you are satisfied with only partial sources you can include a jigle-feature as it is today:
http://jigle.com/background.html#ways2jigle

for the emule hash implementation:

empire, did you ask the emule team? maybe they will be helpfull.

The edonkey hash is the quite known RSA MD4 checksum.

It should be possible to get the source code for an calculator. :)

Sad news.
Could you please keep in mind that not all people will be able to use e-Donkey and BitTorrent :(
I am myself behind a very restrictive firewall...
Anyway, thank you very much for the great service you have provided for years :D

For the edonkey hash (Message Digest 4): http://search.cpan.org/author/OTAKA/...MD4-1.3/MD4.pm

I think it is a good idea, to put share the cracks through p2p networks. And since most cracks are rather small, it is fairly easy to keep them in the share for a while.

And after a few months (when the files are vanishing from ed2k) it should be nor problem to provide them by http (not to much traffic to be expected).

An extension: what about sharing a weekly (or monthly) collection of GCW patches? if the patches are bundled together in larger archives, they typically stand longer in the network, and it is easier to help with a reshare.

Adding an ed2k link in addition to the http/ftp site links would be an excellent idea as I'm sure many of us have the eMule/eDonkey client running 24x7 and would be happy to leave the small(usually <5MB, often just a few KB) files downloaded from GCW shared indefinitely.

Incomplete files won't even be an issue as eDonkey works in 9.28MB chunks, any file smaller than that is a single chunk which is either available or unavailable, and I believe almost all the files on GCW are under 9MB.

All thats needed is for an ed2k link to be added to the download page for every file and they'll probably become available on the eDonkey network over time if you request that eMule/eDonkey users place them in a shared folder after downloading from an http/ftp server, to reduce the load. Provided the contents of the files hosted on each server are identical, the resultant ed2k hash when its shared will be identical regardless of the filename or datestamp; it won't have to be downloaded using ed2k to be shared on it so the number of people sharing it should steadily increase. Theres nothing to lose by giving it a try and its got to be better than the current situation!

Those people unable to use eMule at work, can't you use it at home instead? Also its possible to configure eMule to use whatever ports you want, eg. port 80 which obviously won't be blocked, to get around restrictive firewalls implemented by workplaces, colleges, lame ISPs etc.