[Dev]XTool

[Gupta]

You may store some rare/large duplicated streams in a temp file, while storing small/frequent dupes in RAM - this way the excessive HDD load won't happen,

maybe he can introduce the second phase in compression, then he can store forward reference count for a stream that should ideally decrease requirements memory size beyond window size and that should increase compression too.

HDDs are very slow, I recently upgraded to nvme based storage and I can feel the speed.