Tropico 3/4/5 HPK Archiver

FordGT90Concept · Jan 25, 2010

HPK Archiver is no longer receiving updates. Try nicklec's "hpk" instead.

Original content follows:
---------------------------------------
This application can extract and archive data into and out of the HPK formatted file.

You need .NET Framework 4.6.1 or Mono to run it. Extract before running or else it won't be able to find LZ4.dll.

Works with:

Grand Ages: Rome
Tropico 3
Tropico 4
Omerta: City of Gangsters
Tropico 5
Victor Vran
Surviving Mars

Example, argument command:
-extract "D:\Steam\SteamApps\common\Tropico 4\dlc\Expansion.hpk" "C:\Users\Admin\Desktop\Expansion" -wait

Produced this result:

Commands currently supported (you should be able to do many at once)...
-extract "path to HPK" "path to folder to create/fill"

-archive "path to folder to pack (will be consumed)" "save HPK here"
Optional: Adding -zip to -archive will pack it. Do not do this on Tropico 5 because it won't be able to load it. Note: -zip only has to appear once to apply to ALL -archive commands after it. For example -archive INPUT 1.hpk -zip -archive INPUT 2.hpk would cause 1.hpk to be unzipped and 2.hpk to be zipped.

-unzip "path of file to unzip" "place to save unzipped file"

-wait
Special: This command only needs to appear once and if it does, it will stop the console from closing and show "Press any key to close program."

If no console arguments, the console will hide and the GUI will pop up.

1.1.0: Significant rewrite of code to add support for LZ4.
1.0.14: Archiving would have problems when the deflated size equals chunk size (usually 32768 bytes).
1.0.13: Fixed issue with small files giving inaccurate message/doing stupid things, no longer crashes when encountering a file with no data, and displays message when processing file dates.
1.0.12: Fixed 1333 out of 1334 errors in Tropico 4: Modern Times' Expansion.hpk
1.0.11: Writes data to output that doesn't inflate (fix for DLC).
1.0.10: Separated Tropico 5 game option which disables "Compress It" when checked because the game doesn't support it.
1.0.9: Lots of minor tweaks in the code.
1.0.8: Cleaned up file system code (part of FragmentedFile now) and renamed a lot of variables to more accurately reflect what it represents.
1.0.7: Catches some files that lie about being compressed and dumps the headerless contents.
1.0.6: No longer creates a temporary file for fragmented files and handles all files as fragmented files.
1.0.5: Removed a temp file and cleaned up code relating to all of the temp files (three of them now).
1.0.4: Now handles fragmented files properly.
1.0.3: Tons of bug fixes including the ability to decompress/extract challenges and ShaderCache.hpk.
1.0.2: No longer crashes if the output directory doesn't exist. It will simply create it.
1.0.1: Added bug fixes for abnormal files in Game.hpk.
1.0.0: Fixed a bug with the time stamps when archiving and also added support for Tropico 4 (experimental).
Beta 7: Fixed a bug in the archival code.
Beta 6: Archive feature is now functional, it automatically saves/loads settings, and many other tweaks.
Beta 5: Overhauled the GUI. It now shows statistics about the file and how far along it is while extracting.
Beta 4: It applies the "_filedates" to the files (modified date) and deletes the _filedates file because it is now irrelevant. Made the Zip decompression algorithm more efficient (one output stream per file).
Beta 3: Added Unix support for use on Mono.
Beta 2: Extracts all files successfully via C#ZipLib.
Beta 1: Erred on some files during extracting using zlib.

Implements:

lz4net for handling LZ4 compression.
SharpZipLib for handling ZLIB compression.

FordGT90Concept · Jan 28, 2010

As far as I can tell, Extracting is complete in beta 4. Next is archiving, followed by multithreading and other GUI improvements.

ShiBDiB · Jan 28, 2010

i dont get why we'd need this

FordGT90Concept · Jan 28, 2010

If you want to mod the game, you need it.

Beta 5 uploaded. I'm going to start work on the archiving component soon.

FordGT90Concept · Jan 29, 2010

Beta 6 upload. The application is now fully functional and fully featured.

sharpegg · Oct 1, 2010

Thanks

Thanks for this great tool. I was able to extract the music from the game using this.

FordGT90Concept · Oct 1, 2010

Heh, finally someone uses it. XD

You are welcome.

madames · Oct 4, 2010

HEY FORDGT90!

This is going to make you double-take, but it's Michael Adames. I used to moderate Hardware Analysis. Ya.

Big Tropico 3 player here, really enjoying the game! Found myself quite bothered with El Presidente's annoying sayings as well as some of the ambient files such as the seagull and rooster.

This tool worked perfectly for me and did just as you said. This has made the game infinitely more enjoyable, thanks again.

When I downloaded this and saw your name I was like "OH SHI---" clicked on your name and recognized a few people from Hardware Analysis and knew it was you. Hope all is well with you. Thanks again for the tool.

FordGT90Concept · Oct 4, 2010

Heh, yeah I remember you. Stick around.

Ignasius · Nov 6, 2010

Hello, thank you for this program, but I need two components to the archive file .hpk someone advise me how to do it?

Please reply. Thank

I need to archive two folders.

How do I do?

it must be in format .hpk

FordGT90Concept · Nov 6, 2010

Code:

selected folder
  -> folder 1
    -> file 1
    -> file 2
  -> folder 2
    -> file 1
    -> file 2
  -> file 1
  -> file 2

I'm pretty sure "selected folder" is consumed (it is implied in the format but doesn't actually exist). Just make sure to have multiple subfolders in there and it should work fine.

Ignasius · Nov 6, 2010

Please, try to decompress and re-archiving.

Please try this file. HPK
unpack and wrap

http://www.megaupload.com/?d=S2D8ROSO

after unpacking, you will receive two components

FordGT90Concept · Nov 6, 2010

It says it is unavailable.

Ignasius · Nov 6, 2010

download here: http://no-mercy.byl.cz/Data.hpk

FordGT90Concept · Nov 7, 2010

Ah, I see the problem. Archive should not include the given directory, only everything inside. For instance, if you extracted to C:\Stuff and you archived the contents of C:\Stuff, "Stuff" shouldn't appear in the output HPK file where currently it does.

I am working on it and will get a fix up as soon as possible but it could take a week or two.

Edit: It was easier to fix than I thought but I'm not 100% certain Tropico will like it. I attached the modified version to this post. Try it out and let me know if it works as expected or not (specifically, will Tropico read the outputs from it).

Ignasius · Nov 7, 2010

Yes, this correction is already functional

Thank you so much, now you can, pack more components into a file .hpk

game is fully functional, even after adjusting file
I'm working on another language version in this game

thanks very much
Sincerely, Ignasius

Czech Republic

kenji_03 · Nov 29, 2010

Music Pack

To make sure you know your work is appreciated I wanted to let you know that your tool is the only way to add music to Tropico 3 and that I am very grateful for your efforts. :respect:

I've been playing Tropico 3 for just a few hours and already the (offensively small) music track is getting repetitive, so you just made T3 infinitely more fun by allowing me to add more music to the game.

I am at work right now, but as soon as I get home I'll be installing this program and adding more songs to the game. Thank you!

If you happen to have your own, custom, Tropico3 music pack I'd LOVE for you to upload it btw.

Edit: I almost forgot, people on Steam Forums appreciate your work too btw.

FordGT90Concept · Nov 29, 2010

I don't have a custom music pack but I know there are instructions on the official forum for how to add Tropico (original game) music to Tropico 3 if you want to do that.

FordGT90Concept · Sep 6, 2011

Updated to support Tropico 4, theoretically.

Edit: FYI, the only difference is in the _filedates file. All the filetimes in Tropico 3 are standard Windows FileTimes. In Tropico 4, it appears they divided them by 2000 so that's what I did. The reason why it crashed was because without multiplying by 2000, it would produce an invalid filetime so when it tries to work with it when archiving, the numbers would be all wrong. It was a pretty easy fix but a very strange change they made there.

zbug · Sep 7, 2011

It's a nice tool you've made, but it seems it cant extract big files (1mb+), or is it only bugging on my computer? Been trying to extract some hpk extension files in AppData/Roaming/Tropico 4/Challenges.

FordGT90Concept · Sep 7, 2011

I don't even have an Challenges folder. I'll need specifics in order to debug it (how you obtained one that fails and the name of it).

Frejoh466 · Apr 6, 2012

Are you going to update it so it works with modern times? Trying to extract Game.hpk but the program just keep crashing all the times.

FordGT90Concept · Apr 6, 2012

I don't have Modern Times yet (in transit). I could take a look at it when it arrives.

FordGT90Concept · Apr 11, 2012

I think I see the problem. There is a file (Game\CommonEntities.txt) that is...messed up. The file is supposedly compressed but compressed files require at least 16 bytes just for the header and more for the data. This has 12 bytes total (header + data) making my algorithm run into the next file entry causing it to eventually crash when. The data literally is: 0000000000800000

That 8 in there is suspicious (value of 128) but I don't know if that belongs in the file or if it is part of the broken header. I think what I'm going to do is simply create an empty file and hopefully all will be good.

Will edit this post soon...

Edit: The 8 is the chunksize which is more or less always set to 32768. In other words, it is, in fact, a blank file.

There's a problem with another file...a batch file (HGO\SRP\srp-init.bat) which has the ZLIB header for compression but is, in fact, not compressed. I'm trying to find a solution that problem now. Other than those two hiccups, it appears to be working fine on Game.hpk.

Edit: To address the batch problem, it looks for that file specifically, checks if it is 70 bytes in length, compares it with what it stores internally, if a match, it cuts off the ZLIB header and outputs the remaining 54 bytes. If not a match, it outputs it normally.

In both cases, I would think that Tropico would crash if it tried to access them. Fortunately, the batch file looks like a dev tool and the text file is useless so it doesn't access them. I'll update the op with the updated executable shortly.

Frejoh466 · Apr 11, 2012

Thanks for the update, and for the great tool.

System Name	BY-2021
Processor	AMD Ryzen 7 5800X (65w eco profile)
Motherboard	MSI B550 Gaming Plus
Cooling	Scythe Mugen (rev 5)
Memory	2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s)	AMD Radeon RX 7900 XT
Storage	Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s)	Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case	Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s)	Realtek ALC1150, Micca OriGen+
Power Supply	Enermax Platimax 850w
Mouse	Nixeus REVEL-X
Keyboard	Tesoro Excalibur
Software	Windows 10 Home 64-bit
Benchmark Scores	Faster than the tortoise; slower than the hare.

System Name	BY-2021
Processor	AMD Ryzen 7 5800X (65w eco profile)
Motherboard	MSI B550 Gaming Plus
Cooling	Scythe Mugen (rev 5)
Memory	2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s)	AMD Radeon RX 7900 XT
Storage	Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s)	Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case	Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s)	Realtek ALC1150, Micca OriGen+
Power Supply	Enermax Platimax 850w
Mouse	Nixeus REVEL-X
Keyboard	Tesoro Excalibur
Software	Windows 10 Home 64-bit
Benchmark Scores	Faster than the tortoise; slower than the hare.

System Name	[Daily Driver]
Processor	[Ryzen 7 5800X3D]
Motherboard	[MSI MAG B550 TOMAHAWK]
Cooling	[be quiet! Dark Rock Slim]
Memory	[64GB Crucial Pro 3200MHz (32GBx2)]
Video Card(s)	[PNY RTX 3070Ti XLR8]
Storage	[1TB SN850 NVMe, 4TB 990 Pro NVMe, 2TB 870 EVO SSD, 2TB SA510 SSD]
Display(s)	[2x 27" HP X27q at 1440p]
Case	[Fractal Meshify-C]
Audio Device(s)	[Fanmusic TRUTHEAR IEM, HyperX Duocast]
Power Supply	[CORSAIR RMx 1000]
Mouse	[Logitech G Pro Wireless]
Keyboard	[Logitech G512 Carbon (GX-Brown)]
Software	[Windows 11 64-Bit]

System Name	BY-2021
Processor	AMD Ryzen 7 5800X (65w eco profile)
Motherboard	MSI B550 Gaming Plus
Cooling	Scythe Mugen (rev 5)
Memory	2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s)	AMD Radeon RX 7900 XT
Storage	Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s)	Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case	Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s)	Realtek ALC1150, Micca OriGen+
Power Supply	Enermax Platimax 850w
Mouse	Nixeus REVEL-X
Keyboard	Tesoro Excalibur
Software	Windows 10 Home 64-bit
Benchmark Scores	Faster than the tortoise; slower than the hare.

System Name	BY-2021
Processor	AMD Ryzen 7 5800X (65w eco profile)
Motherboard	MSI B550 Gaming Plus
Cooling	Scythe Mugen (rev 5)
Memory	2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s)	AMD Radeon RX 7900 XT
Storage	Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s)	Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case	Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s)	Realtek ALC1150, Micca OriGen+
Power Supply	Enermax Platimax 850w
Mouse	Nixeus REVEL-X
Keyboard	Tesoro Excalibur
Software	Windows 10 Home 64-bit
Benchmark Scores	Faster than the tortoise; slower than the hare.

Tropico 3/4/5 HPK Archiver

"I go fast!1!11!1!"

Attachments

"I go fast!1!11!1!"

"I go fast!1!11!1!"

"I go fast!1!11!1!"

New Member

"I go fast!1!11!1!"

New Member

"I go fast!1!11!1!"

New Member

"I go fast!1!11!1!"

New Member

"I go fast!1!11!1!"

New Member

"I go fast!1!11!1!"

New Member

New Member

"I go fast!1!11!1!"

"I go fast!1!11!1!"

New Member

"I go fast!1!11!1!"

New Member

"I go fast!1!11!1!"

"I go fast!1!11!1!"

New Member