I think there'll be an update, but mostly for Updating translations and adding new languages.
However, we are still trying to crack the game to allow modding.
I really hope that they'll at least give us a hand at modding, if not a complete mod kit. Simply making the resources a bit more accessible would be great.
I don't think an official DLC is likely. If we do manage to mod the game you'll probably have more than enough mods to play with though.
i'd be happy to assist with modding efforts (but please keep in mind that i've still got a full schedule of little inferno work to do so i apologize if my replies here are a little slower than i'd like them to be!)
the first task would be getting access to the game resources. the big binary blobs that are embedded in the resource section of the game EXE are custom package files that hold all the data for the game. each pak file starts with a header:
struct
{
i32 pakVersion;
i32 numItems;
};
followed by a table of contents which has numItems records - one for each resource contained in the pak:
struct
{
u32 id;
u32 flags;
i32 offset;
i32 size;
};
the offset and size tell where in the pak file the resource's data is. the only valid flag is 0x01 which indicates that the resource data is compressed using zlib. if a resource is compressed, then the resource data starts with a header with info about the compressed data:
struct
{
i32 compressedSizeBytes;
i32 uncompressedSizeBytes;
};
Well I have no idea how to work with binary data (yet), but it looks like enough for davidc or MyGod to start extracting resources with.
I understand that you've still got a full plate with porting and whatnot of Little Inferno so no problem with slow replies, I'm glad you replied at all. Thanks a bunch!
That's likely to be a big help. Thank you! Speaking of work, do you think there would ever be an Android port of the game? It sounds like it would just plain make sense to play this game via touch.
Sup. I spent some time hacking together a simple program to decompress all the files from the given resource blob. Judging from the data it spits out, it strips each file out correctly; but in the couple hours I've spent on this, I haven't got zlib decompressing correctly quite yet. Throwing the code and such here if anybody wants to pick it up and fly with it within the next few hours before I wake up again. I'll keep hacking and see what I come up with tomorrow. EDIT: See comments below for link
No comments or anything sane in the code; just threw it together for now. If you want to mess with the output files, use 7-zip to pull the resources out of the exe- I tried it on .rsrc\20480\1 and the stuff it spits out in out.txt looks right. Then run liDecompress.exe [file] and it'll spit out the individual resources into output\, with the filenames as the ID's of each resource.
Kinda tired atm.
@Allan: It'd also help to know what file format the resources and such are in; are the filenames packed in anyplace, or are they all just .png files? What little zlib testing I've done spits out some pretty large files in comparison, so I'm led to believe you guys didn't use standard .png's in here; more like some uncompressed format, tho I'm not sure what image type to expect, since not a lot of uncompressed image types have an alpha channel.
In any case, cheerio!
-Daxar
EDIT: Couldn't keep my mind off it; made a second attempt. EDIT: See comments below for link. Afaik, this one works... decompresses the output files, renames them .png for lack of anything better. Gonna try poking at the files a bit to see if I can figure out what they are and if I did everything correctly. File sizes and header data sizes agree, so that's looking promising.
Good job Daxar! I think maybe it is .cel file. Checking...
Well I have no idea again. By the way I found there is a System.IO.Compression.DeflateStream class in .NET Framework 4.5 that supports ZLib. Maybe I will try that...
EDIT:
i.imgur.com/Ky8---pa.png
(remove -s)
WAIT! Some of the flags are zero! Allan what does that mean?
By the way, davidc, I can't post this url correctly! Please fix.
EDIT:
I wrote a program and got a lot of errors and warnings.
Source code: http://pastebin.com/MvrSH5jv
Running results: http://pastebin.com/r1hdXE8w
Error details: (System.IO.InvalidDataException) Block length does not match with its complement.
Ok... I think there's something wrong with that. Allan could you give us more information?
if the compression flag isn't set in the table of contents then the resource data is already uncompressed and there is no compression header in front of the data. i think sounds are stored that way because they are vorbis data which wouldn't benefit from being zlib compressed.
there is a resource with id 0xB7E44207 which maps resource ids, which are hashes, back to the resource id strings that they were hashed from. the format of that resource is a header, followed by a variable number of mappings, followed by a string table. the header looks like this:
struct
{
BinHdrPtr maps;
BinHdrPtr stringTableBytes;
};
a BinHdrPtr is something used in a lot of resource headers and is used to describe a section of binary data that is variable length. each BinHdrPtr looks like this:
struct BinHdrPtr
{
i32 count;
i32 offset;
};
after the header comes the mappings. there are header.maps.count of them. each mapping looks like this:
struct
{
u32 resId;
i32 strId;
};
where resId is a resource id and strId is what you need to look up the string in the string table. finally, after the mappings, comes a string table which is another common element used in a lot of resources. a string table starts with a string table header:
struct
{
i32 numStrings;
i32 numPointers;
};
after the string table header comes the root list of string entries. there are header.numStrings of them. each entry looks like this:
struct
{
i32 pointerIndex;
i32 pointerCount;
};
after the list of string entries comes the list of string pointers. there are header.numPointers of them. each pointer looks like this:
struct
{
u32 languageId;
i32 offset;
};
finally after the pointers comes a big blob of null terminated, utf8 encoded strings.
so the process of looking up a resource's id string would be:
1. find the resource id in the mappings
2. use the strId field of the mapping to index into the root list of string table entries
3. use the pointerIndex field of the string table entry to index into the list of string pointers
4. use the offset field of the pointer as an offset into the utf8 string data
that should give you the resource id string for every resource in the game which in most cases is the original file name of the source data that our tools use to build the binary data. you should be able to get an idea of what each resource is for by looking at those file names and extensions (although the compiled binary data stored in the pak files is almost always processed into some other form.)
EDIT: i went to double check that resource 0xB7E44207 was actually included in the shipping version of the game and it looks like it might not be. i remember seeing someone post the list of id strings in another thread though so maybe it was accidentally included in one of the beta builds? not sure, but here is the debug pak file which contains the resource i'm talking about in this post:
Seriously, were you intentionally trying to make this game impossible to mod? Oh, rather than using a type of archive anyone in their right mind is familiar with, let's make our own form of archive. After all, if we totally reinvent the wheel, we're sure to get more compression than any of the other hundreds of archive types there are out there. And let's compile everything into the game's exe, because single-executable games are so common these days, and everyone expects it...
I mean, you guys knew that there's a giant community surrounding World of Goo and making mods for it, even with the encrypted XML. And you knew we'd want to mod this game too, to add new features and other cool stuff. Right now we're having to extract the game's exe (an irreversible process, so we obviously can't do anything so simple as change a single image) and create custom tools to even do something so simple as extract a single sound effect from the game. Did you intentionally make this difficult to mod? Why don't you add a password lock on the resources, to keep everyone out? I mean, I don't see how this would be any easier for you guys to work with, unless this is some sort of archive system that you've been using for a while anyway, and you had tons of tools to work with it to begin with. Are you planning on refactoring it all later to make it more mod-friendly? If so, why all the trouble to make a custom archive format now? Ok, so it's all 180 MB, which is quite small for any game, but I know I personally wouldn't mind a few hundred extra megabytes if it were easier to get into.
I'm a bit confused right now. Nevertheless, I guess I'll keep hacking on trying to get these resources out... :/
But thanks for sharing how to extract the stuff, anyway. That's what's keeping me sane right now...
Ok, ranting aside, thanks for the debug.pak. I managed to find these strings in it: http://pastebin.com/u8SP2TUV
If I can plug all this in right, I may be able to get back the original filenames.
It does seem to me that I'm still missing something crucial. If these are indeed the original filenames, and these files themselves are just compressed using zlib as-is and stuck into the resource blobs, there's something wrong; as PNG images shouldn't compress that much using zlib as they're lzma compressed already, and the headers for each resource item are spitting out a much higher uncompressed than compressed size which matches up with the file sizes I'm getting out. I think I'm doing this all correctly, but I'm not getting meaninful image data (or text data, or anything of that sort; the list above is the first meaningful thing I've seen so far) back out. Is this raw image data, that has a header of its own and raw pixel data following? because that's what it looks like to me. Enough repeating patterns in the data when I look at it with a hex editor to lead me to believe it's an image or such, but there's no standard image header for me to be able to open it with any image program I own. Any feedback on that? EDIT: Yah, you did say this. I didn't read closely enough. Nevermind.
Anyways, thanks. This'll keep me busy for a while.
EDIT: Or not, apparently.
the original file name of the source data that our tools use to build the binary data
basically implies that without said "tools" any modding would be a reverse-engineering nightmare. Unless we somehow managed to reverse engineer a similar set of tools, which would be both difficult and time-consuming. I like hacking stuff, but not quite that much. We could probably use Resource Hacker to patch changes back into the original exe, but getting the resources into packageable form would be difficult. Here's my latest version of my decompressor, which should work fine for extracting uncompressed files, as well. Until we get more info, though, I'm done. EDIT: See comments below for link.
Improved my program. It doesn't work well. I found the actual offset (the last offset) should be added with the offset of the beginning of the string table. Trying again...
P.S. I'm not sure if my program puts everything in the right place but I checked vdata/wordPackDict.dat and that file is correct. (Daxar you have seen it, I extracted this file from the memory and posted it onto the site) I wonder what format other files are using.
EDIT:
Daxar this program doesn't seem to use standard png or flac format so it should be good to pack them.
EDIT:
I think I will post what I got tomorrow. Now going to sleep...
Excellent job, Mygod. I was wondering where you got those words from.
Yeah, seems that they stripped or unpacked the PNG and FLAC's (And XML, and everything else...) somehow before packing them into the resource blobs, since I'm getting the same sort of thing back as well. I imagine they're using some custom format for each resource item. But it's cool to see everything extracted and in the right place with the right filenames, even if in the wrong format.
Awaiting more info from Allan on the format of these resources...
(Such as the compiled XML format, sound format, image format, etc)
EDIT: Oooh, I think I may have figured out the PNG's.
I noticed when hexediting the image files, they had repeating patterns, like rows of an image in raw data (hence why they compress so much, as I said earlier). So I did some rough calculations and here's what I got:
On the right you see me hexediting data/_common/MsNancyBigEyeRightClosed.png. On the left, you see my calculations. First is the size of the image, in bytes, that I got from the editor. Second is the first value you see in the editor, 0xAE, in decimal. Looks about right to be the dimension of the image (dunno if it's width or height). Third is 0xA7 in decimal, again about the right size to be the other dimension. Multiplying them together, you get 116232, exactly the size of the file (minus some header size). I'm not sure what the 1 is in there; probably another version number.
Should be enough to be able to reconstruct the image and use libpng or such to get it back into its original form.
Ok, time for class now.
EDIT2: Just realized that numbering starts at 0. Duh. Anyway, so this means the image file is really 116244 bytes, which means the header is indeed 3 4-byte values, aka 32-bit integers, and the first two are width and height. So our struct for the image header would look like:
struct
{
u32 width;
u32 height;
u32 version;
};
Or similar. Assuming little endian, ofc.
After that, should be fairly easy to reconstruct the image and figure out if it's row/col format or col/row format and which is the width and height visually. (and if it's bgra, argb, whatever format)
EDIT3: It's either bgra or rgba or such; alpha is the last value. It's FF in the middle of the image whenever the first three values are nonzero (Usually-- ofc there's antialiasing). For example, in this image, at 0xE4FC - 0xE4FF, the values are 95 95 95 FF, opaque grey.
Yeah, I know I said I was done hacking for a while, but it is quite a good deal more fun this way...
EDIT4: Yep, above data is correct. Image data is in BGRA format.
Its RAW (simple use IrfanView raw-plugin FORMATS.DLL)
with 32bpp, RGBA,12bytes header (4b width, 4b height, 4b someelse)
Thanks to MyGod and Allan for their work!
EDIT: Still some issues with my handling of the tables and such for loading filenames, so the filenames are jumbled all over the place, but my compressed-to-PNG-image converter is working great. One pic I found that davidc might like, and might be useful for this site (vdata/comboitempics.png): http://i48.tinypic.com/2cru53m.png
Only the combo items, but hey.
EDIT2: Got it all working. https://dl.dropbox.com/u/31816885/liDecompress.zip
I'll call this my 0.1 version; it should be quite usable. See the readme for details on extracting. Currently, it only extracts PNG images correctly, and a few of those end up botched (I think it's different formats or something for the data, haven't looked into it yet), but hey, it's a small price to pay to be able to extract all the images from the game. Source code is provided, though it may be difficult to compile since I'm using three third-party libraries: zlib, libpng, and ttvfs (Tiny Tree Virtual File System- a friend made it).
By the way, there's no UTF-8 support yet, so non-ASCII paths would fail horribly. I haven't seen any yet, so that may be a nonissue for now.
Takes a while (~4 mins on my comp) to extract everything, but it should be fairly robust. Give it a shot if you want to look at all the pretty images.
Oh, and the .png.normal images are standard PNG's as well. liDecompress extracts them correctly, as well.
Oh, and it's all 186 MB total extracted, Allan, so you aren't stripping out any space by having this stuff so hard to get at. *cough cough*
EDIT3: Hmm, seems that some (if not all) of the files without extension are also PNG images. For example, data/items/LittleInfernoBeta/colorbgicon
I'll have to add that as a test case. Later. For now, I'm calling it done until we get more info.
EDIT4: Seems to be the data/items/[item]/colorbgicon, coloritemicon, and greybgicon are images, as well. I made my program also test for those filenames, so they also should turn out as PNG images. Also I seem to keep hacking even after I say I'm done...
EDIT5: Using the table and data format Allan gave us to extract itself, resource 4 of debug.pak is the aforementioned magic resource 0xB7E44207, which extracts to vdata/residmap.dat. Now ya know. There are three other resources in there, too, files for some sort of debug font.
ok, first to address the data format ranting. i completely understand that from the point of view of modding the game, the data formats that we used are inconvenient and create a high barrier to entry. believe me, i get that. it would be really nice if the data were stored in a format that you could work with natively without having to spend time applying pointless transformations. it would be really nice if you didn't need any external dependencies so that the code wouldn't be "difficult to compile since I'm using three third-party libraries." well good news! these things were major design goals! but they were goals from the point of view of the game's runtime code, not from the point of view of modding the game. this is pretty standard stuff for console game development, so if you decide to continue with these modding efforts for little inferno you're going to run into a LOT more of this binary data to sift through. if you decide not to then that's up to you of course but i'll tell you i was really looking forward to seeing what you guys could come up with because i know there was some pretty cool stuff made for world of goo!
and to answer some questions. the third field in the header for textures is another 32 bits of flags. the first 2 are width and height as you already figured out. the reason that a few of your pngs might be coming out looking wrong is that there are a few textures in the game stored using a different pixel format (flag 0x08 should be set for these textures.) they are greyscale images that have a single 16bit color channel and a single 16bit alpha channel per pixel for a total of 32bits per pixel. the first byte of a pixel's data is the low 8 bits of alpha, the second byte is the high 8 bits of color, the third byte is the low 8 bits of color and the fourth byte is the high 8 bits of alpha.
also, if you modify some data, recompress it (optional) and put it back into a pak file, you won't have to also stuff the pak files back into the game exe in order for the game to use them (although you certainly can.) the game will look for external pak files called resource.pak (the 180MB one), frontend.pak (the 5MB one) and embed.pak (the 1MB one) in the same directory as the game exe. the way you get it to use those external files instead of the internal resources is to replace each of those internal resources with 4 bytes of 0s each. check out the win32 api function UpdateResource for an easy way to do that (and also to put the paks back into the exe if that's what you wanted to do.)
RAW2PNG converter injected. Thanks for your useful information. I will post a new one after I finish debugging.
NOW I NEED TO KNOW HOW TO OPEN THE FLAC FILES! I CAN'T WAIT TO LISTEM TO THEM!!!
Awesome Daxar! I'll be sure to check that out soon and fiddle around with all the "pure" images. It might let use make the site a bit better-looking too (instead of having to use screenshots).
@Allan: About external .pak files, that's awesome. That means that we just need to be able to extract the images properly, and from there on we can just make a program like GooTool (InfernoTool?) that does the 0s in the .exe and the .pak organisation. Either way it seems better than having to stuff everything back in the .exe with every mod.
Allan, here are my questions:
1. I didn't find any greyscale images. Could you give me an example?
2. How to open those FLAC files?
3. What does those .png.normal files do?
4. How to open those .xml files?
5. And, why are there two channels for greyscale?
Version 0.2 changelog:
- Added pullpakfiles and WinResource programs for pulling resources directly from the game's executable (so you don't have to mess with 7zip or ResourceHacker anymore)
- Correct handling of greyscale images
@Allan: Ok, that makes a lot more sense now. I reckon that would make porting the game to different platforms a heck of a lot easier, then. Just seemed like a step backward to me, since World of Goo ran basically on anything, though I reckon they put a lot of effort into porting as it was. And I reckon these aren't terribly terse formats or anything, they just require a bit of explanation. Anyways, thanks for mentioning UpdateResource- I hadn't looked into that sort of thing much, and it looks like it'll be quite useful for modding. External pakfiles ftw. Thanks for being willing to explain all this; keep explaining and we'll keep hacking together modding tools. :)
Just for your next game could you please use the same format for everything, so we don't have to hack together a whole new set of tools? Pretty please? ;)
EDIT: I'm attempting to patch the exe to use external .pak files. What's the behavior if these files are missing (i.e. how do I know if I'm patching it correctly, or do I have to modify the external .pak files in order to see any changes)?
@Mygod:
1. As far as I know, there's only seven:
data/animations/Intro/cityFar.png
data/animations/Intro/cityNear.png
data/animations/Intro/glow.png
data/animations/IntroEdges/edges.PNG
data/animations/_common/backdrop.PNG
data/cutscenes/Jingle/tcLogoBg.png
data/animations/ScreenBG/bg.png
Most of these are in frontend.pak ("3", as we've known it up till now)
3. The .png.normal files are standard PNG files used as normal maps for the rendering engine. See http://en.wikipedia.org/wiki/Normal_mapping for a description of what they're for.
By the way, Mygod, thanks for working on your program while I'm working on mine; keeps me from being lazy when you start doing cool stuff. :)
2. the flac files aren't really flac data at all (similar to the pngs) although they originally started off that way. the actual data they contain is mostly a raw vorbis data stream (like the raw pixel data in the textures) but it's not in an ogg wrapper so it's not really a .ogg file either. i can probably provide a tool to convert between a real ogg file and the sound resources the game uses.
3. as Daxar mentioned they are normal maps that the renderer uses to light the items when there is fire nearby. if you get a pirate doll in the fireplace and create a flame with your mouse right next to him, it's the data in the normal maps that allows the surfaces of the pirate to light up in the direction of the fire and give it a pseudo 3d look.
4. the various xml files represent a few different kinds of resources which all have their specific data formats. animations (.anim.xml) and particle effects (.effect.xml) are probably the main ones you might see in there.
5. there are 2 channels but only 1 is for color. these images are used sparingly in a few places where we needed a really dark gradient. if you imagine a full screen gradient that goes from RGB(5,5,5) on the left side of the screen to RGB(9,9,9) on the right, there are only 5 different intensity levels to play with in between and so depending on the brightness and contrast of the display you're using sometimes that can result in really visible bands across the screen. using this fake 16bit format gives us a larger dynamic range to use and much better results, especially on tvs for the wiiu version of the game.
to know if you're patching the exe correctly to look for pak files externally, first patch out the 3 paks inside the exe by updating their resources to be just 4 bytes of 0s each. the game exe should be around 3.7 MB once that's done. now, try running the game without any of the external paks present in its directory. if it pops up a message that it could not open the resource pak file 'whatever.pak' then you know that it's trying to load the paks from the external source. after that, copy the paks with the proper names into the same directory as the exe and the game should run normally but now it's definitely using those external paks.
Daxar can I just say that program is fantastically awesome. All the .pngs, as well as the .xmls and .flacs (even though they're currently unusable) makes me feel like I'm looking at WoG's resources again. I totally can't wait to look at what sort of stuff we can do with the .xmls!
Talking about WoG, I've noticed that some of the gear images (e.g. data/_common/gear1.png) are practically exactly the same as those from WoG: loving it.
Looking forward to progress! I really wish I could help!
Hmm, ok. I figured I was probably patching it wrong. UpdateResource() wasn't behaving properly for me; time to fiddle around with it some more.
EDIT: It's looking like the language ID is hanging me up. FindResourceEx() doesn't appear to work either, while FindResource(), being language-neutral, does. I'm guessing these are language-neutral files, which the UpdateResource() documentation is saying in a terse way that you can update so long as there is no language-specific version of the file, which appears to not be working for me. (Perhaps only an issue since I'm running Win7)
Source code of removeresc program: http://pastebin.com/jd0DAnMS
When run on the Little Inferno executable, it creates the proper resources, which EnumResourceTypes() and EnumResourceNames() show as correct and the right size, and my .pak extraction program pulls out as 4-byte, 0-filled files like it works, but the executable size remains the same and the executable still runs as normal. I feel like it should be an annoying, stupid error on my part but I just can't see it.
have you tried MAKELANGID(LANG_ENGLISH,SUBLANG_ENGLISH_US) ? that's what we use when we build the big final exe (although language-neutral probably would have been more correct)
Now the default resource-extraction program strips the game's executable so that it runs off the .pak files in the same directory (The old program is still there, as well). Just need to add a .pak compression program and the game will actually be moddable! Woot!
Of course, this version in particular should be run off a copy of your game, so make a backup before stripping.
1. Okay thanks. I found them. I just realized I entered the incorrect value. Now my program works well for them too.
2. Instead of providing tools, could you tell us how to do it so we can convert them to ogg or something without running an outer program?
3 & 5. Get it. Thanks.
4. And fonts(.font.xml) too. How can we modify them?
EDIT: Now I'm going to try to extract .pak files...
@Daxar
I made my program because yours doesn't work for me. (older versions, haven't tried new version)
@Allan: I wonder how we can get the console mode?
By the argument to run exe or add debug option in settings.txt or using the keyboard shortcuts?
Especially needed option '[ff]timescale' and 'freecam'
@Allan
Does #11 works for all .dat files? (I think the file structure is same but the meanings are different)
I think I can convert them to XML files so that we can modify the text in the game.
EDIT: Like this one: http://pastebin.com/b16n2hf1
My program only works for residmap.dat, so I guess other .dat files are using a bit different format.
for sounds, yes i can provide some source code instead to show the process of converting between the game's sound data and ogg files. i'll have to spend a little time figuring out exactly how to do that first though : )
font data is yet another resource format that will need to be documented along with the ones i mentioned earlier. i think the most useful data to be able to edit first though might be the item data since it's pretty straight forward to change some simple values and get real changes in game. i figured we could dive into that once repaking is working.
access to the in game console is completely disabled in the shipping version of the game so there's no way to bring it up. it needs the debug font to render anyway and the debug font isn't included in the shipping version of the game either. how did you know that a console even existed? : )
any files that end in .dat are just generic binary data files with a .dat extension because i needed to make up some extension to give them. they all share common building blocks (like the BinHdrPtr structure and the string table structure that i described before) but don't have a single unified format that could be used to automatically convert them to xml or something like that. their data is all formatted to be used directly by the code that works with it in the game with minimal processing once it has been copied into memory from the pak files. you can think of all of these resources as in-memory versions of the game's data which has been serialized out to disk for transport.
My version Little Inferno 1.0
included
DebugFont.page00.png
DebugFont.page01.png
DebugFont.font.xml
and some commands with description
(for example: 'clear - instantly delete all items in fireplace'
and console view:
Little Inferno Version: 1.0 Build Date:... Build Time: ... Command Line: ...
EGG>
command '...' not recognized. type '?' to see a list of commands.)
it means that in my version the game console isn't disabled
and again I ask you to reply to the question - how i can get the console mode?
please give an answer to this vital question... thanks!
Speaking about fonts, I'll help adding Cyrillic letters. I've already emailed Kyle about them.
Allan, could you tell me names of fonts you used in the game. I'll make Russian fonts.
The point is that Cyrillic letters don't exist in Tw Century whatever font, so I have to draw them by hand. This is not that difficult, if someone could then pack them, that would be great.
Here's a link to some fonts that I (and Vladislav) created with respective mappings.
@Vials What is Zog forums?
EDIT: Got it. Ты говоришь по русски
there's no way to access the console in shipping versions of the game. honest! not all traces of every single thing related to the console are removed from the code of course, so the strings for the commands are probably all still in the exe if you look for them, but the ability to actually bring up the console is compiled out of the shipping code (as is the entire function that renders the console so you wouldn't be able to see it even if you were able to activate it.)
anything related to adding cyrillic characters to the font for a russian language version of the game is probably best to discuss over email to contact@tomorrowcorporation.com. if this would actually be for the shipping version of the game then we'd probably need to sort through legal issues that i don't know much about so emailing with all of us is probably a better place to have that discussion. (by the way, i was very happy to discover that although i was not a very good student, my 5 years of studying russian in high school did at least prepare me to be able to understand "Ты говоришь по русски" without having to run it through google translator!)
as for adding new files to the paks, it doesn't really matter where they go. the wiiu version of the game loads the different paks at different times and so it matters what goes where for that, but the pc version of the game is able to load any resource from any pak at any time.
i was trying to come up with a simple candidate resource to try editing first and i think the dictionary for the word pack item might be a good one. it has a really similar structure to the vdata/residmap.dat resource that i laid out before. the dictionary for that item is the resource vdata\wordPackDict.dat. it starts with a header
struct
{
BinHdrPtr words;
BinHdrPtr stringTableBytes;
};
which says the resource contains a section for a list of words and a section for a string table with the actual text. the list of words is one of the following structs for each word:
struct
{
i32 wordStrId;
f32 probability;
};
where wordStrId is the key for looking up the text in the string table and probability is a float in the range 0.0 to 1.0 that gives the probability that the word will be selected when you buy the word pack item. the total probabilities for all words should add up to 1.0. after that list of words there is a string table with the text for all the words in the same string table format that i described before.
any specific requests for resource types to document next? or specific things about the game that you want to change as the first steps toward modding and i could tell you which resources would be involved? i'm going to work on the sounds but they might take a little longer since the process for those is much less straight forward.
Well, anyway I think we'd better make everything in the right pak files since Wii U version matters. So I want to know that anyway.
I think we can generate an index.xml that contains everything that is needed to make pak files and add .png extension for all recognizable pictures whatever it has the extension or not. What do you think, @Daxar?
index.xml example: http://pastebin.com/ndfrgG5U
How to test if a file is a picture fast:
1. Length >= 12
2. Length == Width*Height << 2 + 12
I think simply the decryption of XML files in general would be a huge help, we can probably figure out what most of them do and how they work by looking at the XML itself (as we did with WoG).
I'm curious though, where is the data for the letters and combos held? Also in XML files or elsewhere?
@Mygod: I don't think that's the error you're supposed to get when running the game. That's what I got when I stripped ALL of the resources from the game, rather than just the .pak files. It should say something about being unable to open a .pak file rather than saying that an unhandled exception occurred.
@Allan: Knowing the XML file format next would be pretty tops.
Version 0.3 changelog:
- Added liCompress for compressing everything back into .pak files
- For now, all files are compressed by default, regardless of flac/png etc
- Added util/repack.exe program for repacking .pak files back into the game's exe
- Various bugfixes
To install, do the following:
- first, back everything up. I haven't extensively tested everything, and something might break.
- next, run strip.exe on your Little Inferno executable to pull all the .pak files out, if you haven't already
- next, back up your frontend.pak file
- next, run liDecompress on frontend.pak (You'll need to do this even if you have already, just because my decompress/compress code has changed since last version)
- next, run liDecompress chimneyChanger.pak. This will overwrite the chimney image files for the main menu, hence why you're backing up the frontend.pak first
- next, run liCompress on frontend.pak
You should be good to go. Let me know if you have any trouble.
Finally, you've got first mod evah for Little Inferno! I've tested the mod, it's working and I'm so excited!
There's a lot of work ahead: writing documentation on how it works, adding mod pages on the site, DIscover everything! The gameplay possibilities will be endless! Call davidc here, quick!
Didn't work for me (just got the normal chimneys), but it's quite possible that it's because I'm running a version I copied from Steam, so it might be running the Steam version (since the overlay comes up). I'll download the non-Steam version and try it again.
I'm getting lots of ideas for a GUI for all of this though, to make it all a bit cleaner and user-friendly. I'd volunteer to make one if I wasn't already so busy.
Catalog
Combos
Forum
Get the Game
News
I think there'll be an update, but mostly for Updating translations and adding new languages.
However, we are still trying to crack the game to allow modding.
I really hope that they'll at least give us a hand at modding, if not a complete mod kit. Simply making the resources a bit more accessible would be great.
I don't think an official DLC is likely. If we do manage to mod the game you'll probably have more than enough mods to play with though.
i'd be happy to assist with modding efforts (but please keep in mind that i've still got a full schedule of little inferno work to do so i apologize if my replies here are a little slower than i'd like them to be!)
the first task would be getting access to the game resources. the big binary blobs that are embedded in the resource section of the game EXE are custom package files that hold all the data for the game. each pak file starts with a header:
struct
{
i32 pakVersion;
i32 numItems;
};
followed by a table of contents which has numItems records - one for each resource contained in the pak:
struct
{
u32 id;
u32 flags;
i32 offset;
i32 size;
};
the offset and size tell where in the pak file the resource's data is. the only valid flag is 0x01 which indicates that the resource data is compressed using zlib. if a resource is compressed, then the resource data starts with a header with info about the compressed data:
struct
{
i32 compressedSizeBytes;
i32 uncompressedSizeBytes;
};
and the compressed data would immediately follow.
hope that's helpful to get started with!
Well I have no idea how to work with binary data (yet), but it looks like enough for davidc or MyGod to start extracting resources with.
I understand that you've still got a full plate with porting and whatnot of Little Inferno so no problem with slow replies, I'm glad you replied at all. Thanks a bunch!
That's likely to be a big help. Thank you! Speaking of work, do you think there would ever be an Android port of the game? It sounds like it would just plain make sense to play this game via touch.
I dont mod, I was just hoping for someone to mod. Mygod should be able to get started with that though, Thanks for the help!
Sup. I spent some time hacking together a simple program to decompress all the files from the given resource blob. Judging from the data it spits out, it strips each file out correctly; but in the couple hours I've spent on this, I haven't got zlib decompressing correctly quite yet. Throwing the code and such here if anybody wants to pick it up and fly with it within the next few hours before I wake up again. I'll keep hacking and see what I come up with tomorrow. EDIT: See comments below for link
No comments or anything sane in the code; just threw it together for now. If you want to mess with the output files, use 7-zip to pull the resources out of the exe- I tried it on .rsrc\20480\1 and the stuff it spits out in out.txt looks right. Then run liDecompress.exe [file] and it'll spit out the individual resources into output\, with the filenames as the ID's of each resource.
Kinda tired atm.
@Allan: It'd also help to know what file format the resources and such are in; are the filenames packed in anyplace, or are they all just .png files? What little zlib testing I've done spits out some pretty large files in comparison, so I'm led to believe you guys didn't use standard .png's in here; more like some uncompressed format, tho I'm not sure what image type to expect, since not a lot of uncompressed image types have an alpha channel.
In any case, cheerio!
-Daxar
EDIT: Couldn't keep my mind off it; made a second attempt. EDIT: See comments below for link. Afaik, this one works... decompresses the output files, renames them .png for lack of anything better. Gonna try poking at the files a bit to see if I can figure out what they are and if I did everything correctly. File sizes and header data sizes agree, so that's looking promising.
Oh wow, sounds good. I might check out that program later on.
It's times like this I wish I knew a more advanced language like C++.
Good job Daxar! I think maybe it is .cel file. Checking...
Well I have no idea again. By the way I found there is a System.IO.Compression.DeflateStream class in .NET Framework 4.5 that supports ZLib. Maybe I will try that...
EDIT:
i.imgur.com/Ky8---pa.png
(remove -s)
WAIT! Some of the flags are zero! Allan what does that mean?
By the way, davidc, I can't post this url correctly! Please fix.
EDIT:
I wrote a program and got a lot of errors and warnings.
Source code: http://pastebin.com/MvrSH5jv
Running results: http://pastebin.com/r1hdXE8w
Error details: (System.IO.InvalidDataException) Block length does not match with its complement.
Ok... I think there's something wrong with that. Allan could you give us more information?
if the compression flag isn't set in the table of contents then the resource data is already uncompressed and there is no compression header in front of the data. i think sounds are stored that way because they are vorbis data which wouldn't benefit from being zlib compressed.
there is a resource with id 0xB7E44207 which maps resource ids, which are hashes, back to the resource id strings that they were hashed from. the format of that resource is a header, followed by a variable number of mappings, followed by a string table. the header looks like this:
struct
{
BinHdrPtr maps;
BinHdrPtr stringTableBytes;
};
a BinHdrPtr is something used in a lot of resource headers and is used to describe a section of binary data that is variable length. each BinHdrPtr looks like this:
struct BinHdrPtr
{
i32 count;
i32 offset;
};
after the header comes the mappings. there are header.maps.count of them. each mapping looks like this:
struct
{
u32 resId;
i32 strId;
};
where resId is a resource id and strId is what you need to look up the string in the string table. finally, after the mappings, comes a string table which is another common element used in a lot of resources. a string table starts with a string table header:
struct
{
i32 numStrings;
i32 numPointers;
};
after the string table header comes the root list of string entries. there are header.numStrings of them. each entry looks like this:
struct
{
i32 pointerIndex;
i32 pointerCount;
};
after the list of string entries comes the list of string pointers. there are header.numPointers of them. each pointer looks like this:
struct
{
u32 languageId;
i32 offset;
};
finally after the pointers comes a big blob of null terminated, utf8 encoded strings.
so the process of looking up a resource's id string would be:
1. find the resource id in the mappings
2. use the strId field of the mapping to index into the root list of string table entries
3. use the pointerIndex field of the string table entry to index into the list of string pointers
4. use the offset field of the pointer as an offset into the utf8 string data
that should give you the resource id string for every resource in the game which in most cases is the original file name of the source data that our tools use to build the binary data. you should be able to get an idea of what each resource is for by looking at those file names and extensions (although the compiled binary data stored in the pak files is almost always processed into some other form.)
EDIT: i went to double check that resource 0xB7E44207 was actually included in the shipping version of the game and it looks like it might not be. i remember seeing someone post the list of id strings in another thread though so maybe it was accidentally included in one of the beta builds? not sure, but here is the debug pak file which contains the resource i'm talking about in this post:
https://dl.dropbox.com/s/4c1otxd31t08nwm/debug.pak
...
Seriously, were you intentionally trying to make this game impossible to mod? Oh, rather than using a type of archive anyone in their right mind is familiar with, let's make our own form of archive. After all, if we totally reinvent the wheel, we're sure to get more compression than any of the other hundreds of archive types there are out there. And let's compile everything into the game's exe, because single-executable games are so common these days, and everyone expects it...
I mean, you guys knew that there's a giant community surrounding World of Goo and making mods for it, even with the encrypted XML. And you knew we'd want to mod this game too, to add new features and other cool stuff. Right now we're having to extract the game's exe (an irreversible process, so we obviously can't do anything so simple as change a single image) and create custom tools to even do something so simple as extract a single sound effect from the game. Did you intentionally make this difficult to mod? Why don't you add a password lock on the resources, to keep everyone out? I mean, I don't see how this would be any easier for you guys to work with, unless this is some sort of archive system that you've been using for a while anyway, and you had tons of tools to work with it to begin with. Are you planning on refactoring it all later to make it more mod-friendly? If so, why all the trouble to make a custom archive format now? Ok, so it's all 180 MB, which is quite small for any game, but I know I personally wouldn't mind a few hundred extra megabytes if it were easier to get into.
I'm a bit confused right now. Nevertheless, I guess I'll keep hacking on trying to get these resources out... :/
But thanks for sharing how to extract the stuff, anyway. That's what's keeping me sane right now...
Ok, ranting aside, thanks for the debug.pak. I managed to find these strings in it:
http://pastebin.com/u8SP2TUV
If I can plug all this in right, I may be able to get back the original filenames.
It does seem to me that I'm still missing something crucial. If these are indeed the original filenames, and these files themselves are just compressed using zlib as-is and stuck into the resource blobs, there's something wrong; as PNG images shouldn't compress that much using zlib as they're lzma compressed already, and the headers for each resource item are spitting out a much higher uncompressed than compressed size which matches up with the file sizes I'm getting out. I think I'm doing this all correctly, but I'm not getting meaninful image data (or text data, or anything of that sort; the list above is the first meaningful thing I've seen so far) back out. Is this raw image data, that has a header of its own and raw pixel data following? because that's what it looks like to me. Enough repeating patterns in the data when I look at it with a hex editor to lead me to believe it's an image or such, but there's no standard image header for me to be able to open it with any image program I own. Any feedback on that?EDIT: Yah, you did say this. I didn't read closely enough. Nevermind.Anyways, thanks. This'll keep me busy for a while.
EDIT: Or not, apparently.
basically implies that without said "tools" any modding would be a reverse-engineering nightmare. Unless we somehow managed to reverse engineer a similar set of tools, which would be both difficult and time-consuming. I like hacking stuff, but not quite that much. We could probably use Resource Hacker to patch changes back into the original exe, but getting the resources into packageable form would be difficult. Here's my latest version of my decompressor, which should work fine for extracting uncompressed files, as well. Until we get more info, though, I'm done. EDIT: See comments below for link.
Improved my program. It doesn't work well. I found the actual offset (the last offset) should be added with the offset of the beginning of the string table. Trying again...
EDIT:
YAY!!!!!!!!!!!!!!!! Made it! http://i.imgur.com/IR1SM.png
EDIT:
http://i.imgur.com/JeSnv.png
Unable to play! Argh!
EDIT:
Anyway I'm going to post my working program!
Download: https://dl.dropbox.com/u/94637484/Little%20Inferno%20Resource%20Unpacker...
Source: http://pastebin.com/3KaRXx2B
How to use: Install .NET Framework 4.5 before using this program. Drag these files onto the program at the same time to start unpacking: 1, 2, 3, debug.pak.
P.S. I'm not sure if my program puts everything in the right place but I checked vdata/wordPackDict.dat and that file is correct. (Daxar you have seen it, I extracted this file from the memory and posted it onto the site) I wonder what format other files are using.
EDIT:
Daxar this program doesn't seem to use standard png or flac format so it should be good to pack them.
EDIT:
I think I will post what I got tomorrow. Now going to sleep...
Excellent job, Mygod. I was wondering where you got those words from.
Yeah, seems that they stripped or unpacked the PNG and FLAC's (And XML, and everything else...) somehow before packing them into the resource blobs, since I'm getting the same sort of thing back as well. I imagine they're using some custom format for each resource item. But it's cool to see everything extracted and in the right place with the right filenames, even if in the wrong format.
Awaiting more info from Allan on the format of these resources...
(Such as the compiled XML format, sound format, image format, etc)
EDIT: Oooh, I think I may have figured out the PNG's.
I noticed when hexediting the image files, they had repeating patterns, like rows of an image in raw data (hence why they compress so much, as I said earlier). So I did some rough calculations and here's what I got:
http://i45.tinypic.com/3517gvo.png
On the right you see me hexediting data/_common/MsNancyBigEyeRightClosed.png. On the left, you see my calculations. First is the size of the image, in bytes, that I got from the editor. Second is the first value you see in the editor, 0xAE, in decimal. Looks about right to be the dimension of the image (dunno if it's width or height). Third is 0xA7 in decimal, again about the right size to be the other dimension. Multiplying them together, you get 116232, exactly the size of the file (minus some header size). I'm not sure what the 1 is in there; probably another version number.
Should be enough to be able to reconstruct the image and use libpng or such to get it back into its original form.
Ok, time for class now.
EDIT2: Just realized that numbering starts at 0. Duh. Anyway, so this means the image file is really 116244 bytes, which means the header is indeed 3 4-byte values, aka 32-bit integers, and the first two are width and height. So our struct for the image header would look like:
struct{
u32 width;
u32 height;
u32 version;
};
Or similar. Assuming little endian, ofc.
After that, should be fairly easy to reconstruct the image and figure out if it's row/col format or col/row format and which is the width and height visually. (and if it's bgra, argb, whatever format)
EDIT3: It's either bgra or rgba or such; alpha is the last value. It's FF in the middle of the image whenever the first three values are nonzero (Usually-- ofc there's antialiasing). For example, in this image, at 0xE4FC - 0xE4FF, the values are 95 95 95 FF, opaque grey.
Yeah, I know I said I was done hacking for a while, but it is quite a good deal more fun this way...
EDIT4: Yep, above data is correct. Image data is in BGRA format.
Oh sweet. Yeah, just having the resource tree is already a good step forward. This is getting exciting!
Its RAW (simple use IrfanView raw-plugin FORMATS.DLL)
with 32bpp, RGBA,12bytes header (4b width, 4b height, 4b someelse)
Thanks to MyGod and Allan for their work!
http://i49.tinypic.com/168hs0x.png
Wat? My work doesn't count? *wink wink*
I'm getting close to having this all work...
http://i49.tinypic.com/10f95ox.png
EDIT: Still some issues with my handling of the tables and such for loading filenames, so the filenames are jumbled all over the place, but my compressed-to-PNG-image converter is working great. One pic I found that davidc might like, and might be useful for this site (vdata/comboitempics.png):
http://i48.tinypic.com/2cru53m.png
Only the combo items, but hey.
EDIT2: Got it all working. https://dl.dropbox.com/u/31816885/liDecompress.zip
I'll call this my 0.1 version; it should be quite usable. See the readme for details on extracting. Currently, it only extracts PNG images correctly, and a few of those end up botched (I think it's different formats or something for the data, haven't looked into it yet), but hey, it's a small price to pay to be able to extract all the images from the game. Source code is provided, though it may be difficult to compile since I'm using three third-party libraries: zlib, libpng, and ttvfs (Tiny Tree Virtual File System- a friend made it).
By the way, there's no UTF-8 support yet, so non-ASCII paths would fail horribly. I haven't seen any yet, so that may be a nonissue for now.
Takes a while (~4 mins on my comp) to extract everything, but it should be fairly robust. Give it a shot if you want to look at all the pretty images.
Oh, and the .png.normal images are standard PNG's as well. liDecompress extracts them correctly, as well.
Oh, and it's all 186 MB total extracted, Allan, so you aren't stripping out any space by having this stuff so hard to get at. *cough cough*
EDIT3: Hmm, seems that some (if not all) of the files without extension are also PNG images. For example, data/items/LittleInfernoBeta/colorbgicon
I'll have to add that as a test case. Later. For now, I'm calling it done until we get more info.
EDIT4: Seems to be the data/items/[item]/colorbgicon, coloritemicon, and greybgicon are images, as well. I made my program also test for those filenames, so they also should turn out as PNG images. Also I seem to keep hacking even after I say I'm done...
EDIT5: Using the table and data format Allan gave us to extract itself, resource 4 of debug.pak is the aforementioned magic resource 0xB7E44207, which extracts to vdata/residmap.dat. Now ya know. There are three other resources in there, too, files for some sort of debug font.
ok, first to address the data format ranting. i completely understand that from the point of view of modding the game, the data formats that we used are inconvenient and create a high barrier to entry. believe me, i get that. it would be really nice if the data were stored in a format that you could work with natively without having to spend time applying pointless transformations. it would be really nice if you didn't need any external dependencies so that the code wouldn't be "difficult to compile since I'm using three third-party libraries." well good news! these things were major design goals! but they were goals from the point of view of the game's runtime code, not from the point of view of modding the game. this is pretty standard stuff for console game development, so if you decide to continue with these modding efforts for little inferno you're going to run into a LOT more of this binary data to sift through. if you decide not to then that's up to you of course but i'll tell you i was really looking forward to seeing what you guys could come up with because i know there was some pretty cool stuff made for world of goo!
and to answer some questions. the third field in the header for textures is another 32 bits of flags. the first 2 are width and height as you already figured out. the reason that a few of your pngs might be coming out looking wrong is that there are a few textures in the game stored using a different pixel format (flag 0x08 should be set for these textures.) they are greyscale images that have a single 16bit color channel and a single 16bit alpha channel per pixel for a total of 32bits per pixel. the first byte of a pixel's data is the low 8 bits of alpha, the second byte is the high 8 bits of color, the third byte is the low 8 bits of color and the fourth byte is the high 8 bits of alpha.
also, if you modify some data, recompress it (optional) and put it back into a pak file, you won't have to also stuff the pak files back into the game exe in order for the game to use them (although you certainly can.) the game will look for external pak files called resource.pak (the 180MB one), frontend.pak (the 5MB one) and embed.pak (the 1MB one) in the same directory as the game exe. the way you get it to use those external files instead of the internal resources is to replace each of those internal resources with 4 bytes of 0s each. check out the win32 api function UpdateResource for an easy way to do that (and also to put the paks back into the exe if that's what you wanted to do.)
RAW2PNG converter injected. Thanks for your useful information. I will post a new one after I finish debugging.
NOW I NEED TO KNOW HOW TO OPEN THE FLAC FILES! I CAN'T WAIT TO LISTEM TO THEM!!!
Awesome Daxar! I'll be sure to check that out soon and fiddle around with all the "pure" images. It might let use make the site a bit better-looking too (instead of having to use screenshots).
@Allan: About external .pak files, that's awesome. That means that we just need to be able to extract the images properly, and from there on we can just make a program like GooTool (InfernoTool?) that does the 0s in the .exe and the .pak organisation. Either way it seems better than having to stuff everything back in the .exe with every mod.
V1.0.2.0
Source: http://pastebin.com/6ggipk
Download: https://dl.dropbox.com/u/94637484/Little%20Inferno%20Resource%20Unpacker...
How to use: Same as above.
What's new: Convert all images to PNG automatically. (and keep the original RAW file too)
Allan, here are my questions:
1. I didn't find any greyscale images. Could you give me an example?
2. How to open those FLAC files?
3. What does those .png.normal files do?
4. How to open those .xml files?
5. And, why are there two channels for greyscale?
Version 0.2 changelog:
- Added pullpakfiles and WinResource programs for pulling resources directly from the game's executable (so you don't have to mess with 7zip or ResourceHacker anymore)
- Correct handling of greyscale images
Download link: https://dl.dropbox.com/u/31816885/liDecompress0.2.zip
Source: https://dl.dropbox.com/u/31816885/liDecompress0.2-src.zip
@Allan: Ok, that makes a lot more sense now. I reckon that would make porting the game to different platforms a heck of a lot easier, then. Just seemed like a step backward to me, since World of Goo ran basically on anything, though I reckon they put a lot of effort into porting as it was. And I reckon these aren't terribly terse formats or anything, they just require a bit of explanation. Anyways, thanks for mentioning UpdateResource- I hadn't looked into that sort of thing much, and it looks like it'll be quite useful for modding. External pakfiles ftw. Thanks for being willing to explain all this; keep explaining and we'll keep hacking together modding tools. :)
Just for your next game could you please use the same format for everything, so we don't have to hack together a whole new set of tools? Pretty please? ;)
EDIT: I'm attempting to patch the exe to use external .pak files. What's the behavior if these files are missing (i.e. how do I know if I'm patching it correctly, or do I have to modify the external .pak files in order to see any changes)?
@Mygod:
1. As far as I know, there's only seven:
data/animations/Intro/cityFar.png
data/animations/Intro/cityNear.png
data/animations/Intro/glow.png
data/animations/IntroEdges/edges.PNG
data/animations/_common/backdrop.PNG
data/cutscenes/Jingle/tcLogoBg.png
data/animations/ScreenBG/bg.png
Most of these are in frontend.pak ("3", as we've known it up till now)
3. The .png.normal files are standard PNG files used as normal maps for the rendering engine. See http://en.wikipedia.org/wiki/Normal_mapping for a description of what they're for.
By the way, Mygod, thanks for working on your program while I'm working on mine; keeps me from being lazy when you start doing cool stuff. :)
2. the flac files aren't really flac data at all (similar to the pngs) although they originally started off that way. the actual data they contain is mostly a raw vorbis data stream (like the raw pixel data in the textures) but it's not in an ogg wrapper so it's not really a .ogg file either. i can probably provide a tool to convert between a real ogg file and the sound resources the game uses.
3. as Daxar mentioned they are normal maps that the renderer uses to light the items when there is fire nearby. if you get a pirate doll in the fireplace and create a flame with your mouse right next to him, it's the data in the normal maps that allows the surfaces of the pirate to light up in the direction of the fire and give it a pseudo 3d look.
4. the various xml files represent a few different kinds of resources which all have their specific data formats. animations (.anim.xml) and particle effects (.effect.xml) are probably the main ones you might see in there.
5. there are 2 channels but only 1 is for color. these images are used sparingly in a few places where we needed a really dark gradient. if you imagine a full screen gradient that goes from RGB(5,5,5) on the left side of the screen to RGB(9,9,9) on the right, there are only 5 different intensity levels to play with in between and so depending on the brightness and contrast of the display you're using sometimes that can result in really visible bands across the screen. using this fake 16bit format gives us a larger dynamic range to use and much better results, especially on tvs for the wiiu version of the game.
to know if you're patching the exe correctly to look for pak files externally, first patch out the 3 paks inside the exe by updating their resources to be just 4 bytes of 0s each. the game exe should be around 3.7 MB once that's done. now, try running the game without any of the external paks present in its directory. if it pops up a message that it could not open the resource pak file 'whatever.pak' then you know that it's trying to load the paks from the external source. after that, copy the paks with the proper names into the same directory as the exe and the game should run normally but now it's definitely using those external paks.
Daxar can I just say that program is fantastically awesome. All the .pngs, as well as the .xmls and .flacs (even though they're currently unusable) makes me feel like I'm looking at WoG's resources again. I totally can't wait to look at what sort of stuff we can do with the .xmls!
Talking about WoG, I've noticed that some of the gear images (e.g. data/_common/gear1.png) are practically exactly the same as those from WoG: loving it.
Looking forward to progress! I really wish I could help!
Hmm, ok. I figured I was probably patching it wrong. UpdateResource() wasn't behaving properly for me; time to fiddle around with it some more.
EDIT: It's looking like the language ID is hanging me up. FindResourceEx() doesn't appear to work either, while FindResource(), being language-neutral, does. I'm guessing these are language-neutral files, which the UpdateResource() documentation is saying in a terse way that you can update so long as there is no language-specific version of the file, which appears to not be working for me. (Perhaps only an issue since I'm running Win7)
Source code of removeresc program: http://pastebin.com/jd0DAnMS
When run on the Little Inferno executable, it creates the proper resources, which EnumResourceTypes() and EnumResourceNames() show as correct and the right size, and my .pak extraction program pulls out as 4-byte, 0-filled files like it works, but the executable size remains the same and the executable still runs as normal. I feel like it should be an annoying, stupid error on my part but I just can't see it.
have you tried MAKELANGID(LANG_ENGLISH,SUBLANG_ENGLISH_US) ? that's what we use when we build the big final exe (although language-neutral probably would have been more correct)
Well, derp. Apparently that's what I was missing. I knew it would be something obvious. Thanks!
EDIT: Minor version 0.2.1
Download: https://dl.dropbox.com/u/31816885/liDecompress0.2.1.zip
Source: https://dl.dropbox.com/u/31816885/liDecompress0.2.1-src.zip
Now the default resource-extraction program strips the game's executable so that it runs off the .pak files in the same directory (The old program is still there, as well). Just need to add a .pak compression program and the game will actually be moddable! Woot!
Of course, this version in particular should be run off a copy of your game, so make a backup before stripping.
1. Okay thanks. I found them. I just realized I entered the incorrect value. Now my program works well for them too.
2. Instead of providing tools, could you tell us how to do it so we can convert them to ogg or something without running an outer program?
3 & 5. Get it. Thanks.
4. And fonts(.font.xml) too. How can we modify them?
EDIT: Now I'm going to try to extract .pak files...
@Daxar
I made my program because yours doesn't work for me. (older versions, haven't tried new version)
@Allan: I wonder how we can get the console mode?
By the argument to run exe or add debug option in settings.txt or using the keyboard shortcuts?
Especially needed option '[ff]timescale' and 'freecam'
Making a GUI version...
http://i.imgur.com/5UL6Y.png
@Allan
Does #11 works for all .dat files? (I think the file structure is same but the meanings are different)
I think I can convert them to XML files so that we can modify the text in the game.
EDIT: Like this one: http://pastebin.com/b16n2hf1
My program only works for residmap.dat, so I guess other .dat files are using a bit different format.
EDIT: Little Inferno Resource Manager V1.0.0.1
Screenshot: http://i.imgur.com/u11VE.png
Download: https://dl.dropbox.com/u/94637484/Little%20Inferno%20Resource%20Manager.exe
Source: https://dl.dropbox.com/u/94637484/Little%20Inferno%20Resource%20Manager.rar
How to use: I believe this one is easy enough for you.
Performance: 12s to extract pak files, 2 mins to unpack pak files.
Dependancy: .NET Framework 4.5
Packer module is under development.
P.S. I think I'm going to research about the audio now.
for sounds, yes i can provide some source code instead to show the process of converting between the game's sound data and ogg files. i'll have to spend a little time figuring out exactly how to do that first though : )
font data is yet another resource format that will need to be documented along with the ones i mentioned earlier. i think the most useful data to be able to edit first though might be the item data since it's pretty straight forward to change some simple values and get real changes in game. i figured we could dive into that once repaking is working.
access to the in game console is completely disabled in the shipping version of the game so there's no way to bring it up. it needs the debug font to render anyway and the debug font isn't included in the shipping version of the game either. how did you know that a console even existed? : )
any files that end in .dat are just generic binary data files with a .dat extension because i needed to make up some extension to give them. they all share common building blocks (like the BinHdrPtr structure and the string table structure that i described before) but don't have a single unified format that could be used to automatically convert them to xml or something like that. their data is all formatted to be used directly by the code that works with it in the game with minimal processing once it has been copied into memory from the pak files. you can think of all of these resources as in-memory versions of the game's data which has been serialized out to disk for transport.
My version Little Inferno 1.0
included
DebugFont.page00.png
DebugFont.page01.png
DebugFont.font.xml
and some commands with description
(for example: 'clear - instantly delete all items in fireplace'
and console view:
Little Inferno Version: 1.0 Build Date:... Build Time: ... Command Line: ...
EGG>
command '...' not recognized. type '?' to see a list of commands.)
it means that in my version the game console isn't disabled
and again I ask you to reply to the question - how i can get the console mode?
please give an answer to this vital question... thanks!
You made really nice job guys, especially Daxar!
Speaking about fonts, I'll help adding Cyrillic letters. I've already emailed Kyle about them.
Allan, could you tell me names of fonts you used in the game. I'll make Russian fonts.
http://www.microsoft.com/typography/fonts/family.aspx?FID=148
but without xml-conversion will be difficult to add Russian letters
@movildima: ...and if you want, join to ZoG-forum discussion about cyrillic
The point is that Cyrillic letters don't exist in Tw Century whatever font, so I have to draw them by hand. This is not that difficult, if someone could then pack them, that would be great.
Here's a link to some fonts that I (and Vladislav) created with respective mappings.
@Vials What is Zog forums?
EDIT: Got it. Ты говоришь по русски
@Vials: Just for future reference, please edit your previous posts instead of double-posting. Thanks!
@Allan: Please don't wait for me before explaining stuff. x_x I'll be working on a school project the next few days.
@Mygod: Hmm, strange that it didn't work for you. In any case, kudos on the cool GUI!
Little Inferno Resource Manager V1.0.1.0
Download: https://dl.dropbox.com/u/94637484/Little%20Inferno%20Resource%20Manager.exe
Source: https://dl.dropbox.com/u/94637484/Little%20Inferno%20Resource%20Manager.rar
Dependancy: .NET Framework 4.0 Client Profile
I just realized .NET Framework 4.5 doesn't work for XP so I change the dependancy. : P
Another question: how do I know where I should put a new file, resource.pak, embed.pak or frontend.pak?
EDIT: http://i.imgur.com/UUXd8.png
Modded the game to make it crash successfully.
there's no way to access the console in shipping versions of the game. honest! not all traces of every single thing related to the console are removed from the code of course, so the strings for the commands are probably all still in the exe if you look for them, but the ability to actually bring up the console is compiled out of the shipping code (as is the entire function that renders the console so you wouldn't be able to see it even if you were able to activate it.)
anything related to adding cyrillic characters to the font for a russian language version of the game is probably best to discuss over email to contact@tomorrowcorporation.com. if this would actually be for the shipping version of the game then we'd probably need to sort through legal issues that i don't know much about so emailing with all of us is probably a better place to have that discussion. (by the way, i was very happy to discover that although i was not a very good student, my 5 years of studying russian in high school did at least prepare me to be able to understand "Ты говоришь по русски" without having to run it through google translator!)
as for adding new files to the paks, it doesn't really matter where they go. the wiiu version of the game loads the different paks at different times and so it matters what goes where for that, but the pc version of the game is able to load any resource from any pak at any time.
i was trying to come up with a simple candidate resource to try editing first and i think the dictionary for the word pack item might be a good one. it has a really similar structure to the vdata/residmap.dat resource that i laid out before. the dictionary for that item is the resource vdata\wordPackDict.dat. it starts with a header
struct
{
BinHdrPtr words;
BinHdrPtr stringTableBytes;
};
which says the resource contains a section for a list of words and a section for a string table with the actual text. the list of words is one of the following structs for each word:
struct
{
i32 wordStrId;
f32 probability;
};
where wordStrId is the key for looking up the text in the string table and probability is a float in the range 0.0 to 1.0 that gives the probability that the word will be selected when you buy the word pack item. the total probabilities for all words should add up to 1.0. after that list of words there is a string table with the text for all the words in the same string table format that i described before.
any specific requests for resource types to document next? or specific things about the game that you want to change as the first steps toward modding and i could tell you which resources would be involved? i'm going to work on the sounds but they might take a little longer since the process for those is much less straight forward.
Well, anyway I think we'd better make everything in the right pak files since Wii U version matters. So I want to know that anyway.
I think we can generate an index.xml that contains everything that is needed to make pak files and add .png extension for all recognizable pictures whatever it has the extension or not. What do you think, @Daxar?
index.xml example: http://pastebin.com/ndfrgG5U
How to test if a file is a picture fast:
1. Length >= 12
2. Length == Width*Height << 2 + 12
@Allan
>any specific requests for resource types to document next?
of course TwCen.font.xml description
I think simply the decryption of XML files in general would be a huge help, we can probably figure out what most of them do and how they work by looking at the XML itself (as we did with WoG).
I'm curious though, where is the data for the letters and combos held? Also in XML files or elsewhere?
Wait... I find some of the pictures' flags are set to 2!!!
EDIT: Now working on the packer module. Can't get it work.
I didn't know that you know Russian Allan!
Ok, I'll use the contact@tomorrow to comunicate about this. Vials is working on it too. I've seen him on ZoG forums talking about them.
Nice work guys!
@Mygod: I don't think that's the error you're supposed to get when running the game. That's what I got when I stripped ALL of the resources from the game, rather than just the .pak files. It should say something about being unable to open a .pak file rather than saying that an unhandled exception occurred.
@Allan: Knowing the XML file format next would be pretty tops.
Removing resources works!
Issue: I tested my program on XP. Though it worked but the file size didn't decrease any. (Win7 works fine, after removing, the file size is 3.70 M
V1.0.2.2
Download: https://dl.dropbox.com/u/94637484/Little%20Inferno%20Resource%20Manager.exe
Source: https://dl.dropbox.com/u/94637484/Little%20Inferno%20Resource%20Manager.rar
What's New: Fixed a lot of bugs! And add the exciting packer & injector module! Improved the performance of extractor.
Packer has some bugs. I'm not sure what's the issue. Anyway I'm going to publish the embed.pak it generated: https://dl.dropbox.com/u/94637484/embed.pak (weirdly, my program can unpack it well)
P.S. It doesn't support compressing currently and I will be working on this as soon as the uncompressed version works.
Version 0.3 changelog:
- Added liCompress for compressing everything back into .pak files
- For now, all files are compressed by default, regardless of flac/png etc
- Added util/repack.exe program for repacking .pak files back into the game's exe
- Various bugfixes
Download: https://dl.dropbox.com/u/31816885/liDecompress0.3.zip
Source: https://dl.dropbox.com/u/31816885/liDecompress0.3-src.zip
Which means:
Introducing the very first Little Inferno mod!
Screenshot: https://dl.dropbox.com/u/31816885/BOOYAH.png
Download: https://dl.dropbox.com/u/31816885/chimneyChanger.pak
To install, do the following:
- first, back everything up. I haven't extensively tested everything, and something might break.
- next, run strip.exe on your Little Inferno executable to pull all the .pak files out, if you haven't already
- next, back up your frontend.pak file
- next, run liDecompress on frontend.pak (You'll need to do this even if you have already, just because my decompress/compress code has changed since last version)
- next, run liDecompress chimneyChanger.pak. This will overwrite the chimney image files for the main menu, hence why you're backing up the frontend.pak first
- next, run liCompress on frontend.pak
You should be good to go. Let me know if you have any trouble.
Finally, you've got first mod evah for Little Inferno! I've tested the mod, it's working and I'm so excited!
There's a lot of work ahead: writing documentation on how it works, adding mod pages on the site, DIscover everything! The gameplay possibilities will be endless! Call davidc here, quick!
Oh you are a bit faster than mine. Good job anyway!
Didn't work for me (just got the normal chimneys), but it's quite possible that it's because I'm running a version I copied from Steam, so it might be running the Steam version (since the overlay comes up). I'll download the non-Steam version and try it again.
I'm getting lots of ideas for a GUI for all of this though, to make it all a bit cleaner and user-friendly. I'd volunteer to make one if I wasn't already so busy.
EDIT: Works now!
Daxar, a question. How did you made that ChimneyChanger.pak file?
EDIT: Forgot to read Readme. Got it now.
Pages