Now that the game is out everywhere (More or less)

336 posts / 0 new
Last post
Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

@Daxar
Looks great. I'm eagerly looking forward to the new builds. : )

EDIT: Wait, what? Page 6! : P

By the way, last month I found something about converting matrix back to transforms here: http://stackoverflow.com/questions/5994775/convert-matrixtransform-to-tr...
So I write some codes to help you convert the matrix back to the transforms: http://pastebin.com/vax09G8v
It's written in C# because I know almost nothing about C++ but I think you should understand what I'm trying to do here. ; )
This code should get a output like this if there's a compiler to run it:
<transforms>
  <scale factor='1.5,1.6' />
  <rotate angle='1.570796' />
  <translate offset='10,20' />
</transforms>

The original matrix

0   -1.6 10
1.5 0    20

EDIT: Just realized I assumed all the values are non-zero values. Fixing this issue...

EDIT: Code updated.
This code won't work if it's a singular matrix (_11 = _21 = 0 || _12 = _22 = 0) but I don't think there's one.

EDIT: @allan
Is customized country code supported in 1.2 now?

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

Ok, rather than beating around the bush like I have been about this, here you go, Mygod: http://pastebin.com/By5vRNNw

These are all the matrices in the itemmanifest.dat, hopefully in a format you can use (Let me know if there are any formatting issues so your code can read/write it easily). I agree your proposed XML is quite a lot cleaner than what I have currently; I'm just concerned about edge cases not working. If your code can convert back and forth using that matrix data, and the results are within rounding error of the original values, then let me know and I'll add your code in (C# is very similar to C++, so doing so won't be any problem). I just don't think in matrices yet myself, so I'm a bit hesitant to touch them more than I have to, and I want to make sure the code works for the original game properly so I don't unintentionally break anything. Ideally, a couple of functions (one for converting from matrix to rotate/translate/scale format and a second to convert back) would be nice, so I can plug it in easily. Thanks for doing this!

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

After a few hours of debugging, my program finally passed 15625 tests! : D
This version can convert almost all matrices into transforms and convert them back. The not-supported matrices will be in the xml to support converting back.
Binary: https://dl.dropboxusercontent.com/u/94637484/MatrixConverter.exe
(.NET Framework 4.0 Client Profile or the later version required)
Source: http://paste.ubuntu.com/5976006/
Usage:

  1. Type a matrix like this to get its transforms:
    1 0 0
    0 1 0
  2. Type a transform like this to get its matrix :
    <transforms>
    </transforms>
  3. Type self-test to make the program test itself.

And here's a screenshot: http://s1264.photobucket.com/user/MygodStudio/media/2013-08-12_122525.pn...

P.S. The not-supported matrices are: _11 == 0 || _22 == 0. (only a few of them can be converted to transforms correctly)
They are mostly caused by scale_x == 0 || scale_y == 0 so it's not a really serious problem.

allanblomquist's picture
Offline
Joined: 11/21/2012 - 21:04

with regard to custom languages, there weren't any code changes around localization in version 1.2 so the game will still only work with the languages it was compiled for.

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

So I still need to override the English to get my language pack work. : (
On the plus side, I don't need to change my code. : P

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

@Allen: Looks like I never asked you before; what's the "id" field of the BinLocStrKey structure used for? Is it just for translations, so it can be safely ignored? Or are string literals hardcoded into the game or in some other manifest someplace?

Also, it seems that the "idStrTblIdx" field in the comboRecord structure isn't in combodb.dat, at least not in the 1.0 pakfiles I have. Is this a recent addition?

Oh, and what's the "id" field of the combo records for? Is there some combo manifest someplace, or is this used when building the GUI for the combo names?

@all: First off, a simple change to loctexmanifest.bin.xml: http://pastebin.com/uJFU0Jun
I realized the language IDs were just the two ASCII letters for each language, so I can convert them easily to a human-readable language code, like "en" or "de" or such. Nice!

Next up, thanks to Mygod, we have matrix transformations inside the item XML, and hopefully these make a good deal more sense and are more easily modified. Example Beetle.xml: http://pastebin.com/sibHvYCT
Thanks to Allen, we have the rest of the item data and strings, and I've made some tweaks to the XML as I went along. Notice how few attributes the "itemdata" field now has! Nice, huh? I may split the "iconAnimBounds" out into another field, as well. Also note that down at the bottom, we have three unexplained "burn_[x]" text resources. Looking at the AlarmClock XML ( http://pastebin.com/ZjPgyDrt ), we only have one, named "burn." I imagine this has something to do with what fluid palette is used for the fire flames, but I'm not sure where that all plugs in. Yet!

Finally, and again thanks to Allen, we have a combodb.dat.xml to look at: http://pastebin.com/Tq0PcpS3 Of course, don't look at this one too closely if you haven't found all of the combos yet!

No downloads yet; I'm still finalizing these XML formats, and so far I can only export them but not read them back. I'd also like to make sure liTools supports the latest pak files before releasing another version, in case something drastic has changed. So I'll say the next version will be 'soon.' Of course, if you want to brave link dependencies, the latest code is on GitHub as always.

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

I noticed that there are some lines like this in your XML:
<skew angle="22.996, -22.996"/>
It should be like this one:
<rotate angle="-22.996"/>
I guess you set epsilon too small. 1e-3 should work at this point. And I recommend you to use double (or f64) for these values to decrease the deviation.

allanblomquist's picture
Offline
Joined: 11/21/2012 - 21:04

BinLocStrKey::strId was used for the localization mode that we built for our localizers. it was there so that as the game rendered localized strings on the screen, it could keep track of which strings it was drawing on that frame so that when the localizer paused the game to translate something, they would only see a candidate list of all the strings that were on the screen at that moment.

yes, it does look like BinCombo::idStrTblIdx was added on Jan 3, 2013 (for version 1.2) because we needed an id string to use for game center achievements on iOS. the only other change to the binary data formats since launch that i can see in our svn history is something for animation data (again for iOS) which i don't think i've covered here yet. i think the only meaningful version data that you can look at to know which format to look for would be the win32 version resource in the game EXE itself. one way to get that is using the following win32 api calls (error checks and cleanup omitted for brevity) :

http://pastebin.com/aCmn2Sbp

the combo id is what is saved in your save data to know which combos you've gotten.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

@Mygod: Thanks! And good catch. I'll do that.

@Allan: Ok, so the BinLocStrKey.id fields are important, then. Are they hashed values of some kind of string, by any chance?

Also looks like the combo ID fields are just the combo's English name (Minus the "|COMBO" part) hashed, so that's one thing I can change...

allanblomquist's picture
Offline
Joined: 11/21/2012 - 21:04

the localized string ids are hashes of identifiers that we generate based on which resource the string comes from and what it represents within the resource. so they're things like LETTERS_C1_SUGARPLUMPS1_P07_TEXT, ITEMS_BEARTRAPSITTING_DESC and ANIMS_DATA_ANIMATIONS_CATALOGBUYBOX_CATALOGBUYBOX_ANIM_XML_REGIONS_SHIPTIMETEXT_TEXT.

combo ids are mostly the same as their english names but i think i remember there are a few cases where we had to change the name of the combo (there used to be a plants vs.zombies combo for example that we had to change to zombie garden) at the last minute and so we only updated the name in that case and not the id.

edit: in general ids should be thought of as opaque values that only need to be unique and used consistently. they were all managed by tools and not things that humans interacted with. they probably should have just all been randomly generated GUIDs, but i ended up taking the easy way out and basing them on hashes of unique strings instead which has the downside that the id strings can get kind of out of sync with the resource they're supposed to be identifying like in the combo example above.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

Ok. In that case, I'll leave them untouched for the most part, as well (so long as it makes sense to do so). In the future, we can just hash strings to get IDs or generate them randomly.

Oh, and please don't wait for me before explaining stuff. I can always catch up later. Do continue. : )

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

http://social.msdn.microsoft.com/Forums/silverlight/en-US/f85aa39b-7f11-...
Just found that there are implementation for this in Microsoft Expression. Some people decompiled the dll and got the source but the variable names are messed up. Surprisingly this code passed all my tests without using a matrix. I will try to figure out how it works and post it here some time later. : )

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

Aaaaand.... http://oi40.tinypic.com/e7cb6d.jpg

Lookin' good. Oh, and I modded it to give me 99 coins and 25 stamps for the combo, just for fun. http://oi43.tinypic.com/2gy7q6e.jpg

Onward ho!

EDIT: I've always wondered about this! Turns out if an item is in a combo with itself, simply burning the item will trigger the combo: http://oi44.tinypic.com/jb1o2v.jpg

As it's seen in the combo menu: http://oi43.tinypic.com/9rin9j.jpg

Unfortunately, having more than 99 combos messes up the spacing for combo names farther down the list: http://oi42.tinypic.com/161nu3d.jpg You can dupe this out, however, by adding a space or two to the beginning of the combo name. Here it is with two spaces: http://oi39.tinypic.com/2a6a4nl.jpg
Perhaps I'll have LiTools automatically determine if there are more than 99 combos and add proper spacing if so.

If there's not enough space for the combo name, the game shrinks the text so it fits. I'm liking your engine design, Allan. http://oi43.tinypic.com/28btt7o.jpg

EDIT2: You can have combos with more than three items, it seems, but only the images for the first three are drawn (Both in the combo list and the popup when you get the combo). Burning only these three items together doesn't work, however; all the items in the combo are required, whether they are drawn or not.

Oh, and combos can be safely removed without the game exploding (even if you've gotten them), which is nice. http://oi41.tinypic.com/23s9jkl.jpg

It seems that combos are greyed out in the combo dialog automagically if the catalog in which the item(s) are found isn't unlocked yet. I have no idea if this is just the last item in the combo, or if it's the last item catalog-wise.

On my XML side, I replaced the "texx=x", "texy=y", "texw=w", "texh=h" fields with "texpos=x,y" and "texsize=w,h", which I think is more compact and readable.

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

The new source: http://paste.ubuntu.com/6017565/
This one works much better. : )
The problem is I can't understand some of the code. Anyway it just works.

EDIT: After some research I finally know how it works and optimized it a bit. : P

EDIT: @allan
Is localized word pack supported now in 1.2?

allanblomquist's picture
Offline
Joined: 11/21/2012 - 21:04

word pack still works the same in 1.2. i don't recall making any game changes really, just adding a couple translations.

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

@allan
Okay...
@Daxar
Could you please publish the latest binary now? : )

allanblomquist's picture
Offline
Joined: 11/21/2012 - 21:04

next up i guess i'll do the letter data. all letters are stored together in the resource vdata\letterdb.dat. the header looks like this:

struct
{
BinHdrPtr letters;
BinHdrPtr pages;
BinHdrPtr stringTableBytes;
};

the letters section has a record for each letter in the game. each one looks like this:

struct
{
i32 type;
u32 id;
i32 firstPageIdx;
i32 numPages;
u32 dependsLetterId;
u32 borderAnimExportId;
i32 sendTimeSec;
i32 delayTimeSec;
u32 postLetterSpecialExportId;
u32 depCatalogId;
i32 depCatalogItemCount;
i32 depNoItems;
i32 addTraySlots;
i32 deliverable;
i32 removeAfterRead;
i32 burnableAfterRead;
i32 depAllStars;
i32 depAllCombo;
i32 depAllItemsUsed;
i32 jingleLetter;
i32 blockNextCatalog;
i32 deliverAfterWin;
u32 requestedItemId[3];
u32 reqChildWrongLetterId;
u32 reqParentLetterId;
u32 attachedItemId;
i32 attachedMoneyAmount;
u32 attachedCatalogId;
i32 letterIdStrId;
};

the first field is a letter type. types are 00 = basic, 01 = request, 02 = requestFollowUp, 03 = attachItem, 04 = attachMoney, 05 = attachCatalog. some of the other fields are fairly straight forward and some will probably require some explanation later on. the firstPageIdx and numPages fields refer to the pages section which is next up in the data. the pages section has a record for each individual page of a letter that exists. each record looks like this:

struct
{
u32 senderPicExportId;
u32 specialEventExportId;
u32 pagePicTexResId;
f32 pagePicScale;
u32 musicSoundResId;
f32 musicOffset;
u32 wordsSoundResId;
u32 lastWordSoundResId;
i32 allowSkip;
f32 speedScale;
BinLocStrKey text;
};

most of the fields control various aspects of how the page is presented in the game and the last one is the actual text that goes on the page. after the pages section is a standard string table with all the strings and that's all there is to the letter data.

movildima's picture
Offline
Joined: 11/19/2012 - 01:13

Look! I see there are many unused features like attachMoney! Will come in handy when modding galore starts!

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

@Mygod: Soon. There's a couple of things that I want to finish up first. ; ) I did recently update my GCC version and fix the link errors you were having before, however, so you ought to be able to pull from my GitHub and compile the latest version without a hitch now.

@Allan: Sweet, I'm on it.

@movildima: Honestly, there's nothing keeping anyone from modding the game now. Anyone with some technical know-how and a hex editor can be poking at stuff as soon as Allan describes it here. I'm just pulling the data out into a (hopefully) more easily modifiable form, which is useless if you aren't following the discussion here already.

movildima's picture
Offline
Joined: 11/19/2012 - 01:13

Yeah I know. In fact, I already tried to modify some images, and that was really cool to see changes in-game.
But the real modding galore haven't started yet. For now most users are only able to modify only a few things, like images. Hexediting is a bit difficult to me now, so I'll wait until at least we'll be able to use XML's : )

23boy's picture
Offline
Joined: 11/23/2012 - 18:22

@Daxar and others

Long time no see i have been waiting for you to be active again...
but now my laptop is broken and i cant follow the update...

what are you saying is right "anyone with some technical know-how and a hex editor can be poking at stuff"
but after we do the hard coding there must be a more user friendly program like WOG editor..

or like movildima said, XML and stuff that would be nice

but i know it will came the last...

other than that Great Job!!

P.s Sorry i cant help..

allanblomquist's picture
Offline
Joined: 11/21/2012 - 21:04

next i'll do the data for the catalogs. the resource that holds all data about catalogs is vdata\catalogdb.dat. it starts with a header that looks like this:

struct
{
BinHdrPtr catalogs;
BinHdrPtr items;
BinHdrPtr stringTableBytes;
};

the catalogs section has a record for each catalog in the game. the order of these records determines the order of the catalogs in the game. each record looks like this:

struct
{
u32 id;
i32 cost;
i32 numCombos;
BinTuneDataFloat bgRaysR;
BinTuneDataFloat bgRaysG;
BinTuneDataFloat bgRaysB;
BinTuneDataFloat bgPaperR;
BinTuneDataFloat bgPaperG;
BinTuneDataFloat bgPaperB;
i32 firstItemIdx;
i32 numItems;
u32 coverTexId;
u32 thumbTexId;
u32 lockedTexId;
i32 nameStrId;
};

firstItemIdx and numItems refer to the items section which has a record for each item listed in a catalog. each item record is just a single u32 which is the id for the item to include in the catalog. after those is a string table which i think just has the english names of all the catalogs which aren't actually displayed as text in the game but it looks like we use them as part of the names of some iOS game center achievements.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

Okay! I figure you guys have waited long enough, as have I. I've been tracking down some super-annoying bugs in decompression all morning, but I figured it all out and it works now, so it's time to celebrate! Introducing:

liTools Version 0.3.6

This version requires Little Inferno 1.2, so upgrade before using if you haven't already. Yes, you'll need to wipe everything and re-strip a version 1.2 executable. In the future, I'll probably add a test to see what version of Little Inferno you're using, and decompress accordingly, but for now I'm lazy. So upgrade, or face the terrors of crashing programs. Sorry, I'm full of myself right now. Did I mention I fixed a bug? I love fixing bugs.

Have a download link: https://dl.dropboxusercontent.com/u/31816885/liTools0.3.6.zip
Source is at https://github.com/meh2481/liTools as always.

Changes of note:
* Now (only) supports Little Inferno 1.2. Be sure to upgrade to the latest version before using!
* vdata/itemmanifest.dat is now parsed to/from XML. Hooray!
* vdata/loctexmanifest.dat now parsed to/from XML
* vdata/combodb.dat now parsed to/from XML
* Matrix transformations in item XML thanks to MyGod
* Some Unicode tweaks and such
* Added util/hash.exe for easily getting hashes of filenames or other such text strings

This means you can do fun stuff now, like make up your own combos (You can't add items for now, I'm afraid; I still haven't added support for new files yet), change an item's cost to a negative number so you GAIN money every time you buy it, and beat your screen in frustration because an item's "valueStamps" attribute seems have no effect on the game's code.
Oh, and you can change the ship time. I haven't tried it, but I mean it's all in the XML. There's nothing preventing you, really. Fiddle around. Do cool stuff.

Have fun, guys! Let me know if you have any troubles/issues, and definitely let me know all the cool stuff you figure out.

Onward ho to vdata/letterdb.dat and vdata/catalogdb.dat!

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

Cool!!! I will try it this weekend.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

liTools version 0.3.7

Just catching up on the formats I hadn't added support for yet. Link: https://dl.dropboxusercontent.com/u/31816885/liTools0.3.7.zip

Changelog:
* vdata/catalogdb.dat is now parsed to/from XML
* vdata/letterdb.dat is now parsed to/from XML

And that's about it.

Letters are now fully editable. Hooray! https://dl.dropboxusercontent.com/u/31816885/photos/adopted.png
No, I don't know what I was thinking when I made that, either.

Catalogs are kinda fun to play with. Here's one that totally would have a secret message in the XML if there were an item that started with "Y". But there isn't. https://dl.dropboxusercontent.com/u/31816885/photos/fluttershisbestpon.png
And yes, I changed the background color to pink. Because I'm the kind of guy who can do that and still be awesome.

The game didn't seem to like my "bunnybunnybunnybunnybunny" catalog idea, though: https://dl.dropboxusercontent.com/u/31816885/photos/bunnybunnybunny.png

It seems that having more than 20 items per catalog crashes the game, as well. But the game totally doesn't care if you have less than 20: https://dl.dropboxusercontent.com/u/31816885/photos/missingitems.png
The empty buttons you see there are clickable, but do nothing when clicked.

Look at me, catching up with liTools and stuff. It helps that I'm sick and bedridden. And by bed, I mean a big comfy chair I can code in.

Cheers!
-Daxar

puggsoy's picture
Offline
Joined: 11/20/2012 - 04:25

Awesome! I want to experiment and make a letter and catalog and combo editor and everything, but I don't have the time!

movildima's picture
Offline
Joined: 11/19/2012 - 01:13

Total awesomeness!!! Thank you so much Daxar!

kirdneh's picture
Offline
Joined: 12/22/2012 - 14:01

Amazing! Great job.

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

I recommend you to use &#xA; for \n in XML or my program won't work properly. I'm trying to make a Chinese version of the game.
Replacing (data\=\"[^"]*?)\r\n([^"]*?\") with $1&#xA;$2...

By the way, some changes found in 1.2:
No changes in dialogs/letters.

Items
Manly Razer => Manly Razor
Toy Rocketship => Rocketship of|Learning
MousePad => mousePad (id)

Credits changed a lot.
Added:
mobile programming
GIL CARMEL

Added a lot of translators.
Added:
"Guy on Fire" Pictogram
TRAVIS DOGGETT

Combo list: The title changed from COMBO to COMBOS.

EDIT: It seems everything works PERFECTLY now! THANKS!!!
P.S. Format for .anim.xml DEMANDED!

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

Right, the XML newlines. I noticed those earlier. Unfortunately, that's all handled by the game and TinyXML, and kind of out of my control. For the string data, I'm directly streaming it from the game formats directly to XML, so I'd rather not modify it, since it works both ways.

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

Can you try another xml parser or something like that?

EDIT: It seems that .png files in iOS version are PVR files...
EDIT:
struct
{
u32 width;
u32 height;
u32 pixel_format;
char[] pvr_texture_data;
};

It seems the images in PVR files are stretched. For example, data/_common/0.png is stretched from 40x56 to 64x64.
Other files seemed to be in the same format.
@Daxar Could you please add support for the iOS version? Like: liDecompress abc.pak -ios

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

Switching XML parsers would require rewriting huge amounts of code, which I'm not all that eager to do. What exactly is the problem you're having, and why are the newlines important? I'm not sure I'm following what your program is trying to do.

Didn't realize I had the .ipa file already on my computer because of iTunes. Working on iOS version now.

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

I'm trying to make a Chinese version of the game (the program reads the translations and changes the xml), but the problem was solved by using regular expression replacing and now everything works fine so if you don't want to do that, it's okay for me.
I wonder if the modified .ipa file can be installed onto iOS devices that are not jailbroken.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

I have no idea. I haven't jailbroken my iPod, nor do I really intend to.

From what I gather, PVR is a lossy container format, so I probably won't end up compressing/decompressing the images to/from PNG. I think there are tools for opening and working on them directly, but I haven't looked into any yet, and I'm worried because the headers look different from what the source code examples I have expect. I'll be looking into it, and seeing what I can figure out.

Mygod's picture
Offline
Joined: 11/17/2012 - 08:00

@Daxar
Read what I have said. You should strip the first 12 bytes to get the correct PVR file. I used PVRTexTool to open that stripped file.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

Ah, my bad. I confused myself with the format, and ended up going against what I originally thought was true and what you said.

It seems the image scaling is handled dynamically in the game's code, with the image header tacked onto the beginning of the PVR file. Also, a lot of the textures are quite a good deal smaller, though I'm surprised that didn't affect the overall quality (And the filesize; it's only about 70 megs smaller. I guess they're using uncompressed PVR images here?). Anyhoo, I'll keep poking before I finalize anything.

movildima's picture
Offline
Joined: 11/19/2012 - 01:13

Ok, so I figured out how to add languages to the game.
The only thing is that its needed the support for adding files.
@Alan, I'll talk to you about it through the tomorrow corporation email, ok?
@Daxar, could you add the support of adding files at least for the images in the /data/fonts folder?

Electricut's picture
Offline
Joined: 11/09/2013 - 14:35

Hey, everyone. I sure wish I could help with this, but I don't know the first thing about modding or working with code.

I hope there's still interest in modding this game, because I'd still love to see it. I'm rooting for you.

23boy's picture
Offline
Joined: 11/23/2012 - 18:22

yes we all do!

puggsoy's picture
Offline
Joined: 11/20/2012 - 04:25

I might get started on some tools soon. I'll probably remake Matchbox from scratch, using a different language. It'll be more native, so it won't require you to download any extra runtimes, and it won't require an installer (although I'll probably make one for convenience, once I have a stable version).

First thing's first though, I'll probably want to make a format like .goomod. I know liTools can merge .paks, but we'd want things like metadata. It'll probably be a .zip containing the .pak, a metadata file (probably an addin.xml, again), and... not sure if anything else is needed. The extension could be .infernomod, although that seems a bit long. Maybe something like .matchmod or .flamemod. Or just .match (a matchbox contains matches : P).

In any case I think this needs a bit of brainstorming, I can't really do this on my own. Suggestions for extensions and what kinds of metadata we need is welcome. Name, author, version, description... anything else?

@Daxar: If stuff gets packaged like this, it'd be most convenient for me to feed .pak data directly to the merge program via stdin (as opposed to a path). Is this supported, and if not, could it be added? Extracting it and passing a path is possible, but it would be more convenient to give it the bytes directly.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

Yikes, it's been a while since I worked on this.

I think I could probably add support for that; there'd have to be some kind of commandline switch or another. I think I was trying to make my compression/decompression code as generic as possible, with varying degrees of success (hence why new files still aren't supported; some aspects of compression/decompression don't care, and the sndmanifest.dat part sure does). And I still need to figure out this iOS support...

I'll look into it as I have time (Though it may be a while; exams are coming up in the next couple of weeks).

Name, author, version, description is good, I think. I'm hoping that liTools will be able to manage all the behind-the-scenes stuff like strings and such. The issue for the XML stuff that liTools reads/outputs is that it'd be well-nigh impossible to have XSL in a .pak file, so if two addins try to add a new item, there'd be a conflict with the entire itemmanifest.dat. I'll have to look at it again, but I think that last I left off, I had no idea how to resolve such a conflict.

Long story short is that I was quite excited to be able to decompress, edit, and recompress everything, so I could make changes quickly and see them reflected ingame. My design of liTools reflects this; kinda thrown together as fast as I could to keep up with the formats Allan was describing. As far as actually making the game's assets editable, it does a good job. As far as being easily extensible for modding... not at all. For example, I dump all the filenames of the files in a .pak file into a text file in the same folder. Not really the best way to keep track of what files go where as mods are installed, but quite easy for me to edit and add files to as I test compression/decompression (basically so I don't have to compress/decompress the entirety of resource.pak if I'm testing my code for working correctly on one of the files inside it).

This aside, I'm not certain if having support for multiple mods overwriting the same files is even needed. What I was originally planning on was having mods just be distributed as .pak files, and inside those there's some extra data (The issue here is that it's quite easy to add new resources to a .pak file. After this, however, you don't know the original filename, so you can't extract them back out somewhere sane where they can be edited and compressed again. My current version attempts to put a tiny filename-id mapping list in, like is in debug.pak, but so far I haven't gotten that quite working yet). Then it would be a simple matter to add the new file id's into the original pakfiles, or change old ones, while leaving everything else intact. This is still what I'm kind of hoping for in an addin format, honestly. It's not user-friendly in the slightest, but there's so few of us here that a large percentage of us kind of know what's going on already. And we have enough know-how to back everything up in case it breaks horribly. Because liTools breaks files horribly quite often (As I'm working on it; the releases are ones where nothing's broken in a while, and so I cross my fingers and hope there aren't bugs. And I'm still sure that the hackish UTF-8 support will break horribly sometime).

It's difficult for me coming from World of Goo, because everything there was already out in the open, where I could edit it to my heart's content. Not to mention that we had some awesome editing tools, which were already around by the time I joined GooFans. Everything else from that point on was just icing on the cake; DaB remaking wogedit to actually be newbie-friendly, goomod support for adding strings directly instead of writing XSL to merge them in, that sort of thing. I kind of doubt we'll ever reach that point here, or that we'll even want to.

Bla de bla de bla. What I really need to do is get some time to work on liTools and figure out what I want to do next, and finish my original goal of supporting new files. If you want to work on an addin format/manager, puggsoy, that's great! I'll take a look and see if I can read pakfiles from stdin. I'd also vote for something short, like .limod or .match, if you're planning on a new file extension. The easier it is to type, the better.

Walloftext done. Daxar out. Peace.

movildima's picture
Offline
Joined: 11/19/2012 - 01:13

I'm really glad to hear you guys again! After reading the "walloftext" I understood that I must help in a way or another.
@puggsoy I guess we could use a metadata structure similar to .goomod one.
Like this:

addin
id
name
type (toy or mod)
version
description
author
depends
Maybe adding a thumbnail could be interesting.

EDIT: It seems that I can't put XML tags in comments.

puggsoy's picture
Offline
Joined: 11/20/2012 - 04:25

@Daxar: Aha, I see. Actually, last night I had an idea. It might be overall easier (for me, you, and everyone) that we actually try and keep this as WoG-like as possible. That is, instead of installing mods by merging .paks and all that, what we actually do is first extract out all of them. Then we have sub-folders with everything organised, like WoG does. This is where people make changes and mods are installed. Then, once that's all done, you hit "save changes" (or something) on Matchbox and it'll pack all those changes back into the .paks, then stick the .paks back into the executable.

So the whole mod making/installing shabang is handled in folders, like WoG. Then we only need to worry about .paks when actually putting the changes into the game. That way .paks don't need to be merged either, the mod format can just be organised in subfolders like in .goomods.

@movildima: Thanks! I'd forgotten about the id and depends. I'm not sure if type will be necessary, in WoG this was needed because GooTool installed levels in a specific way so that they appeared in the top-left corner of Chapter 1. Unless Matchbox has to do similar things (although I don't think it will), a mod type is unnecessary. Of course, we could also just have it purely for categorisation purposes.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

@puggsoy: That may indeed be a good idea. The only drawback I can think of is that it may take a while each time to "save changes" (a la GooTool), but we're used to that taking a little while anyway. If we keep track of files that have changed, be it by hashing, or just a list of changed files due to mods, it may be possible to merge changes in quite quickly. I've also been looking at memory mapping for file I/O today, and I'll probably be running some analysis tools on my programs to see where the big speed bottlenecks are currently. I'll keep you posted on what I come up with.

puggsoy's picture
Offline
Joined: 11/20/2012 - 04:25

Yeah, I figured that's probably the biggest issue. I'm not exactly sure what hashing consists of, but hopefully that can work. A list of files mods have changed may not be the best way to deal with it, since it should (ideally) also be able to handle any hand-made changes you make while creating and testing your own mods. Repacking the entire files every single time could get tedious.

I was thinking perhaps there could be 2 "save changes" options; one for when making mods, and one for when installing them. The first one would simply pack the currently existing files into the packs and put them in the .exe, done. The second would, similar to WoG, grab the original files (from a non-custom installation), then apply the mods over that. So for LI it'd extract the original .paks over the already extracted subfolders, apply mods, then pack that into the .paks, then into the .exe.

I'm not sure, just throwing ideas around. I want it to be easy for people to both install mods, but also to make them.

Daxar's picture
Offline
Joined: 11/18/2012 - 14:09

puggsoy wrote:
The second would, similar to WoG, grab the original files (from a non-custom installation), then apply the mods over that. So for LI it'd extract the original .paks over the already extracted subfolders, apply mods, then pack that into the .paks, then into the .exe.

It could actually be simpler than that; my current version of ModManage just reads the original .pak's, the new .pak's, and puts the latter into the former while still in RAM, without bothering to extract or compress (which both take a long time). Quite simple and very fast. Of course, if the files in the mods aren't compressed, the mod files can just be compressed individually, which should be relatively fast as well. I'm liking where this idea is going. Of course, I may need to change some of my code to work on data in RAM rather than on files, but this has been a long time coming, anyway.

Rather than have separate installations (Which I found to be confusing more often than not with GooTool -- I ended up with three. One original install, one modded one for GooTool, and one editing one for WooGLE), my current setup for LI modding is just one folder. In this folder I have the stripped Little Inferno executable, the four main pakfiles (and extracted subfolders), and one subfolder I have labeled "backuppak". In this subfolder, I have the original pakfiles from the game, which I can just copy over whenever anything breaks. Since all the game's original resources are just four files (three, really, since debug.pak isn't read by the game), it'd be very simple to manage without any fancy dual-installation footwork.

For the second "save changes" option: What I mean by hashing is taking the contents of a file and condensing them into a small amount of data. This is what Little Inferno already does with filenames; it condenses one long filename string into a single integer value that can be used as a resource ID, for ease of use by the game code. My idea is to hash an entire file; that way I'll know when a file changes, since the hash would change as a direct result (theoretically, it may not, but the chances are basically nil).
However, since I'm reading the entire file anyway to hash it, that may be slow. What I may end up doing is checking the "date modified" stamp on the files to see if anything changed since the last extract. That way if someone wants to just tweak one or two files, I can easily poll what files have changed, and only recompress those.

Whatever the case, I probably need to do some major code refactoring. so hooray.

puggsoy's picture
Offline
Joined: 11/20/2012 - 04:25

(For readability, I'm going to name the save options option A and option B. A is for when installing mods, B is for when editing the extracted files to make your own.)

Ah, I see. I'll have Matchbox create a folder with the backups itself on the very first extract (although probably in a more secure place, so people don't accidentally mess with or delete them).
The idea for the option A sounds great though. This should work perfectly for when just installing mods.

However, suppose someone were to install some mods with A, then later they edited the extracted files and used option B. These extracted files would contain whatever was within the .paks before the mods were installed. Would they then be added to the .paks, still with the merged mods, or should they be added to the original .paks? I think the latter might be best, if anybody wanted to look at or edit mods they could extract the .paks manually (there will be an option for this).

In any case, I'll wait to see what you can do with liTools. Depending on how much you can optimise it the whole situation might be able to work simpler than I'm seeing it. I don't want to make things more complicated than they have to be.

movildima's picture
Offline
Joined: 11/19/2012 - 01:13

'puggsoy' wrote:
I'm not sure if type will be necessary, in WoG this was needed because GooTool installed levels in a specific way so that they appeared in the top-left corner of Chapter 1.

Yes, I remember that GooTool treated "level" labeled mods in a special way. I thought it would be useful for categorization, yeah.

I don't think it would be a good idea to have two "save changes" in Matchbox. It would be really confusing. What you could instead is making Matchbox "user version" only to install mods, and another "power user" version with editor supplied(obviously when it will be made) used by mod creators.
What do you think?

puggsoy's picture
Offline
Joined: 11/20/2012 - 04:25

Hmm. A good idea. The second "save changes" would also be used just when editing files by hand though, before any editors are made.

I'll probably still keep it as one application, but I can make an "advanced" option. When this is enabled, options for modders will appear and they can use the special "save changes", and so forth. However by default, only the mod installing features will be visible.

CzlowiekDrzewo's picture
Offline
Joined: 12/12/2013 - 10:45

"Then, simply run

liCompress myMod.pak"

Could you explain that point better (or make a program where packing is easier). I made a txt file with directory to edited file but where do I get this 'liCompress myMod.pak'?

P.S I want to make a polish localisation.

Pages