Last update of 2012

Well, not much really. I've put the donate button back, because I am actively working on RPGMaker Trans now. Any bribery to speed me up is greatly appreciated.

Also I believe I've managed to solve all the fundamental problems I had previously encountered with patching XP/VX games. With a bit of luck, this will get rolled out early in 2013.

Hope people have had a good year, and let's hope for a good year to come.

About Trans-locale patching...

Long story short, it ain't working.

As to why, I'm at a bit of a loss. Evidently, there's something in the file somewhere which is still in cp932 locale, but as to what exactly, I don't know. So, for now, I'm suspending work on this feature. Apologies to all who wanted it, but there's at least one RPGMaker VX game coming out in February which I would like to see translated, so my current priority is to get VX support working.

However, one nice note on this subject: XP/VX use Unicode, and hence whilst it probably won't be possible to translate 2k games to non-latin based languages for the forseeable future, once XP/VX support is in those games will be translatable to any language.

UPDATE: I've managed to get a primitive VX dumper/(sort of) assembler going. Looks like XP/VX support should be done sooner rather than later.

RPGMaker Trans v1.99b

Not a big update, unfortunately, but an update to add a feature that was requested and I thought was already in.

• Unused strings are now in a fixed order, rather than the arbitrary order used before. This should make versioning using RPGMaker Trans easier.

I've been playing around with getting XP/VX support working, but nothing concrete is done yet. Partly this is because I don't know much about Ruby programming, but a lot of this is because the scripts out there for reading/writing RPGMaker files aren't written that well (heavy use of global variables, lack of safety checking, highly dependant on Windows platform, and using eval. Eval for crying out loud! gegerlan/rakudayo, you should be ashamed of yourselves - although I am still grateful for you giving me a decent starting point...).

Progress Report

Just figured I'd drop a progress report here...

1. Trans-Locale patching: Targetted for end of year.
2. RPGMaker XP/VX support: Currently investigating msgpack for Python/Ruby communication and learning some more Ruby - which is, annoyingly, pretty much required to actually unpack XP/VX files. 
3. ScriptableQuickAtlas: Learning QT, as my current UI toolkit, wxwidgets doesn't support some features I want on Windows.
4. Speed improvements: Investigating if it's possible to use PyPy instead of vanilla Python; currently struggling with packaging. If it is possible, I'll be able to offer a much, much faster version of RPGMaker Trans.

RPGMaker Trans 1.99 and news

So a new version. Not the one that brings trans-locale support, unfortunately, but just a bug fix and general update. Changelog:

• Fix a bug with translating names or classes set by scripts
• (Hopefully) Fix the bug with the error logger
• Move to Python 2.7 / wxPython 2.8.12.1

So, other news? Well, stuff I'm working on (on and off)

• Trans-locale support - or, in other words, translating games from say Japanese to Russian. Nearly done, but I need to reverse engineer a bit more of the LMU file format before this is done.
• XP/VX support - again, nearly done.
• ScriptableQuickAtlas - A demo of which has been around for a while (for a specific game), but will need some work due to GUI issues. Allows using Scripts to process text hooked by AGTH into something more palatable to ATLAS.
• Translation aid - A tool which aids translation by matching strings supplied by AGTH, and allowing the user to translate games whilst playing them (with ScriptableQuickAtlas support)
• Collaborative patches - Part of the above translation aid, but basically support for crowdsourcing translation work
• (Possibly) A new website

And one other thing... how did this place get popular? Since my last post, hits have gone up at least 10 times, and that was an announcement saying that nothing would be happening for a while.

Hiatus

Mostly due to circumstances beyond my control, I have to take an indefinite hiatus from RPGMaker Trans development. As such, until the hiatus ends, the only work that will be carried out on RPGMaker Trans will be bugfixes, should bugs arise.

Apologies to those waiting for trans-locale support.

RPGMaker Trans v1.98

Right, here is v1.98. It's undergone much more substantial testing than had been the case with the old 1.97, and I'm 99% confident that this one actually works properly. If I'm wrong, then please, bug reports either as comments here or directly e-mail me.

Changes:

• Speed: The speedy patching engine is correct and works properly.
• Speed: The translation folder is constructed in parallel to patching.
• Bugfix: Game folders with Unicode file names should be OK now... (although maybe not if opened using the browse button - this is outside of my ability to control, unfortunately...)
• New UI: OK, it looks like the old one, but this version is better for me as a programmer. Also, it's the cause for the aforementioned bugfix. Oh, and for now, the patch options button won't do anything. That's intentional.

Within the next two weeks, I intend to get a beta version of 1.99 out; this version will add support for trans-locale patches (i.e. Japanese to Korean/Russian). The only substantial item left to do is to add support for "translating" filenames, which requires a modicum of extra reverse engineering. Shouldn't be too hard... I hope...

Oh, and I'm doing a "cash for guaranteed work" thing. For every $10, I will do 1 hours work on RPGMaker Trans. People can specify what feature they'd like to see and I'll work on that. Current list of features under development: XP/VX support, automatic translations via Atlas and Trans-locale patching. Whilst I'm open to ideas, if you have some feature that isn't on this list that you want, please ask before committing funds to it. I'd hate for someone to donate for a feature which I can't implement...

And finally, I will be implementing all of those features anyway. Donations is just a way to try and get me to speed up.

Request

EDIT: It turns out that automatically patching an RPG_RT.EXE file will be impossible. Certain assets, notably the name character dialog, have to be translated, and this is well outside of what I think is possible for me to implement. There may be an alternative later on, but this will not be in any release for the forseeable future. Apologies.


So I haven't been completely idle this last month or so. And hence a request for those wanting to translate into foreign languages: I need RPG_RT.EXE files. Specifically, I need RPG_RT.EXE files from the locales which you want to translate into, so I can try to figure out how to automatically patch the locale of an RPG_RT.EXE file.


Technical details: an RPGMaker 2k is an old pre-UTF-8 program, so it can only display a single code page. Translating across code pages will require either replacing the RPG_RT.EXE file (legally difficult) or on the fly patching - and for on the fly patching, I'll need to know what to change.


Hence, if you can see this, want to translate RPGMaker 2k games to a different locale, and have an RPG_RT.EXE file that is from the correct locale, please send it to me!

Whoops...

I was browing a forum and found that v1.97 of RPGMaker Trans has a pretty major bug. It isn't a crashing bug (as some poeple have complained about here; until I get logs I'm assuming this is file name related), but rather it isn't translating item descriptions. I have no idea what is causing this as it only seems to affect the Windows build; it wasn't an issue when testing it on Linux.

Pending further investigation I've reinstated the older 1.95 version. If you're having trouble with item descriptions, please redownload it. Also if you have problems with an old download of 1.95 complaining about it being expired, you may also have to download this "newer" version of 1.95 which overrides the version check that causes the expired message.

Apologies for all this.

Incommunicado

Just a short update: normally I'm pretty responsive about answering questions etc. However, my smartphone has just died - and hence I'm not getting notifications of questions here at all quickly.

So if I'm late answering a question, it's not because I'm neglecting my userbase or abandoning the project, it's more likely that I just don't know there's a problem. I'll try and get responses within a reasonable time, but don't be surprised if it takes a few days.

RPGMaker Trans v1.97 Released

OK, so this release represents most of my work in the "speedy" line. It isn't all of my work, because I've only dropped the speedy patching engine into the old framework; there's other improvements I'm making in the framework which should make things faster, but they're not ready and I don't want to delay getting this new release out any more.

I also have an apology to the Korean team who wanted to use RPGMaker Trans: testing revealed another problem when changing the locale of the game, specifically that files named in Japanese can no longer be used - with this breaking most of my test games. Automatic file renaming cannot be handled by the current framework, so multi-locale support will have to wait until the next version. Sorry.

Changes

• Patching engine is much faster
• Now able to translate "Set Hero Class" commands
• Pseudo-bug: Events associated with Monster Parties have migrated to the Common Events file. This is due to problems with the current patch format and the new patching engine.

Next Version due by end of month (hopefully - though it seems I'm not good at timescales). Features will be:

• More speed!
• Support for trans-locale (i.e. Japanese to Korean) patching 
• Trying to detect if a game has been unpacked under the wrong locale

Also: on the file hosting situation, it seems that my previous file hosts are well... as far as my auto-upload script is concerned, dead. Defaulting to Mediafire for now,  but liable to change.

Double also: I've set up a donations button, as it was suggested a while back in the comments. As the disclaimer says, donations won't necessarily make me work any faster (i.e. they're old school donations, given charitably, with no expectation of return). This being said, sufficient donations may let me get some private hosting (if just for files and not a full website), which would be nice.

Dangit...

So I couldn't quite keep meet my target...

Basically, lack of time and a few problematic bugs stopped me. I'm finally at the stage where the parser/assembler works without issue, but it isn't yet integrated into the translator or GUI - and nor are the changes to these two parts finished yet either.

I'm very disappointed that I couldn't make it in time for an ironic April fools release though...

Well, new version within the week, I suspect.

So, about that dedicated website...

Well, I've been investigating stuff, and turns out things aren't perhaps as rosy as I thought.

Turns out things are more complex than my initial survey of hosting providers suggested. Long story short, to actually run a website I'd need about $10 a month all told. So, whilst I've very grateful that just over half the respondents to the little survey said they would be prepared to fork out some money to move RPGMaker Trans onto a dedicated site, the finances don't work out.

Damn, that's a shame. Oh well, maybe if I can get some other projects going I'll be able to work something out... but for now, a dedicated site will remain a pipe dream.

Target is for the new version of RPGMaker Trans, with speedy 2k support, to be out by the end of the month. Then I'll move onto either automatic machine translation or XP/VX support.

New Features and Announcing a New Toy

Well, after publishing my roadmap, and making some progress with it, I decided to essentially ignore it and do something else. So, here I'll document exactly what I did with the only time I had to work on RPGMaker Trans recently.

I made a Python binding onto Fujitsu's ATLAS software.

Now, anyone with two brain cells should be able to work out that this means I intend to bring automatic machine translation directly into RPGMaker Trans.

To be clear, I'm not sure when this feature will land. It still isn't on my Roadmap, although logically it shouldn't be that difficult to implement - at least in a primitive form. I'm looking to "do things right" though and enable the creation of some small scripting component which allows scripting around the types of things that ATLAS doesn't translate well. For example, for some games which "announce" character names, ATLAS will decide to merge the character name into the characters dialogue with bad consequences.

But this type of scriptable translation could also be useful for other games, for pretty much the same reason. Hence I'll also be working on a "Scriptable QuickATLAS" style program - name to be decided. There's probably quite a few uses for such a tool, like working around "triangle-ised" text from an AGTH hook (you'll know it if you've seen it), other hook based nonsense, or just generally improving ATLAS translations.

Again, no ETA on this.

Oh, and before I forget, the Python/ATLAS bindings will be released separately under a Public Domain license, once I do a little stress testing on them to make sure they don't fall apart under non-trivial workloads.

And finally, before anyone points it out: I am aware of Vkozyrev and his machine translator for RPGMaker Trans patches. I think, however, that using ATLAS has substantial benefits over his solution of online translators, partly because of speed but also because a lot of people use ATLAS, and hence a lot of people are accustomed to ATLAS-speak. Also, I tend to find that ATLAS with a good custom dictionary simply works better than the online translators. Finally, by integrating machine translation into RPGMaker Trans, Machine Translation becomes much more available (i.e. I'm pretty confident that not many people actually download Vkozyrevs machine translator, otherwise I think we'd be seeing a few more actual patches/pirated games based on it online rather than just the ones he maintains).

Aftermath

Well, Megaupload died. Everyone should know about it by now.

Problem though is that the cyberlockers are now running in fear. Filesonic is no longer offering any services - problem as they're one of my mirrors. Also uploaded.to is blocking the US - and whilst I don't use them, considering that blocking the US is probably a correct way to remove yourself from the jurisdiction of the FBI, I can see other cyberlockers following suit.

Now small rant upcoming (don't want to read, skip to the next paragraph): I think the US exceeded its jurisdiction (getting a German national arrested in New Zealand seems a little over reaching). However, it seems Kim Dotcom (probably - pending trial to establish facts) deserved it. By all accounts it does seem that Megaupload was not responding to requests by copyright holders to remove files, so he was (probably) one of the "bad guys" in the copyright debate. I'm definitely against piracy, as I am a content creator under other names, and if the reports are true on Megaupload then it probably deserved to be shutdown. But arresting a non-national in a foreign jurisdiction seems excessive. It isn't as if New Zealand doesn't have anti-piracy rules, and there shouldn't have been any problem in arresting and trialling him in either New Zealand or Germany.

Needless, regardless of any rants or whatever, everyone has to deal with the aftermath. It seems pretty likely that at some point in the future most cyberlockers will become unreliable (perhaps Mediafire/Rapidshare will stay above board, but the rest, I doubt it). Hence I'm looking to perhaps take RPGMaker Trans onto private hosting. The catch is that this costs money, and I'm pretty unwilling to actually spend money on distributing RPGMaker Trans. That said, private hosting would enable a few neat things to go ahead that up till now haven't been possible.

So I'm putting out a poll on if people would donate to the RPGMaker Trans project. If there's enough people backing me, I'll look to setting up a proper website for RPGMaker Trans and whatever other projects I make. I'm not looking for a huge amount of money - on low end VPS's the price would probably be about $50 a year to cover my costs. So basically, if I get say about 20-30 backers, there'll be a dedicated website. If not.... ah well. What could have been, I suppose.

Progress

And so the new speedy parser is done. Well, done enough that I can get useful performance data out of it. Technically, it needs integrating into the translation engine, but it works pretty well. Also LMU files, but that should be less than 20 minutes work once everything else is ready.

So, the summary of improvements: Much faster (10 times, at least), and now picks out "Change Hero Class Name" (thanks shadepariah for pointing this out and providing a sample of it). Also, (untested) should pick out if a face is being displayed on the dialog - this is useful for translators as a face decreases the amount of space available for text. Finally, the script parser should be much more robust than it was before.

Also note: as this parser uses a different internal representation, there won't be a release until a new serialiser has been written.

The rest of this post is a programming type post, for people interested in the intricacies of programming high performance Python. OK, a lot of this will also apply to many other languages, especially interpreted ones, but mainly Python. If you're interested, look after the break.