Current state of things...

This is just a post to say "I'm not dead". True, I've been working on RPGMaker Trans a lot slower than usual, but I'm not dead. So, with that, here's a little breakdown on what's been going on:

1: Bugfixes

I've found the cause of the bug involving the GUI build and non-ASCII directory names; a bugfix for this should be out soon. There also seems to be a more obscure bug floating around regarding conflicts with existing Python installs - I'm not sure what's going on there, and can't reproduce it. There may be an attempt to fix this, but I'm not sure.

2: Improvements

I've got to say: most of the parsing engine that's presently in use is pretty old code - and a lot of it is from when I was trying to figure out the RPGMaker 2k format. Whilst it's quite easy to work with from the reverse engineering point of view, as I'm sure everyone will have noticed, it isn't fast. I'm presently working on a new parser which is much, much faster - though only about 70% complete so far. It will also need a new rebuilder as well, but this should be fairly straightforward to make once the new parser is done. So expect a speed up in the near future.

3: XP/VX support

The big one, really. I'm still working on memory usage with this; there's some complex issues with regard to how Windows handles pipes (poorly) and Ruby 1.8s memory model (bad). Net result is that when I tried my current build of XP support on Windows, I used all 6GBs of memory before it blew up. I'm working on a new system which should solve the memory issue. So support for newer games is still on the menu, but will take a while.

Warning

It seems that some people are distributing prepatched copies of a certain game. I'm sure everyone involved knows who they are.

This is not allowed. There's a warning prominently displayed on RPGMaker Trans about this.

For all parties involved: Strike 1. 3 Strikes, and I'm afraid bad things will have to happen.

Edit (mk2):
Apologies to Anon4565636: Anon4565636 reposted a pretty offensive comment made on RPGMaker Trans elsewhere and I mistook it for his/her own post. The only thing I'm keeping from my original response though is an answer to the allegation that I am building a botnet (rest from previous response):

Finally, on the serious allegation of me trying to build a botnet: Not a chance. First off, RPGMaker Trans is written in Python - wrong language for a computer virus. Any competent reverse engineer should be able to verify that it is just a Python script packaged with cx_freeze - no funny business. Whenever I referred to "consequences" in the past, it was with reference to either Whitelists or Blacklists being implemented within RPGMaker Trans, but I decided against this as it would be unfair against the majority of legitimate users. At the moment though, consequences means more along the lines of "certain game developers are threatening to upload a 'prepatched' build with a virus in, to teach people who go against their wishes a lesson".

RPGMaker Trans v1.95 released

EDIT: v1.95a is now done and uploaded (delays due to file hosts being awkward). Sorry for any inconvenience.

Well, that was more problematic than it should have been.

Features

• Full GUI: No more text box
• ZIP patches changed: Now the folder inside the zip file is irrelevant; things should just work
• Log files always generated when crashing: If RPGMaker Trans crashes, it should correctly generate log files without entering debug mode.

However, there's a caveat: I had substantial difficulties in compiling RPGMaker Trans to an exe. In the end I changed from cx_freeze (which wasn't working) to bbfreeze to do this; however when I did so Norton Antivirus decided RPGMaker Trans was suspicious.


At present I don't have the time to find out how to fix cx_freeze - which is my goal in the long run. In the meantime, if your antivirus decides that RPGMaker Trans is suspicious, kindly ignore it; it's just responding to the way that RPGMaker Trans spawns multiple processes to utilise all available cores. If ignoring the AV isn't an option, I'm leaving the older version around for now so people can use that until this gets fixed.

I fixed the problem with my original build system; Norton doesn't seem to have a problem now, so the above is a non-issue.

And finally, the other caveat: Some patches that are floating around may not be detected by this version of RPGMaker Trans. This version looks for RPGMKTRANSPATCH files in Zips/folders etc. and some patches that are floating around don't have these. If this is the case for you, either try getting a new version of the patch or prod the person maintaining it.

Progress

So, it turns out that coding whilst on holiday is hard. Especially when your holiday isn't what most people would call a holiday... I would also like to register my hatred of mosquitos.

Anyways, progress so far is basically fixing those crazy error messages and a few GUI faults. It's pretty much ready to go, actually, and would be apart from the fact that the way the patcher is being used by Vkozyrev (comments here under the handle VK) effectively breaks version portability. Normally I'd tell him to fix things on his end, but the specific behaviour (that the directory inside a zipped patch needs to be the same name as the zip file) is not intuitive and really something of an oversight. Unfortunately, it gets worse with the new GUI.

Hence, I'm taking the opportunity to rewrite my ZIP file handler. Presently it's less than ideal (it actually extracts the zip, calls the normal handler, then deletes the folder when done), so the new one will (hopefully) just read stuff out of the ZIP. Anyways, the new ETA is before the end of the week.

Delayed: v1.95

I'm afraid I'm having some problems with getting RPGMaker Trans v1.95 out. My normal build system isn't working well, and compiling it by hand on Windows throws up some pretty crazy error messages.

Basically, it needs more time, and as I'm on a holiday from today I really don't know how much free time I'm going to get. I'll see what I can do (especially on the plane out there), and I'll be trying to get it out as soon as possible.

Sorry for anyone waiting on it, but you'll have to wait a little longer.

Inbound: v1.95

Heads up to people who I've told "wait for version 1.95, it'll be easier to use". Well, wait is nearly over. Just needs a little more testing and packaging, then it'll be here.

ETA: 29th July.

Features: Log files generated without explicitly setting debug mode, a proper GUI.

Big fat public warning for 3rd party tools...

If you're just a user of RPGMaker Trans, please ignore this message. Translators can look forward to a proper GUI and stuff. However, developers... please click through and read on.

GUI Work...

Not too much done on this yet, but future versions of RPGMaker Trans will look something like this:

Plan is that you'll be able to select games from the dropdowns if there in the places that RPGMaker Trans expects them to be in now, or find them with other otherwise. Patches likewise, and if RPGMaker Trans can find the appropriate patch automatically it'll be auto selected.

And yes, the GUI might come earlier rather than later.

RPGMaker Trans Roadmap

EDIT: All dates in that were in this roadmap are now wrong, so I removed them. XP/VX support will be more involved than I thought, and will likely involve creation of an editor GUI program. The GUI version is likely to happen before the speedy version, and is also likely to happen soon.

So, as there were... well, no contributions on future direction... here's the plan and some expected deadline dates. Note that the roadmap is of course subject to change, and extra releases will be made in case of bugs etc.

RPGMaker Trans v1.91-s

• Not a true new version, so no new version bump
• Debut of RPGMaker Trans utilities - RPGMaker Trans will be split into two parts, a small "core program" and a large "utilities pack" which doesn't change as much. RPGMaker 2000 game translation will be possible without the utilities pack, but will be much faster with it. The planned RPGMaker XP/VX support will require the utilities pack
• Utilities pack contains pypy for speed ups.
• Remove some unnecessary code paths for some other speed ups. Probably won't make too much difference, but every little helps.

RPGMaker Trans v1.95

• Add a wxWidgets based GUI
• Add a developer CLI mode - for people building utilities on top of RPGMaker Trans

RPGMaker Trans v2.00

• RPGMaker XP+VX support!
• RPGMaker Trans patch format v3, to cope with XP/VX support
• Utilities pack update to support XP/VX games.

RPGMaker Trans v2.1

• Automatic machine translation support (hopeful translators: ATLAS, Google Translate)
• Possible Utilities pack update to support ATLAS translations

Other

• Google Apps based online translation/collaboration software 

CPython and PyPy: Small and Slow or Big and Fast

This is a heads up for all interested parties.

Presently, RPGMaker Trans is built on CPython, or the bog standard Python distribution. The reason for this is that when I was writing it, CPython was the only Python distribution which had all the Python 2.7 language features in it, and I wanted to use some of them.

Now the situation has changed; PyPy is now fully 2.7 compliant. Playing around with running the source through PyPy I can get substantial speedups - at least on my machine. Given that RPGMaker Trans is a slow program I'm thinking it may be a good idea to look at taking advantage of these speedups.

There's two slight problems I've got at the moment: the first is that PyPy doesn't support freezing - that is creating a special EXE file to run. There's a few ways around this - I'm leaning towards making a basic loader program that launches things. Heck, it might even be a good time to add a proper GUI and proper CLI if I go down that route. The second issue is download size; PyPy is a big binary - something like 27MB. So... perhaps a two version system (small download size but slow, and big download size but fast)?

And for reference, the RPGMaker XP code I'm working on does benefit substantially from PyPy - to the extent that it is a lot more usable. It really may be necessary to switch to a PyPy based system because of this.

If there are any thoughts, I'd be happy to hear them. In fact, I'd really welcome some feedback on where people stand on the small download size and slow or big download size and fast part!

Tweaking the blog...

Well, a few tweaks to the blog. First off, ads have disappeared. Why? Well, three reasons: the first is that by the browser stats, I guess about 70% of people who view this blog have ad blocking. The second is that the majority of ads I get seem to be about enterprise patch management, and that's not exactly anything that people here want. The third is that really, I don't get too much traffic anyway. Those factors mean that obtaining any support for this project via ads is not really feasible.

So my new strategem is to recommend people to download RPGMaker Trans via Easy-Share. The reason for Easy-Share is that all other file hosts pay moneys based on the size of file to be downloaded, and RPGMaker Trans is too small to qualify for any payouts. Easy-Share doesn't take file size into consideration, and so provides a lot more revenue. I suppose I could pad the files or something... but that seems like a bad idea really...

Note that I'm specifically not recommending people to take out premium memberships with Easy-share - unless you really want to. Doing so isn't a way of supporting me however.

But yeah, it'd be nice if everyone could download via Easy-share. I appreciate that for some people it's not possible, and for others they just prefer a different file host, but if everyone downloaded via Easy-Share I'd have gotten $10. Not much, but enough for me to put a donation to some of the 3rd party software that made RPGMaker Trans possible.

EDIT: And now Easy-Share do penalise smaller files. I'd still prefer people to download from them, as the penalties aren't as much as the other hosts, but still annoying.

RPGMaker Trans v1.91

Huzzah! A new version, though this is strictly a bugfix version.

Bugs quashed:

1. Added an undocumented event code that some games use: Thanks to a bug report, I now know that at least one game uses an undocumented event code, E71A. I have no idea what it does, but it's existence was causing the script parser to trip up and die. In any case it's been added to my list of event codes, and the game in question now dumps an extra megabyte of text (or there abouts).
2. Converting RPGMaker Trans format 1.0 to 2.0 should now work properly: Specifically, the LMU files will not import correctly. I don't think anyone actually had a problem with this...
3. Don't dump null strings: Strings which are just a null character are now no longer dumped.

Anyways, have fun with it people.

Anonymous comments are now off

Just a quick note on this. The reason is simple: It's getting really confusing.

If you don't want to link a Google account here for whatever reason (this blog is family friendly, you know!), it's easy enough to just get a spare.

RPGMaker Trans v1.9

This version has been superseded by RPGMaker Trans v1.91

Next version features...

Just a heads up on what the next version will bring: an "Advice" field in the translation files, which should make a translators job somewhat easier - and incidentally bring the new patch file format up to feature parity with the old. It'll also be using the new filehosts.

Also, as you can no doubt see, I've managed to get the header with the download links to look about right.

And finally, even if it's somewhat late: Please consider donating to the Japanese Red Cross, or any other organisation trying to help with the disaster in Japan. They really need the help right now, because in addition to the massive natural disaster, Japan is quite an underinsured country, and of those who have insurance only about half had insurance against a tsunami. A lot of people have lost everything, and aren't going to be getting anything other than whatever kindness others show them.

New file hosts

EDIT: I've finished doing evaluation etc. Automation is still in progress, but the only real challenge presently is Freakshare, which may be more trouble than it's worth. Hence, future releases of RPGMaker Trans will be distributed via Easy-share, Fileserve and Filesonic. Freakshare may be added to this list if automation is practical.

RPGMaker Trans v1.81 Release

This release has been superseded by RPGMaker Trans v1.9.

RPGMaker Trans v1.8 Release

This release has been superseded by RPGMaker Trans v1.81

Stupid damn ISPs...

I'm ready to release stuff.

Unfortunately, I've just found out that the way I was bypassing the IWF proxy (something that the UK has to deal with; overall there's very good reasons for it, but some implementations are problematic) on the file sharing sites I was planning to use no longer works. This results in an upload speed that's being reported in bytes per second, which means I'll have to wait far too long to upload anything, really.

So basically, there'll be a release as soon as I can actually do it. When I get a chance, I'll started poking about with other file hosts, or failing that, other Internet connections.

I would like to apologise to everyone who was waiting patiently, but this is really out of my hands.

Delays...

Annoyingly, things haven't quite gone to plan. There's two main contributing factors to this; first off is that the fix for the security issue is a little hard to actually program (I know what I need to do, but just haven't been able to do it yet). The second is that today a lot of the time I had planned to allocate to this project got consumed by some mandatory DIY I had to do.

So basically, I've got to push the project back a couple of days. New target date is 3rd March, but hopefully sooner.

More features, but possibly controversial

As you'll probably note, there's now a RPGMaker Trans "latest version" thingy at the top of the page, in preparation for a launch soon. Hopefully I'll be able to make the style look a bit better at some point, but that's not important presently.

The latest version part is a little scheme I've cooked up to try and minimise support headaches for me. I have my suspicions that some problems that people had with older versions of RPGMaker Trans was because they used outdated versions. Hence new versions of RPGMaker Trans will check to see if updates are available by looking up this blog - this is one of the other advantages of moving off Hongfire. If an update is available, then RPGMaker Trans will notify the user and will not run until the update is applied. Or as far as that is possible; in practice a user can get around this by simply deleting or editing the configuration file, or blocking RPGMaker Trans with a firewall. There will also be a clause in the RPGMaker Trans license which means that users are only authorised to run the latest version of RPGMaker Trans, so hopefully ethical people will heed the new rules.

On the subject of license changes, I've redone the redistribution clause. The new clause actually works from a legal point of view (the old one did not specify the license that people with a redistributed copy should use, and so did not work), but also there is a requirement to get permission from me first. This is mainly so that I can have a list of people to e-mail updates to, again to minimise support headaches.

... Actually, when I say support headaches, I also mean "serious security flaws". There is a concerning attack vector in older versions of RPGMaker Trans, given it's usage case. Potentially,  it is possible to create a game which can execute pretty much any code when fed through RPGMaker Trans; this could be of concern to unauthorised translations when the creator really strenuously objects, but also if there's just some malicious person wanting to expand a botnet or something. Whilst I've got the solution to this hole planned out and nearly implemented, I think it's wise to get some form of centralisation so I can get updates out in a timely fashion, in case there's some other problem.

And for reference, I'm not being spiteful and making up this security hole because I object to other people sharing the present version of RPGMaker Trans, despite my request not to. I'm not going to be disclosing the problem quite yet (as it would be irresponsible of me to do so when users have no way of fixing the problem), but there will be a full disclosure of the problem once the new version is out.

EDIT: As an addendum, patch files can also be used to exploit the flaw (thanks for the question, Matt). I mentioned game files simply because there is a translation project going on which I'm pretty certain isn't approved of by the creators, and so I think this is perhaps the more likely place for the attack to happen from.

Preliminary launch date

Turns out other projects occupied more of my time than previously thought, but the new version of RPGMaker Trans is nearly ready. The new patch format is coming along nicely - the changes are as follows:

• UTF-8 rather than UTF-16: Reduces file size
• Changing the method of string identification: Previously, a string was identified by the string itself. Now it is identified by the string and a context (e.g. Hero Name, Item Name, etc). The context is shown in the translation file which provides extra info to the translator. It also means that, for example if a character and an item shared a name in Japanese, they no longer match meaning both can have a different translation.
• Currently working on an Advice field which can store for instance, a character limit.

The bad news is that RPGMaker XP support might have to wait a little. I haven't had a chance to complete or test it, so it's still sitting at 50% done.

 Anyhow, the preliminary launch date is 27th February. I'm hoping to launch with XP support, but I can't guarantee that.

First Post

This blog is just a place for me to post some misc programs and stuff.

The first program to be released will be a new version of RPGMaker Trans, when it is finished. This was originally being released through Hongfire, but doing this seemed to give some members of that community the impression that the license could be freely ignored (or at least, that's the impression I got from various flames that came in my general direction) and that my attempts to get people to honour the license / not encourage breaking the license were some kind of capital offence. Part of this might be because some individuals have some misguided sense of community there, as in a sense that everyone has the same values - which seems to be fairly low with regard to respecting copyright, authors wishes etc - something which I actually try to do!

Hence this blogs main purpose is to put distance between my work and Hongfire. These tools are NOT meant as a contribution to the Hongfire community. They are something I created in my own free time, and have released so that other people may use them.

There are also other reasons which are probably better reasons: releasing stuff over at Hongfire does reduce the audience of these tools and editting the posts on Hongfire was becoming a pain, as well as worrying about the thread bumping rules - tricky to manage when the release thread doesn't get many posts.

So, with the gloomy stuff on why this blog exists over, the teaser for new features in RPGMaker Trans 2.0

• Multiprocessing - provides a big speed boost on most machines (Implemented)
• New patch file format - Easier for translators to work with, and necessary for a plan I'm working on (auto insertion of a machine translated script)
• RPGMaker XP support (50% complete)
• Various bug fixes

The date for this release is not set in stone. I'd be surprised if it's later than the end of this month, probably more like a week and a half.