History of the Ruby Parser

So, after my last rant, something a little more positive. Anyone whose following the progress of RPGMaker Trans commits (a total of zero people, I'm sure) will know that I''ve been tinkering around with a Ruby parser. Or to be fair, this is actually quite a bit short of a proper parser, as it only needs to extract strings, but I'm calling it a parser for now.

Basically, I've not posted on progress on the Ruby parser because of... well, not using this blog for blogging really. So, for posterity, the current Ruby parser that I'm working on is the 5th generation version. A quick summary of the first four:

1. (Late 2012) (version in game limited beta VX patcher) Simple Ruby based parser, defeated by a multiline string.
2. (Early 2013) Version based on RubyLexer, defeated by bugs in RubyLexer which probably have no chance of being fixed, and having very unhelpful descriptions (e.g. "appear to be off by 1 sometimes" ... sometimes? Seriously?)
3. (Summer 2013) Version based on RubyRipper: Promising, until I realised that Ripper is only compatible with Ruby 1.9; VX uses Ruby 1.8, and Ruby doesn't really do backwards compatibility. There is an unofficial port to 1.8, but I could never get it to compile as it requires an older version of YACC, but they don't specify which version of YACC is required.
4. (Late 2013/Early 2014) Complex Ruby parser: defeated by realisation that Ruby 1.8 struggles with encoded files, that Ruby 1.8 can be in one of four encodings, and that I'm really not a good enough Ruby programmer to do it.

The current version is written in Python, like most of RPGMaker Trans. Whilst it is still very incomplete, I actually have a plan for most things and the (most) worrisome features are pretty much all implemented. Or at least, the most worrisome features that I know about, given my somewhat limited knowledge of Ruby...

In any case, watch this space... 5th time lucky and all that.

Ruby Custom String Delimiters

So, as I'm in the process of transitioning to Bitbucket, which will eventually entail a proper website-ish thing, I'll actually be posting some more developer style stuff here.

First subject: Ruby Custom String Delimiters. This is a feature of Ruby which just shouldn't be allowed. Basically, Ruby claims it can substitute an arbitrary symbol for a string delimiter, by prefixing the delimiter with %q or %Q (depending on if you want a double or single quoted string, which are different in Ruby). Also using a symbol which has a matching closing symbol will cause the closing symbol to be the end of the string (e.g. using [ causes ] to end the string).

Got that? Well, here's some counter-intuitive stuff you have to deal with when trying to parse the damn thing, when playing around with IRB and Ruby 1.8:

•  %q"Hello\n" == "Hello\n" => False (so it's insufficient to look at quotation mark to determine type of string)
• %q][] does not end the string
• %q][]] is a syntax error
• %q£Hello£ quits IRB  

I just can't conceive of a circumstance where this feature is actually useful, outside of obfuscated code competitions. I've programmed in a fair few languages, and I've never once thought this would be useful enough that it outweighs making the code harder to read. Even if it's inherited  from another language, this is a good example of why I don't like Ruby: too much flexibility in the language can be counterproductive to understanding the language.

RPGMaker Trans v2.06 released, now open source

Hey, I actually managed to keep one of my self set arbitrary deadlines. RPGMaker Trans is now open source, and hosted on Bitbucket. So, noticeable changes:

1. Bug reports should now go on the Bitbucket issue tracker - much more useful, and it has the same sign in requirements as the blog, so not any harder to use (well, probably).
2. Downloads are now served by Bitbucket instead of Dropbox.
3. RPGMaker Trans now has an icon.
4. RPGMaker Trans license changed to GPLv3.
5. Technically, RPGMaker Trans is now fully cross platform. To run on another OS, you need Python 3, PySide and BeautifulSoup4, and then run rpgmakertransmain.py in the source. The value of being cross platform is questionable, given that RPGMaker is only on Windows.
6. If running RPGMaker Trans from the source distribution (emphasis: not the binary download), you can now try the undocumented and probably untested CLI (command line interface; useful for people who want to script the translation process). Start RPGMaker Trans with the --cli option and you'll get help on what the parameters should be. This will be added to the readme soonish, but I only just remembered about this, and I'm not going to put off launch to document it.
7. And finally, the blog gets a new theme. It's a suitably momentous occasion.

 Enjoy. I've still got a few more things to push into the repository (e.g. my initial attempts at VX support), but this should be enough for a good enough release.

EDIT: I initially put the wrong binary on the website. If you're one of the 10 who downloaded it and get the error "TypeError: Type str doesn't support the buffer API", please redownload.

Starting to move to open source

Well, it's been a long time coming, but soon (hopefully within the next month or so), RPGMaker Trans will be open source. Good new for those who care about this sort of thing, which probably isn't many of you.

There's been a lot of issues moving to open source; the biggest practical one was that early on I had some help from a real-life friend of mine who did not (and still does not) want to be identified. Practically, that meant that I wasn't able to release his code as open source, and without that, RPGMaker Trans would mostly not work. However, I've recently got in touch with him and he is now satisfied that his code is no longer present in the current RPGMaker Trans codebase, and hence does not object to it being open sourced.

So the current stuff that needs doing before open sourcing:

1. Documentation: Full documentation won't be available until some way down the line, but at least partial documentation needs to be there if open sourcing is to have any value. Module level documentation is now done, and for the first open source release this will do.
2. Deobfuscation: Some code is... slightly obfuscated. No real reason for it, but it needs to be fixed.
3. General tidy-up: I don't think I'll be following pep8 too strictly (for those who know it), but the code is really ugly in places. A quick spring clean would be nice. I ran autopep8 over the code and I think I'll call it a day. Some pylint based stuff will happen later, but not now.
4. License: I've not yet decided on the license. I'm leaning towards GPLv3, but I've not yet decided. Decided that the license will be GPLv3.
5. Project Logo: I need one of these. I used to have an icon for RPGMaker Trans, but it was rubbish... I've got a logo now. Still pretty bad, but honestly, there's only so many ideas for a 'logo for translation software'

 And for posterity, a list of things that have already been done:

1. Removing code that cannot be distributed openly
2. Migrate from Bazaar to Mercurial for version control
3. Setting up hosting at Bitbucket
4. Saner source organisation

Please don't direct link to Dropbox files

Reasons? Basically, if I release a new version, your link becomes out of date and it's annoying to get share requests for files which no longer exist. Couple that with the fact that the update checker rather means that anyone downloading a direct link would have to come to this page and get the new version anyway... and yeah, direct links aren't so good.

So if you want to share stuff RPGMaker Trans, great, but link to this page please.

RPGMaker Trans v2.05 Released

A minor release. Changes:

• Fixed a crash with opening a zip file where the patch was not included in a subdirectory
• Crash reporter now has a button which copies the traceback to clipboard and opens up the bug reports page

Why the slow progress? Well, I've actually been looking around at RPGMaker Trans usage, and every time I see people redistributing prepatched stuff, sometimes in direct contravention of the original authors wishes, I lose motivation for this project. I'm aware that in making a tool such as this I can't really stop it from happening, but it's still frustrating.

Bug Reports page

So I've added a special page for bug reports, to try and get some order into this site. Link is just below the download link, so hopefully it's noticeable. There's some instructions on the bug reports page about information that is useful for me to fix bugs.