Esoteric Update #295 - ən ʌpˈdeɪt

Alright, it's time for the last update before Mithramas. I'd like to start by saying that I have some things with my family planned and that the next update will not happen before the 30th, just ahead of New Year's.

Now for this update. Well, there has been a decent amount of work done across a few different topics, some of which are technical.

So, let's start with the adventure. That's the simplest one. I've worked on implementing the new approach I talked about last time. An issue to look into was data. Fortunately, it turned out a lot was already there, and I only needed to add some information. This should now give me most of what I need to implement the design as it is currently planned out. So, I'll be working on the implementation and the writing for it now.

I've also merged the new writing implement code with the old writing style concept. Here are some comparisons:

In both cases, the top is the old handwriting code, and the bottom is the new one. The first image is a pen, and the second is a ballpoint. I hope it's visible that the new code handles the whole issue much better and the writing more strongly represents the implement used, whereas the top one remains self-similar.

It's also possible to notice that in the examples, some of the loops on letters get "eaten", but this can be solved by tweaking the parameters, like so:

I've also worked on some image processing code. A lot of it was just tweaks and bug fixes, but I added a new operator for colourising black and white images, which is critical for handling handwriting. I'm also looking into implementing (Extended) Difference of Gaussians, but I'm unsure if I want to go through with it. It's a complex bit of work, and the gain might not be that worth it. Anyway, this is part of the technical stuff, and it's not very interesting, so let's move on. Additionally, you've already seen an example of the code improvements in the three images above! Yes, it's not very obvious what changes there, but some things do, and it's crucial that they indeed change.

Finally, Bear (with my help, a little bit) has been experimenting with a rather strange concept. Essentially, I made a somewhat experimental dictionary-based filter that takes the Britfone dictionary and transforms it from text to an IPA representation. This might not have obvious utility because, obviously, most people don't fluently read or understand IPA, but that's not the point.

Here's a look at the transformation to IPA and then a similar one using the closest Latin equivalents:

So, what is the point of this? Firstly, the above presentation pushes the limits of the filter system, boasting a whopping 15000+ automatically generated rules. Being able to finally test the system's limits was, in its own right, an interesting experience. But there's a bit more to it. I made an offhand comment about how a bespoke tokenisation system with hash dictionaries would be much better, and Bear took it to heart and wrote one. We're not yet sure how fast it is, but those 15000+ rules just breeze past even for large bodies of text. There are some ideas we have for how it can be used for text analysis (consistency checking), building various dictionary-based transformations (like better idiolects, British-to/from-American) and solving ambiguity issues (homograph detection). This is still pretty experimental, though, and under Bear's care, who is free to work on it if and when he feels like it, so we'll see where it gets us.

Ah, right, I almost forgot; I also implemented Bear's solution from the Python library into the JS library, allowing for more seamless interoperation. I also fixed returning arrays from JS into our scripting language, which was previously a problem while I was working on it. It's just another one of those technical things.

Anyway, I guess that because this is my last post before the day our saviour, the Sun, was delivered to us from the North Pole by Saint Nicholas via express shipping... happy Crimbus.