Maps and Dev

Place where I am mapping my journey into mapmaking and some development stuff

[0770man]

Trying the elvish stamps on Inkarnate

[0770man]

Same island region, instead of drawing paths to signal more frequented paths, I added fog and clouds to unknown or less traversed regions

[0770man]

Island region, shades of blue have a meaning of more transit between zones and islands. I gained more experience working with layers

[0770man]

Real-world map created from official INEGI source used as a texture. 

[0770man]

Small country or region map, with borders, political divisions, and towns 

[0770man]

Small town as setting for some magical realism tales, or a small campaign. Town is enclosed by magical forest. 

[0770man]

Small desecrated and abandoned temple. Might work as an encounter map against cultists, a necromancer or Call of Cthulhu.

[0770man]

Map of the geographical elements done in Inkarnate, as an exercise for getting into map making. 

[0770man]

Natural language processing notes 2, tokenization

Tokenization is the natural language processing task where we separate each word and punctuation mark or token inside a text, it is usually considered a pre-processing step along with sentence splitting, normalization and lemmatization.

In the western languages like English or Spanish tokenization is usually done splitting on whitespace, and it works most of the time, except on the case of hyphenization New York-based and enclytics  she’s all that.

The hyphenization case

Here we have to distinguish between hyphens occurring in words like Stratford-upon-Avon and NYC-based. The simplest answer to this problem is to look for the full word in a database, if that fails (the word is not found), we have to split on the hyphen and look each of the two words if the second one is a verb, we can then tokenize the full word and then probably tag it as an adjective.

In the case of enclytics the simplest way is to keep the word together and use the Brown tag set for the tagging step, the Brown Corpus’s tag set is very complete and allows for apostrophes in words, later, this huge tagset can be simplified, but for part of speech tagging purposes attributing a complex tag and simplifying later is a sure way to go and capture meaning more or less correctly.

[0770man]

Why grep is awesome

UNIX and linux grep is the single best tool ever developed in the history of mankind, finding word patterns and regular expressions in text files in a line by line basis, it is one of the greatest inventions for information processing and filtering ever incorporated into the UNIX and linux family of operating systems.

As user knieveltech posted  a few years ago in ycombinator:

Grep is about the single best fucking thing ever written by the hands of man. Shakespeare can suck it, I'm telling you, grep is IT.

Why?

Because sane systems (and sane programmers and users) manage the information generated by programs and users in plain text files or, lacking that there is always a tool that can convert or dump an insane file format (docx, netCDF) to plain text, then we can go to work greping the patterns we are interested in. For example when I was doing my undergraduate thesis and my masters and my PhD, when running long molecular simulations in supercomputers you can always grep for error codes, check if calculations are going fine, later, you can filter those outputs with grep to generate reports, extract atomic coordinates to restart failed calculations or for analysis, when doing natural language processing, grep allows to search for capitalized words, some odd forms of expressions, phrases, and cliched expressions.

It allows to filter csv dump files from databases, remember commands when piping the history command looking in config files for errors, everything done in plain text can be analyzed with grep.

There is even an old UNIX  saying “You can’t grep a dead tree” which means that manuals and documentation should be written in plain text files for easy searching contrary to books, where you have to search page by page.

Anyway, long live grep the most awesome tool ever.

[0770man]

A cool video about influence maps in game AI

[0770man]

Natural Language Processing notes 1, splitting

Splitting sentences is a mostly solved task in Natural Language Processing for languages that use punctuation. Some Machine Learning algorithms achieving 98.5% accuracy on the task.

The main idea for a quick and dirty sentence splitter takes into account common abbreviations, URLs, IP addresses, decimal numbers, interjections, and ellipsis in a rule based fashion.

The idea here is to find and substitute all the cases mentioned above with a tag like , when we reach the end of our text, we look up the remaining punctuation and split on it. It is a very simple and powerful idea for western languages.

to give an example in pseudocode:

func Splitter(text): raw_text=text //remove end of lines one_line_text=remove_eol(raw_text) //use regular expressions to remove the categories we talked about for abbreviation in list_of_abbreviations: regexp.sub(one_line_text,abbreviation,abbreviation_tag) for intejection in list_of_intejections: regexp.sub(one_line_text,interjetion,intejection_tag) regexp.sub(one_line_text,url,url_tag) regexp.sub(one_line_text,number,number_tag) regexp.sub(one_line_text,ellipsis,ellipsis_tag) regexp.sub(one_line_text,initials,initials_tag) //if there is remaining punctuation, split on it sentences=one_line_text.split(punct_list) return sentences

So there is the pseudocode for a simple and fast sentence spitter that achieves high accuracy.

[0770man]

Your focus is your reality

Yoda

[0770man]

Generating design documents for #ProjectEtherling

These weekend @pixkull and me have been in negotiations for a joint venture in game development, he as game designer and me as lead developer, it has been a fun weekend and I am waiting for him to polish up the design document so I can start working.

I am really excited as this is the first time I will not be developing solo, I intend to keep going the development of my nephew's game as a solo adventure (maybe get a designer, I still can't do pixel art I really like). The great thing about this is the options I think this entails @pixkull seems full of ideas and with clear direction about where he wants this to go, for the Rouguelike even though I have a design document, the artistic bits seem to be in thin air, as I can't seem to be able to define an aesthetic.

[0770man]

A great place to learn how to contribute and create.

[0770man]

#PDJam failure

Due to unforseen circumstances I couldn’t finish my entry to PDJam. I wanted to do something like Barbarian but based on the Three Musketeers.

[0770man]

Multi-platform packaging

With Free Pascal, I think I have a greater chance of developing for multiple operating systems, I have been doing experiments and been able to generate code for Windows, Linux and Mac, albeit without graphics, it was some Database app for my dayjob. Packaging and creating an installer doesn’t seem to hard as there is a wiki and an active community.