Discreet Cosine Transform

A few thoughts on ruby, video and other things

Riak-client and :symbolize_keys

Also in the vein of “maybe this saves you a little time”, be careful of riak-client version 1.4.3 or less and setting

1
:symbolize_keys => true

in MultiJson. If you set it for all load opertions, you will break some of riak-client’s methods such as #keys, #buckets and even map-reduce.

For anyone of the “you found it, you fix it” mindset out there I did report the issue but admittedly did not create a pull request with a fix yet. I agree, shame on me!

Dartium’s Fineprint

I am just starting out with Dart and ran into a little hiccup using Dartium that took me longer than it should have to realize what was going on.

Buried on the download page is this line:

The Dartium binary expires after 12 weeks.

And what that line doesn’t detail, is that Chromium will still launch, however, it will just silenty no longer execute Dart scripts.

To fix this, download the latest version of Dartium from the download link above.

Understanding Riak-client’s WalkSpec

I had a little trouble initially grokking riak-client’s options for link walking. There is a wiki page on it on their github and some api docs on it, but I still couldn’t get my mind around the options. Here is a mini walkthrough that might be helpful if you are having the same problem.

Setup

Fire up IRB or your REPL of choice and load riak-client, then connect to your local riak server or wherever you want:

1
2
require 'riak'
client = Riak::Client.new(:protocol => "pbc", :http_backend => :Excon)

I created four objects to toy with, a through d. Note at least when using protocol buffers, riak-client doesn’t let you leave data blank, so I set that to the simplest json doc I could, all we need is the riak links:

1
2
3
4
5
6
7
8
9
10
bucket = client.bucket('test')
a = bucket.get_or_new('a.json')
b = bucket.get_or_new('b.json')
c = bucket.get_or_new('c.json')
d = bucket.get_or_new('d.json')

a.data = '{}'
b.data = '{}'
c.data = '{}'
d.data = '{}'

Then I chained them together with some simple links. This part is pretty easy, thanks to the #to_link method on riak-client’s RObject:

1
2
3
a.links << b.to_link('foo')
b.links << c.to_link('foo')
c.links << d.to_link('bar')

So you can get to c from a through the tag “foo” but you can’t get all the way to d, as that is tagged with “bar”.

Last part of setup, store all your objects:

1
2
3
4
a.store
b.store
c.store
d.store

Walking

There is a shortcut #walk method on RObject, though you can also call #walk from the client itself. I like the shortcut, and at least one use of it is reasonably clear:

1
a.walk('test','_',true)

returns:

1
=> [[#<Riak::RObject {test,b.json} [#<Riak::RContent [application/json; charset=UTF-8]:(4 bytes)>]>]]

So this means walk the links on RObject a, keeping all results to the bucket test, but following all tags (the _ character is the wildcard in riak’s scheme). The true means to return the results of this walk leg, which was one of the parts that was confusing for me at first but makes more sense later.

The returns make sense: it returns b but not c which is one more link away. It only walks the first link, in other words. But why are the results an array of an array? It makes it seem like the return could be two dimensional, but how to achieve that wasn’t real obvious at first.

Note that the api docs do make clear there is an alternate syntax here, using a hash:

1
a.walk(bucket: 'test', tag: '_', keep: true)

Which is nice because its more explicit.

See that syntax and reading the source for the #normalize method on WalkSpec had things making more sense to me. To walk two links out from your current node, just pass it to hashes, both of which will be turned into WalkSpecs:

1
2
a.walk({bucket: 'test', tag: '_', keep: true},{bucket: 'test', tag: '_', keep: true})
=> [[#<Riak::RObject {test,b.json} [#<Riak::RContent [application/json; charset=UTF-8]:(4 bytes)>]>], [#<Riak::RObject {test,c.json} [#<Riak::RContent [application/json; charset=UTF-8]:(4 bytes)>]>]]

Now the keep: true adds up, as does the double array. If the first walk had returned more than 1 result, then the second level of the walk would now be branching, and two dimensions of results would be returned. If we wanted to get to c from a but not return b:

1
2
a.walk({bucket: 'test', tag: '_', keep: false},{bucket: 'test', tag: '_', keep: true})
=> [[#<Riak::RObject {test,c.json} [#<Riak::RContent [application/json; charset=UTF-8]:(4 bytes)>]>]]

And if we want to get to d:

1
2
a.walk({bucket: 'test', tag: '_', keep: false},{bucket: 'test', tag: '_', keep: false},{bucket: 'test', tag: '_', keep: true})
=> [[#<Riak::RObject {test,d.json} [#<Riak::RContent [application/json; charset=UTF-8]:(4 bytes)>]>]]

But that only works because we have allowed for any tag, remember we tagged the link to d with bar so if we try:

1
2
a.walk({bucket: 'test', tag: '_', keep: false},{bucket: 'test', tag: '_', keep: false},{bucket: 'test', tag: 'foo', keep: true})
=> [[]]

I might write a larger test script and test data set to really play with multiple levels of walking and those nested results, but overall the logic of #walk is a lot clearer to me now.

Marshal Ruby Objects to Riak and Back

Thought I would share a little bit of work I have been messing around with to marshal ruby objects to the Riak NoSQL store and back. There is Ripple, a library for exactly this purpose, but it implements the ActiveRecord model and that is not exactly what I wanted.

riak-client

Riak client supports automatic serialization based on what you set as an object’s content-type. So for example, the follow code would automatically store this Ruby hash as JSON in Riak:

1
2
3
4
5
6
client = Riak::Client.new(:protocol => "pbc")
bucket  = client.bucket("my_bucket")
new_one = Riak::RObject.new(bucket, "hash_object.json")
new_one.content_type = "application/json"
new_one.data = one
new_one.store

The difference here is in the use of #data and not #raw_data. There is a way to define your own serializers for different content-types, but that seems to be undocumented by the riak-client team for now. By default it supports both YAML and JSON.

That will work fine for the typical JSON data types such as Hash and Array. But if you want to store a more complex object, and get it back, you will need a little more.

Using the JSON Gem

riak-client uses multi_json as you might expect. So, if you have the JSON gem installed or loaded, you can use their method of storing a ruby object and marshaling it back from JSON.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
class ComplexObject
  attr_accessor :place,:time,:characters

  def initialize(stuff={})
    @place = stuff[:place]
    @time = stuff[:time]
    @characters = stuff[:characters]

  end

  #we have to define our own #to_json and self.json_create methods
  def to_json(*a)
    {'json_class'=>self.class.name,'place'=>@place,'time'=>@time,'characters'=>@characters}.to_json(*a)
  end

  def self.json_create(data)
    new(:place=>data['place'],:time=>data['time'],:characters=>data['characters'])
  end
end


one = ComplexObject.new(:place=>'woods',:time=>'tomorrow',:characters=>'burt')


client = Riak::Client.new(:protocol => "pbc")
bucket = client.bucket("my_bucket")


#in case I have other JSON libraries loaded that MultiJson would favor
MultiJson.use :json_gem

#this is a key, as it does not default to true 
#and without it the JSON gem won't call json_create
MultiJson.load_options = {:create_additions=>true}


new_one = Riak::RObject.new(bucket, "complex_object.json")
new_one.content_type = "application/json"
new_one.data = one
new_one.store

#viola, you get back a new object, not just a Hash
puts client['my_bucket']['complex_object.json'].data

Using Oj

It appears Oj has even more innate support for serializing and deserializing ruby objects to JSON, but I can’t say I am deeply familiar with Oj yet so take that with a grain of salt.

Here is the example above using Oj:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
class ComplexObject
  attr_accessor :place,:time,:characters

  def initialize(stuff={})
    @place = stuff[:place]
    @time = stuff[:time]
    @characters = stuff[:characters]

  end

  #with oj, no need for our own serialize and deserialize methods

end

one = ComplexObject.new(:place=>'woods',:time=>'tomorrow',:characters=>'burt')

client = Riak::Client.new(:protocol => "pbc")
bucket = client.bucket("my_bucket")

#MultiJson would default to oj if present, just being explicit here
MultiJson.use = :oj

#we want oj's object mode to do its automatic serialization
MultiJson.load_options = {:mode=>:object}
MultiJson.dump_options = {:mode=>:object}

#Store a ruby hash as json in riak

new_one = Riak::RObject.new(bucket, "complex_object.json")
new_one.content_type = "application/json"
new_one.data = one
new_one.store

#viola, again, the full ruby object
puts client['my_bucket']['complex_object.json'].data

Planescape

Planescape was my favorite Dungeons and Dragons setting of all-time.

I think admittedly, it was a little more “advanced” than say the Forgotten Realms. What I mean is, if you were playing D&D for the first time with someone new, sitting down and explaining Planescape was perhaps a bit harder, and it meant perhaps a bit more to people that had already played for years with the planes as a backdrop to magical adventures.

But, to contradict myself a bit, the flip side of that is Planescape definitely appeals to a number of core fantasy desires and concepts: hopping into a glowing portal and ending up in a different place is a quintessential fantasy moment. And even someone who has never read a fantasy book or played D&D will probably have a concept of heaven and hell, and know a bit about Greek myth for example - in these ways I think its possible Planescape would be even more digestible then say the Forgotten Realms to some new users.

Either way, the setting had an incredibly strong sense of place for me. That was due to both great writing and fantastic illustrations, in addition to the great core concepts. The illustrations by Tony DiTerlizzi remain some of my favorite to this day.

Also a little contrary to my previous posts, I think Planescape accomplished some of the goals I was bemoaning previously, that Wizards of the Coast brought to Dungeons and Dragons - the push towards fantasy settings that more closely resemble video games like World of Warcraft. It would seem WotC agrees with that, since the Tiefling, first introduced in Planescape, has become a more core part of the game and other settings (and indeed, strikes me as a character that would be right at home in a World of Warcraft setting).

I am surprised it was not more popular at the time of its release, though judging popularity is a relative thing. However, I do think it was far ahead of its time. It was first published in 1994, long before World of Warcraft or the current popularity of concepts like steampunk. However, a (very cursory) ready of current D&D forums online shows its not the most popular setting even now.

That is ok, I am not interested in a popularity contest, I just think the setting deserves more praise. My ideal game is a AD&D 2nd Edition romp set in Planescape.

Object-Oriented Story Telling

Being in both computer programming and role playing (as I would guess many old D&D players find themselves - geek begets geek after all), I have often thought about the concepts of relational data, object-orientation, and story telling.

In relational data, one record relates to another via some common data. There are a million glib examples, like looking up the weather for your hometown via the zip code (a more detailed way to say it might be you can find approximate latitude and longitude via the zip code, and find weather via lat and long - those that have worked with relational data before can almost picture the SQL statement in their minds right now).

Developers then spend a lot of time modeling real-world things and processes into software objects - single units of both functionality and data, with the very common trite examples being something like you have an ”order” object that contains many ”item” objects. Often, you even take it a step further and map those objects onto relational data - orders and items become records in a relational database. Common strategies for doing this exist, such as object-relational mappers (ORMs) and diagramming techniques, while not the fad at the moment, exist, such as relational database diagrams and object models in UML. A common long exists for these types of relationships, that talk about things like the definition of a given specific object (an “is a” relationship), the relationship between different objects (“has one”, “has many”, “has many through”, etc.) and how to relate one object to another. So a zip code has a general latitude and longitude, a town has a zip code, and a bunch of records that describe temperature, barometric pressure and wind speed all have a latitude and longitude to describe where they were measured or are predicted to occur. Viola, I input the zip code, I get out a weather forecast.

Story telling, maybe particularly interactive story telling, strikes me as having some similar ideas. A character has friends and enemies, possess items of power or items other people desire, and resides or adventures in a location. That same location houses other characters, and has traits of its own: geography, weather, and in the case of fantasy perhaps magical properties. It also has history: events that happened there in the past, and those evens in turn involved other characters (or your character). You can even think of useful queries you might perform against that data - say as a dungeon master, or just as author trying to keep all your notes straight. What are all the events that happened here, at a specific location? When was a specific character in this location? If the character possessed that highly valued item of power at the time (the ring!), then logic would apply that the item of power was in that location at the same time (though magic, of course, always jeopardizes logic such as this!).

No, I have not put this grand plan into action yet, but its on my mind per my previously stated goal of finding where my current life still intersects Dungeons and Dragons.

A Sense of Place

One of the things I always particularly loved about D&D, and fantasy in-general, is the sense of another place, the feeling of being there.

However, I think that sometimes who are not fantasy fans might assume that the definition of a “fantastic place” must mean something filled with a lot of danger, or even a place filled with truly fantastical things. I think video games have indeed helped perpetuate this - a “good level” in a video game often is exactly that: filled with danger and fantastical things. Floating islands, erupting volcanoes, enemies around every turn, etc.

That is not what I necessarily enjoyed though. Think instead of the early scenes in Peter Jackson’s interpretation of The Fellowship of the Ring - all those scenes in the Shire. What a wonderful job Mr. Jackson did of visually creating a place that fit the voice-over describing the peacefulness and contentedness of the hobbits. Isn’t that Shire a place you want to be? A place you want to spend time?

There were many such places in AD&D 2nd Edition - the Forgotten Realms, of that edition, had many such places. Yes, it had places filled with danger and intrigue too, but it was very much the type of fantasy that makes it easy to imagine a beautiful forest glen, a high mountain pass in spring, a trading port on the sea on a clear blue day. The more fantastical places had some sense of that too - Sigil, in the Placescape setting, always seemed like a great place to be to me. Yes, a little more “danger around every corner” then perhaps some of the Forgotten Realms, but still, it was easy to imagine an Inn or other gathering in that metropolitan city that was just as much a source of relaxation to its urban inhabitants.

I guess what I really mean by A Sense of Place is anywhere that you can experience with more senses than just your eyes - of course, something that AD&D excelled at because you imagined so much of it (versus say a movie, which I think has a bigger challenge to make you feel that place with more than just your sense of vision). If you can almost feel the breeze on your face, or smell the food vendors, and you get an actual tingle that says “I want to be there”, that is what I mean.

While I do think that is in a strange way harder in a movie, where best efforts can still generate an image that to you feels like looking at a picture, not being there, there are some very strong examples of this in movies as well. Ridley Scott particularly excels at it, in my opinion, and movies like Blade Runner really put you there.

Interestingly, supporting the idea that calling on the viewers imagination makes this more effective, I think some anime movies are perhaps the absolutely strongest film-examples of this. The Place Promised in Our Early Days, Wings of Honneamise, and most especially Spirited Away are some of my favorite examples. If you can’t sense the feeling of being in the bath house in Spirited Away - smell, taste, sight and everything - I might go so far as to say your crazy.

Which is all to say, that at least for me, good fantasy and good role playing are not necessarily about constant danger, and don’t need to be so fantastical as to be unrecognizable. I will go so far as to say video games (which, don’t get me wrong, I do love) are a poor model in some ways for table-top role playing. The pacing, the intent, and the way they confront your senses are generally different. There are a handful of examples of video games that are slower paced and have a true sense of exploration over constant jeopardy, but I doubt those were the games you thought of when I first said “video games”.

Dungeons and Dragons

The d20 Made Me Who I Am Today

Yes, I fit into that geek cliche - I spent a huge portion of my youth, and not-so-youth, playing D&D. I started with 1st Edition but all my significant years were spent playing AD&D 2nd Edition.

We played through high school and through college. I played a wee bit after college too, even after my first child was born, but admittedly young kids and real-life have cut off my playing for now.

But of all those warm, overly-fond memories of youth, few compare to my time with D&D. Other players will know what I mean. It was truly magical for many reasons: time well spent with close friends, the chance to be someone else particularly at a time in your adolescent life when a little escape from what you are feeling is a good thing, and the excercise of your imagination.

While the chance to hang out with your friends was great, I was definitely the type of player that loved the game. I spent untold hours reading material even when not playing, creating characters and monsters and adventures that might never get used. Re-organizing scraps of papers and photocopies from Dragon Magazine in 3-ring binders. To continue the gushing for just one moment more, there was no part of the game I did not love.

I very briefly played the 3.5 Edition and have read the 4.0 Edition rules. It seems, from a little bit of googling, that AD&D 2nd Edition is not necessarily remembered all that fondly. For example, the 1st Edition of the game seems to be much more warmly remember from a nostaglia viewpoint. To me, 2nd Edition was in fact the golden age of the game.

I know THAC0 and other convoluted rules seem to take the brunt of the 2nd Edition dislike, but I don’t think we ever chaffed under the rules. And while I did not know it then, I can definitely see that why it might not have been the most profitable time in TSR’s history with the game - they were publishing a ton of material, its hard to imagine that every book made back its cost to publish (in fact, I did not really know any of the history of the business side of TSR until later in life, but its an interesting read over on wikipedia).

To me, it was exactly these “problems” that I loved and still love about 2nd Edition.

  • The huge volume of material was wonderful to a player like me, that wanted to read and be immersed in the game even when not playing: there was no shortage of material to immerse myself in.
  • While the sprawl of rules across what probably amounts to hudreds of rule books was undoubtly daunting to a new player, to a seasoned player it made for wonderful variation in the game. We played so much and so long that those variations are what really kept us going from a rules and inspiration perspective. Yes, good players really just need a good DM and a great adventure, but some new classes, kits, weapons and monsters are all just inspiration to the imagination side of the game. And after all, what new player didn’t have a seasoned player there showing them the ropes?
  • The creativity that the authors of that generation of the game put out is still astounding to me: a distopian future following environmental disaster fully 10 years before enivronmental disaster would be the disaster du jour to hollywood and other media, steampunk fully 10 years before its current surge in popularity, and so on.

Its not my intent to critize the 4th Edition of the game, Wizards of the Coast or Hasbro in this post. But I will say that in reading the 4th Edition, as well as reading about the changes to the Forgotten Realms setting, these changes don’t appeal to me. It is entirely possible that makes me the cranky old coger of the table top roleplaying world, but the fantasy setting of World of Warcraft (which lets face it, represents a trend in fantasy that undeniably influenced that edition of the game) holds exactly zero appeal for me. Off on a tangent for a second, its interesting to note that Neal Stepheon’s novel Reamde deals, in a tongue-in-cheek way, with this exact same “riff” in the world of fantasy: between those that want floating islands and cat-people, and those that want knights in armor and a good, trusty longsword.

This post, in addition to being a dump of some of these feelings, is a declaration that I want to find some way to recapture some of that feeling of 2nd Edition and what D&D meant to me. I don’t know how yet, I am pretty curious to see the trend in self-written games based on the open-source parts of the d20 system, and in on-line publishing of game material. I need to explore all that further. Even without knowing how though, I wish and want to project that world of 2nd edition forward to future players: a sprawling multiverse of creativity and a sense that variations of the rules can be an inspiration to different play.

In its very small way, this post is also an homage to those minds that made 2nd edition so great: so here’s to Zeb Cook, Rich Baker, Tim Brown, Troy Denning, David Cook, Roger E. Moore and everyone else that worked on 2nd edition!

Creating a 21’ by 9’ Whiteboard

We recently undertook converting one entire wall at the office into whiteboard, using Idea Paint, a paint that goes in one coat and can be written on and erased like a whiteboard.

After an initial test of the Idea Paint system on a smaller area, the biggest challenge we faced was that the wall was covered with wallpaper.  We were faced with a few options: remove the wallpaper, try to sand it down to smooth, or spackle it then sand it down to smooth.  Initially we tested spackling over the rough paper and sanding the combined surface to smooth.  

Luckily, we discovered that the paper was old enough that it pulled right now, and the glue residue underneath was easily sanded off.  This discovery definitely saved us a lot of time.  

Once past our wallpaper problem, we were ready.  We curtained off the area, which turned out to be a good idea with all the sanding we did to remove the glue residue.  

Once the primer was on, we sanded again.  The actual whiteboard paint has to be put up in small batches - they recommend a maximum of 2 kits at once, and each kit covers 50 square feet. I fount it easier to do one kit at once, which split the 21 foot wall up into 5 steps.

The paint has to be mixed, and once mixed you have 1 hour to apply it.  What I learned after the first two batches is that when first mixed, the paint is very watery.  If you wait 10 minutes into your 60 minute limit, the stuff has already started to congeal and is much easier to apply thickly to the wall.  Once I knew that little trick, it was easier to get the paint on in pretty even thickness on the wall.

Also, a word of warning, just throw out everything that has been used with the whiteboard paint - brushes, rollers, etc.  Don’t try to clean anything as you would with latex paint.  The stuff dries to a solid, almost plastic.  Best to just use disposable stuff and toss it all out.

For our first time using the material, we definitely learned a few things we would have done different if we did it again:

  • Do a second coat of primer.  Nothing will paint over the whiteboard paint once applied, so you can’t do two coats (not to mention how hard it is to apply in the first place), so get your target to dead white with primer first.
  • While it would have been a much harder process, purchasing and using their variant of the paint that can be applied with a sprayer would have resulted in a smother surface - the roller itself creates a texture on the wall no matter how smoother you have the wall underneath beforehand. 

Despite the lessons learned, I am very happy with how the wall came out and its starting to see heavy usage as a giant whiteboard.  All in all, the project was not that hard.  Depending on what surface you have available, you could easily create pretty giant whiteboard spaces in your work space.