In JavaScript world, JSON serialization is widely used. When fetching data from server via Ajax, the data is usually represented in JSON; or loading configuration/data from file in Node.js application, the configuration/data is usually in JSON format.

JSON serialization is powerful and convenient, but there is limitation. For security and other reason, behavior and type information are forbidden in JSON. Functions members are removed when stringify a JavaScript object, also functions are not allowed in JSON.

Comparing Yaml to Ruby, this limitation isn’t that convenient when writing JavaScript application. For example, to consume the JSON data fetched via ajax from server, I really wish I can invoke some method on the deserialized model.

Here is simple example:

1
2
3
4
5
6
7
8
9
10
11
12
13

class Rect
  constructor: (width, height) ->
    @width = width if width?
    @height = height if height?
  area: ->
    @width * @height
$.get '/rect/latest', (rectJSON) ->  
  rect = JSON.parse(rectJSON)
  console.log rect.area() # This code doesn't work because there is rect is a plain object

The code doesn’t work, because rect in a plain object, which doesn’t contains any behavior. Someone called the rect DTO, Data Transfer Object, or POJO, Plain Old Java Object, a concept borrowed from Java world. Here we call it DTO.

To add behaviors to DTO, there are variant approaches. Such as create a behavior wrapper around the DTO, or create a new model with behavior and copy all the data from DTO to model. These practices are borrowed from Java world, or traditional Object Oriented world.

In fact, in JavaScript, there could be a better and smarter way to achieve that: Object Mutation, altering object prototype chain on the fly to convert a object into the instance of a specific type. The process is really similar to biologic genetic mutation, converting a species into another by altering the gene, so I borrow the term mutation.

With the idea, we can achieve this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

class Rect
  constructor: (width, height) ->
    @width = width if width?
    @height = height if height?
  area: ->
    @width * @height
$.get '/rect/latest', (rectJSON) ->  
  rect = JSON.parse(rectJSON)
  mutate(rect, Rect)
  console.log rect.area()

The key to implement mutate function is to simulate new operator behavior, alerting object.__proto__ and apply constructor to the instance! For more detail, check out the library mutator , which is available as both NPM package and bower package.

When implementing the mutator, in IE, again, in the evil IE, the idea doesn’t work. Before IE 11, JavaScript prototype chain for instance is not accessible. There is nothing equivalent to object.__proto__ in IE 10 and prior. The most similar workaround is doing a hard-copy of all the members, but it still fails in type check and some dynamical usage.

Background

object.__proto__ is a Mozilla “private” implementation until EcmaScript 6.
It is interesting that most JavaScript support it except IE.
Luckily, IE 11 introduced some features in EcmaScript 6, object.__proto__ is one of them.

This is the second post in this series, previous one discussed the JSON schema migration mechanism.

After finish JSON DSL implementation, the No.1 problem I need to handle is how to upgrading the configuration in CoffeeScript to JSON format.

One of the solutions is to do it manually. Well, it is possible, but… JSON isn’t a language really designed for human, composing configuration for 50+ sites doesn’t sound like a pleasant work for me. Even when I finished it, how can I ensure all the configuration is properly upgraded? The sun is bright today, I don’t want to waste whole afternoon in front of the computer checking the JSON. In one word, I’m lazy…

Since most of the change of DSL is happened on representation instead of structure. So in most cases, there is 1-to-1 mapping between v1 DSL and v2 DSL. So maybe I can generate the most of v2 DSL by using V1 DSL! Then manually handle some exceptions.

Here is a snippet of V1 DSL

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

AdKiller.run ->
  @host 'imagetwist.com', ->
    @clean('#myad', '#popupOverlay')
    @revealImg('.pic')
  @host 'yankoimages.net', ->
    @clean('#firopage')
    @revealImg('img')
  @host 'imageback.info', 'imagepong.info', 'imgking.us', 'imgabc.us', ->
    @revealA('.text_align_center a')
  @host 'imgserve.net',
    'imgcloud.co',
    'hosterbin.com',
    'myhotimage.org',
    'img-zone.com',
    'imgtube.net',
    'pixup.us',
    'imgcandy.net',
    'croftimage.com',
    'www.imagefolks.com',
    'imgcloud.co',
    'imgmoney.com',
    'imagepicsa.com',
    'imagecorn.com',
    'imgcorn.com',
    "imgboo.me",
    'imgrim.com',
    'imgdig.com',
    'imgnext.com',
    'hosturimage.com',
    'image-gallery.us',
    'imgmaster.net',
    'img.spicyzilla.com',
    'bulkimg.info',
    'pic.apollon-fervor.com',
    '08lkk.com',
    'damimage.com',
    ->
      @click('#continuetoimage input')
      @clean('#logo')
      @safeRevealImg('#container img[class]')

In version 1 implementation, @host defines the sites. And in the block of @host method, @click, @clean, @revealImg methods define the actions for the sites. The @host method instantiate new instance of Cleaner. The code block is invoked when cleaner is triggered, which does the actually cleaning.

Now I want to keep this file, since it shares the configuration between version 1 and version 2. And I redefine the behaviors of the cleaning method, such as @clean, @click, etc., I generate the JSON data when it is invoked instead of really altering the DOM. So I got this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

#!/usr/bin/env coffee
fs = require('fs')
AdKiller =
  run: (block) ->
    @scripts = {}
    @hosts = {}
    block.call(this)
    fs.writeFileSync 'seed_data.json', JSON.stringify({version: 1, @hosts, @scripts})
  host: (hosts..., block) ->
    @currentScript = []
    scriptId = hosts[0]
    @scripts[scriptId] = @currentScript
    for host in hosts
      @hosts[host] = scriptId
    block.call(this)
  remove: (selectors...) ->
    selectors.unshift 'remove'
    @currentScript.push selectors
  clean: (selectors...) -> # Backward compatibility
    selectors.unshift 'remove'
    @currentScript.push selectors
  hide: (selectors...) ->
    selectors.unshift 'clean'
    @currentScript.push selectors
  click: (selector) ->
    @currentScript.push ['click', selector]
  revealA: (selector) ->
    @currentScript.push ['revealA', selector]
  revealImg: (selector) ->
    @currentScript.push ['revealImg', selector]
  safeRevealA: (selector) ->
    @currentScript.push ['safeRevealA', selector]
  safeRevealImg: (selector) ->
    @currentScript.push ['safeRevealImg', selector]
AdKiller.run ->
  @host 'imagetwist.com', ->
    @clean('#myad', '#popupOverlay')
    @revealImg('.pic')
  # ...

So now I can easily invoking this piece of code, to convert verion 1 DSL to JSON format.

DSL behavior redefinition is a super powerful trick, we used it on JSON parsing, validation, and generation before on PlayUp project. Which saved us tons of time from writing boring code.

I’m working on the Harvester-AdKiller version 2 recently. Version 2 dropped the idea “Code as Configuration”, because the nature of Chrome Extension. Recompiling and reloading the extension every time when configuration changed is the pain in the ass for me as an user.

For security reason, Chrome Extension disabled all the Javascript runtime evaluation features, such as eval or new Function('code'). So that it become almost impossible to edit code as data, and later applied it on the fly.

Thanks to the version 1, the feature and DSL has almost fully settled, little updates needed in the near future. So I can use a less flexible language as the DSL instead of CoffeeScript.

Finally I decided to replace CoffeeScript to JSON, which can be easily edited and applied on the fly.

After introducing JSON DSL, to enable DSL upgrading in the future, an migration system become important and urgent. (Actually, this prediction is so solid. I have changed the JSON schema once today.) So I come up a new migration system:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

class Upgrader
  constructor: ->
    @execute()
  execute: =>
    console.log "[Upgrader] Current Version: #{Configuration.version}"
    migrationName = "#{Configuration.version}"
    migration = this[migrationName]
    unless migration?
      console.log '[Upgrader] Latest version, no migration needed.'
      return
    console.log "[Upgrader] Migration needed..."
    migration.call(this, @execute)
  'undefined': (done) ->
    console.log "[Upgrader] Load data from seed..."
    Configuration.resetDataFromSeed(done)
  '1.0': (done) ->
    console.log "[Upgrader] Migrating configuration schema from 1.0 to 1.1..."
    # Do the migration logic here
    done()

The Upgrader will be instantiate when extension started, after Configuration is initialized, which holds the DSL data for runtime usage.

When the execute method is invoked, it check the current version, and check is there a upgrading method match to this version. If yes, it triggers the migration; otherwise it succeed the migration. Each time a migration process is completed, it re-trigger execute method for another round of check.

Adding migration for a specific version of schema is quite simple. Just as method 1.0 does, declaring a method with the version number in the Upgrader.

'undefined' method is a special migration method, which is invoked there is no previous configuration found. So I initialize the configuration from seed data json file, which is generated from the version 1 DSL.

The seed data generation is also an interesting topic. Please refer to next post(Redfine DSL behavior) of this series for details.

Today we met a weird problem when return a string via json.

Here is the coffee script code:

1
2
3
4

$.post serverUrl, data, (status) ->
	console.log status

And here is our controller:

1
2
3
4
5
6
7

def action
	# do some complex logic
	render json: "success"
end

Code looks perfect, but we found that the callback is never called! When we check the network traffic, you will found that server does send its response “success”, but the callback is not called!

After spending half an hour to struggle against the jQuery, we finally find the problem!

The reason is that success is not a valid json data! A valid json string should be quoted with “”, or JSON parser will treat it as token, like true or false or nil.

So to fix the problem, we need to change our action code:

1
2
3
4
5
6
7

def action
	# do some complex logic
	render json: '"success"'
end

This is really a pitfall, since the wrong code looks so nature!