This is the second post in this series, previous one discussed the JSON schema migration mechanism.

After finish JSON DSL implementation, the No.1 problem I need to handle is how to upgrading the configuration in CoffeeScript to JSON format.

One of the solutions is to do it manually. Well, it is possible, but… JSON isn’t a language really designed for human, composing configuration for 50+ sites doesn’t sound like a pleasant work for me. Even when I finished it, how can I ensure all the configuration is properly upgraded? The sun is bright today, I don’t want to waste whole afternoon in front of the computer checking the JSON. In one word, I’m lazy…

Since most of the change of DSL is happened on representation instead of structure. So in most cases, there is 1-to-1 mapping between v1 DSL and v2 DSL. So maybe I can generate the most of v2 DSL by using V1 DSL! Then manually handle some exceptions.

Here is a snippet of V1 DSL

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

AdKiller.run ->
  @host 'imagetwist.com', ->
    @clean('#myad', '#popupOverlay')
    @revealImg('.pic')
  @host 'yankoimages.net', ->
    @clean('#firopage')
    @revealImg('img')
  @host 'imageback.info', 'imagepong.info', 'imgking.us', 'imgabc.us', ->
    @revealA('.text_align_center a')
  @host 'imgserve.net',
    'imgcloud.co',
    'hosterbin.com',
    'myhotimage.org',
    'img-zone.com',
    'imgtube.net',
    'pixup.us',
    'imgcandy.net',
    'croftimage.com',
    'www.imagefolks.com',
    'imgcloud.co',
    'imgmoney.com',
    'imagepicsa.com',
    'imagecorn.com',
    'imgcorn.com',
    "imgboo.me",
    'imgrim.com',
    'imgdig.com',
    'imgnext.com',
    'hosturimage.com',
    'image-gallery.us',
    'imgmaster.net',
    'img.spicyzilla.com',
    'bulkimg.info',
    'pic.apollon-fervor.com',
    '08lkk.com',
    'damimage.com',
    ->
      @click('#continuetoimage input')
      @clean('#logo')
      @safeRevealImg('#container img[class]')

In version 1 implementation, @host defines the sites. And in the block of @host method, @click, @clean, @revealImg methods define the actions for the sites. The @host method instantiate new instance of Cleaner. The code block is invoked when cleaner is triggered, which does the actually cleaning.

Now I want to keep this file, since it shares the configuration between version 1 and version 2. And I redefine the behaviors of the cleaning method, such as @clean, @click, etc., I generate the JSON data when it is invoked instead of really altering the DOM. So I got this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

#!/usr/bin/env coffee
fs = require('fs')
AdKiller =
  run: (block) ->
    @scripts = {}
    @hosts = {}
    block.call(this)
    fs.writeFileSync 'seed_data.json', JSON.stringify({version: 1, @hosts, @scripts})
  host: (hosts..., block) ->
    @currentScript = []
    scriptId = hosts[0]
    @scripts[scriptId] = @currentScript
    for host in hosts
      @hosts[host] = scriptId
    block.call(this)
  remove: (selectors...) ->
    selectors.unshift 'remove'
    @currentScript.push selectors
  clean: (selectors...) -> # Backward compatibility
    selectors.unshift 'remove'
    @currentScript.push selectors
  hide: (selectors...) ->
    selectors.unshift 'clean'
    @currentScript.push selectors
  click: (selector) ->
    @currentScript.push ['click', selector]
  revealA: (selector) ->
    @currentScript.push ['revealA', selector]
  revealImg: (selector) ->
    @currentScript.push ['revealImg', selector]
  safeRevealA: (selector) ->
    @currentScript.push ['safeRevealA', selector]
  safeRevealImg: (selector) ->
    @currentScript.push ['safeRevealImg', selector]
AdKiller.run ->
  @host 'imagetwist.com', ->
    @clean('#myad', '#popupOverlay')
    @revealImg('.pic')
  # ...

So now I can easily invoking this piece of code, to convert verion 1 DSL to JSON format.

DSL behavior redefinition is a super powerful trick, we used it on JSON parsing, validation, and generation before on PlayUp project. Which saved us tons of time from writing boring code.

I’m working on the Harvester-AdKiller version 2 recently. Version 2 dropped the idea “Code as Configuration”, because the nature of Chrome Extension. Recompiling and reloading the extension every time when configuration changed is the pain in the ass for me as an user.

For security reason, Chrome Extension disabled all the Javascript runtime evaluation features, such as eval or new Function('code'). So that it become almost impossible to edit code as data, and later applied it on the fly.

Thanks to the version 1, the feature and DSL has almost fully settled, little updates needed in the near future. So I can use a less flexible language as the DSL instead of CoffeeScript.

Finally I decided to replace CoffeeScript to JSON, which can be easily edited and applied on the fly.

After introducing JSON DSL, to enable DSL upgrading in the future, an migration system become important and urgent. (Actually, this prediction is so solid. I have changed the JSON schema once today.) So I come up a new migration system:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

class Upgrader
  constructor: ->
    @execute()
  execute: =>
    console.log "[Upgrader] Current Version: #{Configuration.version}"
    migrationName = "#{Configuration.version}"
    migration = this[migrationName]
    unless migration?
      console.log '[Upgrader] Latest version, no migration needed.'
      return
    console.log "[Upgrader] Migration needed..."
    migration.call(this, @execute)
  'undefined': (done) ->
    console.log "[Upgrader] Load data from seed..."
    Configuration.resetDataFromSeed(done)
  '1.0': (done) ->
    console.log "[Upgrader] Migrating configuration schema from 1.0 to 1.1..."
    # Do the migration logic here
    done()

The Upgrader will be instantiate when extension started, after Configuration is initialized, which holds the DSL data for runtime usage.

When the execute method is invoked, it check the current version, and check is there a upgrading method match to this version. If yes, it triggers the migration; otherwise it succeed the migration. Each time a migration process is completed, it re-trigger execute method for another round of check.

Adding migration for a specific version of schema is quite simple. Just as method 1.0 does, declaring a method with the version number in the Upgrader.

'undefined' method is a special migration method, which is invoked there is no previous configuration found. So I initialize the configuration from seed data json file, which is generated from the version 1 DSL.

The seed data generation is also an interesting topic. Please refer to next post(Redfine DSL behavior) of this series for details.

I’m working on a project that need some complicated html snippets for test, which cannot be easily generated with factory. So I put these snippets into fixture files.

RSpec provides a very convenient DSL keyword let, which allow us to define something for test and cached it in the same test. And I want I could have some similar keyword for my html fixtures. To achieve this goal I decide to extend DSL.

So I created module which contains the new DSL I want to have:

1
2
3
4
5
6
7
8
9
10
11

# spec/support/html_pages.rb
module HtmlPages
  def load_html(name)
    let name do
      file = Rails.root.join('spec/html_pages', "#{name}.html")
      Nokogiri::Html File.read(file)
    end
  end
end

Put this file into the path spec/support, by default, spec_helper.rb would require this file for you.
Then we should tell rspec to load the DSL into test cases.

1
2
3
4
5

RSpec.configure do |config|
	# ...
	config.extend HtmlPages
	# ...
end

By telling config to extend the module, our DSL will be loaded as the class methods of RSpec::Core::ExampleGroup, where let is being defined.

HINT: Rspec config has another way to extend DSL by calling config.include. Then the DSL methods will be injected into the test example group instance, then these methods can be used in the test cases. That’s how runtime DSLs like FactoryGirl work.