Is Android API document on ConsumerIrManager lying?

Just found a shocking fact that Android API document on ConsumerIrManger.transmit method is wrong!

KitKat has realised its own Infrared blaster API, which is incompatible with legacy Samsung private API. So I was working on Android Infrared Library to make it adapted automatically on both Samsung private API and Kit Katofficial API.

After I finished the coding according the document, I found the app broke on my Galaxy Note 3 with Kit Kat. It works perfect when running on Jelly Bean.

And I figured out an issue that it takes longer time to transmit the same seqeunce when I upgraded API. (When IR blaster is working, the LED indicator on the phone turns blue. And I found the time of indicator turning blue is significant longer than before.) And my IRRecorder cannot recognize the sequence sent by my phone any longer.

After spent several hours, I figured out the reason. The pattern was encoded in a wrong way. But I’m pretty sure that I strictly followed the API document.

So I get a conculusion that the ConsumerIrManager implementation on Samsung Note 3 is different to what described in Android API document. However I’m not sure the reason is that the Android document is lying or Samsung implemented the driver in a wrong way.

Here is the technical details of the issue and its solution:

IR Command is trasmitted by turnning the IR blaster LED on and off for a certain period of time. So each IR command can be represented by a series of time periods, which indicates how long the led is on or off. The difference between Samsung API and Kit Kat APi is that how the time is mesured.

carrierFrequency The IR carrier frequency in Hertz.
pattern The alternating on/off pattern in microseconds to transmit.

According to the Android Developer Refernece), the time in KitKat is measured in the unit of microseconds.

But for Samsung, the time is mesured by the number of cycles. Take NEC encoding as example, the frequency is 38kHz. So the cycle time T ~= 26us. BIT_MARK is 21 cycles, the period of time is around 26us x 21 ~= 546us.

So ideally, regardless of lead-in and lead-out sequence, to send the code 0xA in NEC encoding, Samsung API needs 21 60 21 21 21 60 21 21; and Kit Kat API needs 560 1600 560 560 560 1600 560 560.

But accroding to my experience, the Android Developer Reference is wrong. Even in KitKat, the time sequence is also measure by number of cycles instead of the number of microseconds!

So to fix the issue, you need some mathmatical work. Here is the conversion formula:

1
2
3
4
5
6
7
n = t / T = t * f / 1000
n: the number of cycles
t: the time in microseconds
T: the cycle time in microseconds
f: the transmitting frequency in Hertz

Use Jade as client-side template engine

Jade is a powerful JavaScript HTML template engine, which is concise and powerful. Due to its awesome syntax and powerful features, it almost become the default template engine for node.js web servers.

Jade is well known as a server-side HTML template, but actually it can also be used as a client-side template engine, which is barely known by people! To have a better understanding of this issue, firstly we should how Jade engine works.

When we’re translating a jade file into HTML, Jade engine actually does 2 separate tasks: Compiling and Rendering.

Compiling

Compiling is almost a transparent process when rendering jade files directly into HTML, including, rendering a jade file with jade cli tool. But it is actually the most important step while translating the jade template to HTML.
Compiling will be translate the jade file into a JavaScript function. During the process, all the static content has been translated.

Here is simple example:

Jade template
1
2
3
4
5
6
7
8
9
10
11
12
13
doctype html
html(lang="en")
head
title Title
body
h1 Jade - node template engine
#container.col
p You are amazing
p.
Jade is a terse and simple
templating language with a
strong focus on performance
and powerful features.
Compiled template
1
2
3
4
5
6
function template(locals) {
var buf = [];
var jade_mixins = {};
buf.push('<!DOCTYPE html><html lang="en"><head><title>Title </title></head><body><h1>Jade - node template engine</h1><div id="container"class="col"> <p>You are amazing</p><p>Jade is a terse and simple\ntemplating language with a\nstrong focus on performance\nand powerful features.</p></div></body></html>');
return buf.join("");
}

As you can see, the template is translated into a JavaScript function, which contains all the HTML data. In this case, since we didn’t introduce any interpolation, so the HTML content has been fully generated.

The case will become more complicated when interpolation, each, if statement is introduced.

Jade template with interpolation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
doctype html
html(lang="en")
head
title =title
body
h1 Jade - node template engine
#container.col
ul
each item in items
li= item
if usingJade
p You are amazing
else
p Get it!
p.
Jade is a terse and simple
templating language with a
strong focus on performance
and powerful features.
Compiled template with interpolation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
function template(locals) {
var buf = [];
var jade_mixins = {};
var locals_ = locals || {}, items = locals_.items, usingJade = locals_.usingJade;
buf.push('<!DOCTYPE html><html lang="en"><head><title>=title </title></head><body><h1>Jade - node template engine</h1><div id="container"class="col"></div><ul>');
(function() {
var $$obj = items;
if ("number" == typeof $$obj.length) {
for (var $index = 0, $$l = $$obj.length; $index < $$l; $index++) {
var item = $$obj[$index];
buf.push("<li>" + jade.escape(null == (jade.interp = item) ? "" : jade.interp) + "</li>");
}
} else {
var $$l = 0;
for (var $index in $$obj) {
$$l++;
var item = $$obj[$index];
buf.push("<li>" + jade.escape(null == (jade.interp = item) ? "" : jade.interp) + "</li>");
}
}
}).call(this);
buf.push("</ul>");
if (usingJade) {
buf.push("<p>You are amazing</p>");
} else {
buf.push("<p>Get it!</p>");
}
buf.push("<p>Jade is a terse and simple\ntemplating language with a\nstrong focus on performance\nand powerful features.</p></body></html>");
return buf.join("");
}
Data for interpolation
1
2
3
4
5
6
7
8
9
{
"title": "Jade Demo",
"usingJade": true,
"items":[
"item1",
"item2",
"item3"
]
}
Output Html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<!DOCTYPE html>
<html lang="en">
<head>
<title>=title </title>
</head>
<body>
<h1>Jade - node template engine</h1>
<div id="container" class="col"></div>
<ul>
<li>item1</li>
<li>item2</li>
<li>item3</li>
</ul>
<p>You are amazing</p>
<p>
Jade is a terse and simple
templating language with a
strong focus on performance
and powerful features.
</p>
</body>
</html>

Well, as you can see, the function has become quite complicated than before. It could become more complicated when extend, include or mixin introduced, you can trial it on your own.

Rendering

After the compiling, the rendering process is quite simple. Just invoking the compiled function, the return string is rendered html. The only thing need to mentioned here is the interpolation data should be passed to the template function as locals.

Using Jade as front-end template engine

Still now, you probably have got my idea. To use jade a front-end template, we can compose the template in jade. Later compile it into JavaScript file. And then we can invoke the JavaScript function in front-end to achieve dynamic client-side rendering!

Since Jade template has been precompiled at server side, so there is very little runtime effort when rendering the template at client-side. So it is a cheaper solution when you have lots of templates.

To compile the jade files into JavaScript instead of HTML, you need to pass -c or --client option to jade cli tool. Or calling jade.compile instead of jade.render while using JavaScript API.

Configure Grunt

Well, since Grunt is popular in node.js world. So we can also use Grunt to do the stuff for us.
Basically, use grunt for jade is straightforward. But it is a little bit tricky when you want to compile the back-end template into HTML as well as to compile the front-end template into JavaScripts.

I used a little trick to solve the issue. I follow the convention in Rails, that prefix the front-end template files with underscore.
So

1
2
3
4
5
/layouts/default.jade -> Layout file, extended by back-end/front-end templates, should not be compiled.
/views/settings/index.jade -> Back-end template, should be compiled into HTML
/views/settings/_item.jade -> Front-end template, should be compiled into JavaScript
Gruntfile.coffee
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
module.exports = (grunt) ->
grunt.initConfig
pkg: grunt.file.readJSON('package.json')
jade:
options:
pretty: true
compile:
expand: true
cwd: 'views'
src: ['**/*.jade', '!**/_*.jade']
dest: 'build/'
ext: '.html'
template:
options:
client: true
namespace: 'Templates'
expand: true
cwd: 'views'
src: ['**/_*.jade']
dest: 'build/'
ext: '.js'
grunt.loadNpmTasks('grunt-contrib-jade')

I distinguish the layouts and templates by file path. And distinguish the front-end/back-end templates by prefix. The filter !**/_*.jade excludes the front-end templates when compiling the back-end templates.

This approach should work fine in most cases, but if you are facing more complicated situation, and can’t be handled with this trick, try defining your own convention, and recognizing it with custom filter function to categorize them.

Create shortcut for your project Mingle with Alfred

A few days ago, I post a blog(Create shortcut for your project Mingle with Chrome OmniBox) about that we can access specific Mingle story card directly from Chrome “OmniBox”. The shortcut is created by registering Mingle search as custom search in Chrome.

Today I found applying the same trick to launch apps, we can have one step further. In Mac OS, almost all the launcher apps support custom search, such as Aflred, QuickSilver, Launcher. Even in Windows, we also have Launchy. For Linux, I believe there should be similar stuff. So the trick is environment independent.

To add custom search query in different launch app is quite different, but should be straightforward.
I’ll take Alfred as example:

  1. Open Alfred Preference

    Alfred Preferences
  2. Register Mingle as custom search

    Alfred Preferences

The url for custom search can be get with the same approach described in previous post. Alfred uses {query} as placeholder, so you should replace the %s with {query} when coping the url from chrome to Alfred.

Upgrading DSL From CoffeeScript to JSON: Part.2. Redefine DSL behavior

This is the second post in this series, previous one discussed the JSON schema migration mechanism.

After finish JSON DSL implementation, the No.1 problem I need to handle is how to upgrading the configuration in CoffeeScript to JSON format.

One of the solutions is to do it manually. Well, it is possible, but… JSON isn’t a language really designed for human, composing configuration for 50+ sites doesn’t sound like a pleasant work for me. Even when I finished it, how can I ensure all the configuration is properly upgraded? The sun is bright today, I don’t want to waste whole afternoon in front of the computer checking the JSON. In one word, I’m lazy…

Since most of the change of DSL is happened on representation instead of structure. So in most cases, there is 1-to-1 mapping between v1 DSL and v2 DSL. So maybe I can generate the most of v2 DSL by using V1 DSL! Then manually handle some exceptions.

Here is a snippet of V1 DSL

Ver 1 DSL snippet
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
AdKiller.run ->
@host 'imagetwist.com', ->
@clean('#myad', '#popupOverlay')
@revealImg('.pic')
@host 'yankoimages.net', ->
@clean('#firopage')
@revealImg('img')
@host 'imageback.info', 'imagepong.info', 'imgking.us', 'imgabc.us', ->
@revealA('.text_align_center a')
@host 'imgserve.net',
'imgcloud.co',
'hosterbin.com',
'myhotimage.org',
'img-zone.com',
'imgtube.net',
'pixup.us',
'imgcandy.net',
'croftimage.com',
'www.imagefolks.com',
'imgcloud.co',
'imgmoney.com',
'imagepicsa.com',
'imagecorn.com',
'imgcorn.com',
"imgboo.me",
'imgrim.com',
'imgdig.com',
'imgnext.com',
'hosturimage.com',
'image-gallery.us',
'imgmaster.net',
'img.spicyzilla.com',
'bulkimg.info',
'pic.apollon-fervor.com',
'08lkk.com',
'damimage.com',
->
@click('#continuetoimage input')
@clean('#logo')
@safeRevealImg('#container img[class]')

In version 1 implementation, @host defines the sites. And in the block of @host method, @click, @clean, @revealImg methods define the actions for the sites. The @host method instantiate new instance of Cleaner. The code block is invoked when cleaner is triggered, which does the actually cleaning.

Now I want to keep this file, since it shares the configuration between version 1 and version 2. And I redefine the behaviors of the cleaning method, such as @clean, @click, etc., I generate the JSON data when it is invoked instead of really altering the DOM. So I got this:

seed_data_generator
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
#!/usr/bin/env coffee
fs = require('fs')
AdKiller =
run: (block) ->
@scripts = {}
@hosts = {}
block.call(this)
fs.writeFileSync 'seed_data.json', JSON.stringify({version: 1, @hosts, @scripts})
host: (hosts..., block) ->
@currentScript = []
scriptId = hosts[0]
@scripts[scriptId] = @currentScript
for host in hosts
@hosts[host] = scriptId
block.call(this)
remove: (selectors...) ->
selectors.unshift 'remove'
@currentScript.push selectors
clean: (selectors...) -> # Backward compatibility
selectors.unshift 'remove'
@currentScript.push selectors
hide: (selectors...) ->
selectors.unshift 'clean'
@currentScript.push selectors
click: (selector) ->
@currentScript.push ['click', selector]
revealA: (selector) ->
@currentScript.push ['revealA', selector]
revealImg: (selector) ->
@currentScript.push ['revealImg', selector]
safeRevealA: (selector) ->
@currentScript.push ['safeRevealA', selector]
safeRevealImg: (selector) ->
@currentScript.push ['safeRevealImg', selector]
AdKiller.run ->
@host 'imagetwist.com', ->
@clean('#myad', '#popupOverlay')
@revealImg('.pic')
# ...

So now I can easily invoking this piece of code, to convert verion 1 DSL to JSON format.

DSL behavior redefinition is a super powerful trick, we used it on JSON parsing, validation, and generation before on PlayUp project. Which saved us tons of time from writing boring code.

Chrome Extension RPC Router

When developing chrome extension, communication between background script and content scripts is very typical use case. Chrome provides messaging APIs to achieve this goal. But this API has limitation that every message goes to the same listener.

Background script in Chrome extension usually works as a function hub for the whole extension, so background scripts usually required to process different types of messages. Then the limit of Chrome messaging API become an issue we need to face.

There are several approaches to resolve the limitation. Since API allow multiple listener, a simple and cheap solution is Responsibility Chain; adding listener for each message type, and checking message type at beginning of the listener.

Responsbility Chain
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
chrome.runtime.onMessage.addListener(function(message, sender, sendResponse){
if(message.type != 'reloadData')
return;
// reload data logic here
});
chrome.runtime.onMessage.addListener(function(message, sender, sendResponse){
if(message.type != 'updateBrowerAction')
return;
// update browser action logic here
});

This approach works, but not that graceful. And there is potential performance issue when message types increases.

So I come up a new more graceful solution: RPC Message Router

RCPRouter
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
extractResponseHandler = (args) ->
return undefined if args.length == 0
last = args.pop()
if typeof last == 'function'
return last
else
args.push last
return undefined
class @RPCRouter
constructor: ->
chrome.runtime.onMessage.addListener @handleMassage
handleMassage: (message, sender, sendResponse) =>
{method: methodName, args} = message
method = this[methodName]
unless method?
console.error "Unknown RPC method: %s", methodName
return
args.push sendResponse
console.log "RPC Call: %s, args: %o", methodName, args
method.apply(this, args)
callBackground: (method, args...) ->
responseHandler = extractResponseHandler(args)
console.log "RPC Call to Background: %s, args: %o", method, args
chrome.runtime.sendMessage {method, args}, responseHandler
callTab: (tabId, method, args...) ->
responseHandler = extractResponseHandler(args)
console.log "RPC Call to Tab \"%s\": %s, args: %o", tabId, method, args
chrome.tabs.sendMessage tabId, {method, args}, responseHandler

So this new RPCRouter wraps the chrome original messaging API and provides a more convenient way to invoke a remote method. To create a specific RPC Router for background page or content page is quite easy.

Responsbility Chain
1
2
3
4
5
6
7
8
9
10
11
12
class BackgroundRPCRouter extends RPCRouter
refreshData: (isForceUpdate, dataLoadedCallback) ->
# reload data logic here
dataLoadedCallback(data)
updateBrowerAction: (icon, hintText) ->
# update browser action logic here
return false # protection: avoid channel leak

HINT:
Using messaging API in chrome here should be careful. Coffee script will return the last executed statement result as function result, which could be potentially truthy, such as non-zero number, object. The truthy return value will make the channel become a async channel, which won’t be closed until the sendResponse callback is invoked. But just as the handler updateBrowserAction, the handler doesn’t need a sendResponse callback, the issue will keep the channel alive forever. So do add false or return false at the end of the method unless you can ensure the function will never yield truthy value in last statement.

Upgrading DSL from CoffeeScript to JSON: Part.1. Migrator

I’m working on the Harvester-AdKiller version 2 recently. Version 2 dropped the idea “Code as Configuration”, because the nature of Chrome Extension. Recompiling and reloading the extension every time when configuration changed is the pain in the ass for me as an user.

For security reason, Chrome Extension disabled all the Javascript runtime evaluation features, such as eval or new Function('code'). So that it become almost impossible to edit code as data, and later applied it on the fly.

Thanks to the version 1, the feature and DSL has almost fully settled, little updates needed in the near future. So I can use a less flexible language as the DSL instead of CoffeeScript.

Finally I decided to replace CoffeeScript to JSON, which can be easily edited and applied on the fly.

After introducing JSON DSL, to enable DSL upgrading in the future, an migration system become important and urgent. (Actually, this prediction is so solid. I have changed the JSON schema once today.) So I come up a new migration system:

Upgrader
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class Upgrader
constructor: ->
@execute()
execute: =>
console.log "[Upgrader] Current Version: #{Configuration.version}"
migrationName = "#{Configuration.version}"
migration = this[migrationName]
unless migration?
console.log '[Upgrader] Latest version, no migration needed.'
return
console.log "[Upgrader] Migration needed..."
migration.call(this, @execute)
'undefined': (done) ->
console.log "[Upgrader] Load data from seed..."
Configuration.resetDataFromSeed(done)
'1.0': (done) ->
console.log "[Upgrader] Migrating configuration schema from 1.0 to 1.1..."
# Do the migration logic here
done()

The Upgrader will be instantiate when extension started, after Configuration is initialized, which holds the DSL data for runtime usage.

When the execute method is invoked, it check the current version, and check is there a upgrading method match to this version. If yes, it triggers the migration; otherwise it succeed the migration. Each time a migration process is completed, it re-trigger execute method for another round of check.

Adding migration for a specific version of schema is quite simple. Just as method 1.0 does, declaring a method with the version number in the Upgrader.

'undefined' method is a special migration method, which is invoked there is no previous configuration found. So I initialize the configuration from seed data json file, which is generated from the version 1 DSL.

The seed data generation is also an interesting topic. Please refer to next post(Redfine DSL behavior) of this series for details.

Create shortcut for your project Mingle with Chrome OmniBox

Mingle is a common choice of story wall management system in ThoughtWorks internal projects.

Mingle uses rest style url to identify card or entities in the system, so the url contains the account id and project id in it.
Hosted Mingle is used in most project, to distinguish between accounts and projects, the url of the project mingle site is usually long, e.g., https://minglehosting.thoughtworks.com/telstra_de/projects/telstra_24x7_apps/. And other resources in the project, such as cards, whose url will be even longer.

When working in the team, open a specific card is a very common scenario, such as kicking off story, moving card, update story ACs, etc. To open a card, user usually need to open the project mingle home page, and use the search feature in the project. To open the home page, usually you need to save the url into your bookmark.

Today I found a better way to solve this issue, using the Chrome OmniBox! Chrome OmniBox is usually configured by Open Search Description. But it also allow user to manually add search engine configuration.

After the registration, we can use Mingle search direct from OmniBox, and we can open a specific card by typing “#card number”.

  1. Keyword Mingle in OmniBoxKeyword mingle in OmniBox
  2. Search in OmniBoxSearch in OmniBox
  3. Open the pageOpen the page

To register Mingle to Chrome OmniBox, you can use following approach:

Find out the Mingle search page url first:

  1. Open Mingle project home page, type test in the Search Box. Press enter, copy the url, and replace word test with %s. That’s the search url.Find Search Url
  2. Open url chrome://settings, find Manage search engines.Find Manage search engines
  3. Register new search engine.
    The first box is the display name of item, Mingle, project_name Mingle is recommended The second box is the keyword triggers the search, mingle is recommended. In most cases, you will work with only 1 mingle instance, so it will not cause confusion. But if it does, add more specific keyword to it.
    ** The third box is the url search url, paste the url you just foundRegister mingle