We found a computer name issue when building our private cloud environment on CloudStack over KVM. We found that KVM doesn’t support to rename new created instance automatically.
As a result, all the instance booted from the same disk image have the exactly same computer name, same administrator password.

So we need to do some manual provision work before user can use the new booted instances.
Administrator password can be changed in several ways, it is not a difficult job. But to rename the computer running Windows is not an easy work. The typical way is to call the WMI interface to rename the computer, which is taken as the most “formal” and “documented” way to rename the computer. But this approach require to reboot the instance, which is what we don’t like.

So we try some “hack” way to solve this problem, we use powershell scripts to hack the registry. By doing this, we can rename the computer without rebooting, and it works fine on our environment.
But since it is a hacking way, it only changed the most common values in registry, which means it won’t modify the rare ones and all kind of cached values. Such as the values cached in the SQL Server or MSMQ service, etc. So there might be some unknown potential issues. Take if on your own risk:

Here is the gist:

This is a script to populate computer hardware configuration in Linux.

1
2
3
4
5
6
7
8
9

#!/bin/bash
# CPU
cat /proc/cpuinfo |  grep -e "model name" -e "cores" -m 2 && cat /proc/cpuinfo | grep -e "physical id"| sort  | uniq -c
# Memory
free -got
# Disk
df -lh -t ext4 --total

Here is a bash script that acts as a TextMate command, which enables blogger to create new post without leaving TextMate.
It depends on “Post” task of the Rakefile in Jekyll Bootstrap.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

cd $TM_PROJECT_DIRECTORY
title=$(CocoaDialog standard-inputbox \
 --title "New Post" \
 --informative-text "Title of the new post:")
[[$(head -n1 <<<"$title") == "2" ]] && exit_discard
title=$(tail -n1 <<<"$title")
output=$(rake post title="$title")
expr="^\(in (.*)\) Creating new post: (.*)$"
if [[$output =~ $expr ]]; then
   	project=${BASH_REMATCH[1]}
	post=${BASH_REMATCH[2]}
	echo "new post file created at $post"
	exit_show_tool_tip
else
   	echo "Error"
	exit_show_tool_tip
fi

I have spent almost a week to fight against the code highlight system in jekyll, but finally failed.

At very beginning, I tried the “legendary” “github flavored markdown”, which supports use “```” to quote and highlight code. But soon, I failed.

Then I tried pygements, but sooner I found the perfectly local rendered code fails to be rendered on github pages, the reason seems caused because the github use a older version of the pygements.

Since I cannot fully control pygements as I expected, so I turned to a javascript version code highlighter, SyntaxHighliter.

Soon, I found SyntaxHighlighter cannot cooperate with jekyll’s markdown compiler well, so I tried several approaches to make them work together. I change part of the SyntaxHighlighter’s implementation and added several new includes liquid template based on Jekyll Bootstrap.
But it still cannot work perfect.

The problem is that jekyll markdown parser cannot distinguish the html snippet perfect, it usually try to parse the code as html.
Then I tried to quote my code with CData tag, but I found the generated html are not consistent. Some of which the “<” and “>” are escaped with “&gt;” and “&lt;”. And some other are not escaped but quoted with “”. It is really hard to deal with so many cases in javascript.

So I gave up. I gave up to use the github to render my pages. So I guess Octopress must be a good option for me.
Since Octopress is based on jekyll but with a lot of new goodies. These new liquid markups are powerful and can fulfill my requirements.

I’ve been working on MetaPaas project for a while. And composing deploy script is one of the my major tasks.
Here are some script snippets I found or I compose, which could be very useful in my daily work.

Download File from Web

This piece of script is useful when the script need to download package from CI server or configuration from configuration server etc.
Especially when wget or curl available.

NOTICE: This piece of script enable you to download the file from web in Powershell without any 3rd party dependences. But its performance is not as good as wget. Wget is still recommended if there are quite a lot of files to be downloaded.

Original Source

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

## Get-WebFile (aka wget for PowerShell)
##############################################################################################################
## Downloads a file or page from the web
## History:
## v3.6 - Add -Passthru switch to output TEXT files
## v3.5 - Add -Quiet switch to turn off the progress reports ...
## v3.4 - Add progress report for files which don't report size
## v3.3 - Add progress report for files which report their size
## v3.2 - Use the pure Stream object because StreamWriter is based on TextWriter:
##        it was messing up binary files, and making mistakes with extended characters in text
## v3.1 - Unwrap the filename when it has quotes around it
## v3   - rewritten completely using HttpWebRequest + HttpWebResponse to figure out the file name, if possible
## v2   - adds a ton of parsing to make the output pretty
##        added measuring the scripts involved in the command, (uses Tokenizer)
##############################################################################################################
function Get-WebFile {
   param(
      $url = (Read-Host "The URL to download"),
      $fileName = $null,
      [switch]$Passthru,
      [switch]$quiet
   )
   $req = [System.Net.HttpWebRequest]::Create($url);
   $res = $req.GetResponse();
   if($fileName -and !(Split-Path $fileName)) {
      $fileName = Join-Path (Get-Location -PSProvider "FileSystem") $fileName
   }
   elseif((!$Passthru -and ($fileName -eq $null)) -or (($fileName -ne $null) -and (Test-Path -PathType "Container" $fileName)))
   {
      [string]$fileName = ([regex]'(?i)filename=(.*)$').Match($res.Headers["Content-Disposition"] ).Groups[1].Value
      $fileName = $fileName.trim("\/""'")
      if(!$fileName) {
         $fileName = $res.ResponseUri.Segments[-1]
         $fileName = $fileName.trim("\/")
         if(!$fileName) {
            $fileName = Read-Host "Please provide a file name"
         }
         $fileName = $fileName.trim("\/")
         if(!([IO.FileInfo]$fileName).Extension) {
            $fileName = $fileName + "." + $res.ContentType.Split(";")[0].Split("/")[1]
         }
      }
      $fileName = Join-Path (Get-Location -PSProvider "FileSystem") $fileName
   }
   if($Passthru) {
      $encoding = [System.Text.Encoding]::GetEncoding($res.CharacterSet )
      [string]$output = ""
   }
   if($res.StatusCode -eq 200) {
      [int]$goal = $res.ContentLength
      $reader = $res.GetResponseStream()
      if($fileName) {
         $writer = new-object System.IO.FileStream $fileName, "Create"
      }
      [byte[]]$buffer = new-object byte[] 4096
      [int]$total = [int]$count = 0
      do
      {
         $count = $reader.Read($buffer, 0, $buffer.Length);
         if($fileName) {
            $writer.Write($buffer, 0, $count);
         }
         if($Passthru){
            $output += $encoding.GetString($buffer,0,$count)
         } elseif(!$quiet) {
            $total += $count
            if($goal -gt 0) {
               Write-Progress "Downloading $url" "Saving $total of $goal" -id 0 -percentComplete (($total/$goal)*100)
            } else {
               Write-Progress "Downloading $url" "Saving $total bytes..." -id 0
            }
         }
      } while ($count -gt 0)
      $reader.Close()
      if($fileName) {
         $writer.Flush()
         $writer.Close()
      }
      if($Passthru){
         $output
      }
   }
   $res.Close();
   if($fileName) {
      ls $fileName
   }
}

Extract File with Windows Explorer

This piece of code enable you to extract zip package without any 3rd party dependencies. It is useful environment provision script, which is usually executed on a Windows without anything being installed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Function Extract-Zip ([string] $zipPath, [string]$destination) {
	$progressbar = 4
	$shellApplication = New-Object -com shell.application
	$zipPackage = $shellApplication.NameSpace($zipPath)
	$destinationFolderName = Join-Path $destination "tiger"
	if (Test-Path $destinationFolderName ) {
		Remove-Item $destinationFolderName -Recurse -Force
	}
	New-Item $destinationFolderName -type directory
	$destinationFolder = $shellApplication.NameSpace($destinationFolderName)
	$destinationFolder.CopyHere($zipPackage.Items(), $progressbar)
}

Interpolate value into configuration

This piece of script enable you to replace the place-holders in configuration text with actual values.

1
2
3
4
5
6
7
8
9
10

Function Apply-Config([hashtable]$config) {
	foreach($line in $input) {
		foreach($key in $config.Keys) {
			$line = $line -replace $key, $config[$key]
		}
		$line
	}
}

This piece of script can be used in different scenarios:

Prepare Configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14

$configuration = @{
	"initialCatalog" = "lion"
	"logInitialCatalog" = "lion_log"
	"webServers" = @("192.168.122.46")
	"dbServer" = "192.168.122.65"
	"SQLServerDataDir" = "C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA"
	"SQLServerDataFileSize" = "10MB"
	"dataset" = "prod"
	"appPoolUser" = "$($env:COMPUTERNAME)\Lion"
	"appPoolPassword" = "1zhlmcl..oostp"
	"createAppPoolUser" = $true
}

Apply config to in-memory configuration template

1
2
3

$configurationText = $configurationTemplate | Apply-Config($configuration)

Create configuration file according to template

1
2
3

cat $configurationTemplateFile | Apply-Config($configuration) > $configurationFile

Download and apply configuration from configuration server

1
2
3

Get-WebFile -url $configTemplateUrl -Passthru | Apply-Config($configuration) > $configurationFile

In Node.js world, we usually encapsulate a service into a module, which means the module need to export the façade of the service. In most case the service could be a singleton, all apps use the same service.

But in some rare cases, people might would like to create several instances of the service ,which means the module also need to also export the service constructor.

A very natural idea is to export the default service, and expose the constructor as a method of the default instance. So we could consume the service in this way:

1
2

var defaultService = require('service');
var anotherService = service.newService();

So we need to write the module in this way:

1
2
3
4
5

function Service() { }
module.exports = new Service();
moudle.exports.newService = Service;

But for some reason, node.js doesn’t allow module to expose object by assigning the a object to module.exports.
To export a whole object, it is required to copy all the members of the object to moudle.exports, which drives out all kinds of tricky code.
I misunderstood how node.js require works, and HERE is the right understanding. Even I misunderstood the mechanism, but the conclusion of this post is still correct. To export function is still a more convenient way to export both default instance and the constructor.

And things can become much worse when there are backward reference from the object property to itself.
So to solve this problem gracefully, we need to change our mind.
Since it is proved that it is tricky to export a object, can we try to expose the constructor instead?

Then answer is yes. And Node.js does allow we to assign a function to the module.exports to exports the function.
So we got this code.

1
2

function Service() { }
module.exports = Service;

So we can use create service instance in this way:

1
2

var Service = require('service');
var aService = new Service();

As you see, since the one we exported is constructor so we need to create a instance manually before we can use it. Another problem is that we lost the shared instance between module users, and it is a common requirement to share the same service instance between users.

How to solve this problem? Since as we know, function is also kind of object in javascript, so we can kind of add a member to the constructor called default, which holds the shared instance of the service.

This solution works but not in a graceful way! A crazy but fancy idea is that can we transform the constructor itself into kind of singleton instance??!! Which means you can do this:

1
2
3
4

var defaultService = require('service');
defaultService.foo();
var anotherService = service();
anotherService.foo();

The code style looks familiar? Yes, jQuery, and many other well-designed js libraries are designed to work in this way.
So our idea is kind of feasible but how?

Great thank to Javascript’s prototype system (or maybe SELF’s prototype system is more accurate.), we can simply make a service instance to be the constructor’s prototype.

1
2
3

function Service() { }
module.exports = Service;
Service.__proto__ = new Serivce;

Sounds crazy, but works, and gracefully! That’s the beauty of Javascript.

Javascript is weakly typed, and its type system always behaves different than your expectation.
Javascript provide typeof operator to test the type of a variable. it works fine generally. e.g.

1
2
3
4
5
6

typeof(1) === 'number'
typeof('hello') === 'string'
typeof({}) === 'object'
typeof(function(){}) === 'function'

But it is not enough, it behaves stupid when you dealing with objects created by constructors. e.g.
if you expected

1

typeof(new Date('2012-12-12')) === 'date' // Returns false

1

typeof(new Date('2012-12-12')) === 'object' // Returns true

How can we make the typeof() operator works in the way as we expected?
As we know when we create a object, the special property of the object constructor will be set to the function that create the object. which means:

1

(new Date('2012-1-1')).constructor // Returns [Function Date]

So ideally we can retrieve the name of the function as the type of the variable. And to be compatible with javascript’s native operator, we need to convert the name to lower case. So we got this expression:

1
2
3
4
5

function typeOf(obj) { // Use capital O to differentiate this function from typeof operator
	return obj ? obj.constructor.name.toLowerCase() : typeof(obj);
}
typeOf(new Date('2012-1-1')) === 'date' // Returns true

And luckily, we can also apply this to other primitive types, e.g:

1
2
3

typeOf(123) === 'number'; // Returns true
typeOf('hello') === 'string'; // Returns true
typeOf(function(){}) === 'function';  // Returns true

or even

1

typeOf({}) === 'object'; // Returns true

So in general, we use this expression as new implementation of the typeof() operator! EXCEPT One case!

If someone declare the object constructor in this way, our new typeof() implementation will work improperly!

1
2
3
4
5

var SomeClass = (function() {
	return function() {
		this.someProperty='some value';
	}
})();

or even define the constructor like this

1
2
3

var SomeClass = function() {
	this.someProperty = 'some value';
}

And we will find that

1

typeOf(new SomeClass) === ''; // Returns true

the reason behind this is because the real constructor of the SomeClass is actually an anonymous function, whose name is not set.

To solve this problem, we need to declare the name of the constructor:

1
2
3
4
5

var SomeClass = (function() {
	return function SomeClass() {
		this.someProperty='some value';
	}
})();

or even define the constructor like this

1
2
3

var SomeClass = function SomeClass() {
	this.someProperty = 'some value';
}

The most common task when operating the servers is to distribute a file to multiple servers.
So I wrote a piece of shell script to solve this problem:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

#!/bin/bash  
echo "mscp <source file> <target dir>"  
SourceFile=$1  
TargetDir=$2  
echo "Copy $SourceFile to $TargetDir as $RemoteUser"  
echo "Enter the servers:"  
if [-f $SourceFile ]  
then  
  printf "File found, preparing to transfer\n"  
  while read server  
  do  
  scp -p $SourceFile ${server}:$TargetDir  
  done  
else  
  printf "File \"$SourceFile\"not found\n"  
  exit 1  
fi  
exit 0

call the script mscp <source file> <target dir>, then the script will ask you the list of target servers. So you can type them one by one. If the remote user is different than you current user, you can also explicitly identify it by typeing user@server

Beside the previous scenario, there is a more common sceanrio, that you have got a server list stored in afile already. Then instead of type the servers line by line, you can pipe the file content to the script.
e.g:

1

cat server_list.txt > mscp src_files dest_path

I downloaded the TextMate bundle for node.js.
But this bundle doesn’t work properly.
When i clicked cmd+R to run javascript, it reports that it cannot get variable “TM_FILE” from undefined.
And the console output contains a warning that module “sys” is renamed as “util”.

To fix this two issues:
Some fix these 2 issues, some modification to command script is needed:

1. Open Command Editor in TextMate, and edit the command script of “Run File or Spec” under “JavaScript Node” category:
2. Change var sys = require("sys"); to var sys = require("util"); to fix the warning.
3. Replace all instances of process.ENV. with process.env.

After modification, close the Bundle Editor. Then ask TextMate to reload all the bundles.
Then the command will work perfectly now.

There is another Trick, since there had been a bundle called javascript in TextMate. So this node js bundle doesn’t activated when you editing .js file.
You need to press ctrl + alt + n to activate the bundle manually.

This problem can be fixed by changing scope selector of all the snippets and commands. You can change it from “source.js.node” to “source.js”

尝试了一下用 LinqPad 把各种诡异的字母转成 Html 编码～结果发现不是左右字符都能转过去～
.net 内置的工具并不能完美的处理所有的 Html 编码～

字符来源

Query

1
2
3

"A,a,À,à,Á,á,Â,â,Ã,ã,Ä,ä,Å,å,Ā,ā,Ă,ă,Ą,ą,Ǟ,ǟ,Ǻ,ǻ,Æ,æ,Ǽ,ǽ,B,b,Ḃ,ḃ,C,c,Ć,ć,Ç,ç,Č,č,Ĉ,ĉ,Ċ,ċ,D,d,Ḑ,ḑ,Ď,ď,Ḋ,ḋ,Đ,đ,Ð,ð,DZ,dz,DŽ,dž,E,e,È,è,É,é,Ě,ě,Ê,ê,Ë,ë,Ē,ē,Ĕ,ĕ,Ę,ę,Ė,ė,Ʒ,ʒ,Ǯ,ǯ,F,f,Ḟ,ḟ,ƒ,ff,fi,fl,ffi,ffl,ſt,G,g,Ǵ,ǵ,Ģ,ģ,Ǧ,ǧ,Ĝ,ĝ,Ğ,ğ,Ġ,ġ,Ǥ,ǥ,H,h,Ĥ,ĥ,Ħ,ħ,I,i,Ì,ì,Í,í,Î,î,Ĩ,ĩ,Ï,ï,Ī,ī,Ĭ,ĭ,Į,į,İ,ı,IJ,ij,J,j,Ĵ,ĵ,K,k,Ḱ,ḱ,Ķ,ķ,Ǩ,ǩ,ĸ,L,l,Ĺ,ĺ,Ļ,ļ,Ľ,ľ,Ŀ,ŀ,Ł,ł,LJ,lj,M,m,Ṁ,ṁ,N,n,Ń,ń,Ņ,ņ,Ň,ň,Ñ,ñ,ʼn,Ŋ,ŋ,NJ,nj,O,o,Ò,ò,Ó,ó,Ô,ô,Õ,õ,Ö,ö,Ō,ō,Ŏ,ŏ,Ø,ø,Ő,ő,Ǿ,ǿ,Œ,œ,P,p,Ṗ,ṗ,Q,q,R,r,Ŕ,ŕ,Ŗ,ŗ,Ř,ř,ɼ,S,s,Ś,ś,Ş,ş,Š,š,Ŝ,ŝ,Ṡ,ṡ,ſ,ß,T,t,Ţ,ţ,Ť,ť,Ṫ,ṫ,Ŧ,ŧ,Þ,þ,U,u,Ù,ù,Ú,ú,Û,û,Ũ,ũ,Ü,ü,Ů,ů,Ū,ū,Ŭ,ŭ,Ų,ų,Ű,ű,V,v,W,w,Ẁ,ẁ,Ẃ,ẃ,Ŵ,ŵ,Ẅ,ẅ,X,x,Y,y,Ỳ,ỳ,Ý,ý,Ŷ,ŷ,Ÿ,ÿ,Z,z,Ź,ź,Ž,ž,Ż,ż"
.Split(',')
.ToDictionary(k=>k,HttpUtility.HtmlEncode)

Result

Dictionary<String,String>
(304 items)