The Packer 2.0 Threat

• Date: January 24, 2008
• Author: Don Jackson, Senior Security Researcher

Web 2.0 features rely on JavaScript, the "J" in "AJAX". The popularity of Web 2.0-style sites and applications is driving a dramatic increase in the amount of JavaScript code required to implement features. Many Web 2.0 features are packaged together with a consistent API and distributed as a JavaScript "library". However, these libraries have to be loaded and reinterpreted for every page that uses a feature implemented in the library, and these libraries can be relatively large for web content -- tens or hundreds of kilobytes.

Although other, generally better, methods exist for reducing the number of bytes downloaded, the use of JavaScript "packers" has become accepted and widespread.

Computer hackers have taken advantage of the acceptance of these packers as suboptimal network optimization tactics and are using them as a way to evade and bypass security controls on the gateway and at the host. Consequently, exploits or other malicious code is delivered successfully because of the packer's ability to bypass anti-virus and IDS/IPS and directly to a user's vulnerable system.

Much of the debate around the use of JavaScript packers is analogous to the debate around the use of Windows PE executable (EXE file) packers used by malware distributors. While lessons can be found there, the information below focuses on the use of JavaScript packers specifically.

It's important that we have some basic knowledge of the most common packers and the tools that use them.

The Dojo Toolkit and Dojo ShrinkSafe

One of the most popular JavaScript libraries is the Dojo Toolkit. To reduce unnecessary code, developers build a Dojo library for the functions and languages they use in their site. The last step is compressing the library files, for which Dojo uses ShrinkSafe. "Safe" because:

1. It does not change things that break the API, like renaming class names to something short in order to save space.

2. It actually uses a custom build of the Rhino JavaScript engine to work its magic in the parser stream, reducing the chance that it will break code because of some pattern in the input that is not accounted for in the packer's custom text parser/regex functions.

It doesn't use anything that makes detection of malicious scripting too difficult. It keeps code debuggable. It saves space by removing comments and whitespace. Because of the way it does this, it doesn't produce the very smallest of files, and it doesn't impact performance. Many web developers are oblivious to the size vs. performance equation, but not the Dojo team. They even suggest not using it at all: "Gzipping content on the wire [mod_gzip, for example] is the next obvious improvement for those deploying applications." They have an excellent write up here:

http://shrinksafe.dojotoolkit.org/

MOOtools

"My Object Oriented" JavaScript Tools library source is available from the mootools.net Subversion server. However, most developers download it in the form of a "build", a .js file which includes only the functions they need. A developer visits the mootools page, checks the boxes next to the library functions needed and downloads the build that contains only the required code. An optional radio button allows users to package the .js file using a choice of:

• JavaScript Packer: The highest compression ratio. Uses the PHP5 version of Dean Edwards Packer. The output uses eval.
• YUI Compressor: Uses YUI Compressor to clean whitespace and rename internal variables to shorter strings. Does not use eval.
• JsMin Compression: Removes comments and whitespace.
• No Documentation: Uncompressed, removes only documentation/comments.
• Full Source: Includes documentation/comments. Recommended while testing & building.

According to Mootool, versions of their library are used on web sites for W3C, CNET, Ubuntu, Joomla, Gamespot, and Chrysler Jeep. Almost all of them use a compressed version. Gamespot, for example, uses the library as a file called mootools.js which has been packed with the Dean Edwards packer.

A build which contains all the Mootools v1.11 functionality and is not compressed at all weighs in at 180 KB.

More information on the Dean Edwards packer appears later. Let's compare the others:

The YUI Compressor

The "Yahoo! User Interface" Compressor also uses Rhino. Because it works in the Rhino parsing stream and has some sense of scope and context of variables, almost the same way as Dojo ShrinkSafe, it will not mangle the public API.

It is more aggressive on renaming local variables and classes and compressing literal values. It has more of a client performance impact than code packed with Dojo ShrinkSafe. This also makes it more difficult to audit and to decipher benign code from malicious code, especially in "Trojaned libraries" where one or more commonly used functions are used to call locally-scoped exploit code which has had it's namespace squished and jumbled.

More information can be found here: http://developer.yahoo.com/yui/compressor/

That page provides useful information such as a study on numbers of cache hits and misses, important data if one is determining impact of compression on networks. However, like all other compressor developers except the Dojo team, they ignore the fact that better options for network optimization exist.

JSMin

Douglas Crockford's "JavaScript Minifier" is a relatively simple pattern-based tool.

Originally implemented in C as an MS-DOS program, it has been ported to many other languages by third parties.

Because of the way it handles whitespace and some confusing patterns, "a + ++b", which it will turn into "a+++b" and is interpreted according to the ECMAScript standard as "a++ + b", which is not what the original code intended.

Therefore, this compressor is not considered safe because it can cause unintended changes in your code. It works best for simple, well written scripting that follows one of the standard best practices coding styles. One option is to first pipe the input through JSLint (written by the same author). Using our previous example, this would produce "a + (++b)". That output could then be piped into JSMin. Ironically, the addition of the parentheses makes the linted-and-minified version of this expression exactly the same size as the original: the strings "a + ++b" and "a+(++b)" are both seven characters .

While producing relatively small files and having no client-side performance impact, the unsafe behavior of its brittle pattern substitution design makes it more likely to break code than packers that operate on a parser stream. This is typically undesirable, as output must have its functionality verified separately from the source (that is to say two testing phases are required).

JSMin does not make it much more difficult to tell benign script from malicious script. However, that is not the reason it's not used by many cyber criminals.

Many attackers using JavaScript exploits like to pack pre-obfuscated versions of their exploits. That is all that's typically available to people who buy or simply use -- rather than actually write -- the exploit kits. That code is purposefully not styled nicely and makes for a flood of confusing warnings from JSLint. The attacker isn't sure if the malicious obfuscated code going through JSMin behaves in the same way or not. It's difficult to iteratively test true exploit code. Rather than take chances on the reliability of their exploits, attackers generally avoid this packer.

Dean Edwards Packer and IE7/IE8 Patches

Dean Edwards' most popular code is his IE7 (and now, IE8) JavaScript libraries. These are designed to normalize behavior related to HTML, DOM objects, and CSS cross IE5/IE6 and IE7 and IE8.

By including this JavaScript "patch" in a browser-based application, developers do not have to recode to account non-standard quirks in different IE versions. That is, an app written for IE6 can be made compatible with the newest IE versions simply by plugging this script into the app. Newer IE features, such as support for PNG alpha channels (transparency), will work correctly in older IE versions. All popular IE versions are on equal grounds as far as standard behaviors.

This is a huge time and money saver, and because of that, the file IE7.js might well be one of the most common JavaScript files on the Web.

The default download links lead to versions that are packed using Dean Edwards' own packer. Unpacked versions are available but are described as "source code". JavaScript, being interpreted, is by definition its own source code. The use of this term leads people who want to just "plug in" the script to avoid the unpacked version, as they may think the packer is some sort of compiler without which the script does not function (not true!).

The Dean Edwards "/packer/" is a tool which operates as a sort of dictionary-based compressor similar to LZ compression tools, except instead of working on bits and byte patterns, it works on JavaScript tokens/atoms. It does this in a way that is considered (but not proven) to be "safe" in the fact that the original and packed code will function identically, except for the initial unpacking function.

Parsing out the original code -- as a large string from another large string based on dictionary lookups in an array of strings created from another large string -- takes a heavy toll on client-side performance, especially on IE7, where the extra seconds it takes to load and render the page and execute its JavaScript are very noticeable to the human user. This is counter to the quick, intuitive interfaces which are supposed to be a hallmark of Web 2.0.

Additionally, this packer has to include a significant amount of unpacking logic. That logic is implemented in code attached to every packed script, which means there are many common blocks of script for which the packed script is actually larger than the original.

A small (less than 50 KB) packed JavaScript file is smaller than typical packed benign libraries and the size itself is a sign of possily malicious code.

The unpacking stub itself uses some dubious functions. The YUI Compressor developers link to information on "evil features" as described by the developer of JSLint, an ECMAScript standard-compliant tool which aims to reduce coding errors due to poor coding practices. From the JSLint documentation section titled "eval is evil":

"The eval function (and its relatives, Function, setTimeout, and setInterval) provide access to the JavaScript compiler. This is sometimes necessary, but in most cases it indicates the presence of extremely bad coding. The eval function is the most misused feature of JavaScript."

- Douglas Crockford, 2002, http://www.jslint.com/help.html

The eval function lies at the heart of the Dean Edwards packer. It is seen so often in obfuscated malicious JavaScript that many security companies protect clients from new JavaScript threats via heuristics based on statistical analysis of the use of eval in combination with other functions, the combination of which is seen almost exclusively in code designed to bypass other filters (ad blockers or anti-virus, for example).

The Dean Edwards packer makes it difficult to tell benign code from malicious code. Although it does not operate on a parser stream, its design allows it to safely pack pre-obfuscated exploits which may include patterns ambiguous to other pattern-based packers. This makes it attractive to attackers.

Because of its use of eval combined with other certain methods (such as String.fromCharCode) and layout/context of certain types of tokens, the output of the Dean Edwards packer is very similar, statistically, to other known malicious code. This would normally mean that attackers would gain nothing towards evasion by using it. However, it is now being used more in known malicious scripts than in known benign code. Why?

Because the Dean Edwards packer output is so similar to malicious code, but so widely used in benign code -- consider the popularity of IE7.js -- security companies have basically whitelisted its output based on its signature unpacking stub code which begins, cleverly:

eval(function(p,a,c,k,e,d...

Researchers have verified the whitelist status of this packer in at least one product by placing this stub around known exploit code. The exploit code alone generated an alert, which the (unpacked) exploit code surrounded by the packer stub sailed right through.

For years, in detecting malicious code using this packer, the risk of a false negative was very small compared to a risk of a false positive. It simply wasn't used by the cyber underground. That all changed with the success of an XSS worm which infected 700,000 users of the Orkut social networking service in November 2007.

In December 2007, a couple of security companies' products began blocking packed scripts with new signature updates. Because of complaints of false alarms from end users, both relented, leaving their customers at risk of encountering new packed exploits. In fact, shortly after re-whitelisting packed scripts, a SQL injection flaw in the web site of one of the security companies, allowed attackers to place script directing visitors to, ironically (or not?) , a packed RealPlayer exploit which installed a rootkit and backdoor from China.

Would-be attackers took notice, documented the whitelist status, and began packing their wares in order to successfully evade filters. One actual attack used both packed and unpacked versions of another RealPlayer exploit so the attackers could study the relative effectiveness of the use of this packer to evade filters. Heuristics often fail when encountering "Trojaned libraries" which are actually copies of common libraries laced with exploit code. These look and even function -- except under one unique circumstance or trigger -- identical to their benign counterparts. The files could be named the same as well. Packing increases the chance that heuristics will fail to detect previously unknown exploits, such as the packed 0-day QuickTime exploit uncovered in January 2008.

Today, there are more documented packed malicious scripts than packed benign scripts.

Summary

While the use of packers is widespread, all have drawbacks. These include:

• The inability to easily verify and audit code
• The administrative overhead of repacking code for each change
• Suboptimal compression
• The increased risk of false negatives which may lead to a site being used to spread malicious code
• The increased risk of false positives, which may lead to a site or some of its functions being blocked by security controls
• Noticeable negative impact on client-side performance.

Site owners, operators, developers and administrators can achieve intended results -- typically reducing the number of bytes downloaded from the server -- with greater degree of success and fewer side-effects using one or more alternative tactics:

• Not compressing JavaScript code at all
• Reliance on increases in average available bandwidth
• Reliance on local and network caching
• Not using more JavaScript functions than necessary (smaller library "builds")
• Using only safe whitespace/comment reduction techniques
• Automatic application of safe techniques as a last step in the publishing process
• The use of mod_deflate/mod_gzip for compressing the HTTP response data
• The use of jar files to package JavaScript (these can be cryptographically signed to further enhance authenticity of code, and hence improve security)

Replacing packed code is not generally an expensive proposition. Usually, libraries are available in unpacked versions. Checking the packed code out of a source code control/versioning system, copying over the unpacked code, and checking it back in to be published just like any normal code change is a quick procedure with immediate benefits.

This would improve both the security and quality of the abundance of JavaScript used in the world of Web 2.0.