The Nifty Dom Fuzz-Off Of 2017

Posted past times Ivan Fratric,

Introduction

Historically, DOM engines conduct keep been 1 of the largest sources of spider web browser bugs. And piece inward the recent years the popularity of those kinds of bugs inward targeted attacks has somewhat fallen inward favor of Flash (which allows for cross-browser exploits) in addition to JavaScript engine bugs (which oft final result inward rattling powerful exploitation primitives), they are far from gone. For example, CVE-2016-9079 (a põrnikas that was used inward Nov 2016 against Tor Browser users) was a põrnikas inward Firefox’s DOM implementation, specifically the portion that handles SVG elements inward a spider web page. It is also a rare instance that a vendor volition issue a safety update that doesn’t comprise fixes for at to the lowest degree several DOM engine bugs.

An interesting belongings of many of those bugs is that they are to a greater extent than or less tardily to uncovering past times fuzzing. This is why a lot of safety researchers every bit good every bit browser vendors who tending almost safety invest into edifice DOM fuzzers in addition to associated infrastructure.

As a result, later joining , 1 of my get-go projects was to assay out the electrical flow nation of resilience of major spider web browsers against DOM fuzzing.

The fuzzer

For this projection I wanted to write a novel fuzzer which takes some of the ideas from my previous DOM fuzzing projects, but also improves on them in addition to implements novel features. Starting from scratch also allowed me to halt upwards amongst cleaner code that I’m open-sourcing together amongst this weblog post. The destination was non to create anything groundbreaking - every bit already noted past times safety researchers, many DOM fuzzers conduct keep begun to expression similar each other over time. Instead the destination was to create a fuzzer that has decent initial coverage, is easily understandable in addition to extendible in addition to tin hold upwards reused past times myself every bit good every bit other researchers for fuzzing other targets likewise exactly DOM fuzzing.

We named this novel fuzzer Domato (credits to Tavis for suggesting the name). Like most DOM fuzzers, Domato is generative, pregnant that the fuzzer generates a sample from scratch given a laid of grammars that describes HTML/CSS construction every bit good every bit diverse JavaScript objects, properties in addition to functions.

The fuzzer consists of several parts:

• The base of operations engine that tin generate a sample given an input grammar. This portion is intentionally fairly generic in addition to tin hold upwards applied to other problems likewise exactly DOM fuzzing.
• The primary script that parses the arguments in addition to uses the base of operations engine to create samples. Most logic that is DOM specific is captured inward this part.
• A laid of grammars for generating HTML, CSS in addition to JavaScript code.

One of the most hard aspects inward the generation-based fuzzing is creating a grammer or some other construction that describes the samples that are going to hold upwards created. In the past times I experimented amongst manually created grammars every bit good every bit grammars extracted automatically from spider web browser code. Each of these approaches has advantages in addition to drawbacks, then for this fuzzer I decided to exercise a hybrid approach:

1. I initially extracted DOM API declarations from .idl files inward Google Chrome Source. Similarly, I parsed Chrome’s layout tests to extract mutual (and non then common) names in addition to values of diverse HTML in addition to CSS properties.
2. Afterwards, this automatically extracted information was heavily manually edited to brand the generated samples to a greater extent than probable to trigger interesting behavior. One illustration of this are functions in addition to properties that conduct keep strings every bit input: Just because a DOM belongings takes a string every bit an input does non hateful that any string would conduct keep a pregnant inward the context of that property.

Otherwise, Domato supports features that you’d await from a DOM fuzzer such as:

• Generating multiple JavaScript functions that tin hold upwards used every bit targets for diverse DOM callbacks in addition to lawsuit handlers

• Implicit (through grammer definitions) back upwards for “interesting” APIs (e.g. the Range API) that conduct keep historically been prone to bugs.

Instead of going into much technical details here, the reader is referred to the fuzzer code in addition to documentation at https://github.com/google/domato. It is my promise that past times open-sourcing the fuzzer I would invite community contributions that would comprehend the areas I powerfulness conduct keep missed inward the fuzzer or grammer creation.

Setup

We tested five browsers amongst the highest marketplace position share: Google Chrome, Mozilla Firefox, Internet Explorer, Microsoft Edge in addition to Apple Safari. We gave each browser exactly about 100.000.000 iterations amongst the fuzzer in addition to recorded the crashes. (If nosotros fuzzed some browsers for longer than 100.000.000 iterations, exclusively the bugs institute inside this number of iterations were counted inward the results.) Running this number of iterations would conduct keep too long on a unmarried machine in addition to so requires fuzzing at scale, but it is nonetheless good inside the pay arrive at of a determined attacker. For reference, it tin hold upwards done for almost $1k on Google Compute Engine given the smallest possible VM size, preemptable VMs (which I recollect piece of work good for fuzzing jobs every bit they don’t demand to hold upwards up all the time) in addition to 10 seconds per run.

Here are additional details of the fuzzing setup for each browser:

• Google Chrome was fuzzed on an internal Chrome Security fuzzing cluster called ClusterFuzz. To fuzz Google Chrome on ClusterFuzz nosotros only needed to upload the fuzzer in addition to it was run automatically against diverse Chrome builds.

• Mozilla Firefox was fuzzed on internal Google infrastructure (linux based). Since Mozilla already offers Firefox ASAN builds for download, nosotros used that every bit a fuzzing target. Each crash was additionally verified against a release build.

• Internet Explorer eleven was fuzzed on Google Compute Engine running Windows Server 2012 R2 64-bit. Given the lack of ASAN build, page heap was applied to iexplore.exe procedure to displace into easier to grab some types of issues.

• Microsoft Edge was the exclusively browser nosotros couldn’t easily fuzz on Google infrastructure since Google Compute Engine doesn’t back upwards Windows 10 at this fourth dimension in addition to Windows Server 2016 does non include Microsoft Edge. That’s why for fuzzing it nosotros created a virtual cluster of Windows 10 VMs on Microsoft Azure. Same every bit amongst Internet Explorer, page heap was applied to MicrosoftEdgeCP.exe procedure earlier fuzzing.

• Instead of fuzzing Safari directly, which would require Apple hardware, nosotros instead used WebKitGTK+ which nosotros could run on internal (Linux-based) infrastructure. We created an ASAN construct of the release version of WebKitGTK+. Additionally, each crash was verified against a nightly ASAN WebKit construct running on a Mac.

Results

Without farther ado, the number of safety bugs institute inward each browsers are captured inward the tabular array below.

Only safety bugs were counted inward the results (doing anything else is tricky every bit some browser vendors create non-security crashes piece some don’t) in addition to exclusively bugs affecting the currently released version of the browser at the fourth dimension of fuzzing were counted (as nosotros don’t know if bugs inward evolution version would hold upwards caught past times internal review in addition to fuzzing procedure earlier release).

Vendor	Browser	Engine	Number of Bugs	Bug IDs
Google	Chrome	Blink	2	994, 1024
Mozilla	Firefox	Gecko	4**	1130, 1155, 1160, 1185
Microsoft	Internet Explorer	Trident	4	1011, 1076, 1118, 1233
Microsoft	Edge	EdgeHtml	6	1011, 1254, 1255, 1264, 1301, 1309
Apple	Safari	WebKit	17	999, 1038, 1044, 1080, 1082, 1087, 1090, 1097, 1105, 1114, 1241, 1242, 1243, 1244, 1246, 1249, 1250
Total			31*

*While adding the number of bugs results inward 33, ii of the bugs affected multiple browsers

**The rootage displace of 1 of the bugs institute inward Mozilla Firefox was inward the Skia graphics library in addition to non inward Mozilla source. However, since the relevant code was contributed past times Mozilla engineers, I consider it fair to count here.

All of the bugs listed hither conduct keep been fixed inward the electrical flow transportation versions of the browsers. As tin hold upwards seen inward the tabular array most browsers did relatively good inward the experiment amongst exclusively a yoke of safety relevant crashes found. Since using the same methodology used to final result inward significantly higher number of issues exactly several years ago, this shows clear progress for most of the spider web browsers. For most of the browsers the differences are non sufficiently statistically important to justify proverb that 1 browser’s DOM engine is improve or worse than another.

However, Apple Safari is a clear outlier inward the experiment amongst significantly higher number of bugs found. This is specially worrying given attackers’ involvement inward the platform every bit evidenced past times the exploit prices in addition to recent targeted attacks. It is also interesting to compare Safari’s results to Chrome’s, every bit until a yoke of years ago, they were using the same DOM engine (WebKit). It appears that later the Blink/Webkit split upwards either the number of bugs inward Blink got significantly reduced or a important number of bugs got introduced inward the novel WebKit code (or both). To effort to address this discrepancy, I reached out to Apple Security proposing to part the tools in addition to methodology. When 1 of the members decided to transfer to Apple, he contacted me in addition to asked if the offering was nonetheless valid. So Apple received a re-create of the fuzzer in addition to volition hopefully exercise it to improve WebKit.

It is also interesting to uncovering the outcome of MemGC, a use-after-free mitigation inward Internet Explorer in addition to Microsoft Edge. When this mitigation is disabled using the registry flag OverrideMemoryProtectionSetting, a lot to a greater extent than bugs appear. However, Microsoft considers these bugs strongly mitigated past times MemGC in addition to I concur amongst that assessment. Given that IE used to hold upwards plagued amongst use-after-free issues, MemGC is an illustration of a useful mitigation that results inward a clear positive real-world impact. Kudos to Microsoft’s squad behind it!

When interpreting the results, it is rattling of import to greenback that they don’t necessarily reverberate the safety of the whole browser in addition to instead focus on exactly a unmarried element (DOM engine), but 1 that has historically been a source of many safety issues. This experiment does non conduct keep into concern human relationship other aspects such every bit presence in addition to safety of a sandbox, bugs inward other components such every bit scripting engines etc. I tin also non disregard the possibility that, inside DOM, my fuzzer is to a greater extent than capable at finding sure types of issues than other, which powerfulness conduct keep an outcome on the overall stats.

Experimenting amongst coverage-guided DOM fuzzing

Since coverage-guided fuzzing seems to create rattling expert results inward other areas nosotros wanted to combine it amongst the DOM fuzzing. We built an experimental coverage-guided DOM fuzzer in addition to ran it against Internet Explorer. IE was selected every bit a target both because of the author's familiarity amongst it in addition to because it is rattling tardily to boundary coverage collection to exactly the DOM element (mshtml.dll). The experimental fuzzer used a modified Domato engine to generate mutations in addition to used a modified WinAFL's DynamoRIO customer to stair out coverage. The fuzzing flow worked roughly every bit follows:

1. The fuzzer generates a novel laid of samples past times mutating existing samples inward the corpus.
2. The fuzzer spawns IE procedure which opens a harness HTML page.
3. The harness HTML page instructs the fuzzer to start measurement coverage in addition to loads 1 of the samples inward an iframe
4. After the sample executes, it notifies the harness which notifies the fuzzer to halt collecting coverage.
5. Coverage map is examined in addition to if it contains unseen coverage, the corresponding sample is added to the corpus.
6. Go to stair three until all samples are executed or the IE procedure crashes
7. Periodically minimize the corpus using the AFL’s cmin algorithm.
8. Go to stair 1.

The next laid of mutations was used to create novel samples from the existing ones:

• Adding novel CSS rules
• Adding novel properties to the existing CSS rules
• Adding novel HTML elements
• Adding novel properties to the existing HTML elements
• Adding novel JavaScript lines. The novel lines would hold upwards aware of the existing JavaScript variables in addition to could so reuse them.

Unfortunately, piece nosotros did run across a steady growth inward the collected coverage over fourth dimension piece running the fuzzer, it did non final result inward whatever novel crashes (i.e. crashes that would non hold upwards discovered using dumb fuzzing). It would seem to a greater extent than investigation is required inward social club to combine coverage information amongst DOM fuzzing inward a meaningful way.

Conclusion

As stated before, DOM engines conduct keep been 1 of the largest sources of spider web browser bugs. While this type of põrnikas are far from gone, most browsers demonstrate clear progress inward this area. The results also highlight the importance of doing continuous safety testing every bit bugs larn introduced amongst novel code in addition to a relatively brusque catamenia of evolution tin significantly deteriorate a product’s safety posture.

The large query at the halt is: Are nosotros at nowadays at a phase where it is to a greater extent than worthwhile to expression for safety bugs manually than via fuzzing? Or make to a greater extent than targeted fuzzers demand to hold upwards created instead of using generic DOM fuzzers to attain improve results? And if nosotros are non at that topographic point yet - volition nosotros hold upwards at that topographic point presently (hopefully)? The answer sure depends on the browser in addition to the somebody inward question. Instead of attempting to answer these questions myself, I would similar to invite the safety community to allow us know their thoughts.