The Dandy Dom Fuzz-Off Of 2017

Posted past times Ivan Fratric,

Introduction

Historically, DOM engines have got been 1 of the largest sources of spider web browser bugs. And acre inwards the recent years the popularity of those kinds of bugs inwards targeted attacks has somewhat fallen inwards favor of Flash (which allows for cross-browser exploits) too JavaScript engine bugs (which oft outcome inwards really powerful exploitation primitives), they are far from gone. For example, CVE-2016-9079 (a põrnikas that was used inwards Nov 2016 against Tor Browser users) was a põrnikas inwards Firefox’s DOM implementation, specifically the component that handles SVG elements inwards a spider web page. It is also a rare representative that a vendor volition issue a safety update that doesn’t comprise fixes for at to the lowest degree several DOM engine bugs.

An interesting belongings of many of those bugs is that they are to a greater extent than or less slow to discovery past times fuzzing. This is why a lot of safety researchers equally good equally browser vendors who tending close safety invest into edifice DOM fuzzers too associated infrastructure.

As a result, afterward joining , 1 of my outset projects was to assay out the electrical flow nation of resilience of major spider web browsers against DOM fuzzing.

The fuzzer

For this projection I wanted to write a novel fuzzer which takes some of the ideas from my previous DOM fuzzing projects, but also improves on them too implements novel features. Starting from scratch also allowed me to terminate upwards amongst cleaner code that I’m open-sourcing together amongst this weblog post. The destination was non to create anything groundbreaking - equally already noted past times safety researchers, many DOM fuzzers have got begun to facial expression similar each other over time. Instead the destination was to create a fuzzer that has decent initial coverage, is easily understandable too extendible too tin ship away live on reused past times myself equally good equally other researchers for fuzzing other targets too precisely DOM fuzzing.

We named this novel fuzzer Domato (credits to Tavis for suggesting the name). Like most DOM fuzzers, Domato is generative, important that the fuzzer generates a sample from scratch given a laid of grammars that describes HTML/CSS construction equally good equally diverse JavaScript objects, properties too functions.

The fuzzer consists of several parts:

• The base of operations engine that tin ship away generate a sample given an input grammar. This component is intentionally fairly generic too tin ship away live on applied to other problems too precisely DOM fuzzing.
• The principal script that parses the arguments too uses the base of operations engine to create samples. Most logic that is DOM specific is captured inwards this part.
• A laid of grammars for generating HTML, CSS too JavaScript code.

One of the most hard aspects inwards the generation-based fuzzing is creating a grammer or some other construction that describes the samples that are going to live on created. In the past times I experimented amongst manually created grammars equally good equally grammars extracted automatically from spider web browser code. Each of these approaches has advantages too drawbacks, too so for this fuzzer I decided to utilisation a hybrid approach:

1. I initially extracted DOM API declarations from .idl files inwards Google Chrome Source. Similarly, I parsed Chrome’s layout tests to extract mutual (and non too so common) names too values of diverse HTML too CSS properties.
2. Afterwards, this automatically extracted information was heavily manually edited to brand the generated samples to a greater extent than probable to trigger interesting behavior. One representative of this are functions too properties that have got strings equally input: Just because a DOM belongings takes a string equally an input does non hateful that any string would have got a important inwards the context of that property.

Otherwise, Domato supports features that you’d await from a DOM fuzzer such as:

• Generating multiple JavaScript functions that tin ship away live on used equally targets for diverse DOM callbacks too lawsuit handlers

• Implicit (through grammer definitions) back upwards for “interesting” APIs (e.g. the Range API) that have got historically been prone to bugs.

Instead of going into much technical details here, the reader is referred to the fuzzer code too documentation at https://github.com/google/domato. It is my promise that past times open-sourcing the fuzzer I would invite community contributions that would encompass the areas I mightiness have got missed inwards the fuzzer or grammer creation.

Setup

We tested five browsers amongst the highest marketplace share: Google Chrome, Mozilla Firefox, Internet Explorer, Microsoft Edge too Apple Safari. We gave each browser or too so 100.000.000 iterations amongst the fuzzer too recorded the crashes. (If nosotros fuzzed some browsers for longer than 100.000.000 iterations, exclusively the bugs constitute inside this number of iterations were counted inwards the results.) Running this number of iterations would have got too long on a unmarried machine too hence requires fuzzing at scale, but it is even too so good inside the pay attain of a determined attacker. For reference, it tin ship away live on done for close $1k on Google Compute Engine given the smallest possible VM size, preemptable VMs (which I retrieve piece of work good for fuzzing jobs equally they don’t demand to live on upwards all the time) too 10 seconds per run.

Here are additional details of the fuzzing setup for each browser:

• Google Chrome was fuzzed on an internal Chrome Security fuzzing cluster called ClusterFuzz. To fuzz Google Chrome on ClusterFuzz nosotros precisely needed to upload the fuzzer too it was run automatically against diverse Chrome builds.

• Mozilla Firefox was fuzzed on internal Google infrastructure (linux based). Since Mozilla already offers Firefox ASAN builds for download, nosotros used that equally a fuzzing target. Each crash was additionally verified against a release build.

• Internet Explorer xi was fuzzed on Google Compute Engine running Windows Server 2012 R2 64-bit. Given the lack of ASAN build, page heap was applied to iexplore.exe procedure to locomote inwards easier to grab some types of issues.

• Microsoft Edge was the exclusively browser nosotros couldn’t easily fuzz on Google infrastructure since Google Compute Engine doesn’t back upwards Windows 10 at this fourth dimension too Windows Server 2016 does non include Microsoft Edge. That’s why for fuzzing it nosotros created a virtual cluster of Windows 10 VMs on Microsoft Azure. Same equally amongst Internet Explorer, page heap was applied to MicrosoftEdgeCP.exe procedure earlier fuzzing.

• Instead of fuzzing Safari directly, which would require Apple hardware, nosotros instead used WebKitGTK+ which nosotros could run on internal (Linux-based) infrastructure. We created an ASAN create of the release version of WebKitGTK+. Additionally, each crash was verified against a nightly ASAN WebKit create running on a Mac.

Results

Without farther ado, the number of safety bugs constitute inwards each browsers are captured inwards the tabular array below.

Only safety bugs were counted inwards the results (doing anything else is tricky equally some browser vendors create non-security crashes acre some don’t) too exclusively bugs affecting the currently released version of the browser at the fourth dimension of fuzzing were counted (as nosotros don’t know if bugs inwards evolution version would live on caught past times internal review too fuzzing procedure earlier release).

Vendor	Browser	Engine	Number of Bugs	Bug IDs
Google	Chrome	Blink	2	994, 1024
Mozilla	Firefox	Gecko	4**	1130, 1155, 1160, 1185
Microsoft	Internet Explorer	Trident	4	1011, 1076, 1118, 1233
Microsoft	Edge	EdgeHtml	6	1011, 1254, 1255, 1264, 1301, 1309
Apple	Safari	WebKit	17	999, 1038, 1044, 1080, 1082, 1087, 1090, 1097, 1105, 1114, 1241, 1242, 1243, 1244, 1246, 1249, 1250
Total			31*

*While adding the number of bugs results inwards 33, two of the bugs affected multiple browsers

**The rootage crusade of 1 of the bugs constitute inwards Mozilla Firefox was inwards the Skia graphics library too non inwards Mozilla source. However, since the relevant code was contributed past times Mozilla engineers, I consider it fair to count here.

All of the bugs listed hither have got been fixed inwards the electrical flow transportation versions of the browsers. As tin ship away live on seen inwards the tabular array most browsers did relatively good inwards the experiment amongst exclusively a span of safety relevant crashes found. Since using the same methodology used to outcome inwards significantly higher number of issues precisely several years ago, this shows clear progress for most of the spider web browsers. For most of the browsers the differences are non sufficiently statistically important to justify maxim that 1 browser’s DOM engine is ameliorate or worse than another.

However, Apple Safari is a clear outlier inwards the experiment amongst significantly higher number of bugs found. This is peculiarly worrying given attackers’ involvement inwards the platform equally evidenced past times the exploit prices too recent targeted attacks. It is also interesting to compare Safari’s results to Chrome’s, equally until a span of years ago, they were using the same DOM engine (WebKit). It appears that afterward the Blink/Webkit dissever either the number of bugs inwards Blink got significantly reduced or a important number of bugs got introduced inwards the novel WebKit code (or both). To effort to address this discrepancy, I reached out to Apple Security proposing to part the tools too methodology. When 1 of the members decided to transfer to Apple, he contacted me too asked if the offering was even too so valid. So Apple received a re-create of the fuzzer too volition hopefully utilisation it to improve WebKit.

It is also interesting to discovery the lawsuit of MemGC, a use-after-free mitigation inwards Internet Explorer too Microsoft Edge. When this mitigation is disabled using the registry flag OverrideMemoryProtectionSetting, a lot to a greater extent than bugs appear. However, Microsoft considers these bugs strongly mitigated past times MemGC too I concord amongst that assessment. Given that IE used to live on plagued amongst use-after-free issues, MemGC is an representative of a useful mitigation that results inwards a clear positive real-world impact. Kudos to Microsoft’s squad behind it!

When interpreting the results, it is really of import to authorities annotation that they don’t necessarily reverberate the safety of the whole browser too instead focus on precisely a unmarried ingredient (DOM engine), but 1 that has historically been a source of many safety issues. This experiment does non have got into concern human relationship other aspects such equally presence too safety of a sandbox, bugs inwards other components such equally scripting engines etc. I tin ship away also non disregard the possibility that, inside DOM, my fuzzer is to a greater extent than capable at finding sure types of issues than other, which mightiness have got an lawsuit on the overall stats.

Experimenting amongst coverage-guided DOM fuzzing

Since coverage-guided fuzzing seems to make really skillful results inwards other areas nosotros wanted to combine it amongst the DOM fuzzing. We built an experimental coverage-guided DOM fuzzer too ran it against Internet Explorer. IE was selected equally a target both because of the author's familiarity amongst it too because it is really slow to boundary coverage collection to precisely the DOM ingredient (mshtml.dll). The experimental fuzzer used a modified Domato engine to generate mutations too used a modified WinAFL's DynamoRIO customer to stair out coverage. The fuzzing flow worked roughly equally follows:

1. The fuzzer generates a novel laid of samples past times mutating existing samples inwards the corpus.
2. The fuzzer spawns IE procedure which opens a harness HTML page.
3. The harness HTML page instructs the fuzzer to start mensuration coverage too loads 1 of the samples inwards an iframe
4. After the sample executes, it notifies the harness which notifies the fuzzer to halt collecting coverage.
5. Coverage map is examined too if it contains unseen coverage, the corresponding sample is added to the corpus.
6. Go to stair iii until all samples are executed or the IE procedure crashes
7. Periodically minimize the corpus using the AFL’s cmin algorithm.
8. Go to stair 1.

The next laid of mutations was used to make novel samples from the existing ones:

• Adding novel CSS rules
• Adding novel properties to the existing CSS rules
• Adding novel HTML elements
• Adding novel properties to the existing HTML elements
• Adding novel JavaScript lines. The novel lines would live on aware of the existing JavaScript variables too could hence reuse them.

Unfortunately, acre nosotros did meet a steady growth inwards the collected coverage over fourth dimension acre running the fuzzer, it did non outcome inwards whatever novel crashes (i.e. crashes that would non live on discovered using dumb fuzzing). It would seem to a greater extent than investigation is required inwards gild to combine coverage information amongst DOM fuzzing inwards a meaningful way.

Conclusion

As stated before, DOM engines have got been 1 of the largest sources of spider web browser bugs. While this type of põrnikas are far from gone, most browsers demo clear progress inwards this area. The results also highlight the importance of doing continuous safety testing equally bugs larn introduced amongst novel code too a relatively brusque menstruum of evolution tin ship away significantly deteriorate a product’s safety posture.

The large enquiry at the terminate is: Are nosotros at nowadays at a phase where it is to a greater extent than worthwhile to facial expression for safety bugs manually than via fuzzing? Or make to a greater extent than targeted fuzzers demand to live on created instead of using generic DOM fuzzers to attain ameliorate results? And if nosotros are non at that spot yet - volition nosotros live on at that spot shortly (hopefully)? The response sure depends on the browser too the somebody inwards question. Instead of attempting to response these questions myself, I would similar to invite the safety community to allow us know their thoughts.