After sending the request to the server, Fiddler begins reading its response. Some common web scenarios (particular-ly streaming audio and video) are impacted negative(particular-ly by response buffering, so Fiddler permits streaming of responses. Streamed responses cannot be modified using Fiddler’s Inspectors. By default, only audio and video responses are configured to stream. You can configure all responses to stream using the toggle on the Fiddler toolbar, or you may stream selectively on a per-response basis by setting the Session’s bBufferResponse property to false using FiddlerScript.
When streaming is enabled for a response, each block of data read from the server is immediately passed to the client application. By default, if the client application closes its connection to Fiddler, Fiddler will continue to read the response from the server to permit collection of the entire response. If you’d prefer Fiddler to abort the download if the client disconnects, set the fiddler.network.streaming.AbortIfClientAborts preference to true.
When buffering is disabled for a response, FiddlerScript’s OnBeforeResponse method runs after the response is fully returned to the client. This (perhaps surprising) behavior allows operations that don’t modify the response (e.g.
logging the response body to a database) to work properly for all Sessions. If your code needs to determine whether a given response was streamed, it can test the Session’s BitFlags:
bool bWasStreamed = oSession.isFlagSet(SessionFlags.ResponseStreamed));
If you need to modify the response body before it is sent to the client, you must disable streaming by setting oSession.bBufferResponse=true in either the OnBeforeRequest or OnPeekAtResponseHeaders handlers.
COMET
Fiddler’s Inspectors will only show the response’s body after the response data has been completely read from the server. But what happens if the server’s response never ends? Such responses occur with streaming radio stations and on sites that use a web programming technique called COMET.
95 | Configuring Fiddler and Clients With COMET, a server uses a “hanging frame” or other mechanism to push data to the client using a long-held HTTP connection that trickles data to the client as needed. HTML5 introduces a similar mechanism called “Server Sent Events” which works using the same general mechanism. Because a server’s COMET response never really “finish-es,” the data returned on that connection is not normally visible to Fiddler’s Inspectors until the connection closes.
To enable such data to be inspected, Fiddler offers the COMETPeek command on the Web Session’s context menu.
When you invoke the command, Fiddler will take a “snapshot” of the in-progress response, allowing you to inspect the partial response read from the server.
Fiddler’s default behavior of buffering responses wreaks havoc with sites that depend on COMET. That’s because the server’s response never ends and therefore Fiddler’s buffering of a COMET stream will usually cause the web application to stall. Unfortunately, COMET responses aren’t marked in any way, so you will need to manually exempt such responses from buffering if you do not wish to enable the Streaming option globally.
96 | Configuring Fiddler and Clients
HTM L 5 W E B S O C K E T S
The HTML5 specification introduces WebSockets, a technology that enables real-time socket communication between a client and server. To create a WebSocket, the client first establishes a HTTP(S) connection to the server. Next, the client and server handshake and agree to use the WebSocket protocol for subsequent traffic on the connection. The ws:// and wss:// URI schemes are used for plaintext and secure WebSockets respectively, even though the initial handshakes take place over HTTP or HTTPS.
To help ensure that WebSocket traffic flows unimpeded through an proxy (like Fiddler), clients will first establish a CONNECT to the proxy, requesting a tunnel to the destination server. If the WebSocket URI uses the wss:// scheme, a HTTPS handshake then occurs. If the unsecure ws:// scheme was used, then the HTTPS handshake is skipped.
The client then uses the newly-established connection to send a HTTP GET request with an Upgrade header propos-ing a switch to the WebSocket protocol. If the server agrees to change protocols, it sends a HTTP/101 Switchpropos-ing Protocols response. You can easily observe this process using the Web Sessions list and Inspectors:
After the handshake, the client and server may send each other WebSocket messages in any order. Unlike HTTP’s request-response pattern, the server may send WebSocket messages to the client without the client first sending a WebSocket message to the server. Because HTTP messages are no longer exchanged on a connection which has been upgraded to the WebSocket protocol, you will not see the WebSocket messages in the Web Sessions list or the request or response Inspectors.
However, Fiddler does have the ability to parse WebSocket messages, and in future versions of Fiddler, a new type of Inspector will be introduced to enable display and modification of WebSocket traffic. For now, WebSocket messages
97 | Configuring Fiddler and Clients from the client and server are shown in the Log tab. The bytes of these messages are parsed and displayed in
plaintext:
Upgrading Session #47 to websocket
[WebSocket #47] Server->Client (12 bytes) TYPE: TEXT.
MESSAGE: connected,
FLAGS: 10000001 DATA: 10 bytes.
---
[WebSocket #47] Client->Server (14 bytes) TYPE: TEXT.
MESSAGE: timer,
FLAGS: 10000001 DATA: 6 bytes, masked using KEY: BD-36-84-D5.
---
[WebSocket #47] Server->Client (23 bytes) TYPE: TEXT.
MESSAGE: time,2012/4/30 14:50:38 FLAGS: 10000001 DATA: 23 bytes.
---
[WebSocket #47] Client->Server (8 bytes) TYPE: CLOSE.
CLOSE REASON: 1001
FLAGS: 10001000 DATA: 2 bytes, masked using KEY: 25-A2-56-AE.
---
To help prevent attacks against transparent proxies or other buggy but well-meaning intermediaries, a client’s WebSocket messages are typically “masked” by XOR’ing the message text against a masking key that is not visible to JavaScript running on the client. When logging the client’s WebSockets messages, Fiddler automatically unmasks the text for ease of reading.
As the standard finalizes and browsers implement the specification, more websites will begin building upon WebSockets. As that happens, Fiddler will be updated to better support this exciting new protocol.
98 | Configuring Fiddler and Clients
F I D D L E R A N D HT TPS
When visiting a HTTPS site in Fiddler, there’s not a lot to see by default. Instead of the expected list of requests and responses, you’ll only see one or more CONNECT tunnels:
The HTTPS protocol sandwiches an encrypted (SSL or TLS) connection between HTTP requests and the underlying TCP/IP network connection upon which those requests are sent. Network intermediaries or observers are thus prevented from viewing or modifying the HTTP traffic thanks to the use of the cryptographic protocols.
You might then be surprised to learn that Fiddler can both view and modify HTTPS traffic if configured appropriate-ly. Fiddler achieves this by using a “man-in-the-middle” approach to HTTPS, which means that when talking to the client, it pretends to be the server, and when talking to the server, it pretends to be the client.
The HTTPS protocol is explicitly designed to block this attack by using digital certificates to authenticate the identity of a HTTPS server (and optionally the client). When a client receives a certificate from the server, it validates that the certificate itself is trustworthy by determining whether it is chained to a Root Certification Authority that is trusted by the client or the operating system. Since you are typically running Fiddler on your own computer, you can reconfigure your browser or operating system to trust Fiddler’s root certificate. After you do so, the client application will not complain when it detects that traffic is being protected by Fiddler-generated certificates.
To enable HTTPS traffic decryption in Fiddler, click Tools > Fiddler Options. On the HTTPS tab, tick the box labeled Decrypt HTTPS traffic.
When you enable HTTPS decryption, Fiddler will generate a self-signed root certificate and a matching Private Key.
Fiddler will use this root certificate to generate HTTPS server certificates (also called “End Entity” certificates) for each secure site that you visit.
99 | Configuring Fiddler and Clients