Olivier Langlois's blog

It is an ok Gigabit router with its wireless functionality working fine. However when connecting a PS3 on the router and let it test the router capabilities, the router fails the UPnP testing. In my opinion, this is inexcusable when you sell a product at premium price and you market it as a "gaming" router, you would expect the company to have tested it with all the mainstream consoles.

I have contacted Cisco/Linksys customer support chat service to report the problem and get assistance and basically, I have been told something like that the problem was my fault and that I had to change some obscure and unrelated WiFi settings to fix my problem.

I was not satisfied by the answer and I decided to see for myself why the PS3 is reporting a UPnP failure with that router.

2 tools have been required for the analysis.

1. Wireshark
2. An Ethernet hub

A hub is a piece of networking hardware allowing you to share an Ethernet link between more than 2 network devices. Those were common in the 90s when processing power and memory were expensive. Today they have been replaced by the more robust and powerful Ethernet switches. My setup has been to plug my PC between the Linksys router and my PS3 and let Wireshark running on my PC sniff the UPnP exchange between them.

Before going in the details of the PS3 UPnP test, I want to share some interesting details about the whole PS3 Internet testing which is close to be undocumented on the Internet

Testing Internet Connection:

It does it by doing a HTTP request on fus01.ps3.update.playstation.net.


GET /update/ps3/list/us/ps3-updatelist.txt HTTP/1.1
Host: fus01.ps3.update.playstation.net
Connection: Keep-Alive
Accept-Encoding: identity
User-Agent: PS3Update-agent/1.0.0 libhttp/1.0.0

HTTP/1.1 200 OK
Server: Apache
ETag: "4d8c4dbf774c6349ad778577e53bd8c7:1285037260"
Last-Modified: Tue, 21 Sep 2010 02:47:40 GMT
Accept-Ranges: bytes
Content-Length: 252
Content-Type: text/plain
Date: Mon, 06 Dec 2010 03:22:33 GMT
Connection: keep-alive

# US
Dest=84;CompatibleSystemSoftwareVersion=3.5000-;
Dest=84;ImageVersion=0000b437;SystemSoftwareVersion=3.5000;CDN=http://dus01.ps3.update.playstation.net/update/ps3/image/us/2010_0921_0215e26d1dadeb950471a9c3397a140a/PS3UPDAT.PUP;CDN_Timeout=30;


Test PSN connection:

It does so by establishing an HTTPS connection with auth.np.ac.playstation.net. Obviously since it is encrypted, I cannot comment much about it.

NAT Type testing.

The PS3 sends a series of STUN (Session Traversal Utilities for NAT) (I am mentionning STUN in another blog entry for the curious) requests to us.np.stun.playstation.net

So now, lets get back to the UPnP testing. Here is a short overview of the whole procedure:

• Broadcast a request to find all network devices that are a InternetGatewayDevice
• Query its capability by requesting a XML file
• If the service WANIPConnection is supported, continue the test
• Call GetExternalIPAddress to obtain the router public Internet IP address
• Call AddPortMapping to open a public port on which all inbound traffic will be forwarded to the PS3

Here is how goes the exchange with the WRT330N router:


PS3 Broadcast:

M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 5
ST: urn:schemas-upnp-org:device:InternetGatewayDevice:1

2 seconds timeout. Retransmit the same request

After about 1.5 seconds after the 2nd retransmission, the router finally reply:

HTTP/1.1 200 OK
EXT:
SERVER: ipOS/7.2, UPnP/1.0, ipSSDPDevice/1.0
ST: urn:schemas-upnp-org:device:InternetGatewayDevice:1
LOCATION: http://192.168.1.1/root.sxml
CACHE-CONTROL: max-age=1800
USN: uuid:1C7AE0B4-AF9D-3FD9-AC27-04FAE3357DD5::urn:schemas-upnp-org:device:InternetGatewayDevice:1
Content-Length: 0

PS3 request this:

GET /root.sxml HTTP/1.1
HOST: 192.168.1.1:80

PS3 UPnP request:

POST /wipconn HTTP/1.1
HOST: 192.168.1.1:4444
Content-Length: 290
Content-Type: text/xml; charset="utf-8"
SOAPACTION: "urn:schemas-upnp-org:service:WANIPConnection:1#GetExternalIPAddress"

Linksys Router:

HTTP/1.1 200 OK
SERVER: ipOS/6.8 UPnP/1.0 IGD/1.0
EXT:
Transfer-Encoding: Chunked

197
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><soap:Body><u:GetExternalIPAddressResponse xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1"><NewExternalIPAddress>xxx.xxx.xxx.xxx</NewExternalIPAddress></u:GetExternalIPAddressResponse></soap:Body></soap:Envelope>
0

2nd PS3 UPnP request:

POST /wipconn HTTP/1.1
HOST: 192.168.1.1:4444
Content-Length: 644
Content-Type: text/xml; charset="utf-8"
SOAPACTION: "urn:schemas-upnp-org:service:WANIPConnection:1#AddPortMapping"

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:AddPortMapping xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
<NewRemoteHost></NewRemoteHost>
<NewExternalPort>3658</NewExternalPort>
<NewProtocol>UDP</NewProtocol>
<NewInternalPort>3658</NewInternalPort>
<NewInternalClient>192.168.1.109</NewInternalClient>
<NewEnabled>1</NewEnabled>
<NewPortMappingDescription>192.168.1.109:3658 to 3658 (UDP)</NewPortMappingDescription>
<NewLeaseDuration>0</NewLeaseDuration>
</u:AddPortMapping>
</s:Body>
</s:Envelope>

Linksys Router:

HTTP/1.1 200 OK
SERVER: ipOS/6.8 UPnP/1.0 IGD/1.0
EXT:
Transfer-Encoding: Chunked
151
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><soap:Body><u:AddPortMappingResponse xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1"></u:AddPortMappingResponse></soap:Body></soap:Envelope>
0


By looking the exchange it looks all good and in the router log, there is even an entry that shows that the Port mapping occured:


[INFO] Sun Dec 05 20:40:09 2010 UPnP renew entry 255.255.255.255 <-> 24.37.208.168:3658 <-> 192.168.1.109:3658 UDP timeout:-1 '192.168.1.109:3658 to 3658 (UDP)'


I initially believed that the error is the returned length of the router reply for the AddPortMapping request. It reports a length of 151 bytes but if you count them, I come to something around 340 chars! The GetExternalIPAddress call has the same problem. The reported size is 197 while in fact, I count around 410 chars!

However, a coworker of mine pointed out to me that the chunk size values are in hex as described in the RFC 2616. So I can only conclude that the problem comes from the PS3 that does not handle correctly HTTP chunked transfer coding in the context of UPnP exchange.

Another indication that the chunked transfer encoding in the replies is the culprit of the problem, it is that at the end of both exchange between the PS3 and the WRT330N is that the PS3 is sending back TCP RST segments to the router which means that the PS3 is closing its connection with the router before the router having finished to send its replies.

For your reference, here is an exchange with my new NetGear router, the WNDR37AV that works like a charm!


In less than 2 ms the reply to the M-SEARCH query comes back.

HTTP/1.1 200 OK
Cache-Control: max-age=1800
ST: urn:schemas-upnp-org:device:InternetGatewayDevice:1
USN: uuid:12345678-0000-0000-0000-00000000abcd::urn:schemas-upnp-org:device:InternetGatewayDevice:1
EXT:
Server: Linux/2.6.15-1.2054_FC5 UPnP/1.0 miniupnpd/1.0
Location: http://192.168.1.1:5555/rootDesc.xml

PS3 request this:

GET /rootDesc.xml HTTP/1.1
HOST: 192.168.1.1:5555

PS3 UPnP request:

POST /ctl/IPConn HTTP/1.1
HOST: 192.168.1.1:5555
Content-Length: 290
Content-Type: text/xml; charset="utf-8"
SOAPACTION: "urn:schemas-upnp-org:service:WANIPConnection:1#GetExternalIPAddress"

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:GetExternalIPAddress xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
</u:GetExternalIPAddress>
</s:Body>
</s:Envelope>

Netgear reply:

HTTP/1.1 200 OK
Content-Type: text/xml; charset="utf-8"
Connection: close
Content-Length: 356
Server: Linux/2.6.15-1.2054_FC5 UPnP/1.0 miniupnpd/1.0
Ext:

<?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body><u:GetExternalIPAddressResponse xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1"><NewExternalIPAddress>xxx.xxx.xxx.xxx</NewExternalIPAddress></u:GetExternalIPAddressResponse></s:Body></s:Envelope>

2nd PS3 UPnP request:
POST /ctl/IPConn HTTP/1.1
HOST: 192.168.1.1:5555
Content-Length: 644
Content-Type: text/xml; charset="utf-8"
SOAPACTION: "urn:schemas-upnp-org:service:WANIPConnection:1#AddPortMapping"

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:AddPortMapping xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
<NewRemoteHost></NewRemoteHost>
<NewExternalPort>3658</NewExternalPort>
<NewProtocol>UDP</NewProtocol>
<NewInternalPort>3658</NewInternalPort>
<NewInternalClient>192.168.1.109</NewInternalClient>
<NewEnabled>1</NewEnabled>
<NewPortMappingDescription>192.168.1.109:3658 to 3658 (UDP)</NewPortMappingDescription>
<NewLeaseDuration>0</NewLeaseDuration>
</u:AddPortMapping>
</s:Body>
</s:Envelope>

Netgear reply:

HTTP/1.1 200 OK
Content-Type: text/xml; charset="utf-8"
Connection: close
Content-Length: 260
Server: Linux/2.6.15-1.2054_FC5 UPnP/1.0 miniupnpd/1.0
Ext:

<?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body><u:AddPortMappingResponse xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1"/>
</s:Body>
</s:Envelope>


In conclusion, I am disapointed by the quality of the Linksys product and the lack of attention of details that the company did put in the QA before the release of that product since the UPnP bug could have been caught by a very simple test in less than 5 minutes. Apparently the bug is in the PS3 that does not support UPnP interaction with HTTP replies using chunked transfer coding but I was expecting to purchase a router compatible with my PS3 when I bought a gaming router. For that reason, I prefer the NetGear router.

Happy networking!

AAC is a new audio codec standardized to supersede the MP3 codec. When I hear about new technologies, I am always a little skeptic since I have long understood that newer technology is too often not synonym with better technology. However, at the company that I am working for, StreamTheWorld, we have developed the first system that stream AAC through Flash players. On StreamTheWorld website, there is a demo where the player connects to one MP3 stream and to one AAC stream with the same bitrate of 48 kbps. Both streams contain the same song and the demo consists of slider that allows you to switch between the AAC version and the MP3 version. You must absolutely check the demo! (PS: Make sure that you have the latest Adobe Flash player installed as AAC playback as being added only in the latest release which is 9.0.115) The difference is so convincing that you will probably want to start converting all your music to AAC!

One note about AAC, is that there exist many versions of AAC. I am listing them in the order that they have been released:

• MPEG2 AAC
• MPEG4 AAC
• AACPlus
• AACPlus v2

All versions provide improvements over the previous versions and also provide backward compatibility. That is you could listen an AACPlus v2 stream with an MPEG2 AAC decoder. However, the additional information contained for AACPlus v2 will be discarded and the resulting audio will only be as good as an MPEG2 AAC audio stream can be. Players supporting AACPlus v2 includes:

• Adobe Flash player
• WinAmp and every player using the open source AAC decoder FAAD2.

iTunes support AAC but not AACPlus v2. Sony with its PS3 and Nintendo with the Wii gaming console have made AAC their official audio codec but I do not know which version they are supporting.

Today is my last day at Quazal, the online gaming middleware provider. I have been working there for the last year and a half and have been involved in many projects such as the Relic Company of heroes lobby server, Supreme Commander, Atari Test Drive unlimited M.O.O.R. (Massively Open Online Racing) and Ubisoft Splinter Cell Conviction.

I am also the person who designed and implemented the generalized routing system that enhances peer-to-peer connectivity in Net-Z that Quazal should announce in the next few days. There are many purposes for this new feature. First, it allows developers to create online games in a client/server topology. Secondly, routing can be used in a peer-to-peer setup to enable the connectivity between players in the event that they would be behind NAT routers that make the connection impossible (any combination of strict and moderate NAT routers). As an extra bonus, Net-Z bandwidth usage has been optimized during the implementation of the new feature.

In two weeks, I will start to work at Stream the world on their audio/video streaming server. Its going to be very a very cool challenge!

A NAT router is a device that allow many computers to share the same IP address. You can learn more about NAT itself from Internetworking with TCP/IP, Vol 1 (5th Edition). One drawback of using NAT is that to communicate with someone outside the private network, you must initiate the communication from the private network. This is all fine for contacting a web server but this is making peer to peer applications such as file sharing or gaming harder. There are some techniques to perform NAT traversal but what makes the matter worse is that NAT routers have a different NAT behavior from a model to the other and as if it was not bad enough, these behaviors are totally undocumented. Things are about to change as the RFC 4787 describes a set of behaviors that a router should have to be gaming friendly. So my advice as a person who has worked at Quazal, a big well known multiplayer online middleware company, if you want the best NAT router for online gaming for your video game console (Xbox, Xbox360, Nintendo Wii, Playstation 2, Playstation 3 or PSP), you should look for a router that is RFC 4787 compliant. They might be hard to find now as the RFC has been released just few months ago but if I personnally was looking to purchase a new router, that is what I would be looking for.

The basic principle to perform NAT traversal is that you need a server on the public network that will be used as a traversal probe relay. Lets say that client A and client B that are located behind 2 distinct NAT routers. They first have to contact the server. The server must keep the address from which it receive data from the clients. These addresses are the public address of the clients behind the NAT router as seen from the server. When client A wants to establish a connection with client B, it will first request the client B public address from the server and start sending probes to B public address. At the same time, client A will send a request to B by using the server as a relay to start sending probes to itself. What will happen is since both clients try to establish a connection with each other at the same time, there should be a midair collision where a hole has been created for the other on each of their respective router. This technique is called hole punching and there is an RFC describing a protocol that implement this principle. The protocol name is STUN (Simple Traversal of UDP for NAT).

NAT traversal for TCP is much harder because TCP handshaking for establishing a connection is asymmetric. To workaround that problem, you have to trick the TCP protocol and the router that might monitor the TCP handshake. There is not yet an official protocol but some proposals exist. One of them is STUNT (STUN for TCP) but the name is funny because it has also a second meaning. The way STUNT works is that both side will initiate a connection by sending a SYN packet with a short TTL (Time To Live). The TTL value must be carefully choosen so the packet goes out of the private network but does not reach the destination. Then the stack itself or a packet sniffer must read the sequence number contained in the SYN message and send the read value to the STUNT server. The server will then impersonate the other endpoint and spoof the SYN/ACK reply. Then the last step of the connection which is to send the final ACK can be sent as usual and if everything has worked fine, the TCP connection will be established normally. I will stop the NAT traversal explanation here as there are a lot of good resources on the net. Here are the most interesting:

• RFC 2663
• RFC 3022
• RFC 3027
• RFC 3489
• RFC 4787
• http://en.wikipedia.org/wiki/STUN
• http://en.wikipedia.org/wiki/UDP_hole_punching
• http://nutss.gforge.cis.cornell.edu/publications.php

Update (12/05/2010): I have written a review of Linksys and a Netgear gaming router if it can help you to choose the right router for your needs.