Olivier Langlois's blog

I have submitted a patch for the bugzilla bug 3606:

http://bugzilla.dre.vanderbilt.edu/show_bug.cgi?id=3606

Fix function ACE::handle_ready + SOCK_Test modif to test the fix

The patch include the following:

1. Fix for the bug 3606 in function ACE::handle_ready
2. Modif of the unittest SOCK_Test to validate the fix
3. Removal of the define ACE_HAS_LIMITED_SELECT

There has been a very long thread about it last summer on the ACE mailing list and I was arguing that it was dangerous to use select() when poll() is available because if you pass an handle whose value that is higher than FD_SETSIZE (typically 1024), it causes a buffer overflow on the stack which is hard to debug.

At the end, a consensus emerged that ACE_HAS_LIMITED_SELECT should be removed. Actually if some platform would benefit from that, it would be safer to define ACE_HAS_UNLIMITED_SELECT instead.

4. Remove the duplicated code from ACE::handle_ready() at different locations to replace it with the appropriate ACE namespace function call.

For those who do not know what RAII is, you are probably using it without knowing it. One common use of this C++ idiom is to keep code manipulating mutexes exception safe. While I was writing a thread safe getter with a scoped lock like this:

A B::getA() const
{
  ScopedLock sc(m_lock);
  return m_a;
}

A doubt sparkled in my mind about whether or not the object copy performed by the return statement was protected by the lock. It must be since I have seen tons of functions using similar pattern but I could not explain why. I asked to another senior developer and he could not tell neither. By the way, this is what I love about the C++ language. You can have years of experience with it, there are millions of small details and any day, you may stumble on a mind boggling detail that force you to stop and think about it.

So if we come back to the original question, I guess that I could have checked what the standard document says about it but it is boring to do so. I prefer to do small scientific experiments to figure out myself what is the answer. So I wrote this small program:

#include <iostream>

class RAIIObj
{
  public:
  RAIIObj() { std::cout << "RAIIObj()\n"; }
  ~RAIIObj() { std::cout << "~RAIIObj()\n"; }
};

class A
{
  public:
  A() {}
  A( const A & ) { std::cout << "A( const A & )\n"; }
};

class B
{
  public:
  B() {}
  A getSyncAByVal() const;
  private:
  A m_a;
};

A B::getSyncAByVal() const
{
  RAIIObj scopedLock;
  return m_a;
}

int main( int argc, char *argv[] )
{
  B b;
  A retVal = b.getSyncAByVal();
  return 0;
}

and here is the output confirming that the getter code is correct:

RAIIObj()
A( const A & )
~RAIIObj()

Now that I know the answer, I have found another proof confirming that return statement object copies are performed before the local lock goes out of scope. If you were returning by value a local object, it absolutely must be copied before going out of scope.

Upon reception of RST segment, the receiving side will immediately abort the connection. This statement has more implications than just meaning that you will not be able to receive or send any more data to/from this connection. It also implies that any unread data still in the TCP reception buffer will be lost. This information can be found in TCP/IP Internetworking volume 2 and Unix Network Programming Volume 1 third edition but in my opinion those books do not put enough emphasis on that detail and if you are not reading these books to find this exact detail, it might slip away from your attention.

Now, what are the conditions to receive a RST segment? The easiest way is to enable the SO_LINGER option with a timeout value of 0 on a server socket. As soon as the connection will be closed by the server, it will send a RST. You can learn more about SO_LINGER in the book Unix Network Programming Volume 1 third edition. The other possibility is if a server receives data after having closed the connection. If you enable the LINGER option, sending RST is the expected behavior but you can easily be bitten by the second possibility and here is an example on how it can happen.

Imagine a HTTP server that limits the number of simultaneous connections. Upon accepting a new connection, it might compare a global connection counter against a configured limit and if the counter has reached the limit then immediately send back a 503 error and close the connection. Do you see the problem? The server is closing the connection before receiving the HTTP request. There is a high probability that the client will never receive the 503 reply because a RST segment will immediately follow the reply.

A not too good solution would be to call shutdown(SHUT_WR) before closing the socket. What this will do is force the server to send a FIN segment before the RST segment. I have tried this solution and this seems to greatly improve the probability that the client application will receive the data before receiving the RST segment.

A better way is to program the server to read the request even if it has no intention to actually process it. This ensures that no RST will be sent unintentionally.

There remains one question. What exactly means data reception on a closed connection? Does that only include new received TCP segment or could data still in the TCP receive buffer of a closing connection be considered as data arrived after the connection has been closed? My intuition is that this question is open to interpretation and the TCP behavior will vary from one platform to the other. However, even if your platform does not react with a RST if a TCP connection receive buffer is not empty when it is closing it; my opinion is that you should not rely on that since this is a race condition. You could be closing the connection before or after receiving the client request depending on the RTT (Round Time Trip) of the connection.