Tuesday, September 29, 2009

Implementing Traceroute with System.Net - Part IV

In the previous parts, I showed how to implement traceroute using System.Net.Sockets. We used the Socket class to create a Raw socket, and use that to do a traceroute.

Implementing Traceroute with System.Net Part III
Implementing Traceroute with System.Net Part II
Implementing Traceroute with System.Net Part I

It turns out, that System.Net.NetworkInformation already has a Ping class, that encapsulates all the functionality of sending ICMP ECHO requests and processing replies. It has the necessary switches to tweak the TTL settings as well.

I wrote a small program to demonstrate how we can do a Traceroute using the Ping class.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Net.NetworkInformation;

namespace PingClassTest
{
    class Program
    {
        static void Main(string[] args)
        {
            Ping pinger = new Ping();
            String data = "0123456789ABCDEF";
            byte[] buffer = Encoding.ASCII.GetBytes(data);
            PingOptions options = new PingOptions();

            IPHostEntry target = Dns.GetHostEntry(args[0]);
            bool isDestination = false;
            for(int i = 1; i <= 30; i++)
            {
                String intermediateHost = null;
                for (int j = 0; j < 3; j++)
                {
                    options.Ttl = i;
                    PingReply reply = pinger.Send(target.AddressList[0], 30, buffer, options);
                    switch (reply.Status)
                    {
                        case IPStatus.TimedOut:
                        case IPStatus.TtlExpired:
                        case IPStatus.Success:
                            Console.Write("<{0}ms\t", reply.RoundtripTime);
                            if (reply.Address.Equals(target.AddressList[0]))
                            {
                                isDestination = true;
                            }
                            intermediateHost = reply.Address.ToString();
                            break;
                        default:
                            Console.Write("*\t");
                            break;

                    }
                }

                Console.WriteLine("\t{0}", intermediateHost);

                if (isDestination)
                {
                    break;
                }
            }javascript:void(0)
        }

        pinger.Dispose();
    }
}
The output of this program, when pinging www.yahoo.com is as follows:
<0ms    <0ms    <0ms            192.168.1.1
<0ms    <0ms    <0ms            98.117.116.1
<0ms    <0ms    <0ms            130.81.138.218
<0ms    <0ms    <0ms            130.81.28.166
<0ms    <0ms    <0ms            130.81.17.56
<0ms    <0ms    <0ms            130.81.17.231
<0ms    <0ms    <0ms            130.81.14.90
<0ms    <0ms    <0ms            216.115.107.57
<0ms    <0ms    <0ms            209.131.32.23
<27ms   <26ms   <26ms           209.131.36.158
Press any key to continue . . .
As you can see, this program gives the exact same functionality as the program that we wrote using the Socket class. The Ping class also supports asynchronous functionality which makes it very useful for high performance applications.

Thursday, September 17, 2009

Implementing Traceroute with System.Net - Part III

In the previous part http://ferozedaud.blogspot.com/2009/09/implementing-traceroute-with-systemnet_07.html I showed a simple implementation of traceroute, that built upon my earlier implementation of Ping.

As promised, in this article, we will see how to make the utility more robust.

One of the things I didnt like about the previous implementation, was that there was no verification of the received request packet. The implementation was not verifying whether the type of the response was correct - an ICMP Echo request should elicit an ICMP Echo response (see the ICMP RFC at http://www.ietf.org/rfc/rfc0792.txt)

So, we add the code for verifying that. According to the RFC, the Echo reply message has a Type Field = 0. Also, since we are using the TTL mechanism for detecting hosts along the route to the destination, we expect that every host but the final one, will reply with a ICMP Time Exceeded message, which has a Message Type = 11.

Also, the other issue with the tool, is that it does not resolve the IPAddress of the intermediate hosts. In a traceroute output, we are mostly interested in the hostnames of the intermediate routers, and not just the IP address. So, we will add that capability as well.

With these changes, here is the changed code. I have only put the changes to the while(true) loop in my original implementation.
while (true)
{
Console.Write("{0}", hop);
bool allTimedOut = true;
for (int i = 0; i < 3; i++)
{
ICMP_PACKET packet = ICMP_PACKET.CreateRequestPacket(111, 222, data);

IPEndPoint epRemote = new IPEndPoint(ipTarget, 0);

pingSocket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.IpTimeToLive, hop);


stopWatch.Start();
pingSocket.SendTo(packet.Serialize(), epRemote);



byte[] receiveData = new byte[1024];
int read = 0;
try
{
epResponse = new IPEndPoint(0, 0);
read = pingSocket.ReceiveFrom(receiveData, ref epResponse);
stopWatch.Stop();

ICMP_PACKET recvPacket = new ICMP_PACKET(receiveData, 20, read);

ipepResponse = epResponse as IPEndPoint;

if (recvPacket.PacketType == 11)
{
// this is an ICMP Time exceeded message
if (ipepResponse.Address.Equals(ipTarget))
{
// do not expect an ICMP Time Exceeded message from the
// final destination.
Console.Error.Write("\t!!");
}
}
else if (recvPacket.PacketType == 0)
{
// this is an ICMP Echo reply message
// validate that this is coming from the destination host.
if (!ipepResponse.Address.Equals(ipTarget))
{
// do not expect an ICMP Time Exceeded message from the
// final destination.
Console.Error.Write("\t!!");
}
}

Console.Write("\t<<{0}ms", stopWatch.ElapsedMilliseconds);
allTimedOut = false;
}
catch (SocketException e)
{
Console.Write("\t*");
}
}

if (allTimedOut)
{
Console.WriteLine("\tRequest timed out");
}
else
{
String intermediateHost = null;
// now try to resolve the IPAddress
try
{
IPHostEntry hostEntry = Dns.GetHostEntry(ipepResponse.Address);
intermediateHost = hostEntry.HostName;
}
catch (SocketException e)
{
}

Console.WriteLine("\t{0} {1}", ((IPEndPoint)epResponse).Address.ToString(), intermediateHost);

}
++hop;

ipepResponse = epResponse as IPEndPoint;
if (hop > maxHops || ipepResponse.Address.Equals(ipTarget))
{
break;
}
}


In the next part, I will throw some ideas out there, on how this can be extended even more.

Friday, September 11, 2009

Network Programmers Toolchest

In this post, I want to talk about some of the tools that a programmer doing network programmer might want to learn, in order to do his job better.

First, let us talk about logging tools. No matter which library you use, whether it is System.Net from .NET framework, or CommonsHTTP from the Apache Commons project, you need to have a way of enabling logging to see what is going on in the library.

If you are using System.Net, then you can enable logging using the Logging facility provided in the .NET framework.

http://ferozedaud.blogspot.com/2009/08/tracing-with-systemnet.html

For Apache ComonsHttp library, you can enable logging using the Log4J logging library:

http://logging.apache.org/log4j/index.html

Next, you need a way to capture network traffic in order to see what is going on in the network. Many tools exist that can allow you to do that. I prefer to use Wireshark, but on windows, Netmon is also pretty good.

http://www.wireshark.org
http://blogs.technet.com/netmon

Imagine that you are trying to write code that can simulate loggin in to a website. For eg, Facebook or a line of business application. However, it isnt working - you tried using logging but cant seem to get to the problem. In this case, you might want to use a browser and try out the same scenario, and see what the browser is doing differently.

You can do that by using Wireshark/Netmon to sniff the browser session, but it is not very user friendly. Plus, if the session is encrypted (HTTPS) you cannot even make sense of the network sniff.

In this case, it helps to have a browser extension that can sniff the outgoing requests and incoming responses for you. I like to use Firebug on Firefox, and there are equivalent extensions for Internet Explorer as well.

http://getfirebug.com/

Here are the ones for Internet Explorer.

http://www.fiddler2.com/Fiddler2/version.asp
http://www.httpwatch.com/


Readers: Do you have any favorite tools that I have not covered here? I would like to know about them. Add a comment to this post with the information.

Thursday, September 10, 2009

The case of multiple NTLM challenges with IIS7

A reader posed a question on Stack Overflow about a problem with authentication. The scenario was as follows:

This was a typical 3-tier application, in which the middle-tier (ASP.NET web server) was making a HttpWebRequest to a backend IIS-7 server that required authentication.

When authentication method was set to Digest/Basic/Negotiate, the server worked fine, and the authentication succeeded. However, if the auth method was set to NTLM, the server started to challenge twice.

The reader investigate this, and found that this was caused by a Microsoft Security Update http://support.microsoft.com/kb/957097. Also, he came up with a solution for the problem.

Read more about the problem and it's solution at: http://www.tinyint.com/index.php/2009/08/24/401-error-on-httpwebrequest-with-ntlm-authentication/

Monday, September 7, 2009

Implementing Traceroute with System.Net: Part-II

In this part, we will implement traceroute according to the principles we discussed in the last post:

http://ferozedaud.blogspot.com/2009/09/implementing-traceroute-with-systemnet.html

First, start with the code for Ping, which is at http://blogs.msdn.com/feroze_daud/archive/2005/10/26/485372.aspx and modify the Main() method as follows:

    static void Main(string[] args)
    {

        if (args.Length != 1)
        {
            Usage();
        }



        if (0 == String.Compare(args[0], "/?", true, CultureInfo.InvariantCulture)

        || 0 == String.Compare(args[0], "-h", true, CultureInfo.InvariantCulture)

        || 0 == String.Compare(args[0], "-?", true, CultureInfo.InvariantCulture))
        {
            Usage();
        }

        string target = args[0];

        IPAddress[] heTarget = Dns.GetHostAddresses(target);

        IPAddress ipTarget = null;

        foreach (IPAddress ip in heTarget)
        {

            if (ip.AddressFamily == AddressFamily.InterNetwork)
            {
                ipTarget = ip;
                break;
            }
        }



        IPAddress[] heSource = Dns.GetHostAddresses(Dns.GetHostName());

        IPAddress ipSource = null;

        foreach (IPAddress ip in heSource)
        {
            byte[] addressBytes = ip.GetAddressBytes();
            if (addressBytes[0] == 169 && addressBytes[1] == 254)
            {
                continue;
            }
            if (ip.AddressFamily == AddressFamily.InterNetwork)
            {
                ipSource = ip;
                break;
            }
        }

        IPEndPoint epLocal = new IPEndPoint(ipSource, 0);

        Socket pingSocket = new Socket(AddressFamily.InterNetwork, SocketType.Raw, ProtocolType.Icmp);

        pingSocket.Bind(epLocal);

        byte[] data = Encoding.ASCII.GetBytes("1234567890abcdef");

        Console.WriteLine("Ping {0}({1}) with {2} bytes of data...", target, ipTarget.ToString(), data.Length);

        Console.WriteLine();

        int hop = 1;
        int maxHops = 30;

        pingSocket.ReceiveTimeout = 30;
        System.Diagnostics.Stopwatch stopWatch = new System.Diagnostics.Stopwatch();
        EndPoint epResponse = (EndPoint)new IPEndPoint(0, 0);

        while (true)
        {
            Console.Write("{0}", hop);
            bool allTimedOut = true;
            for (int i = 0; i < 3; i++)
            {
                ICMP_PACKET packet = ICMP_PACKET.CreateRequestPacket(111, 222, data);

                IPEndPoint epRemote = new IPEndPoint(ipTarget, 0);

                pingSocket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.IpTimeToLive, hop);


                stopWatch.Start();
                pingSocket.SendTo(packet.Serialize(), epRemote);

                byte[] receiveData = new byte[1024];
                int read = 0;
                try
                {
                    epResponse = new IPEndPoint(0, 0);
                    read = pingSocket.ReceiveFrom(receiveData, ref epResponse);
                    stopWatch.Stop();
                    ICMP_PACKET recvPacket = new ICMP_PACKET(receiveData, 20, read);

                    Console.Write("\<<{0}ms", stopWatch.ElapsedMilliseconds);
                    allTimedOut = false;
                }
                catch (SocketException e)
                {
                    Console.Write("\t*");
                }
            }

            if (allTimedOut)
            {
                Console.WriteLine("\tRequest timed out");
            }
            else
            {
                Console.WriteLine("\t{0}", ((IPEndPoint)epResponse).Address.ToString());
            }
            ++hop;

            IPEndPoint ipepResponse = epResponse as IPEndPoint;
            if (hop > maxHops || ipepResponse.Address.Equals(ipTarget))
            {
                break;
            }
        }

        pingSocket.Close();
    }

As discussed earlier, we have now put a loop around the actual Socket.SendTo() call. This loop will run three times. We will continue sending the packet until we reach 30 hops max, or we have reached the destination. As you have noticed, since we did all the heavy lifting in the Ping program, all we had to do here, was to set the TimeToLive Socket option so that the routers/gateways on the path to the destination respond with a "Time Exceeded" message. When I run it on my machine, here is the output:
C:\>traceroute.exe www.yahoo.com
Ping www.yahoo.com(209.131.36.158) with 16 bytes of data...

1       <0ms    <1ms    <1ms    192.168.1.1
2       <5ms    <10ms   <14ms   <Address removed>
3       <18ms   <23ms   <27ms   <Address removed>
4       <33ms   <38ms   <43ms   <Address removed>
5       <68ms   <92ms   <116ms  <Address removed>
6       <143ms  <171ms  <197ms  <Address removed>
7       <223ms  <250ms  <276ms  <Address removed>
8       <304ms  <331ms  <365ms  <Address removed>
9       <391ms  <417ms  <445ms  <Address removed>
10      <473ms  <510ms  <538ms  <Address removed>


Note, that I have removed the actual IPAddresses from the output, for privacy reasons.

In the next part, we will discuss how we can make this utility more robust.

Friday, September 4, 2009

Implementing traceroute with System.Net: Part-I

In the last part, I started by giving links to my implementation of the Ping utility, that used System.Net.Sockets.


http://ferozedaud.blogspot.com/2009/08/implementing-traceroute-with-systemnet.html


In this part, we will talk about traceroute. Traceroute is a general purpose utility that is used to discover a network path from the source to the destination. Traceroute is similar to Ping, in that it uses the ICMP protocol to send ICMP Echo request packets to the server. However, one key difference, is that it additionally uses the IP Time To Live (TTL) mechanism to specify the lifetime of the outgoing packets.


When the TTL expires on a packet, the receiving host must send an ICMP "Time Exceeded" message to the sender. The sender looks at this packet, and gets the IPAddress/HostName of the host that responded.


So, the basic algorithm for this goes as follows:

while (reply.address != dest.address)
int ttl = 1;
for i = 1 to 3
send a packet to the host at address dest.address with TTL=ttl
wait for reply
if (timeout) then
print "*"
else
print ipaddress of host thatresponded
// increase the TTL
ttl = ttl + 1;
end

This should be pretty easy to implement, given that we have already implemented a PING client in previous episodes, that does all the heavy lifting for us, in terms of marshalling the ICMP packet from managed code to network byte order, etc.

http://blogs.msdn.com/feroze_daud/archive/2005/10/20/483088.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/23/483976.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/24/484260.aspx
http://blogs.msdn.com/feroze_daud/archive/2005/10/26/485372.aspx

In the next part, we will modify the Ping utility to convert it into a Traceroute implementation.