Via JonesBlog - Apples Retina Display

Brian Jones analyzes Apple's claim that the Iphone4's display resolution is better than a human eye. Interesting reading.

Multipart Form Upload Helper for System.Net.WebClient

Implementing Multipart-MIME upload helper using WebClient

The WebClient class is a higher level abstraction on top of HttpWebRequest, that abstracts away the common tasks of uploading files, as well as doing Form Posts to HTTP servers. However, it does not provide an easy way to do multipart form uploads.

Multipart form upload is defined by the W3C specification.

Multipart HTML Form Design

Let us start off by designing the HTML Form that will be the target of the multipart upload. As per the HTML spec, the form should have an enctype attribute set to "multipart/form-data". Also, we are going to have a File input control embedded in the form.

	<form action="form.aspx" enctype="multipart/form-data" method="post" runat="server">
	Your name: <input name="fname" />
	Your id: <input name="id" />
	What files are you sending?
	<input id="pics1" name="pics1" runat="server" type="file" />
	<input id="pics2" name="pics2" runat="server" type="file" />
	<input type="submit" />
	</form>

view raw Form.html hosted with ❤ by GitHub

MIME Part Requirements

In a multipart form upload, each mime part begins with some MIME headers. The following headers are mandatory:

• Content-Disposition header
• A name attribute that specifies the name of the corresponding control from the HTML form.

For eg, in the form above, if I specify "John" for the name, and "1" for the ID, then I will have the following in the MIME upload

--BD37EC1
Content-Disposition: form-data; name="fname";

john
--BD37EC1
Content-Disposition: form-data; name="id";

acme
--BD37EC1

Ideally, our helper class should be able to write out the boundaries itself, and delegate the task of writing out the headers to each MIME part that is part of the upload.

Our multipart form has two kinds of controls: normal Text input as well as FILE input controls. Therefore, our helper class needs to support both of these types of MIME parts.

Let us start off by thinking about the layour of the MIME part classes.

MIME Part Design

As discussed, we are going to have two types of MIME parts - a normal FORM upload type where we want to specify Name/Value pairs, as well as a FILE part where we want to embed the contents of a file.

In the previous section we saw the basic requirements of a MIME part. It has to have a "Content-Disposition" header. In addition to this, a MIME part that contains file contents should also have a "Content-Type" header.

With this, we can now write out the layout of the base class for these two MIME parts.

	///
	/// MimePart
	/// Abstract class for all MimeParts
	///
	abstract class MimePart
	{
	public string Name { get; set; }
	public abstract string ContentDisposition { get; }
	public abstract string ContentType { get; }
	public abstract void CopyTo(Stream stream);
	public String Boundary
	{
	get;
	set;
	}
	}

view raw mimepart.cs hosted with ❤ by GitHub

This is a good abstraction of a MIME part. Derived classes can define the Content-Type and Content-Disposition, we well as write out the contents of the MIME part into the Stream that is provided. The MIME helper class will do the job of formatting the MIME part and uploading it to a HTTP endpoint.

With this, we can now write out a derived class that can upload textual input data (i.e not file contents)

	class NameValuePart : MimePart
	{
	private NameValueCollection nameValues;
	public NameValuePart(NameValueCollection nameValues)
	{
	this.nameValues = nameValues;
	}
	public override void CopyTo(Stream stream)
	{
	string boundary = this.Boundary;
	StringBuilder sb = new StringBuilder();
	foreach (object element in this.nameValues.Keys)
	{
	sb.AppendFormat("--{0}", boundary);
	sb.Append("\r\n");
	sb.AppendFormat("Content-Disposition: form-data; name=\"{0}\";", element);
	sb.Append("\r\n");
	sb.Append("\r\n");
	sb.Append(this.nameValues[element.ToString()]);
	sb.Append("\r\n");
	}
	sb.AppendFormat("--{0}", boundary);
	sb.Append("\r\n");
	//Trace.WriteLine(sb.ToString());
	byte [] data = Encoding.ASCII.GetBytes(sb.ToString());
	stream.Write(data, 0, data.Length);
	}
	public override string ContentDisposition
	{
	get { return "form-data"; }
	}
	public override string ContentType
	{
	get { return String.Empty; }
	}
	}

view raw NameValuePart.cs hosted with ❤ by GitHub

NameValuePart is an implementation of the MimePart abstract class. It does the job of serializing the Name/Value pairs as a MIME encapsulated part.

Next, we write the implementation of the class the encapsulates file uploads as a MIME part.

	class FilePart : MimePart
	{
	private Stream input;
	private String contentType;
	public FilePart(Stream input, String name, String contentType)
	{
	this.input = input;
	this.contentType = contentType;
	this.Name = name;
	}
	public override void CopyTo(Stream stream)
	{
	StringBuilder sb = new StringBuilder();
	sb.AppendFormat("Content-Disposition: {0}", this.ContentDisposition);
	if (this.Name != null)
	sb.Append("; ").AppendFormat("name=\"{0}\"", this.Name);
	if (this.FileName != null)
	sb.Append("; ").AppendFormat("filename=\"{0}\"", this.FileName);
	sb.Append("\r\n");
	sb.AppendFormat(this.ContentType);
	sb.Append("\r\n");
	sb.Append("\r\n");
	// serialize the header data.
	byte[] buffer = Encoding.ASCII.GetBytes(sb.ToString());
	stream.Write(buffer, 0, buffer.Length);
	// send the stream.
	byte[] readBuffer = new byte[1024];
	int read = input.Read(readBuffer, 0, readBuffer.Length);
	while (read & gt; 0)
	{
	stream.Write(readBuffer, 0, read);
	read = input.Read(readBuffer, 0, readBuffer.Length);
	}
	// write the terminating boundary
	sb.Length = 0;
	sb.Append("\r\n");
	sb.AppendFormat("--{0}", this.Boundary);
	sb.Append("\r\n");
	buffer = Encoding.ASCII.GetBytes(sb.ToString());
	stream.Write(buffer, 0, buffer.Length);
	}
	public override string ContentDisposition
	{
	get { return "file"; }
	}
	public override string ContentType
	{
	get { return String.Format("content-type: {0}", this.contentType); }
	}
	public String FileName { get; set; }
	}

view raw FilePart.cs hosted with ❤ by GitHub

According to the HTML Form upload specification, when the form defines more than one FILE INPUT field, then all the files in the form can be uploaded as one MIME type (of type multipart/mixed) that encapsulates each file, and each file will have it's own MIME encapsulation. In order to support this, we need to create a special class that can encapsulate FILE types.

	/// Helper class that encapsulates all file uploads
	/// in a mime part.
	/// </summary>
	class FilesCollection : MimePart
	{
	private List<filepart> files;
	public FilesCollection()
	{
	this.files = new List<filepart>();
	this.Boundary = MultipartHelper.GetBoundary();
	}
	public int Count
	{
	get { return this.files.Count; }
	}
	public override string ContentDisposition
	{
	get
	{
	return String.Format("form-data; name=\"{0}\"", this.Name);
	}
	}
	public override string ContentType
	{
	get { return String.Format("multipart/mixed; boundary={0}", this.Boundary); }
	}
	public override void CopyTo(Stream stream)
	{
	// serialize the headers
	StringBuilder sb = new StringBuilder(128);
	sb.Append("Content-Disposition: ").Append(this.ContentDisposition).Append("\r\n");
	sb.Append("Content-Type: ").Append(this.ContentType).Append("\r\n");
	sb.Append("\r\n");
	sb.AppendFormat("--{0}", this.Boundary).Append("\r\n");
	byte[] headerBytes = Encoding.ASCII.GetBytes(sb.ToString());
	stream.Write(headerBytes, 0, headerBytes.Length);
	foreach (FilePart part in files)
	{
	part.Boundary = this.Boundary;
	part.CopyTo(stream);
	}
	}
	public void Add(FilePart part)
	{
	this.files.Add(part);
	}
	}

view raw FilesCollections.cs hosted with ❤ by GitHub

This class does the job of encapsulating FILE parts inside a MIME container.

Now, we just need to write the code that can put this all together.

	/// Helper class to aid in uploading multipart
	/// entities to HTTP web endpoints.
	/// </summary>
	class MultipartHelper
	{
	private static Random random = new Random(Environment.TickCount);
	private List<namevaluepart> formData = new List<namevaluepart>();
	private FilesCollection files = null;
	private MemoryStream bufferStream = new MemoryStream();
	private string boundary;
	public String Boundary { get { return boundary; } }
	public static String GetBoundary()
	{
	return Environment.TickCount.ToString("X");
	}
	public MultipartHelper()
	{
	this.boundary = MultipartHelper.GetBoundary();
	}
	public void Add(NameValuePart part)
	{
	this.formData.Add(part);
	part.Boundary = boundary;
	}
	public void Add(FilePart part)
	{
	if (files == null)
	{
	files = new FilesCollection();
	}
	this.files.Add(part);
	}
	public void Upload(WebClient client, string address, string method)
	{
	// set header
	client.Headers.Add(HttpRequestHeader.ContentType, "multipart/form-data; boundary=" + this.boundary);
	Trace.WriteLine("Content-Type: multipart/form-data; boundary=" + this.boundary + "\r\n");
	// first, serialize the form data
	foreach (NameValuePart part in this.formData)
	{
	part.CopyTo(bufferStream);
	}
	// serialize the files.
	this.files.CopyTo(bufferStream);
	if (this.files.Count & gt; 0)
	{
	// add the terminating boundary.
	StringBuilder sb = new StringBuilder();
	sb.AppendFormat("--{0}", this.Boundary).Append("\r\n");
	byte[] buffer = Encoding.ASCII.GetBytes(sb.ToString());
	bufferStream.Write(buffer, 0, buffer.Length);
	}
	bufferStream.Seek(0, SeekOrigin.Begin);
	Trace.WriteLine(Encoding.ASCII.GetString(bufferStream.ToArray()));
	byte[] response = client.UploadData(address, method, bufferStream.ToArray());
	Trace.WriteLine("----- RESPONSE ------");
	Trace.WriteLine(Encoding.ASCII.GetString(response));
	}
	/// Helper class that encapsulates all file uploads
	/// in a mime part.
	/// </summary>
	class FilesCollection : MimePart
	{
	private List<filepart> files;
	public FilesCollection()
	{
	this.files = new List<filepart>();
	this.Boundary = MultipartHelper.GetBoundary();
	}
	public int Count
	{
	get { return this.files.Count; }
	}
	public override string ContentDisposition
	{
	get
	{
	return String.Format("form-data; name=\"{0}\"", this.Name);
	}
	}
	public override string ContentType
	{
	get { return String.Format("multipart/mixed; boundary={0}", this.Boundary); }
	}
	public override void CopyTo(Stream stream)
	{
	// serialize the headers
	StringBuilder sb = new StringBuilder(128);
	sb.Append("Content-Disposition: ").Append(this.ContentDisposition).Append("\r\n");
	sb.Append("Content-Type: ").Append(this.ContentType).Append("\r\n");
	sb.Append("\r\n");
	sb.AppendFormat("--{0}", this.Boundary).Append("\r\n");
	byte[] headerBytes = Encoding.ASCII.GetBytes(sb.ToString());
	stream.Write(headerBytes, 0, headerBytes.Length);
	foreach (FilePart part in files)
	{
	part.Boundary = this.Boundary;
	part.CopyTo(stream);
	}
	}
	public void Add(FilePart part)
	{
	this.files.Add(part);
	}
	}
	}

view raw MultipartHelper.cs hosted with ❤ by GitHub

Since the job of serializing to MIME has been delegated to each concrete implementation of MimePart class, the MultipartHelper only has to make sure that it uses the different parts and serializes them into a stream. The stream is then uploaded using the supplied WebClient.

Note that I have put the FilesCollection class as an inner class of the MultipartHelper class, because it is meant to be a helper class that should not be directly accessed by the clients.

The client program for this looks as follows:

	class Program
	{
	static void Main(string[] args)
	{
	Trace.Listeners.Add(new ConsoleTraceListener());
	try
	{
	using (StreamWriter sw = new StreamWriter("testfile.txt", false))
	{
	sw.Write("Hello there!");
	}
	using (Stream iniStream = File.OpenRead(@"c:\platform.ini"))
	using (Stream fileStream = File.OpenRead("testfile.txt"))
	using (WebClient client = new WebClient())
	{
	MultipartHelper helper = new MultipartHelper();
	NameValueCollection props = new NameValueCollection();
	props.Add("fname", "john");
	props.Add("id", "acme");
	helper.Add(new NameValuePart(props));
	FilePart filepart = new FilePart(fileStream, "pics1", "text/plain");
	filepart.FileName = "1.jpg";
	helper.Add(filepart);
	FilePart ini = new FilePart(iniStream, "pics2", "text/plain");
	ini.FileName = "inifile.ini";
	helper.Add(ini);
	helper.Upload(client, "http://localhost/form.aspx", "POST");
	}
	}
	catch (Exception e)
	{
	Trace.WriteLine(e);
	}
	}
	}

view raw MultipartHelperUsage.cs hosted with ❤ by GitHub

If you want to understand what is going on at the wire level, you can create a system.net trace log which will show the details.

Complete Library

You can find the complete library at multipart-upload-helper repository in my Github.

Conclusion

We implemented a helper library that uses System.Net.WebClient to upload multipart form data.