On IIS 6, Content-Location, Internal IP Addresses, and URL Redirection

One of our automated security scans of our Web site turned up a low priority notice that our server was revealing its internal IP address on some redirect requests. It’s a “feature” of IIS that occurs when a client requests a resource via HTTP 1.0 (or without the Host header) that results in a redirection. The output of a telnet session to such a resource looks something like this:

[npiaseck ~]$ telnet www.example.com 80
Trying 10.1.10.38...
Connected to www.example.com (10.1.10.38).
Escape character is '^]'.
GET /Media HTTP/1.0
 
HTTP/1.1 301 Moved Permanently
Content-Length: 152
Content-Type: text/html
Location: http://10.1.10.38/Media/
Server: Microsoft-IIS/6.0
Date: Fri, 05 Dec 2008 13:50:00 GMT
Connection: close
 
<head><title>Document Moved</title></head>
<body><h1>Object Moved</h1>
This document may be found <a HREF="http://10.1.10.38/Media/">here</a>
</body>
 
Connection closed by foreign host.

I’ve made up an internal IP address, but the point is that IIS is inserting the internal IP address in the body of the document as well as the Location header. If you make the same request with the Host header supplied, such as in Host: www.example.com, then IIS would use www.example.com instead of supplying the IP address.

The recommended Microsoft solution is detailed in KB834141. Namely, you edit the SetHostName metabase entry for the Web site to www.example.com, and now IIS will use that instead of the IP address in the above redirection scenario. Easy as pie, right?

Well, that change broke some code. For reasons that are not clear to me, here’s the mechanism by which IIS 6 determines what to use in these redirect requests:

  1. Is the UseHostName property set to true? If so, use the machine’s host name.
  2. Is the SetHostName property set to some value? If so, use that value.
  3. Is a Host header supplied in the HTTP request? If so, use that value.
  4. If all of the above fail, just use the IP address, which is probably internal.

For search engine optimization reasons, all of the requests on our Web site redirect all requests that come in without the www subdomain to ones with the www subdomain. This prevents Google from dinging us for having duplicated content. The code was implemented as an ASP.NET HTTP module that ran before every request and inspected the incoming URL by looking at the HttpContext.Current.Request.Url.OriginalString property. If the domain didn’t match, then it issued a permanent redirect.

The problem with the above precedence order is that IIS will substitute the value of HTTP_HOST and other server variables passed into ASP.NET with the SetHostName value, not the Host header if so supplied. The net effect is that our redirection method suddenly stopped working because the code was always seeing the SetHostName value, www.example.com, regardless of the URL that was actually used to make the request. To make matters worse, if we were to set the UseHostName property, the code would always see a mismatch and enter a redirect loop. Neither are desirable scenarios.

The solution is to change the code to inspect the Host header directly, if present, instead of looking at the URL as passed into the server variables array. It ends up looking something like this:

private void OnContextBeginRequest(object sender, EventArgs e)
{
	HttpContext context;
	string originalRequestHost;
	int portIndex;
	HttpRequest request;
	string redirectUri;
 
	context = HttpContext.Current;
	request = context.Request;
 
	originalRequestHost = request.Headers["Host"];
 
	// Some clients will pass in the port number in the Host header if
	// it's not going over port 80
	portIndex = originalRequestHost.LastIndexOf(':');
	if (portIndex != -1)
	{
		originalRequestHost =
			originalRequestHost.Substring(0, portIndex);
	}
 
	if (originalRequestHost != null && originalRequestHost != mPreferredSubdomain)
	{
		redirectUri = string.Format(
			"http://{0}{1}{2}",
			mPreferredSubdomain,
			request.Url.Port != 80 ? ":" + request.Url.Port.ToString() : string.Empty,
			request.RawUrl);
		cLog.InfoFormat("Redirecting from to {0} (domain: {1}, preferred: {2}).",
			redirectUri, originalRequestHost, mPreferredSubdomain);
		context.Response.Redirect(redirectUri, true);
	}
}

The code looks at the host header. If it’s set, and doesn’t match, then it redirects. (It does a little bit of finagling to strip off the port number if a non-standard port is used in the request.)

With this code in place, we can now use the SetHostName property and still have our URL redirection work properly for HTTP/1.1 requests. We’re out of luck when it comes to HTTP/1.0 requests, but this is something that we can live with.

Working around USPS Web Tools SSL Certificate Errors

Our internal application calls the USPS Web Tools APIs to generate customs forms automatically for us. This integration was working untouched and swimmingly for about a year, when suddenly this morning it started throwing exceptions saying that the SSL certificate at the USPS servers is not trusted. Sure enough, visiting the https://secure.shippingapis.com/ShippingAPI.dll URL in Google Chrome or Internet Explorer popped up an untrusted certificate warning message.

It would appear that they are using “AAA Certificate Services” now (not sure if they were using this before or if some Windows Update removed this intermediate certificate, but the error suddenly started occurring on all machines in the office), which doesn’t have its root certificate installed in Windows XP.

Here’s the workaround: Download the AAA Intermediate certificate file. Right-click, Install, click Next a few times, and then you should be right as rain again.

I could go on about the USPS Web Tools in general, but suffice to say that this is but one example of how it Simply Does Not Work.

Free spell checking in Windows Forms

So our back-end application at work runs on Windows Forms. Windows Presentation Foundation looks cool, and I’ll certainly want to learn it some day, but it didn’t exist at the time that our back-end application was started, it’s hard for a developer without a good sense of the aesthetic to build a decent interface, it pretty much requires Visual Studio 2008 (which I don’t have the luxury of purchasing at this time), and it flies in the face of over a decade of user32 programming. So we’ll be stuck with Windows Forms for a while.

Why do I bring WPF up? Well, the default text controls in WPF have “wavy red line spell checking” built right in, which is awesome. But there’s no joy for the Windows Forms world. There are lots of components out there for this, but they’re either expensive or bundled together with much larger control suites that I don’t need. (By expensive, I generally mean anything over fifty bucks.)

Being used to being able to Google just about anything and find an open source version somewhere, I was pretty surprised that I haven’t been able to track down a free version of a spell checker. I found plenty of free spell checking engines (one of which I used in my implementation) as well as snippets from people who have written wavy red line algorithms for MFC-based controls, but there was no love for Windows Forms. I was, however, able to cobble together a solution from these snippets and piece together something that works. It’s not very heavyweight or robust, but it’s great in small text boxes, such as the product description field in our application.

This is my implementation. You attach a provider to a text box and you'll get wavy red lines underneath misspelled words. Right-click the word and you'll see a pop up menu with some spelling suggestions.

This is my implementation. You attach a provider to a text box and you'll get wavy red lines underneath misspelled words. Right-click the word and you'll see a pop up menu with some spelling suggestions.

An Overview of What We’re Building

To make this as painless to use as possible, I built the spell checker as an IExtenderProvider. If you’ve used things like Tooltip or ErrorProvider in Windows Forms before, it works the same way: you drag and drop a SpellChecker onto your form in the Visual Studio visual designer. Then, any control on that form that inherits from TextBoxBase magically gets a SpellCheckEnabled on mSpellChecker property. Set that to true and the visual designer adds some magic in the designer file that registers the textbox with the spellchecker, enabling it to get the wavy red line spellchecking behavior that I’ve programmed.

So you drag the (1) SpellChecker onto the form. Here I've named it "mSpellChecker". Then when you click on a (2) TextBox control, it gets a new (3) "SpellCheckEnabled" property. That's it!

So you drag the (1) SpellChecker onto the form. Here I've named it mSpellChecker. Then when you click on a (2) TextBox control, it gets a new (3) SpellCheckEnabled property. That's it!

A Spell Checking Interface

When I first used the control, I piggybacked onto Microsoft Word to do the spellchecking for me. It worked well for the spike, but I quickly discovered that it was a pain to have to ensure that every client has the right version of Word installed on their computer. So I switched the engine implementation over to NetSpell, a free and open source spell checking engine. In the process, however, I abstracted out the spell checking engine that my new SpellChecker uses and made it pluggable.

Similarly, you just drag a spell checking engine provider onto the form and hook it up to the SpellChecker component.

Similarly, you just drag a spell checking engine provider onto the form and hook it up to the SpellChecker component.

So, without any further ado, here’s the interface that these spell checking engine providers need to adhere to:

    /// <summary>
    /// For a service to provide spell checking capabilities to the SpellChecker
    /// extender provider, it needs to implement this interface.
    /// </summary>
    public interface ISpellCheckerProvider : IDisposable
    {
 
        #region Methods
 
        /// <summary>
        /// Adds a user-defined word to the dictionary.
        /// </summary>
        /// <param name="word">the word</param>
        void AddWord(string word);
 
        /// <summary>
        /// Tests to see if the given word is in the dictionary
        /// (that is, not misspelled).
        /// </summary>
        /// <param name="word">the word to test</param>
        /// <returns>whether or not the word is in the dictionary</returns>
        bool IsWordInDictionary(string word);
 
        /// <summary>
        /// Returns an array of suggested spellings for the given misspelled
        /// word.
        /// </summary>
        /// <param name="word">the word</param>
        /// <returns>the suggested spellings</returns>
        string[] SuggestSpellings(string word);
 
        #endregion
 
        #region Properties
 
        /// <summary>
        /// Gets whether or not the object has been disposed.
        /// </summary>
        bool IsDisposed
        {
            get;
        }
 
        /// <summary>
        /// Gets whether or not the provider supports adding user-defined
        /// words to its dictionary.
        /// </summary>
        bool SupportsAddingWords
        {
            get;
        }
 
        #endregion
 
    }

As you can see, it’s a pretty simple interface: you can add words to your local dictionary (which not all implementors may support), you can test to see if a word is misspelled, and you can suggest some alternative spellings for a misspelled word.

Here’s how the implementation that uses NetSpell looks:

using System;
using System.Collections.Generic;
using System.Text;
using System.ComponentModel;
using NetSpell.SpellChecker;
using NetSpell.SpellChecker.Dictionary;
 
    public class NetSpellSpellCheckerProvider : Component, ISpellCheckerProvider
    {
 
        #region Members
 
        private Spelling mSpelling;
 
        private readonly object mSpellingLock = new object();
 
        #endregion
 
        #region Constructors
 
        public NetSpellSpellCheckerProvider()
        {
            mSpelling = new Spelling();
            mSpelling.Dictionary = new WordDictionary();
            mSpelling.Dictionary.EnableUserFile = true;
        }
 
        public NetSpellSpellCheckerProvider(IContainer container) : this()
        {
            container.Add(this);
        }
 
        #endregion
 
        #region Public Methods
 
        public void AddWord(string word)
        {
            lock (mSpellingLock)
            {
                if (SupportsAddingWords)
                {
                    mSpelling.Dictionary.Add(word);
                }
            }
        }
 
        public bool IsWordInDictionary(string word)
        {
            bool isInDictionary;
 
            lock (mSpellingLock)
            {
                isInDictionary = mSpelling.TestWord(word);
            }
 
            return isInDictionary;
        }
 
        public string[] SuggestSpellings(string word)
        {
            string[] suggestions;
 
            lock (mSpellingLock)
            {
                mSpelling.Suggest(word);
                suggestions = new string[mSpelling.Suggestions.Count];
                mSpelling.Suggestions.CopyTo(suggestions);
            }
 
            return suggestions;
        }
 
        #endregion
 
        #region Properties
 
        public string DictionaryFileName
        {
            get { return mSpelling.Dictionary.DictionaryFile; }
            set { mSpelling.Dictionary.DictionaryFile = value; }
        }
 
        public bool IsDisposed
        {
            get { return false; }
        }
 
        public bool SupportsAddingWords
        {
            get { return mSpelling.Dictionary.UserFile != null; }
        }
 
        #endregion
    }

How simple is that? The only thing that makes it slightly complicated is being sure to do a lock whenever we’re dealing with the spelling dictionary. It turns out that NetSpell isn’t happy if, say, one thread is adding a word to the dictionary while another thread is trying to read one. Not too difficult to work around, however.

Implementing the IExtenderProvider

So now let’s write the bit of code that lets the Visual Studio visual designer add those magic SpellCheckEnabled properties to our textboxes.

Let’s start at the beginning:

    /// <summary>
    /// An extender provider that will provide as-you-type spell checking (with
    /// wavy red underlines) to a control that inherits from TextBoxBase.
    /// </summary>
    [ProvideProperty("SpellCheckEnabled", typeof(TextBoxBase))]
    public class SpellChecker : Component, IExtenderProvider, ISupportInitialize
    {
            // TODO
    }

The ProvideProperty attribute is what tells Visual Studio to add a SpellCheckEnabled property to controls that inherit from TextBoxBase. When we change the default property value on one of those controls, Visual Studio will call GetSpellCheckEnabled() and SetSpellCheckEnabled() methods. (It does this by well-known name mangling. If I had called it DeathDefyingStuntsEnabled, then Visual Studio would be calling methods named GetDeathDefyingStuntsEnabled() and SetDeathDefyingStuntsEnabled().) So let’s add those to our new class:

 
        /// <summary>
        /// A mapping of the TextBoxBase controls that are being extended to a
        /// class that contains various information about the control.
        /// </summary>
        private Dictionary<textBoxBase, SpellCheckInfo> mControls;
 
        /// <summary>
        /// Gets whether or not spell checking is enabled for the given control.
        /// </summary>
        /// <param name="control">the control to test</param>
        /// <returns>whether or not spell checking is enabled for the control</returns>
        public bool GetSpellCheckEnabled(TextBoxBase control)
        {
            return control != null && mControls.ContainsKey(control);
        }
 
        /// <summary>
        /// Sets whether or not spell checking is enabled for the given control.
        /// </summary>
        /// <param name="control">the control to enable or disable</param>
        /// <param name="enabled">the enabled status</param>
        public void SetSpellCheckEnabled(TextBoxBase control, bool enabled)
        {
            if (control == null)
            {
                throw new ArgumentNullException("control");
            }
 
            if (enabled)
            {
                mControls.Add(control, new SpellCheckInfo());
            }
            else
            {
                mControls.Remove(control);
            }
        }

What I’m doing is having the SpellChecker instance store a dictionary that maps the TextBoxBases that are added by a call to SetSpellCheckEnabled to their corresponding SpellCheckInfo objects. The SpellCheckInfo object is just a quick and dirty class (it could have been a structure, really) that’s used when drawing the wavy red lines underneath the control. I’ll get to how that’s used later, but essentially all you need to know is that it holds the list of misspelled words in the textbox.

Drawing the wavy red lines

So how do we alter the behavior of these textboxes to draw wavy red lines underneath the misspelled words? We obviously can’t just inherit from TextBox. Otherwise, you’d have to drag and drop a new control on your form instead of a regular TextBox, and that would defeat the whole purpose of having the extender provider.

The secret is to use NativeWindow to listen to the messages that get pumped to each registered TextBox. Whenever they repaint themselves, we’ll paint on top of them the wavy red lines that are necessary. The textboxes won’t even know what hit them!

Our extension of NativeWindow, which I’ve called SpellCheckingTextBox, takes a TextBoxBase and our own SpellCheckInfo object in its constructor and attaches itself so that its WndProc method will see all of the messages that are getting pumped to that TextBoxBase. Here’s what our WndProc looks like:

        /// <summary>
        /// Processes Windows messages.
        /// </summary>
        /// <param name="m">the message to process</param>
        protected override void WndProc(ref Message m)
        {
            switch (m.Msg)
            {
                case (int)WindowsMessages.WM_PAINT:
                    mControl.Invalidate();
                    base.WndProc(ref m);
                    Repaint();
                    break;
                case (int)WindowsMessages.WM_CONTEXTMENU:
                    Point pt;
 
                    pt = GetPointFromLParam(m.LParam);
                    pt = mControl.PointToClient(pt);
                    if (!DisplaySpellingSuggestionsMenu(pt))
                    {
                        base.WndProc(ref m);
                    }
                    break;
                default:
                    base.WndProc(ref m);
                    break;
            }
        }

There’s nothing too exotic going on here. When the TextBoxBase is told by Windows to paint itself, we go ahead and let it do that. But then we call our own Repaint() method.

Similarly, if Windows is asking the TextBoxBase control to draw a popup menu (usually because the user right-clicked their mouse), then we intercept that little message and decide if we want to pop up our own menu of spelling suggestions. If we do, we show it at the correct position on screen with our own DisplaySpellingSuggestionsMenu() method and swallow the message. Otherwise, we pass the message onto the TextBoxBase itself by calling base.WndProc().

The Repaint() method is where the fun happens:

        /// <summary>
        /// Draws the red wavy lines under misspelled words.
        /// </summary>
        private void Repaint()
        {
            Graphics g;
 
            g = Graphics.FromHwnd(mControl.Handle);
            g.Clip = new Region(mControl.ClientRectangle);
 
            lock (mMisspelledWords)
            {
                foreach (Match word in mMisspelledWords)
                {
                    int endIndex;
                    Point start;
                    Point stop;
 
                    endIndex = word.Index + word.Length;
                    start = mControl.GetPositionFromCharIndex(word.Index);
 
                    // For some reason, GetPositionFromCharIndex() freaks out if called
                    // on the last character in the textbox (that is, the misspelled word
                    // is also the last word). We have to do one character prior and figure
                    // out how long the line should be.
                    if (endIndex == mControl.TextLength)
                    {
                        Size charSize;
 
                        charSize = TextRenderer.MeasureText(word.Value.Substring(word.Length - 1), mControl.Font);
 
                        stop = mControl.GetPositionFromCharIndex(endIndex - 1);
                        stop.Offset(charSize.Width, 0);
                    }
                    else
                    {
                        stop = mControl.GetPositionFromCharIndex(endIndex);
                    }
 
                    start.Offset(0, mControl.Font.Height);
                    stop.Offset(0, mControl.Font.Height);
 
                    DrawWavyLine(g, start, stop);
                }
            }
 
            g.Dispose();
        }

There’s nothing too hard going on here since the TextBoxBase provides some great methods for determining the pixel positions of words that are contained within it. On a paint event, we just loop through our SpellCheckInfo object (called mMisspelledWords here). For each of those words, we measure the size of the word in the text box and then we call DrawWavyLine at those points, a neat method that I totally lifted from some guy’s blog:

        /// <summary>
        /// Draws a red wavy line at the given start and stop points with the given
        /// graphics context. Got to give credit where it's due -- this function (and
        /// pretty much the inspiration for this entire class) is lifted straight
        /// from the great work of Andrei Alecu at
        /// http://www.codedblog.com/2007/09/17/owner-drawing-a-windowsforms-textbox/.
        /// </summary>
        /// <param name="g">the graphics context</param>
        /// <param name="start">the start point</param>
        /// <param name="stop">the stop point</param>
        private void DrawWavyLine(Graphics g, Point start, Point stop)
        {
            Pen pen;
 
            pen = Pens.Red;
 
            if ((stop.X - start.X) > 4)
            {
                List<point> points;
 
                points = new List<point>();
 
                for (int i = start.X; i <= (stop.X - 2); i += 4)
                {
                    points.Add(new Point(i, start.Y));
                    points.Add(new Point(i + 2, start.Y + 2));
                }
 
                g.DrawLines(pen, points.ToArray());
            }
            else
            {
                // If the line is really short, don't bother with
                // any waves
                g.DrawLine(pen, start, stop);
            }
        }

That’s pretty much the meat of the NativeWindow implementation–it’s concerned about the display and painting of the information in the SpellCheckInfo object. The actual behind-the-scenes spellchecking is performed by a background thread that’s maintained by the SpellChecker extender provider:

        /// <summary>
        /// Performs the spell checking on the control.
        /// </summary>
        /// <param name="control">the control</param>
        private void PerformAsYouTypeSpellChecking(TextBoxBase control)
        {
            SpellCheckInfo info;
            List<match> misspelledWords;
            string controlText;
            MatchCollection words;
 
            if (control == null)
            {
                throw new ArgumentNullException("control");
            }
 
            if (control.InvokeRequired)
            {
                IAsyncResult result = control.BeginInvoke((ReturnControlTextDelegate)delegate()
                {
                    return control.Text;
                });
                controlText = (string)control.EndInvoke(result);
            }
            else
            {
                controlText = control.Text;
            }
 
            if (mControls.TryGetValue(control, out info))
            {
                misspelledWords = new List<match>();
                words = mReWords.Matches(controlText);
 
                foreach (Match word in words)
                {
                    string text;
 
                    text = word.Captures[0].Value;
 
                    try
                    {
                        if (!mSpellCheckerProvider.IsWordInDictionary(text))
                        {
                            misspelledWords.Add(word);
                        }
                    }
                    catch (ApplicationException)
                    {
                        // If the spell checking provider went down, let's not
                        // bother checking it any more this session
                        break;
                    }
 
                    if (mIsRefreshPending)
                    {
                        break;
                    }
                }
 
                // If the text has changed during the time it took us to process this
                // request, let's just forget everything and try again
                if (mIsRefreshPending)
                {
                    mIsRefreshPending = false;
                    PerformAsYouTypeSpellChecking(control);
                }
                else
                {
                    lock (info.MisspelledWords)
                    {
                        info.MisspelledWords.Clear();
                        info.MisspelledWords.AddRange(misspelledWords);
                    }
                    control.Invalidate();
                }
            }
        }

Is it the cleanest code? Probably not–there’s some issues with disposing that are kind of nasty. And there are some optimizations that could be made: for example, the wavy red lines disappear while you’re typing. And it probably breaks down for very long passages of text since it basically rescans the entire textbox every time the text changes. But in general, it works pretty well, especially for an in-house app. Download it and see how well it works for you!

The sample application shows how to use the code that I've presented here.

The sample application shows how to use the code that I've presented here.

Spell Check Sample Application

If you have any suggestions for improvement, drop me a line. Good luck!

Programmatically selecting complex printer options in C#

At Skiviez, we print out a full-color eight page customized flyer for every one of our orders. As soon as the order is done being processed, our internal processing application generates the flyer and shoots it off to one of the copier machines. The copier machine prints out the flyer (duplex), folds it, and staples it. This all happens silently and automatically; the employee does not have to twiddle with printer configuration or driver settings.

Getting this to happen automatically, however, was not easy. If you’re used to using the PrinterSettings and PrintDocument classes in Windows Forms, you’ll know that you have some options like a Duplex option available. But in our case, using that Duplex property seemed to make no difference in the printed output; the ol’ copier stubbornly went simplex every single time. It Simply Did Not Work. Not to mention that the .NET-native API doesn’t even begin to provide support for specifying paper folding, saddle stitching, and stapling. In a quest to find out how I could automate the selection of all these settings, I learned a bit more about printer drivers on Windows than I cared to know.

To get down and dirty with printer settings, I had to get familiar with DEVMODE. Being a .NET weenie who really doesn’t know his way around the Windows API very well, discovering this was progress; this structure exposes some of the options that I needed (such as collation), but it still didn’t provide any programmatic way to access folding and stapling settings.

Let’s go into the Printers section of the Control Panel and look at the Printing Preferences for one of these copiers. It turns out that this is a good way to be able to tell where a configuration setting lives. Some of the universal, standard settings are displayed on the “Layout” and “Paper/Quality” tabs. But if I want to get to any of the fancy features such as folding or stapling, well, they’re specific to our unique printer driver (and indeed, they’re exposed on a custom tab called “Fiery Printing” that the printer driver provides). There’s approximately zero documentation out there for configuring Fiery printer drivers programmatically, so how the hell was I going to communicate with this black box?

The Layout and Paper/Quality tabs can be configured by setting fields of the DEVMODE structure to documented values. But many of the Advanced "Printer Features" live in a driver-specific, undocumented part of memory just beyond the DEVMODE structure.

The Layout and Paper/Quality tabs can be configured by setting fields of the DEVMODE structure to documented values. But many of the Advanced "Printer Features" live in a driver-specific, undocumented part of memory just beyond the DEVMODE structure.

I’m not sure if my solution is genius, expected, or insane, but it’s been working for over a year now. The secret is a curious little member of the DEVMODE structure called dmDriverExtra. MSDN has the following to say about this little member:

Contains the number of bytes of private driver-data that follow this structure. If a device driver does not use device-specific information, set this member to zero.

Private driver data, I thought. How interesting. It’s perfectly logical to assume that since the stapling and folding features are unique to this printer, then the configuration for those features is stored in some unknown format. That data lives in a block of memory at the end of the DEVMODE structure. The size of that block of memory is the value given in the dmDriverExtra field.

Here’s a crazy idea, I continued. What if I configured the default printer settings via the Printer Preferences pane in the Control Panel to the exact settings that I want the fliers to print with. Then, I’ll write a program that dumps the default DEVMODE structure and its private data for the printer to a file. Then, I can revert the default printer settings in the Control Panel back to normal.

When I want to print the flyer, I’ll just obtain the default DEVMODE, overwrite its spot in memory with the version that I had saved to disk, and then send the document to the printer. That way, I can specify those proprietary folding and stapling settings without actually knowing exactly how they’re defined in memory!

The printer-specific configuration data lives just beyond the DEVMODE structure.

The printer-specific configuration data lives just beyond the DEVMODE structure.

It turns out that this ended up working quite well. Obviously, things will blow up if we ever change the version of the printer driver that’s installed on the server (since they’ll probably have changed the layout of their private data section). Luckily, though, we don’t really plan on changing things once they’re working.

Dumping the DEVMODE

So, first I went into the Control Panel and selected the duplexing, folding, and stapling options that I wanted in the print driver-supplied tab. Then I ran a quick and dirty console application like the following (I swiped the P/Invoke version of DEVMODE from pinvoke.net:

    class Program2
    {
        #region P/Invoke
 
        [DllImport("kernel32.dll", ExactSpelling = true)]
        public static extern IntPtr GlobalFree(IntPtr handle);
 
        [DllImport("kernel32.dll", ExactSpelling = true)]
        public static extern IntPtr GlobalLock(IntPtr handle);
 
        [DllImport("kernel32.dll", ExactSpelling = true)]
        public static extern IntPtr GlobalUnlock(IntPtr handle);
 
        #endregion
 
        #region Constants
 
        private const string DUMP_PATH = "DevModeDump.bin";
 
        #endregion
 
        static void Main(string[] args)
        {
            IntPtr hDevMode;                        // handle to the DEVMODE
            IntPtr pDevMode;                        // pointer to the DEVMODE
            DEVMODE devMode;                        // the actual DEVMODE structure
            PrinterSettings printerSettings;        // our flyer's printer settings
            Fall2008LargeFlyerPrintDocument flyer;  // our custom subclass of PrintDocument
 
            flyer = new Fall2008LargeFlyerPrintDocument();
 
            flyer.PrintController = new StandardPrintController();
            printerSettings = flyer.PrinterSettings;
            printerSettings.PrinterName = "Fiery X3E";
 
            // Get a handle to a DEVMODE for the default printer settings
            hDevMode = printerSettings.GetHdevmode(printerSettings.DefaultPageSettings);
 
            // Obtain a lock on the handle and get an actual pointer so Windows won't
            // move it around while we're futzing with it
            pDevMode = GlobalLock(hDevMode);
 
            // Marshal the memory at that pointer into our P/Invoke version of DEVMODE
            devMode = (DEVMODE)Marshal.PtrToStructure(pDevMode, typeof(DEVMODE));
 
            // Read the bytes starting at that pointers position in memory into our
            // file stream
            using (FileStream fs = new FileStream(DUMP_PATH, FileMode.Create))
            {
                for (int i = 0; i < devMode.dmSize + devMode.dmDriverExtra; ++i)
                {
                    fs.WriteByte(Marshal.ReadByte(pDevMode, i));
                }
            }
 
            // Unlock the handle, we're done futzing around with memory
            GlobalUnlock(hDevMode);
 
            // And to boot, we don't need that DEVMODE anymore, either
            GlobalFree(hDevMode);
        }
    }

Using the saved DEVMODE for printing

Now what do I do when I want to crank these suckers out? I just take my saved DEVMODE and overwrite the one that I get in memory when I’m about to print. It’s something like this:

    class Program3
    {
        #region P/Invoke
 
        [DllImport("kernel32.dll", ExactSpelling = true)]
        public static extern IntPtr GlobalFree(IntPtr handle);
 
        [DllImport("kernel32.dll", ExactSpelling = true)]
        public static extern IntPtr GlobalLock(IntPtr handle);
 
        [DllImport("kernel32.dll", ExactSpelling = true)]
        public static extern IntPtr GlobalUnlock(IntPtr handle);
 
        #endregion
 
        #region Constants
 
        private const string DUMP_PATH = "DevModeDump.bin";
 
        #endregion
 
        static void Main(string[] args)
        {
            IntPtr hDevMode;                        // a handle to our current DEVMODE
            IntPtr pDevMode;                        // a pointer to our current DEVMODE
            PrinterSettings printerSettings;        // our flyer's printer settings
            Fall2008LargeFlyerPrintDocument flyer;  // our custom subclass of PrintDocument
            byte[] savedDevMode;                    // will hold the DEVMODE we saved earlier
            Stream savedDevStream;                  // used to read the DEVMODE we saved earlier
 
            flyer = new Fall2008LargeFlyerPrintDocument();
 
            renderedFlyer.PrintController = new StandardPrintController();
            printerSettings = renderedFlyer.PrinterSettings;
            printerSettings.PrinterName = "Fiery X3E";
 
            // Obtain the current DEVMODE position in memory
            hDevMode = printerSettings.GetHdevmode(printerSettings.DefaultPageSettings);
 
            // Obtain a lock on the handle and get an actual pointer so Windows won't move
            // it around while we're futzing with it
            pDevMode = GlobalLock(hDevMode);
 
            // Load the saved DEVMODE that we dumped earlier
            savedDevStream = new FileStream(DUMP_PATH, FileMode.Open, FileAccess.Read);
            savedDevMode = new byte[savedDevStream.Length];
            savedDevStream.Read(savedDevMode, 0, savedDevMode.Length);
            savedDevStream.Close();
            savedDevStream.Dispose();
 
            // Overwrite our current DEVMODE in memory with the one we saved.
            // They should be the same size since we haven't like upgraded the OS
            // or anything.
            for (int i = 0; i < savedDevMode.Length; ++i)
            {
                Marshal.WriteByte(pDevMode, i, savedDevMode[i]);
            }
 
            // We're done futzing
            GlobalUnlock(hDevMode);
 
            // Tell our printer settings to use the one we just overwrote
            printerSettings.SetHdevmode(hDevMode);
 
            // It's copied to our printer settings, so we can free the OS-level one
            GlobalFree(hDevMode);
 
            // Print and we're done!
            renderedFlyer.Print();
            renderedFlyer.Dispose();
        }
    }

Is it insane? Probably. But watching it duplex, fold, and staple automatically is great. I love it when something Simply Works!

Paging Phones with Asterisk and C#

Working as a programmer at a small e-commerce business means that, well, I’m not exactly programming all day, sitting at a computer with our order processing application open. Sometimes we’re all back in the warehouse picking orders, checking in items, or rearranging things. So it’s nice to have some sort of cue when new orders arrive during the middle of the day that need to be picked.

For about a year, I had things set up so that our order processing application polls the server for new orders every 5 minutes. If it detects that new orders have been placed since the last time it checked, then the software would use the Windows Speech API (SAPI) to speak an announcement like “There are six new orders” aloud. Pretty cool.

But there was one annoyance. Since each application was polling individually, you’d here a small echo as each employee’s individual copy of our application polled the server at a slightly different time and made the announcement through their computer’s speakers. Also, if you were on the phone with a customer and you forgot to disable the announcement in your copy of the application, the customer would hear a loud, disembodied voice saying “There are nine new orders” in a dull monotone. This was usually followed by the customer asking, “What the hell was that?!”

Since the feature was implemented, I built a VOIP phone system for our internal use in the office based on Asterisk. Asterisk has provided some great integration points; for example, when a customer calls us, Asterisk reports the caller ID to our application running on an employee’s machine, which can then display the correct customer account automatically. It’s pretty neat to have the customer’s orders on your screen before even saying hello. In a roundabout way, I was able to integrate Asterisk in the reverse: to have our software tell Asterisk to call some phones and play a message.

The easiest way to solve the problem of everyone’s machines making the new orders announcement at different times is to, well, only have one machine do that. So I wrote a quick and dirty console application that runs as a scheduled task on a server. That scheduled task then “pokes” Asterisk such that Asterisk pages the warehouse phones (leaving the office phones quiet, so as not to disturb employees helping customers) and announces something like “BEEP! Four new. BEEP!” before automatically disconnecting.

How did I accomplish this? The secret is to use Asterisk’s “.call” file capability. A call file is simply a plain text INI-esque file that you drop into a directory that Asterisk is always monitoring. If you’re familiar with the PickUp directory in Microsoft Exchange, it’s the exact same thing except for a phone system, located in /var/spool/asterisk/outgoing/ on our phone server.

Here’s an example of what a generated call file looks like for our scenario:

Channel: Local/99@incoming-from-server/n
CallerID: Skiviez Paging Service <9999>
Context: incoming-from-server
Extension: 1
MaxRetries: 0
Set: NumOrders=8

The scheduled task generates a call file like the one above and then just copies it over to the outgoing directory on the phone server (which I just shared via Samba). This file is a little confusing, so here’s the lowdown on what everything means:

  • Channel is the device that is doing the “dialing.” In our paging scenario, it’s the device that’s doing the talking. Here, I’ve used “local,” which means to create a made-up, temporary device that is “speaking” the dialplan at extension 99 of context “[incoming-from-server]“. More on this in a bit. (The “n” parameter tells it not to optimize away any context-level variables. If we left it out, then we wouldn’t be able to access our “NumOrders” variable described below from extension 99.)
  • CallerID is just what the phones that are being paged will display while connected.
  • Context and Extension indicate the context and extension that represents the device that “picks up” the dialed call. Note that this is NOT the same thing as the phones that we are trying to page with our message. This will all make sense in a minute.
  • Finally, MaxRetries tells Asterisk not to care if the page fails for any reason, and the Set is a variable that we pass to the context of the device that “dials” the call. In this case, we’re passing the number of orders that should be announced aloud over the page.

To see how this makes sense, let’s look at the corresponding entry in the extensions.conf file:

[incoming-from-server]
; This is what "picks up" the call file. It pages the available
; phones.
exten => 1,1,Answer()
exten => 1,2,SIPAddHeader(Call-Info: answer-after=0)
exten => 1,3,Page(SIP/9500&SIP/9501&SIP/9502|q)
exten => 1,4,Hangup()
 
; This is the local extension that the call file dials. All it
; has to do is pick up; it'll the dump the call into extension 1
; of this section as instructed by the call file.
exten => 99,1,Answer()
exten => 99,2,Wait(3)
exten => 99,3,Playback(beep)
exten => 99,4,SayNumber(${NumOrders})
exten => 99,5,Playback(local/new)
exten => 99,6,Wait(1)
exten => 99,7,Playback(beep)
exten => 99,8,Hangup()

So here’s what happens when the call file gets dropped. Asterisk picks up and then pretends to be a phone at extension 99. That phone dials another pretend phone at extension 1.

The extension 1 phone adds a header that’s necessary for our Grandstream GXP-2000 phones to pick up the page immediately, and then it pages the warehouse phones SIP/9500, SIP/9501, and SIP/9502 (these numbers aren’t real, but are useful for this example). They connect and start listening in on the conversation going on with the pretend phone at extension 1.

Meanwhile, the phone at extension 99 has waited around doing nothing for 3 seconds (since it takes a second or two for the warehouse phones to all pick up the page call from extension 1). Then it plays a beep, says the number of orders, says the word new, beeps again, and hangs up. The phone at extension 1 hears this, and since the (real) paged phones are listening to the pretend phone at extension 1, they playback the message that the pretend phone at extension 99 just said: “BEEP! 8 new. BEEP!”

Phew! It certainly took me a few minutes to figure it out, let me tell you, because I sure got myself spun around a few times. (This fake phone dials this fake phone which pages real phones which listen to the fake phone….) But, in the end, I took something that worked (our individual application instances making announcements) to something that works better (a single device paging phones simultaneously, which sounds more professional and is louder).

Is it Rube Goldberg-esque? Sure. But it Simply Works, and that’s good enough for me.

Reading a Stamps.com USB Scale from C#

An innocuous, cheap looking plastic postal scale.

I’ve managed to use Stamps.com’s PDK to integrate Skiviez’s USPS shipping down to a pretty automatic process. First, you dump a package onto an old serial scale. Second, you scan the order number barcode. Our software (which is constantly polling that scale) reads the weight, loads the order, and passes this information off to the Stamps.com COM object (try saying “Stamps.com COM object” without sounding like an idiot; this is why I hate companies with “.com” in their name). Then, third, you usually just have to smash the Enter key and a delivery label comes shooting out of the printer. Great.

But we haven’t been using the little USB scale that was supplied with our Stamps.com account; instead, the software has been reading from an old serial scale that we got from UPS. Why? I knew how to read data from the serial scale (which is simple enough: write a newline to the serial port and get an ASCII string with the weight back), but the Stamps.com USB scale was a black box. First, I had no idea how even to begin to access data from a USB device in C#. Second, I wouldn’t know how to read data coming back from the scale even if I did.

Here’s my story of how I figured this out. I’m sure that there are more intelligent ways that I could have done this; if there are, I’d love to hear from you.

First, I sat down at my computer with Device Manager open and plugged in the scale. I notice that it appeared in Windows as a “USB Human Interface Device.” Now that’s interesting, I thought to myself. In all my years of futzing around with Windows, I see things pop up as a USB Human Interface Device all the time. I wonder what in the blue hell that actually means?

A quick trip to Google reveals that devices that are HIDs can send and receive data according to a common specification that is detailed in a several hundred page tome that I would care not to read. That means that the operating system can provide one generic HID driver that all of these devices can share. That’s a relief because it means that I don’t need to learn how to interact with some low-level, custom driver that was written specifically for this USB scale.

Since it appears that there is no native .NET way to interact with these HID devices, however, I began searching for a pre-built library. I struck gold with Mike O’Brien’s USB HID library, which has an easy-to-use API that enabled me to complete this project without having any clear idea of what exactly I was doing. These HID devices can send “reports” according to some common protocol; this library lets me read the reports, but figuring out the content of the report is still up to me to figure out.

So, having the scale hooked up, and using the example code on Mike O’Brien’s page, I hammered out some code that looked something like the following (I looked up the product and vendor ID by using the Properties pane in Device Manager):

HidDeviceData inData;
HidDevice[] hidDeviceList;
HidDevice scale;
 
hidDeviceList = HidDevices.Enumerate(0x1446, 0x6A73);
 
if (hidDeviceList.Length > 0)
{
	int waitTries;
 
	scale = hidDeviceList[0];
	waitTries = 0;
 
	scale.Open();
 
	if (scale.IsConnected)
	{
		inData = scale.Read(250);
 
		for (int i = 0; i < inData.Data.Length; ++i)
		{
			Console.WriteLine("Byte {0}: {1:X}", i, inData.Data[i]);
		}
	}
 
	scale.Close();
	scale.Dispose();
}

At this point, I was fairly flabbergasted because I actually got a reasonably small amount of data back:

Byte 0: 0x3
Byte 1: 0x4
Byte 2: 0xB
Byte 3: 0x0
Byte 4: 0x38
Byte 5: 0x0

Unfortunately, I had no idea what this data meant. So I stared at the numbers. During this test, I had been weighing my iPhone, and the LCD display on the scale was displaying 5.6 ounces.

Hmmm, I thought. I’m pretty much a moron when it comes to hexadecimal, so let me convert these numbers to decimal and see if they make any more sense to me. Here goes:

Byte 0: 3
Byte 1: 4
Byte 2: 11
Byte 3: 0
Byte 4: 56
Byte 5: 0

Well, that didn’t help a whole–wait a minute! Byte 4 says “56″ and there are 5.6 ounces on the scale? Coincidence? I think not!

So I added a Sharpie on top of my iPhone and ran the application again. 7.8 ounces and byte 4 says 78.

Like any good engineer, I think to myself, if it happens three times, it must be true! Sure enough, replacing the items with a dead hard drive that was laying on my desk resulted in 12.3 ounces == 123 in byte 4.

Great. So byte 4 is returning the weight in tenths of an ounce. So all I have to do is take byte 4, move the decimal point once to the left, and there I have it. But what happens when byte 4 overflows? To discover this, I pile up everything onto the scale–iPhone, Sharpie, and two dead hard drives–and the scale displays 2 lb 13.5 oz, or 37.5 oz. The data returned is

Byte 0: 3
Byte 1: 4
Byte 2: 11
Byte 3: 0
Byte 4: 119
Byte 5: 1

This doesn’t seem to make a lick of sense, until I noticed that byte 5 is now displaying a 1 where it always had returned a 0 before. I take a wild guess: if byte 4 is returned tenths of an ounce, could byte 5 be an overflow bit? 1 * 256 = 256 + 119 = 375 … yes! 37.5 ounces!

What happens when byte 5 overflows? Who knows–the scale is only rated for 25 lb, and overflowing bytes 4 and 5 would be like, what, 4,112 lb? Let’s not find out.

After some more twiddling, I discovered that byte 1 returns “4″ when the scale is stable (e.g., not bouncing up and down, trying to figure out the weight) and “not 4″ when it is sliding. Therefore, my code to read from a Stamps.com Model 2500i scale using Mike O’Brien’s library is as follows:

private void GetStampsComModel2500iScaleWeight(out decimal? ounces, out bool? isStable)
{
	HidDeviceData inData;
	HidDevice[] hidDeviceList;
	HidDevice scale;
 
	isStable = null;
	ounces = null;
 
	hidDeviceList = HidDevices.Enumerate(0x1446, 0x6A73);
 
	if (hidDeviceList.Length > 0)
	{
		int waitTries;
 
		scale = hidDeviceList[0];
		waitTries = 0;
 
		scale.Open();
		// For some reason, the scale isn't always immediately available
		// after calling Open(). Let's wait for a few milliseconds before
		// giving up.
		while (!scale.IsConnected && waitTries < 10)
		{
			Thread.Sleep(50);
			waitTries++;
		}
 
		if (scale.IsConnected)
		{
			inData = scale.Read(250);
			ounces = (Convert.ToDecimal(inData.Data[4]) +
				Convert.ToDecimal(inData.Data[5]) * 256) / 10;
			isStable = inData.Data[1] == 0x4;
		}
 
		scale.Close();
		scale.Dispose();
	}
}

I still have no idea what bytes 0, 2, or 3 signify, but I’ve followed the first rule of coding: when it’s working, stop coding.

I hope this helps somebody take a USB scale that Simply Does Not Work and turn it into one that Does. Good luck!