AVS Information In Magento

We’ve recently set up a new e-commerce store at Men’s Underwear Discounters that represents a division of the company that I work for, Skiviez. We decided to save time for this discount store by using Magento, the current darling child in the open source e-commerce platform market segment.

A brief interlude on consulting, documentation, and open source

Now, to be clear, it doesn’t take much to exceed expectations in the “open source e-commerce platform” market because prior to Magento, your options were pretty much limited to osCommerce, which is quite possibly the most reckless, idiotic excuse of an e-commerce platform that I have seen (and I don’t like to insult code), and its slightly-less-retarded sibling Zen Cart. (I will save the osCommerce rants for another post.)

Magento seems to be a dream come true: thousands of man-hours of development into an e-commerce platform for free. So what’s the catch? While Varien (“the Magento company”) has done a great service to the world and the open source community by creating Magento, they’re still in it to make a buck–and that’s not a bad thing–but you might not realize how they make those bucks until you’ve started playing around with your own Magento store. They make money on support, consulting, and customization because Magento is huge and poorly documented.

Actually, “poorly documented” would imply that there was at least some documentation, and while there truly is some documentation available in the source code and on Varien’s Magento Web site, in reality, that amount of documentation is a statistical rounding error. And when you actually view your Magento installation in an FTP program, you will be absolutely astounded at the number of directories and files. You can try inspecting the source files to try to put it all together, but a lot of it is “magic”: a framework erected at runtime by XML configuration files and PHP’s idiotic __call mechanism for magic functions that appear at runtime out of thin air. You can see references to a function called getOrder(), for example, grep the entire installation for “getOrder()”, and not get any results because the function doesn’t actually exist at compile time. Personally, I’ve always considered magic methods to be an extremely unwise idea, a “too clever” approach to programming. But for people new to PHP looking to do a little customization to their Magento store, it makes the codebase all the more impenetrable. That’s more consulting dollars.

You could go to the Magento forums, but you won’t find too many helpful answers because Magento obviously doesn’t want to give help for free, since giving help is central to their business model. So if you need help, your best bet is to generally (a) pay for it or (b) hope that you stumble across somebody’s blog who took the time to post a solution to your exact problem.

Why do I need to do this, anyway?

Magento has a lot of stuff built in. But we need to remember that Magento was built by software developers, and software developers generally don’t understand the problem domain of running an e-commerce business very well. Sure, every software developer has built the pet shop–products, categories, inventory, orders, how hard could it be?–but they stop or fail to understand that there’s a whole class of operations that need to happen after the order has been placed.

One of those operations is fraud analysis. The Address Verification System (AVS) and the Card Verification Number (CVN) system for credit cards enable merchants to help determine if a credit card has been fraudulently used on an order. AVS will tell the merchant whether or not the billing address specified by the customer matches or partially matches the billing address on the credit card’s billing statement. And the CVN number represents the number that (supposedly) exists in only two places in the world: on the physical credit card and in the issuing bank’s files. If a CVN doesn’t match and the AVS doesn’t match, then there’s a good bet that there is something wrong with the order.

Because software developers are nerds, this information doesn’t currently display by default in Magento’s administration section. So let’s add it.

Can’t I just edit the template?

I’m talking about adding this AVS display functionality in the context of the CyberSource extension for Magento. After spelunking through the myriad files and directories, I discovered that the Varien-provided CyberSource extension is saving the AVS (setCcAvsStatus()) and CVN (setCcCidStatus()) information when it authorizes the card, but it doesn’t display in the information block in the administration section.

While you could just modify app/design/adminhtml/default/default/template/cybersource/info.phtml directly (yes, that is the actual path to the file that displays the little “Payment” block), your changes would be lost and overwritten in the future if you ever upgraded the CyberSource extension via Magento Connect. That’s obviously less than ideal.

A better way would be to use Magento’s “rewrite” mechanism. We’ll create our own Magento module and instruct Magento to replace the CyberSource extension’s information block with our own.

Creating our own module

First, we’ll create a new empty directory to hold our files. Let’s call it avsdisplay. Then we’ll create code, etc, and design subdirectories. These directories mirror the subdirectories in the app folder of your Magento installation.

In the etc directory, we need to create an XML configuration file. Magento scans this directory at runtime for XML files and reads them to know which modules to load. I’ve named my configuration file Mud_Avsdisplay.xml, and the naming is very important. (Magento uses a mixture of convention and configuration.) The part of the filename before the underscore is your “namespace,” since PHP doesn’t support namespaces. And the part of the filename after the underscore is the name of your module. Mine is “Avsdisplay.” Don’t try to use something like “AvsDisplay,” however, because that would break Magento’s convention as we’ll see in a moment.

Inside this file, we tell Magento to load our module in the local pool and to actually run it (by setting “active” to “true”). (The other pools are core, for those modules provided by Varien, and community, for those modules that you can download via Magento Connect.)

<?xml version="1.0"?>
<config>
	<modules>
		<Mud_Avsdisplay>
			<active>true</active>
			<codePool>local</codePool>
		</Mud_Avsdisplay>
	</modules>
</config>

Now it’s time to create the meat of the module. In the code folder, create a local directory. Within that, create a directory named after your chosen namespace (Mud in my example) and within that yet another subdirectory named after your module (Avsdisplay in my example). (This is why naming was important. If I had called it “Mud_AvsDisplay,” with a capital ‘D’, then Magento would have looked in app/code/local/Mud/Avs/Display for my module instead of app/code/local/Mud/Avsdisplay.)

Within your module directory, you’ll need to create three more subdirectories: Block, etc, and Helper. When it’s all said and done, you’ll end up with a hierarchy that looks like this (ignore the “design” directory, we’ll get to that later):

Directory Structure

Creating the block

Next, we need to create the “block” that we’ll be replacing. After searching through the codebase, I figured out that I wanted to replace the Mage_Cybersource_Block_Info block (for reference, by Magento’s convention, that file lives in app/core/Mage/Cybersource/Block/Info.php — see the convention?). But I really only want to add stuff to it, not replace it outright, so I’ll create a new file in my Block folder called Info.php that extends that class. Something like:

<?php
 
class Mud_AvsDisplay_Block_Info extends Mage_Cybersource_Block_Info
{
	// Yes, these really have to be protected, not private, due to more Magento
	// "magic"
	protected $avsDescriptions = array(
		// Snipped for brevity; download file at end of blog post
	);
 
	protected $ccCidDescriptions = array(
		// Snipped for brevity; download file at end of blog post
	);
 
	protected function _construct()
	{
		parent::_construct();
		$this->setTemplate('avsdisplay/info.phtml');
	}
 
	public function getAvsStatusDescription()
	{
		$info = $this->getInfo();
		$avsCode = $info->getCcAvsStatus();
		$avsDescription = 'Unrecognized response code.';
 
		if (array_key_exists($avsCode, $this->avsDescriptions))
		{
			$avsDescription = $this->avsDescriptions[$avsCode];
		}
 
		return $avsDescription;
	}
 
	public function getCcCidDescription()
	{
		$info = $this->getInfo();
		$ccCidCode = $info->getCcCidStatus();
		$ccCidDescription = 'Unrecognized response code.';
 
		if (array_key_exists($ccCidCode, $this->ccCidDescriptions))
		{
			$ccCidDescription = $this->ccCidDescriptions[$ccCidCode];
		}
 
		return $ccCidDescription;
	}
}

Okay, this makes sense. I’m doing everything the old CyberSource info block did, except I added two new functions that give me human-friendly descriptions of the AVS and CVN codes, and I’m setting the template to something called avsdisplay/info.phtml instead of cybersouce/info.phtml. How this comes into play will make sense in a minute.

Creating the helper

In my Helper directory, I need to add a Data.php file that looks like this:

<?php
 
class Mud_Avsdisplay_Helper_Data extends Mage_Core_Helper_Abstract
{
}

That’s right, it does nothing except inherit from a Magento-provided abstract class. What is a helper? I have no idea, but you have to do it–Magento looks for this file.

Telling Magento to do the swap

In my etc directory, I need to add a config.xml file that will tell Magento some information about my module and, most importantly, that I want to be swapped with the CyberSource extension’s info block:

<?xml version="1.0"?>
<config>
	<modules>
		<Mud_Avsdisplay>
			<version>0.0.1</version>
		</Mud_Avsdisplay>
	</modules>
	<global>
		<helpers>
			<avsdisplay>
				<class>Mud_Avsdisplay_Helper_Data</class>
			</avsdisplay>
		</helpers>
		<blocks>
			<cybersource>
				<rewrite>
					<info>Mud_Avsdisplay_Block_Info</info>
				</rewrite>
			</cybersource>
		</blocks>
	</global>
</config>

Under <modules>, I’m just telling Magento the version number of my module. Under <global>, I say that I want to “rewrite” the CyberSource info block with my own class named Mud_Avsdisplay_Block_Info, which, by convention, Magento will look for in [module-root]/Mud/Avsdisplay/Block/Info.php. That is the file that I just created. Great.

Finally, writing the template that displays the information

But we haven’t actually changed any HTML at this point! Remember in my Info.php file where I set the template to something called info.phtml? Well, I need to create that file. At the top of my module folder–the one with the code and etc subdirectories–I’ll need to create some more subdirectories. A lot of them. In fact, I’ll need to create a bunch of empty directories that look like this:

In the bottom-most subdirectory, I’ll create the info.phtml file. This is a PHP file, except that when I use the $this pseudo-variable in this file, $this will be pointing to an instance of my Mud_Avsdisplay_Block_Info class. So I’ll be able to call my getAvsStatusDescription() method here among other things:

<?php
	if ($info = $this->getInfo())
	{
		?>
			<strong><?php echo $this->getMethod()->getTitle(); ?></strong><br />
			<strong>Type:</strong> <?php echo $this->__($this->getCcTypeName()); ?><br />
			<strong>Number:</strong> xxxx-<?php echo $this->__($info->getCcLast4()); ?><br />
			<strong>Expiry:</strong> <?php echo $this->__('%s/%s', $this->getCcExpMonth(), $info->getCcExpYear()); ?><br />
			<strong>AVS:</strong> (<?php echo $this->__($info->getCcAvsStatus()); ?>) <?php echo $this->__($this->getAvsStatusDescription()); ?><br />
			<strong>CCID:</strong> (<?php echo $this->__($info->getCcCidStatus()); ?>) <?php echo $this->__($this->getCcCidDescription()); ?><br />
		<?php
	}
?>

Conclusions and delusions

And that’s it. Upload this entire directory structure to your Magento installation’s app directory, merging the contents as necessary, and you should be able to see AVS information for your CyberSource orders in the administration section.

How did I figure out how to learn all of this? It was mostly by lots of Googling, picking apart existing modules to see how they were organized, and piecing together lots of blog posts. I hope this helps someone out there. Good luck!

Download the code used in this article.

I can't log in!

What has happened here is that my OpenID provider, Verisign Labs, is down for some reason while StackOverflow, the site that I’m trying to log on to, is still up.

A single point of failure

While this isn’t a big deal for a site that I use recreationally, it is kind of a big deal in principle: all sites that use OpenID are inaccessible to me, since I only have this single OpenID provider. It is a single point of failure.

Sure, I could have multiple OpenID accounts, such as a “backup” OpenID account at another provider, but this defeats the purpose of OpenID, of having a central point of authority regarding my online identity. If I have to create a second OpenID provider, I might as well just make a native username and password at the site that I’m trying to log into. I could also be my own OpenID provider, but I shouldn’t have to do that, and the average OpenID consumer is not going to know how to do that, anyway. It’s faster just to create a “native account” at the Web site that I’m trying to access and be done with it.

An unnecessary third-party dependency

For any e-commerce business, it’s usually unwise to take on any third-party dependency. Speaking from experience, that third-party will go down for some reason at some point and you will be the one fielding angry customer support calls for something that isn’t your problem. Even worse, if something as critical as authentication through OpenID goes down, then you’re losing sales, and that means that you’re not doing your job.

If you’re going to add OpenID to an e-commerce Web site or a “mission-critical” Web site, then you should implement it as a red-headed step-child, “convenience only” implementation; this way, your users can fall back onto their “native credentials” (those stored in your database) when their preferred OpenID provider explodes. In this way, OpenID is a shortcut, not a single authoritative source, much how like customers use PayPal’s Express Checkout to save time when entering payment and addressing information during checkout. If PayPal’s Web Service API goes down (and believe you me, it’s gone down several times this year alone), customers can still begrudgingly use the “native checkout” to complete their purchase. Some lost conversions due to the inconvenience–and for incorrectly faulting the integrity of your Web site for the third party’s failure–but at least the sale is still actually possible. If OpenID is your only authentication mechanism, and a user’s OpenID provider goes down, then that user is simply screwed. The onus should not be on the consumer to provide a backup authentication mechanism to your Web site.

It’s just complicated

OpenID is still just too complicated. I’m not the smartest software developer in the world by any means, but if I can barely wrap my mind around it enough just to get my first account, then I am 100% certain that our customers would have problems with it:

• Why do I have to go to some third-party provider to sign in? That doesn’t make any sense.
• Why should I trust that third-party provider? I’m not even sure that I trust you!
• I still don’t know which third-party provider to use! Oh well, I guess I’ll use Verisign, at least I’ve heard of them.
• What happens to my credentials when that third-party provider goes under?
• Can I call your business to reset my password when I forget it? I will anyway!

Even with our “native registration” of e-mail addresses and passwords, we had to use an Amazon.com style login form because users kept filling out the “new customer” form even when they already had an account:

Login screen. Amazon’s sign-in screen remains a model to be emulated, minimizing the common problem of new customers who try to log in without having registered. Amazon presents two questions in linear order: (1) “What is your email address?” and (2) “Do you have an Amazon.com password?” For the second question, users can select one of two radio buttons: “No, I am a new customer,” or “Yes, I have a password.” Many other sites present the new- and established-user sections side-by-side, and thereby divert new users to the established-user section through the magnetic attraction of type-in fields. — Jakob Niesen, useit.com

When our users have trouble navigating two fields and a radio button, you can imagine the hilarity that ensues when they are presented with multiple authentication mechanisms. Especially one that doesn’t ask for a password.

A leaky abstraction

While the idea of consolidating your online identity to a single source sounds like a good idea, it simply does not work in practice. Not even the United States government has a consistent picture of my identity: my identity is stored with the IRS, the Social Security Administration, the United States Postal service, the Virginia Department of Taxation, the Virginia Department of Motor Vehicles, and the City of Richmond’s Voter Registrar. These are all independent identity stores that, in general, agree that I am who I say that I am, but each identity is stored and updated independently of one another. Consider OpenID the RealID act of the Internet: what happens when your centralized identity gets screwed up? It’s an existentialist crisis and you just have to deal with it: identity is what other people say you are; it’s not what you say you are. Better to have multiple voices confirming your identity than just one.

Conclusions and Delusions

If you’re implementing a social-oriented Web site that is designed to integrate with Facebook or some of the Web-2.0-savvy consumers, then OpenID might make sense. Those nerds may have heard of OpenID and know how to use it, foibles and all. Until the dust settles on OpenID, though, I wouldn’t add it to a commercial site even if you paid for it: nobody has asked for it; it will cause problems.

My general recommendation would be to have the usual in house username and password mechanism that can be supplemented by OpenID. But realize that each alternative identification mechanism that you add runs the risk of customer confusion, bugs, security holes, and increased customer support. When in doubt, leave it out.

[npiaseck ~]$ telnet www.example.com 80
Trying 10.1.10.38...
Connected to www.example.com (10.1.10.38).
Escape character is '^]'.
GET /Media HTTP/1.0
 
HTTP/1.1 301 Moved Permanently
Content-Length: 152
Content-Type: text/html
Location: http://10.1.10.38/Media/
Server: Microsoft-IIS/6.0
Date: Fri, 05 Dec 2008 13:50:00 GMT
Connection: close
 
<head><title>Document Moved</title></head>
<body><h1>Object Moved</h1>
This document may be found <a HREF="http://10.1.10.38/Media/">here</a>
</body>
 
Connection closed by foreign host.

I’ve made up an internal IP address, but the point is that IIS is inserting the internal IP address in the body of the document as well as the Location header. If you make the same request with the Host header supplied, such as in Host: www.example.com, then IIS would use www.example.com instead of supplying the IP address.

The recommended Microsoft solution is detailed in KB834141. Namely, you edit the SetHostName metabase entry for the Web site to www.example.com, and now IIS will use that instead of the IP address in the above redirection scenario. Easy as pie, right?

Well, that change broke some code. For reasons that are not clear to me, here’s the mechanism by which IIS 6 determines what to use in these redirect requests:

1. Is the UseHostName property set to true? If so, use the machine’s host name.
2. Is the SetHostName property set to some value? If so, use that value.
3. Is a Host header supplied in the HTTP request? If so, use that value.
4. If all of the above fail, just use the IP address, which is probably internal.

For search engine optimization reasons, all of the requests on our Web site redirect all requests that come in without the www subdomain to ones with the www subdomain. This prevents Google from dinging us for having duplicated content. The code was implemented as an ASP.NET HTTP module that ran before every request and inspected the incoming URL by looking at the HttpContext.Current.Request.Url.OriginalString property. If the domain didn’t match, then it issued a permanent redirect.

The problem with the above precedence order is that IIS will substitute the value of HTTP_HOST and other server variables passed into ASP.NET with the SetHostName value, not the Host header if so supplied. The net effect is that our redirection method suddenly stopped working because the code was always seeing the SetHostName value, www.example.com, regardless of the URL that was actually used to make the request. To make matters worse, if we were to set the UseHostName property, the code would always see a mismatch and enter a redirect loop. Neither are desirable scenarios.

The solution is to change the code to inspect the Host header directly, if present, instead of looking at the URL as passed into the server variables array. It ends up looking something like this:

private void OnContextBeginRequest(object sender, EventArgs e)
{
	HttpContext context;
	string originalRequestHost;
	int portIndex;
	HttpRequest request;
	string redirectUri;
 
	context = HttpContext.Current;
	request = context.Request;
 
	originalRequestHost = request.Headers["Host"];
 
	// Some clients will pass in the port number in the Host header if
	// it's not going over port 80
	portIndex = originalRequestHost.LastIndexOf(':');
	if (portIndex != -1)
	{
		originalRequestHost = 
			originalRequestHost.Substring(0, portIndex);
	}
 
	if (originalRequestHost != null && originalRequestHost != mPreferredSubdomain)
	{
		redirectUri = string.Format(
			"http://{0}{1}{2}",
			mPreferredSubdomain, 
			request.Url.Port != 80 ? ":" + request.Url.Port.ToString() : string.Empty,
			request.RawUrl);
		cLog.InfoFormat("Redirecting from to {0} (domain: {1}, preferred: {2}).",
			redirectUri, originalRequestHost, mPreferredSubdomain);
		context.Response.Redirect(redirectUri, true);
	}
}

The code looks at the host header. If it’s set, and doesn’t match, then it redirects. (It does a little bit of finagling to strip off the port number if a non-standard port is used in the request.)

With this code in place, we can now use the SetHostName property and still have our URL redirection work properly for HTTP/1.1 requests. We’re out of luck when it comes to HTTP/1.0 requests, but this is something that we can live with.