Drupal Commerce Alpha 4

Well, Christmas is already over for a lot of you, but there's still an hour left here in the States for me to wish everyone a Merry Christmas with a Drupal Commerce Alpha 4 release. I drove down to Louisiana to celebrate with my wife's family, and for once I took the passenger seat and got a good bit of work in to tighten up our Entity / Rules integration. Éowyn had a meltdown after 8 hours, too, so thanks to the Super 8 Motel in Wheatley, AR, I made even more headway on the dynamic Add to Cart form and its dependent attribute display.

However, integration and maintenance issues aren't the most exciting things in this release. I'm much more excited to announce the extensive work done on Drupal Commerce's Rules powered dynamic pricing system. We now have a centralized system empowering site builders and store administrators to implement pricing rules for discounts, tax calculation and display, and currency conversion through the user interface. While some of the subsystems still need attention, the Rules are in and working on the Calculating the sell price of a product event. You can see the results immediately on product field displays and in the shopping cart.

For example, I might want to give members to my Wombat Dating site a 10% discount. Never mind the absurdity of the scenario (if that's even possible) and see the basic product display node showing the product image and price fields:

The price field is displayed with the Formatted amount display formatter, which knows that modules are enabled that allow for dynamic price calculation:

This particular calculation option allows product sell prices to be calculated through Rules, so I've setup a 10% Members Discount using this simple rule configuration:

The end result, as you would expect, is a much cheaper night out on the town with our lovely wombat:

This system is a huge step forward from what we attempted in Ubercart, which lacked a baked in UI, touched every price on the site, and involved a pretty heavy caching system that couldn't address the problem of executing queries based on calculated prices. The system eventually got the boot, but we got a chance to tackle the challenges afresh with Drupal Commerce. Through some good discussion at DrupalCon Copenhagen with Damien, Miro Dietiker, and others, we realized the problem was actually in how we understood our dynamic pricing needs.

The first thing we did was restrict our initial scope to calculating product sell prices. That solved a lot of performance problems and API confusion on its own. However, we still had to solve the querying problem since we're depending on Rules to perform the calculations. This means we require PHP to execute to find the actual sell price of a product for any given customer. Simple queries that order and filter lists of products based on the sell price would be inaccurate once the price was altered for display, so a customer with a special discount on a product won't see it at the top of a catalog View when he orders by price. We've run into this problem several times at Commerce Guys, especially when displaying prices with tax included.

What we determined was that we needed to pre-calculate dynamic prices so the data would exist in the database before a customer ever needed it. Since we know the Rules attached to the price calculation event, we can pre-calculate prices for every combination of applicable Rules. The pre-calculated price data can then be joined into queries for any given set of Rules, enabling ordering, filtering, and faceting to work with actual sell prices instead of just the base price stored on the field. There are some guidelines to follow when constructing Rules for pre-calculation, and we've also devised a few simple ways for larger sites to keep the calculated price table from breaching epic row counts.

If I lost you, my apologies. It's very exciting stuff for the project, and I tend to talk too much about it. I've already explained it to all of my in-laws, including my 75 year-old grandfather-in-law whom I'll be teaching Drupal while I'm down here for his personal website.

I'll be blogging more about the dynamic price pre-calculation system, especially once it gets an actual user interface. For now it's a small API and a database table, but it's itching to be put through its paces. I'll be testing it first with Views and welcome additional eyes on the code. For more information on how it functions, refer to the development specification and the copious amount of comments on the pertinent functions in the Product Reference module.

Anyways, I've been closeted up with the laptop enough for one Christmas, so I'm going to join the rest of the family with some doctored eggnog and a seat in front of the fire. I hope you all had a Merry Christmas, and I look forward to feedback on the new hotness in Alpha 4.

Check out the release notes for the full changelog since Alpha 3, If you're thinking of resolving to visit Paris for the New Year, check out the details on our Paris Commerce Sprint, January 17-21, 2011 and consider joining us in Paris to push this code from a planned beta to 1.0 in time for DrupalCon Chicago. We'd love to have you.

Drupal Commerce Alpha 3

This last week's development on Drupal Commerce has been a flurry of exciting commits for me. Ever since the first demos I've been talking about how the Add to Cart form will be able to use field data to let customers choose products by their field values instead of their titles. Similarly, I've talked about dynamically updating field data on the page so images, prices, and other fields are re-rendered to reflect the currently selected product.

Until now these features have been relatively low priority, as we focused on the other major core systems. However, with the release of 7.x-1.0-alpha3, they have finally moved from the hypothetical realm into code awaiting your testing.

Example Add to Cart form
Inherited product fields, dynamic options, and dependent attributes... oh my!

The screenshot shows three features of Drupal Commerce product displays:

  1. Product fields from the default product are pulled into the node display via the Product Reference field. The pictured display actually references 12 separate products representing different color / size combinations.
  2. The Add to Cart form display formatter turns product reference fields into forms. It creates attribute select lists using field data from the referenced products, only including options that actually exist on products referenced by the field.
  3. As options are selected on the form, the product fields displayed on the page are updated to reflect the currently selected product. The options on the form also update to show newly available options, allowing for attribute dependencies to any depth in their order of appearance on the form.

In other words, as I choose a color option, the size select list updates to only list sizes of the shirt available in that color and the image can change to show me wearing a different shirt. If the size is changed to a more expensive option, the price on the page updates accordingly. I'm sure this description hardly does the features justice, so I'll get a screencast and live demo up next week for you to play with.

This behavior is all thanks to the new #ajax property in Drupal 7 and required no custom JavaScript to work. When a select list's value changes, the form is submitted and rebuilt server side giving me a chance to change the available options. My #ajax callback uses a set of AJAX commands to direct the replacement of the existing form with the newly rebuilt form and works alongside the product reference field to re-render and replace any product fields on the page as necessary.

(On a side note, if you need to do full form replacement yourself, be sure not to target the form using its id. Each time it is rebuilt, the HTML id is incremented by drupal_html_id(), so the replacement will only work the first time. Instead, either target a container within the form or some other selector that won't change each refresh. This cost me a few hours. )

Other features of note in this release include:

  • The order edit form has taken shape with vertical tabs, revision logging, and working customer profile reference fields.
  • All currencies have been defined with updates to the Price field accommodating multi-currency stores. Currency rounding now accommodates currencies like Swiss Francs, which rounds to the nearest twentieth (thanks to das-peter). Only known currency formats are implemented, so you may need to open an issue for yours to have proper formatting (i.e. $100.00 vs. 100.00 USD).
  • The Payment module received more work since the last release to support credit card payment methods. I've created Commerce Authorize.Net as a proof-of-concept and example implementation for other developers to follow. (Some code specific to that module right now will be abstracted to Payment, like a general handler for prior authorization captures.)

More information and a full commit log are available in the release notes. I'm very grateful for the efforts of community contributors, especially das-peter for his help on the currency issues and our Entity API and Rules integration.

There's still plenty of work to be done, so feel free to join is in the issue queue.

Drupal Commerce Alpha 2

Blog's been a bit sleepy as of late... so sleepy it managed to miss out on two trips to Europe for DrupalCon Copenhagen and E-commerce Paris 2010, the first Drupal Commerce alpha release, and my daughter's first steps and first birthday.

Éowyn looking up.
"What gives, dad?"

Does it work to just say they were all awesome and move on? My wife can testify that many things in my life back up to a point where a dearth of activity stymies said activity's resumption... or simply put, when I stop doing something I want to do for an extended period of time, I find it hard to start again.

Ok, </introspection>. I'm breaking the ice with a happy announcement - I managed to package up Drupal Commerce 1.0-alpha2 in the wee hours of the night.

The release notes provide an overview of changes with a full commit log since the alpha 1 at DrupalCon CPH. The new alpha and my development installation profile have been updated for Drupal 7.0-beta1, though I still need to update our Fields to take advantage of display formatter settings.

The highlights of the release are:

  • Functioning Payment module with integration points for on-site methods (demonstrated by the core Null payment module) and redirected payment services (demonstrated by PayPal WPS integration in Commerce PayPal). Payment service integrators would do well to jump on board so we can take care of any workflow issues, especially for redirected payment services.
  • SimpleTest base class for Commerce testing provided by Jeremy Blanchard, complete with documentation and related issues. It's just begging for you to start using it...
  • Updates to the price object to allow Rules based transformation of prices prior to purchase and/or display; still requires some implementation as outlined in a rambling Google Doc.
  • The Specification handbook at DrupalCommerce.org has received some love to get it up to speed. A major goal for the project was to not have undocumented major releases, so I've begun documenting the various system components and will then move onto the system overviews. For now only the sections on our info hooks and core API utilization have meaningful content. Feedback on the structure is most welcome.
  • A host of bugfixes and API / variable naming standardization. The documentation effort really helped here, so I expect similar results as I catalog the Commerce Entities and Fields.

The release is also accompanied by an alpha release of Address Field, a dependency the Customer module uses to collect name and address information during checkout according to the xNAL standard. This module took some massaging to get working, because it defines a Field that uses #ajax to update address form elements based on the country chosen. It now works no matter where I embed the widget, even if I'm doing something crazy like embedding the customer profile entity's field widgets in the checkout form... but that's a topic for a follow-up blog post.

That's all to report for now. There's plenty more activity and good contributions coming through the issue tracker. As this alpha's release was dependent on a fully functioning Payment module, the next alpha release will be dependent on the full implementation of dynamic pricing as outlined in the doc above. We'll also push to get as far as we can on the order administration UI and show some love to product and line item display in the meantime. We'll stay busy!

Book Review: Drupal E-commerce with Ubercart 2.x

Developing Ubercart was a wonderful experience. I learned a lot about software development, Drupal, web technologies, and worldwide communities. At the end of my time leading the project, I could also see that thousands of people were using the software to sell products for profit, bring people together at events, and raise money for many worthy causes and organizations. Seeing now a book written to help even more people do these things is an added treat.

I was asked to review Drupal E-commerce with Ubercart 2.x published by Packt and received an e-book and three physical copies to give away at DrupalCon San Francisco. I started reading the e-book on that trip and finally finished on the plane home from DrupalCamp Colorado.

What follows is my review with a chapter by chapter analysis of the content. What I saw was very encouraging, and I applaud the authors, George Papadongonas and Yiannis Doxaras, for their effort in delivering this work to the community. My review picks up on what's great about the book, what could use some work, and who I think would benefit most from owning it.

Before the in-depth analysis, here's a quick summary:

  • This book provides a very comprehensive look at Ubercart's major core and contributed features, modules, and themes (free and commercial). It's the most expansive summary of all that Ubercart offers I've seen. The value in this exposure can't be overstated, and it's for this aspect of the book alone that I'd recommend it to anyone diving into building e-commerce sites with Ubercart.
  • The book includes chapters and content introducing the user to Drupal itself. I suppose this was to make it a standalone resource for building e-commerce sites on Drupal, but I think the book would have benefited from a sharper focus on Ubercart itself. Referencing one of the many general Drupal books, even titles published by Packt itself, would have been sufficient for topics such as installation, search engine optimization, and theming.
  • The book introduces all of Ubercart's features but doesn't often address the reasons for a particular feature's existence or the best practices for implementing it. For example, chapter four could have provided a lengthier introduction to product attributes so readers have a fuller understanding of the reasoning behind the system, when and how to use them, and what to look out for (like losing adjusted SKUs when attributes are modified on a product). If the book were made shorter by removing the general Drupal instruction, the content could easily be made up in those sorts of discussions.
  • In other areas, the discussion is particularly well thought out and helpful. I thought the payment discussion was great.

Read on for the chapter by chapter summary and review. As I said in the first point, I'd recommend this book to any newcomer to Ubercart. Experienced users might not find much new here but could still employ the book as a desk reference. Anyone looking to write modules and customizations for Ubercart won't find much here - but that wasn't really the point of the book.

For more information, resources, and errata, you should refer to the book's own website, http://www.drupalubercartbook.com/.

Chapter One: Getting Started

The first chapter provides a quick introduction to both Drupal and Ubercart that hits all the high points of using the software to build your store. It then takes the approach of showing the end product first - there are screenshots and a feature walkthrough demonstrating the store that the reader will build by the end of the book. Good idea!

Chapter Two: Installation of Drupal and Ubercart

The second chapter covers installation of both Drupal and Ubercart and should successfully guide a newcomer with some technical experience through the installation process. If anything, it might be a little too thorough. Because the book isn't really a Drupal primer, many of the details and "gotchas" for general Drupal installation could have been left out entirely. Furthermore, instructing readers to download Ubercart from Ubercart.org instead of http://drupal.org/project/ubercart seems risky, as the Ubercart downloads page has a history of lagging behind releases.

I was impressed that the book included a section on the UberDrupal installation profile, and in the first update of the text it would be good to mention that drupal.org now automatically packages distributions based on installation profiles that include all the necessary modules. To install Drupal and Ubercart now, one would simply have to grab the latest distribution from http://drupal.org/project/uberdrupal ... but now I'll need to go make sure all the modules in that package are up to date!

Chapter Three: Basic Configuration

The third chapter runs through various administration pages and configuration forms describing what a lot of the forms and settings do. As with the previous chapter, I would've left the general Drupal information out, but the chapter doesn't suffer from having it in there. The main problem with the approach as I see it is the chapter introduces all the various Ubercart settings, but at this stage in the book they're divorced from any greater context. Descriptions of some of the settings aren't helpful without greater knowledge of how Ubercart works in general, so the discussion would have fit better in a chapter specifically introducing the topic (e.g. introducing order panes, invoices, and settings in a chapter on Ubercart orders).

Chapter Four: Managing Categories, Products, and Attributes

Chapter four is spot on when it begins by emphasizing planning. Ubercart offers multiple ways to simplify product catalog creation and administration, and the authors provide a good overview of how to get it done. I was pleased to see a description of product classes, but it would have been greatly improved by a discussion of how to take advantage of their default attribute settings. Those sorts of tips should save store administrators plenty of time.

I did feel like the introduction of Tagadelic was a little distracting and might recommend placing all the contributed module discussions in the later chapters on customizing / optimizing the store. I also think it would be helpful for the order of the information to be adjusted, so that readers are instructed on how to make attributes and product classes before bothering with information on product kits and especially product importing (as that will often require classes and such to be configured in advance). In fact, given the poor support for product importing in general, I'd almost want to just punt that whole section off into an Appendix or at least a separate chapter where the pitfalls and strategies for the topic can be addressed.

Chapter Five: Managing Shipping and Packaging

Chapter five introduces the configuration of shipping quotes and Ubercart's limited product packaging functionality. It includes a brief description of the Conditional Actions system, but in my experience with training, there is not near enough material here. You could really include an entire chapter on navigating the CA interface, explaining the concepts, and walking through screenshots of setting up predicates. Most people won't understand how to add conditions and actions, especially when it comes to grouping, logical operators, and argument selection. While I wouldn't expect an introductory book to be a full manual on the system, the instructions and few screenshots scattered through the chapters seem insufficient for getting started.

Chapter Six: Managing Taxes and Payments

Chapter six covers taxes and payment, two essential parts of any online store. Figuring out taxes is one of the trickiest things for any online store, especially stores selling physical goods across multiple tax jurisdictions. The chapter provides a sound overview of Ubercart's tax features, but this is one of the rare places in the book where appropriate contributed modules aren't mentioned. Users should be aware of tax modules providing out of the box support for specific tax jurisdictions (like some countries / states) and VAT.

The payment discussion shines in this chapter as one of the most thorough and helpful of the entire book. It includes a lot of the background information and best practices instruction that I felt would've benefited other discussions in the book. Many readers, especially those new to e-commerce in general, will find the payment section extremely helpful.

Chapter Seven: Managing Customers and Orders

Chapter seven begins with a helpful introduction to the order management features of Ubercart, including a nice explanation and graphic explaining the difference between order states and statuses. The chapter is full of screenshots and instruction for administering orders and reminds me that in spite of Ubercart's limitations, there's still quite a lot you can do with it.

The chapter does mostly gloss over the packaging and shipping features on order screens, but those are seldom used features and the mention seems sufficient. Readers should get a firm grasp on how to create and administer orders from this thorough chapter. There are some minor errors in the chapter, like saying that Ubercart's reports depend on Views (they're actually stand alone and worse off for it), but nothing that should confuse newcomers to Ubercart. Also, the chapter helpfully introduces readers to CiviCRM but fails to mention how to setup an ongoing integration between it and Ubercart using the contributed integration modules.

Chapter Eight: Customizing the Frontend

Chapter eight is something of an enigma. It's full of useful information, but it feels more like a brain dump of the many ways to theme a Drupal site (from free and commercial themes to rolling your own using Zen or from scratch) than actually customizing a frontend for an Ubercart store. I'd expect to find information on theming product pages, the shopping cart, the checkout form, and order invoice templates. Instead, the chapter offers many different ways to prepare a custom Drupal theme without really going deep into any particular topic. This is one of those more general Drupal introductions where recommending a reference book on Drupal theming at the beginning and then explaining how to specifically customize the Ubercart frontend would've been more helpful.

Chapter Nine: User Interface Enhancements and Techniques

Chapter nine takes a look at using contributed modules to improve the user experience of your store for the purpose of increasing sells. I didn't find the cross-selling section useful beyond finding out that the Recommender module has an Ubercart integration module. It never went beyond introducing a couple methods without really applying them, and I didn't see mention of the UC Upsell module which was written specifically for thus purpose. However, the chapter moves from there to a much more useful section introducing using Panels, Views, and the Ubercart Views module to merchandise products.

The rest of the chapter introduces some key contributed modules that enhance the shopping cart / checkout form and others for offering discounts and coupons to your customers. The AJAX cart block module mentioned is a great way to get around the limitations of Ubercart's static cart block when page caching is enabled. The UC Discount Framework module doesn't get mentioned (likely due to its development status), but it provides an additional way to provide discounts using the Conditional Actions system. You might never use all the modules mentioned in this chapter (though you're bound to use some), but simply knowing they exist will help you greatly as you build out different types of sites.

Chapter Ten: Optimizing and Promoting Your Store

Chapter ten walks through various optimization topics regarding search engine optimization, marketing your site, improving performance, and enhancing security. It starts by walking through configuration of various modules recommended by the SEO Checklist module. Following the chapter on cross-selling products, I expected Packt to mention the Drupal SEO Book here.

There are certainly more ways to market your site and improve performance than are mentioned, but the chapter should provide a good start for new store owners to get a leg up. In what seems to be an oversight, the Ubercart Google Analytics module isn't mentioned in the discussion on Google Analytics, but enabling that module will cause actual sales data to be reported to Google Analytics for review in addition to goal completion.

Appendices

The appendices offer some nice extra information, like how to take advantage of the UC Hotel module to sell hotel reservations using Ubercart in Appendix A. It won't be useful for many people, but for those who need it I'm sure it will be helpful. That's why it's an appendix I guess! The remaining appendices list all the modules referenced in the book (there's a lot!) and a host of free and commercial Drupal themes that are ready for e-commerce use and/or specifically designed with Ubercart in mind.

Pages