Getting to work on a stronger Drupal Commerce 2.x

I don't like being sandy but I love building sand castles. The potential is high but the stakes are low. You have unlimited raw material to build whatever you can imagine, and if you screw up you can just knock it over (to the delight of your children) and start again.

When your constant is impermanence, you adjust your expectations accordingly. If the sands shift or the tide rises, you shrug it off and start again or go back to bocce. However, if you were somehow bound by the fruit of your labor, you'd think twice about the when, where, and how of your building.

Lyle and I started developing Ubercart when Drupal 5 was still in beta, and we put up with the impermanence because we were just two dudes learning a lot and working from a clean slate.

Moving that forward to Drupal 6 took even longer because our codebase grew unwieldy, so I decided at Commerce Guys to trim the fat and start over. We didn't learn every lesson moving to Drupal 7, though, as it was still unstable when we began developing Drupal Commerce. With an unstable Views module. And an unstable Rules module. And an incomplete Entity API. Tongue

I don't think we would've done any differently, as the flip side of the instability is the opportunity to positively impact the development of the projects you depend on. For Drupal 8 we ended up contributing to core in other ways while tackling a full Drupal Commerce rewrite more slowly than we hoped. Even the code that we did develop against Drupal 8 is now outdated, as we had to juggle managing our existing ecosystem, writing new code, and rewriting that code to track changes in Drupal 8 itself.

However, we've recently been given the opportunity to host a variety of developers / documenters in Paris from June 30 - July 4 to re-evaluate our Commerce 2.x roadmap with the direct help and guidance of key members of the Symfony project. Specifically, we'll be looking to both move our code "upstream" from our Drupal modules into generalized libraries and take advantage of existing Symfony projects where possible.

Our target is an even leaner codebase with connections to the broader world of PHP based eCommerce. If you have any insights in this direction, please comment or contact me directly and consider joining us in Paris to learn and contribute.

Photo credit: j.s. clark

You'll Never Know Enough

Now he would never write the things that he had saved to write until he knew enough to write them well. Well, he would not have to fail at trying to write them either. Maybe you could never write them, and that was why you put them off and delayed the starting. Well he would never know, now.

-- Ernest Hemingway, The Snows of Kilimanjaro

I've been working my way through The Complete Short Stories of Ernest Hemingway, and this quote nailed me. I have half a dozen unwritten stories and a dozen unwritten blog posts. I delay writing about technical topics until I know more. I defer penning my fictional fancies for fear of screwing up what seems to me to be a decent idea.

I imagine there's a more knowledgeable, more experienced version of me just waiting to take up the topic at some point in the future, but the reality is he only gets that way through the "practice of." In Kilimanjaro, Harry dies never knowing if he was competent to write the things he planned to write or not. Here's hoping I don't do the same.

My current strategy (at least for the blog) is to write less on any given topic but do it more often. Any other ideas?

Beware InnoDB's auto_increment reset on reboot

Earlier this year I helped my friend Samuel bring his used cell phone resell business online using Drupal Commerce. The site is still in maintenance mode while we finish the self-service features, but his staff currently uses it logged in from their various locations as their point of sale system. Knowing the ins and outs of Commerce, I didn't have any problems tailor making an eCommerce application for his business, but I did have one hiccup during deployment that I'd never seen before.

We built Wikiwoo.com on Pantheon, a Drupal Platform as a Service, using a free developer site until it was ready for use in stores. Pantheon really helped us collaborate on the site build, with me doing the coding and configuring while he filled the product catalog. We did everything on the site's dev environment, including letting his partners take a look around to find things worth fixing, until we were ready to go live.

One of the last things I did to prepare for the launch was update the auto_increment value of the commerce_order table to account for the number of orders they processed in the previous year and a half. However, we weren't really migrating old eCommerce data, so I just expected the first order on the new site to start where we wanted and we'd watch them grow from there. A quick test showed it working as expected, so I deleted the dummy orders and sent him a link to upgrade the account to a paid plan to take it live.

Unfortunately, when I went back to the site the next day, I saw that orders were being created with IDs starting back at 1. I knew there was nothing in Commerce that would effect such a change, so I hit up Pantheon support and got a quick confirmation that nothing they do would intentionally reset auto_increment values either.

Sidebar: I really should emphasize quick. Any time I've ever contacted Pantheon support, they've responded right away. "Groovy," said Josh Koenig in this particular instance when we nailed down what was happening. "Groovy," I say to Pantheon's customer service. Cool

It turns out what I experienced was a result of InnoDB's treatment of auto_increment values. The auto_increment counter is only stored in memory, not on disk, and it is recalculated on server startup. InnoDB simply looks for the highest used ID and sets the counter to the next value, explaining why my order IDs shrunk back down to 1 after I cleared out all of our dummy orders.

In our case, it was the upgrade from a free account to a paid account that restarted the database server, triggering the reset of the counter. However, with cloud based Platforms as a Service, I imagine there are other scenarios where an expected alteration to an auto_increment value is apparently "lost" on migration between environments or builds. This is probably mostly an eCommerce issue with respect to Drupal sites, as merchants often want or need order IDs to account for historical sales, but perhaps the tip can save someone else a bit of head scratching.

To get around my issue, I simply reset the auto_increment to where I wanted it to be, created a cart order for myself, and waited for a real order to be created before deleting my dummy order.

Problem solved, it's been fun to watch the order count grow from there.

Photo credit: Max Barnes

Beyond Wombats

I accidentally started publishing open source software in 2006, the first integration of the QuickBooks Web Connector with anything. This was pre-Ubercart when I was just cutting my teeth on PHP / MySQL development at Prima Supply, and I thought it would be fun to claim the code was written by wombats while I just published it online. I decided to own the silliness and start blogging on bywombats.com using Drupal 4.7 at the time - and immediately picked up a freelance contract doing QuickBooks integration work. Tongue

Fast forward a few years, a couple of eCommerce projects, and a Drupal startup, and the moniker has apparently run its course. A lot of folks over the years assumed "By Wombats" was the name of a Drupal shop and tried to contact me for work. Little did they know my site's contact form hasn't been working for years. Shock

This past week I finally got around to fixing that by moving from Rackspace to Digital Ocean and configuring my server to send mail through Mandrill. I upgraded to Drupal 7 (about time?) and played around with Mike Crittenden's Meedjum theme until I liked the look. He's presently trying to bribe me to help maintain the theme, but I'm holding out for better beer.

Migrating was the fun part, as it was my first opportunity to make use of the Drupal-to-Drupal data migration module. I couldn't get moving files to work for some reason, but I figured out how to make options settings work in the configuration form (leave the select lists alone and put the value in the "Default value" text field) to make use of the preserve_files option. I copied the old files to the Drupal 7 files directory and ran the migration with that flag set for it to start managing the files that were already in the directory.

Migrating taxonomy terms was straightforward, but it took me a while to get blog posts right. I learned I had to select the taxonomy term migration in the "Source migration" select list to ensure blog posts got created in the Drupal 7 site referencing the correct term ID (since they changed during migration). I had to do the same for files, choosing the "Upload [upload]" source field and selecting the file migration I used as the source migration. I also had to set the optional MigrateFile class to "MigrateFileFid" to ensure the new blog's file field referenced the appropriate file IDs.

Perhaps the best thing about Migrate is its ability to rollback migrations and re-import them to debug the process. I did this several times for the various migrations. I also made use of its ability to import a representative subset of the awaiting content while I was debugging the term / file association process. Clever module - Mike Ryan rocks.

The future for this blog is a consolidation of various other things I'm doing online, including blogging about life in Greenville, SC (hint: it rocks) and Roguelikes / indie games (again with the Wordpress?). Once that's done, I'll carry on with the blogging about Drupal, startups, and a few other things, including my recent explorations with R and Commerce Guys's Platform.

I hope to hear from you soon. Smile

Pages