Innovating within Drupal’s Core Competency

The dominant theme of DrupalCon Pittsburgh was innovation. It featured heavily in our Drupal Association Board retreat as we considered how the non-profit might better support and contribute to continuous innovation in Drupal. Project lead Dries Buytaert discussed the topic at length in his keynote presentation, which ended with the audience selection of proposals that would receive “Pitch-Burgh” innovation grants. It echoed in myriad discussions that generated buzz around initiatives like Single Directory Components that makes theming Drupal easier.

Furthermore, the Drupal Association Board ratified an Open Web Manifesto at our public meeting that situates our work within the broader context of the people and projects advancing an innovative, inclusive, open web. We want an open web to win against walled gardens, proprietary technologies, and platforms that force you to trade your privacy or intellectual property rights to reach a captured audience. By extension, I personally want the future of commerce to be one where merchants don’t have to trade their customers’ privacy to do business on eCommerce platforms like the notoriously data hungry Shopify.

As much as you can be, I’m on board. I’m revising my company’s vision and web copy to more closely align it with the Drupal Association’s Open Web Manifesto, and I encourage others to do the same. The more unified we are in presenting our vision for the future of the web the better! I’m also leading innovation discussions with my team. We don’t want our work on Drupal Commerce to be mere maintenance but the pursuit of victory over our proprietary competition.

Reflecting on discussions to date, I do want to challenge us to focus even more clearly on Drupal’s unique contributions to an open web and to inspire innovation within our core competency. In a world of limitless possibilities and very well-funded competition, it would be easy for us to spend all of our effort and energy “keeping up with the Joneses.” That would look like solely innovating at the edges. Being the first CMS to natively integrate ChatGPT for content creation or delivering an open source layout building experience that rivals that of our SaaS competitors are important initiatives. As we explain in our manifesto, “to achieve its full potential … the open web must prove it’s a better web.” However, innovations at the edge must be paired with improvements to our core, lest some other platform become better than Drupal at what we do best and replace us.

What is Drupal’s core competency?

That obviously begs the question, “What is Drupal’s core competency?” I don’t have a definitive answer, but I consider it to be something along the lines of “content modeling, organization, and distribution.” We are a content management system, after all, and in each of these three areas, we have distinct advantages. Where Drupal loses to proprietary competitors, it isn’t likely owing to limitations in our entity data model, our unparalleled Search API, our systems of access control or API based access to content. Where we lose to other open source solutions, our competitors are more often than not working to catch up to us in these areas, but they’ve done a better job appealing to end users in other areas.

We can survive lost opportunities to both proprietary and open source competitors so long as we aren’t losing due to deficiencies in our core competency. Indeed, we shouldn't be so much concerned with Drupal’s overall site count, maximized in the past through mass adoption by casual content creators, as we are with its success in more rigorous contexts demanding our unique strengths. So long as we’re the best at what we do best, we’ll always have a market and the opportunity to advance our values and vision for the web within dynamic, influential organizations.

Maintaining our unique advantages

What exactly will it take to maintain our unique advantages? I doubt anyone can say for sure, which is why Dries’s appeal at Pittsburgh for us to help a thousand flowers bloom is poignant. My observations are slanted toward Drupal Commerce use cases, where our content challenges relate to complex product data models, catalog and faceted search optimizations, and managing composite entities (e.g. orders with their order items and profiles or products with their variations and attributes). I do have a few “first draft” thoughts and invite others to contribute their own.

First, following Dries’s demonstration of the OpenAI integration module, I couldn’t help but wonder what it might look like to apply LLMs like ChatGPT to the unique challenges faced by ambitious site builders, our target audience, not just content creators. Instead of prompting ChatGPT to create content that still has to be fact-checked and edited, what if we applied LLMs to analyzing content libraries and business rulesets to propose Drupal optimized content architectures? It’s not uncommon for Centarro to spend weeks analyzing the business rules, existing datasets, and search requirements related to products to arrive at an appropriate product data architecture. If we could speed up this analysis and definition phase for every Drupal user, how much more productive would our community be?

I asked Jamie Abrahams to review this post prior to publication when we were chatting recently. Little did I know he and his team at FreelyGive were already working in this exact problem space. I’m excited to see what they’re developing in the Search API AI project. You can learn more about their work and vision from Jamie’s Drupal Dev Days session (cf. the session description and video recording).

I also reflected with Jonathan Sacksick from our team at Centarro on the needs of headless applications, especially in light of the success of Next Drupal and the fact that our largest Drupal Commerce sites incorporate at least some decoupled components if not fully decoupled front-ends. What if we made it even easier to model content for headless applications? So much of our configuration related to entities, bundles, and fields primarily or exclusively serves full-stack Drupal sites. I’m thinking of things like server-side defined optionality, allowed values, form widgets and display modes, etc. Many headless apps disregard most or all of this configuration, and at least some would prefer to fully validate input in the client. Is it time to re-approach document storage not just as a way to more efficiently store field data but even to change the way we model it in the first place?

I was encouraged to see Brad Jones’s pitch in this problem space get selected in Pittsburgh. He proposes to add JSON field schemas and querying in core, unlocking improvements for many projects within our ecosystem. This could lead not just to optimizations in the management and querying of complex fields (e.g. addresses, where we adopted the highly structured xNAL specification) but hopefully to the improved management of composite entity structures.

For an example of this initiative’s potential in Centarro’s domain, we considered serializing order items into the order table to improve data management and performance in Commerce Core, but we passed due to lack of support throughout Drupal for storing and utilizing entities in this fashion. There are many use cases where it would be advantageous and unique to Drupal to be able to manage entities only in the context of other entities - loading and altering, checking access, REST resource processing, etc. - while maintaining the ability to customize the schema and make full use of the entity throughout Drupal’s other systems (e.g. search indexes, Views, layouts, etc.).

Evaluating innovation opportunities

Bill Gates famously coined the phrase “Content is King” in his 1996 essay, and we might generalize that to “Information is King” today. All of the services, protocols, and other technologies that sit “beneath” Drupal in the open web stack (DNS, HTTP, SSL, HTML, etc.) exist to facilitate the communication of information, whether it’s human consumable multimedia content or the contextual data that explains the content’s meaning and impact.

Whether Drupal functions as a full-stack website provider or just feeds structured content to apps, search engines, LLMs, or technologies yet to be developed, being the best platform in the world to model, organize, and distribute content will ensure we remain relevant regardless of how the web evolves. Maintaining our relevance is not merely about preserving our jobs; it’s how we will create and preserve a future where an open web prioritizing our values facilitates “information, connection, commerce, and political participation … [for] everyone in the world, regardless of background, identity, ability, wealth, or status.”

Accordingly, with Dries’s help reviewing and organizing my thinking, I believe we should work to discover and evaluate opportunities for innovation based on their:

1. Contribution to our vision for an open web,
2. Relevance to the needs of ambitious site builders, and
3. Proximity to Drupal’s core competency, ensuring our long term defensibility.

I wish I had time to flesh this rubric out further or demonstrate its use by ranking the ideas and initiatives I mentioned above, others we funded through Pitch-Burgh, or novel ideas of my own. Absent the time and inspiration, though, I’m leaving it here for now and am curious to see if it resonates with anyone else.

If you’re already dreaming up or working on innovations that satisfy these criteria, I’d love to hear from you and amplify your work!