elgentos commerce & configurator agency

13 min read

A personal message from Jeroen Boersma and why my name is on the Open Letter.

TLDR

Why a fork could be the future. It would be awesome if it could go hand-in-hand with Magento. Some insights into my years working with the system.

Introduction

After years of working with Magento, a part of the community finally voiced a clear signal. Things need to change and people and companies need to wake-up! We all have great ideas, but it feels like we are swimming upstream. Currently there is a pile of work, in the form of Open Issues (I don't even know where to start), unmerged Merge Requests (some that I created myself) and stale branches. There is no clear perspective on what the future will bring (also known as a roadmap) or what to expect as an agency to do some proper planning.

A lot already has been said. I know for sure that Adobe is awake and paying attention. For the first time I have the feeling that I need to put some words down. It'll be a long read. I think it is that important! A trigger for me was the latest panel discussion.

Panel discussion

Last Monday (September 20th, 2021) there was a panel discussion at Meet Magento Poland (great to see they made room for the voice of the community) which I think made clear what the open letter is about! Community members from all over the world booked tickets to be there in person and to support that statement. I think it was a pity to see only two members from the Magento Association there in person. However, they came empty-handed. What I hoped for was that they reached out and said, "Hear, hear our beloved community, we heard you, we spoke to our stakeholders, we have done our stinking best and have set up the following schedule to get everything on track". Someone stated that Adobe couldn't be there because of COVID restrictions. Although I understand that, I'm flabbergasted that while in 2021, with people working from home, which are able to have online video calls and chats, Adobe has issues with doing a simple video call. I think this was a missed opportunity from Adobe to get in touch with their community (update: it was by choice of the organization to not do a mixed panel) It was great to see someone from an existing community (Typo3 <3) drop in to share their thoughts and give real and valuable input. This is also something that we shared already with the Magento Association.

About me

For years I've been working in the Magento ecosystem, the wonderful community, where I feel at home. I'm so happy about all those people I met over years and more in the years to come. I started off as early as Magento 1.3 (played around with 1.0, killed some servers doing so). Started giving back and contributing through visiting Hackathons, meetups, Meet Magento events, sleepovers, (un-)conferences and starting fresh chats with those lovely people around me. When Magento 2.0 was released it started a whole new era (of problems) with better technology and a better stack (or so they said). As a company, we already had some existing experience with the IT industry and made a wise decision to not jump the Magento 2.0 train yet for commercial implementations.

I still know a Hackathon in Groningen (It'll be fun they said) and I was able to see in real life the whole bunch of experienced Magento developers needing a whole day to just install a fresh Magento 2 development version. After that first hands-on experience we upgraded all our hardware for all developers to make working with Magento 2 feasible. We started creating our first MRs, creating issues and having fun!

As much as I loved hackathons, I always felt a little uncomfortable attending Magento Contribution days, it felt like just doing normal work, processing issues, but in our spare time. For me, hackathons were kickstarters for great new ideas and added value to the existing product. Contribution Days weren't focused on that.

Current state from a technical point of view

I want to sum up a few things which have been bothering me for years, for most of us things we have accepted, but are not actually normal. As everything needs to be on the table, this also needs to be addressed.

Slow release cycle

Magento 2 chose to maintain one Git repository to rule them all, so all (important) stuff is in there. I guess this is the reason that my name is on the letter. It is also something I already addressed several times in in-person meetings but so far, no clear answer was given as to why. It is the cause why we have planned (big bang) releases, use patches (why??), have bundled unused crapware and errors which means we have big replaces sections in our composer.json files. All in all, this has brought a whole new level of security risks because of merchants choosing not to upgrade even if this means that they run vulnerable software.

Patches

Magento 2 uses package managers for distributing code. The Composer implementation for Magento via their repo.magento.com built into shippable packages from the Monolith is just weird to me. The whole idea behind Composer is that you can quickly apply semeantic versioning and release patches and/or minor/major releases to your project without you thinking about it.

However, Magento chose their own approach. Everything needs to be released and signed off and scripts need to run to update the private repository. A two-line bug fix needs to go through the whole release pipeline where everything is tagged and released at once and this can quickly take up to three months before being available upstream. People got fed up with this and composer patches specifically tailored for Magento were born in return. Magento released "quality patches" to apply to your project before the next big bang release. Currently we also have half of the packages served via packagist.org and also available in repo.magento.com, which can and may differ or be outdated, causing conflicts to occur.

The good thing about the curated repository is that they can run a marketplace, where you can buy/sell stuff quickly to merchants via a token based authentication. IMHO everything else could just be served via existing channels if you ask me.

Version numbers

So you tell me you're running Magento 2.4.1-p1. Yes, that is the commercial version convention. Internally it starts at 100 + the minor of the commercial release number. AFAIC it makes sense knowing that if applying semver, 2.3 to 2.4 is a major version bump is, so technically, 103.* to 104.*. So a package running version 102.0.1-p1 should line up with 2.2.0-p1. I found examples where this didn't match, so it's technical debt.

Replaces

Magento ships a whole lot of awesomeness by default, however there are also paying vendors (which could be fine) we don't need in our installations. There is no other solution than adding all these packages to the replaces section in composer.json. You tell composer, "I handle this myself". In this case it creates a black hole. So in the next release it becomes problematic if new code is released and depends on this black-holed code, the build process breaks. Another reason to put packages in the replace section is because not all packages need to go to production.

Fixing the release cycle

When Magento at first announced the "headless" future for Magento 2 I was very happy - this would be an opportunity to break the current Git repository into smaller packages, deprecate and decouple a lot, which then could be moved and maintained by the community. We were talking about this three years ago and currently this idea is on hold. We need to break the big Git repository down into smaller packages, making the release cycle a lot smoother because we can just apply regular semver. No more scripting involved, no more patching. Let people run RenovateBot or dependabot and update that one package as soon as it is available, like most community packages. The Magento Multi Source Inventory modules are an example of this effort.

These fixes should also be done if a fork would happen to remove a lot of technical debt. We can run a build script to fill the private repository, why not create or update all individual packages once in new Git repositories, create a reference from the root projects composer.json and everything will keep working as expected.

We gain from this that we can still have commercial releases on 2.4.4(-p1) on the metapackage but also can update a patch for a package invoking composer update {PACKAGE} because packages are tagged individually. No more patching required! Root package would still be able to run all tests like before.

Fixing the replaces section

If the repository is fixed, new meta packages can be created, where we finally can have a Magento bare|minimal|default, without packages like the rest API, or GraphQL, or a backend. or a CMS. Upgrades would work because all classes can be found and dependencies are correct, also patches could always be applied, because there aren't any left.

The Community

In the last couple of years I have seen people whom I consider friends leaving the Magento community in favor of other platforms or other work. Some even went through time of uncertainty in terms of work and money. Some even needed to find a whole new way of making a living. I think that people leaving for something else is more scary than people standing up writing an open letter because the latter still care before walking away!

We are not only talking about individuals. If your company makes a decision to move elsewhere and your services and knowledge is no longer needed, you can feel left alone or lost. These voices are being heard and ignored for a long time. This has nothing to do with a fork. If a fork would make people feel more involved again, that would directly feed into the success of the community, not for Magento, an Adobe product. This is why we reach out to you, Adobe, I think we got your attention, now it's time to act.

It worries me hearing voices in favor of leaving (again; reasons stated over and over); "not feeling heard", "development pace is slow", "I'm fed up waiting for my MR to be merged", "today I'll be adding my 23rd patch".

People I trust working with Magento for years leave Adobe and all I read is a brief Twitter message or LinkedIn update. It breaks my trust in the product and it raises questions about the future. These people made me feel included, where I could reach out too if I had a question. One tip I would make to Adobe is to make sure that your community is well-informed.

The Open Letter

So why did I sign the open letter to the Community? I just knew it was the right and only thing to do. On behalf of the whole community, it's when you want to act now. The way it is evolving, having it set up correctly and doing this the right way! Seeing people care about this too, as of moment of writing, more than 1400 additional people signed the letter. It confirms my first thought signing the letter.

Upstream compatible

We want to create an upstream compatible fork. Here's a quote from the letter:

The fork will be upstream-compatible with Magento Open Source as long as it is supported by Adobe. That means, when the monolith is ultimately deprecated, all companies who want to remain on the monolith platform will be able to do so.

New features in a fork can also be fed back into the main repo, just delivered faster. This could mean that the fork will evolve more quickly as the main repository, a change I am waiting for for years but where everyone profits.

Our company

We do Magento development and we will do so in years to come. I wrote this with my personal view on things, also how I see that we as a company will benefit from this. We cannot run this alone, we need to do this together, that's why we as a company signed, and also want to invest in this community, with organizing events, hackathons, meetups or even making a fork.

Wrap up

I hope that writing this gives more insight into why I think that there needs to be a change.It's up to you to form your own opinions. If you are with me and haven't signed yet, please do so.

Thanks for bearing with me to the end,

Jeroen

What others say

Finally I want to share all good reads from last couple of weeks. I think all people are awesome and I sorted the list alphabetically.

Social interactions:

Updates

Adobe couldn't be there because of travel restrictions, MMPL decided not to do a mixed panel, updated section.