Server Team Report for 2017
January 23, 2018
With this report we want to inform you about the achievements of the TYPO3 server team in 2017.
Team Structure and Communication
The TYPO3 Server Admin Team currently consists of six active members:
- Andreas Beutel (mehrwert)
- Andri Steiner (snowflake Ops)
- Bastian Bringenberg (pottkinder)
- Michael Stucki (team leader, michaelstucki.ch + snowflake Ops)
- Steffen Gebert (EMnify)
- Stephan Großberndt (side by site)
Stephan joined the team just recently in December and we are happy to have such a dedicated community member now with us.
The team is reachable in the public Slack channel #typo3-server-team and holds bi-weekly calls, currently scheduled on Tuesdays at 17:00 CET. The meeting notes are publicly available and interested folks are invited to join our meetings. In general we are doing our best to have an open and transparent communication style.
The team plus some contributors met four times for sprints during the last year, once per quarter:
Those sprints are intensive and exhaustive, but they give us the chance to work together on bigger projects that we like to move forward. After all, we see those sprints as an advanced training for each one of us - and we always hope to meet people from outside the team at one of these events. If you are interested to join one of our next sprints, please get in touch with us so we can include you for finding more dates and locations...
Projects and Progress
A large share of our available time is - unfortunately - spent on invisible changes and usual operations toil. The major topics that were tackled by our team and its contributors in 2017 include:
Moving more servers away from OpenVZ to KVM
We have moved multiple servers away from the old OpenVZ platform onto a relatively new KVM/libvirt based environment. The process took much longer than we expected and still is not finished. It's disappointing but true that we could not finish this project even though budget was granted for it.
Improving and modernizing the web hosting stack
Until last year, our own web hosting stack was managed using a bunch of scripts. This was not only difficult to manage, it also led to problems with scalability, performance and, of course, the availability of latest PHP versions.
Finally, our team member Andri came up suggesting that we integrate the publically available web hosting stack of snowflake Ops into our environment. The main problem here was that the snowflake Ops stack is based on Puppet while the TYPO3 server infrastructure is managed through Chef. It took one sprint and some more evenings to integrate the two tools within each other. The result is the site-webhosttypo3org cookbook which allows us to manage all of our websites with a single Git repository.
Upgrading Redmine from the _very_ old 1.4 to version 3.3
It was one of our longest lasting tasks to move our forge.typo3.org website away from Redmine 1.4 to the (back then) recent version 3.3.
Additionally, we also replaced a custom single-signon implementation with a standardized LDAP interface.
Upgrading Gerrit from the not-so-old 2.12 to 2.14
It's just a small jump when you look at the numbers, but again it took us a lot of time to upgrade from Gerrit 2.12 to version 2.14. This included a major cleanup of content in the Gerrit database for extensions which have been abandoned or are no longer maintained in Gerrit.
Once again, we replaced the custom single-signon implementation with LDAP access. During that process, we also cleaned up some tens of thousands of users that never logged in, but were synchronized by the old authentication system.
Supporting the launch of the new TER site
The new TYPO3 Extension Repository (TER) was launched in August 2017 by the great t3o team which is lead by Thomas Löffler. Our team spent a lot of time and efforts to complete this launch and support the migration of the content.
Hosting improvements and call for offers
In Summer 2017 we started a call for offers for our hosting infrastructure. Our goal was to find a new partner for the next few years, if possible operated on a cloud hosting platform.
Unfortunately, we stopped the project without deciding for a new offer because paying per VM seemed to be a lot more expensive than what we expected (and what we pay today using bare metal hardware). It will be a major task for us in 2018 to discuss the worthiness and the advantages of a cloud based infrastructure together with the TYPO3 Association. Within the team, we have already started these discussions about our current hosting and ideas to improve it.
The last bullet point brings us to the topic. We migrated to KVM and now (finally) have most services running in the same data center. Yet, we're still not convinced. The team is currently investigating hosted Kubernetes solutions from public cloud providers and having a trial to move some of our "legacy" applications to it. We hope, this again lowers our burden to fix low-level infrastructure components (hard disk replacements, firewalls, DNS servers etc).