TYPO3 CMS Active Contributors Meeting, 2014
April 03, 2014
The TYPO3 CMS team just came back from the Active Contributors Meeting ("ACME" for short). This is an annual and very important gathering of all actively involved developers of the TYPO3 CMS project.
We met for three days in Nuremberg/Germany, just like we did last year in May for the kick-off of the TYPO3 6.2 release development. The location was organized by Thomas Maroschik, the meeting was organized by the TYPO3 CMS team, represented by Oliver Hader, Ernesto Baschny, Helmut Hummel and Christian Kuhn. Benjamin Mack, Steffen Ritter and Xavier Perseguers weren't able to attend. We also have as guests Matthias Schreiber representing the "non developers" and Martin Wiederkehr representing the EAB (Experts Advisory Board). Later on we also had the visit of Ben van 't Ende - our Community Manager.
The focus of this gathering is not "coding", but deciding upon organizational and technological matters related to our product. It is also a good opportunity to get everybody in sync again for future development.
Feelings and Expectations
Olly started gathering the feelings and the expectations from the group. Overall the most mentioned feeling was "happy" tightly coupled with "tired" mainly due to being an exhausting phase shortly before the just released TYPO3 CMS 6.2.0.
As for the expectations, most of the group expected us come up or at least grasp a common vision and a roadmap for the future development of TYPO3 CMS. Some want to work on the team spirit. This first round was a good opportunity to share the fears and desires of the group.
Short term decisions
Some decisions needed to be tackled soon, and this were the first discussion we had:
We quickly agreed that it is still too early to open up a new "6.2" branch. We decided to keep the current master as our only "6.2" stabilization branch at least until the Code Sprint in Bremen in the beginning of May.
Travis Bot Voting
We figured that Travis could add votes to our reviewing system automatically, as it is being tested by the Flow team (see an example here). The discussion was about potential performance issue, Travis' queue and it's delay (which is especially noticeable during code sprints with lots of merging activity). About the bot voting, we decided that if we are going to implement this feature, Travis should only issue down-votes ("-1 Verify") just like Jenkins currently does and not up-voting reviews (thus, not really replacing the review process by human beings). Christian and Helmut will check with the Neos team and work on this topic, if it turns out to be doable.
We drifted a bit into the discussion about Jenkins and Travis and if it's not worth building our own "Travis infrastructure" around Jenkins (thus performing our unit and functional tests on our own machines). The reasons not to do so, are that the costs of maintenance of such a system - which we currently get "for free" from Travis and is doing such an amazing job already. We could think about moving the CGL testing and PHP linting from Jenkins also to Travis to have one central tool for all tests.
Christian also explained that there is still work to be done on the Travis test execution platform, because currently it times out sometimes and is not as performant as we need it.
The Friendly Ghosts project was kick started some years ago, but was ever difficult to maintain. Jigal explained that he tried hard to work on that, but lack of active participation made it drift away. We recognized in the group that this kind of activity is something that cannot be planned in a "calendar", but needs to be embraced by the whole team and has some very individual components.
We came to the conclusion that we should drop the general concept of dedicated friendly ghosts but instead focus on the "fields of responsibility" of each individual contributor. To aid keeping the issue tracker clean, we had the following suggestions:
- Closing bugs shall happen automatically after 90 days if there is not feedback and the status is on "needs feedback". Here is a text suggestion for this auto-closing. Olly will contact the Server team on implementing this change on Forge.
- Automatically asking for feedback before closing on bugs which are reported for versions that are not supported anymore.
- Tracker categories need to be updated and cleaned up.
- Fields of responsibilities ("competences") needs to be updated and more public and should be mapped to forge categories for easier association.
The idea is not new, but we wanted to finally try it out. The extension repository contains a huge amount of extensions. It is difficult to understand which ones adhere to some common development rules. The past idea was to classify the most recommended ones as "A-Class" extensions. This idea was difficult to realize because it would require an actual reviewing process and a team working on such a review system. It is already very difficult to tackle the security issue parts of the extensions.
Thus, the new concept switches the responsibilities: the community sets up the rules and the extension authors decide to stick to them and give their own extensions the "A-Badge". The community acts as an auditing instance and reports on abuse of the badge. The exact details will be formulated and discussed during this next days by Xavier, Jigal and Philipp - representing the Extension Coordination Team (ECT).
TYPO3 6.2 retrospective
Ernesto then made a summary of what happened during the 6.2 development phase, recapitulating the multiple events, code sprints and achievements. We compared the goals which were set during kick off in the last years ACME to the accomplishments.
The team then collected a list of good and bad sides of the past development phase. The topics were sticked on a board - pink for bad, green for good - which we then clustered in major groups. Many individual feature achievements were listed as good aspects. Other positive topics were the concept of the Workpackages, the work of the release management and our team culture. On the bad side many listed problems with quality control (rushed in merges, too many regressions during development and follow-ups, breaking tests, etc). We also identified as problems the lack of UX improvements and little to no involvement of customers and agencies in the development process. The release cycle with the pressure added by the 3-year LTS and the lack of adoption interest for the releases "in between" (TYPO3 CMS versions 4.6, 4.7, 6.0 and 6.1) were also on the negative side.
We discussed about ideas on how to improve on these matters, and especially reminded ourselves on how it "used to be" in our projects community.
- We need to improve connection with the community
- We need to communicate more in general, especially to non developers
- We got lazy writing meaningful documentation, articles: need to improve on that
- We need more nice communication about features (podcasts, screencasts)
The future of TYPO3 CMS
After this very long discussion we started the next discussion round about the product TYPO3 CMS in general, our own motivation and the market for it. We talked about it in smaller groups, asking ourselves some fundamental questions about how each one of us see TYPO3 CMS in three years.
We came up with pretty strong reasons to believe that TYPO3 CMS will still be around and kicking in at least the next three years.
Why should customers build on TYPO3 CMS then (in three years)? And what makes TYPO3 CMS unique at that time?
Maturity, broad eco-system with a large base functionality, great interoperability, reliable, integrators can solve almost 90% of the tasks without writing custom applications (just by plain core functionality, i.e. awesome flexible TCA), great backwards compatibility and upgrade paths.
A huge and great user base, huge community, spirit and huge amount of events. TYPO3 CMS is a real community driven project, there's no company steering the project. There are already tons of books about the system and any trained integrator can work on any TYPO3 installation.
Lots of satisfied customers which will still be able to satisfy their needs with TYPO3 CMS by then. The product will still fulfill a multitude of users requirements. Customers with large amount of content and custom applications still enjoy TYPO3 CMS and benefit from the structural design (backend/frontend, pagetree, etc), concepts which we should not drop. TYPO3 Neos still needs time to evolve and TYPO3 CMS is established in the market.
"We just need a mobile ready UI and everything is fine".
Why should we as developers contribute to TYPO3 CMS then?
Most have strong reasons to believe that they will still use the product and even earn money with it, so they continue contributing to it. The knowledge is in our heads and doesn't need to be thrown away. There is enough trust and commitment inside the community to solve even complex task with TYPO3 CMS.
Some developers currently lack a common vision or roadmap to decide whether to continue contributing to it. The motivation also might depend on whether TYPO3 Neos is a competitor that solves the demands just like current TYPO3 CMS does for them. This also boils down to the question of whether TYPO3 CMS can co-exist besides TYPO3 Neos. Some questioned if TYPO3 CMS shall concentrate on a more specific market (to shape the products to different target markets). There was some consensus that we might not be able to solve this "product shaping" in this developer centric team, but would require a more specialized group (including people from marketing, CMS and Neos teams).
There is still a lot of enhancements possible in TYPO3 CMS: we identified lots of drawbacks already and there is enough motivation and man-power to get those tackled.
Do we see TYPO3 CMS phasing out then?
Not in the next three years. By then we can ask the same question again.
The second day started with a flash-light review on the feelings of every participant about the previous day. Some had some moments of confusion and reflection, most managed to turn that into excitement and joy after understanding that we somehow were again on the same track after all.
Summary of the Communication Workshop
Mattes presented a very short overview of lessons learned from the OSS Watch in the communication workshops at the marketing sprint in Altleiningen in November 2013.
The value of communication, the channels, the potential problems, how to deal with trolls, how to use the communication sandwich technique. In general most communication issues can be solved with self-reflection of own actions, including stepping down, being humble, agreeing to disagree in the end (which is quite natural for individuals) or to just accept other opinions. It's quite natural that community includes diversity which should not only be accepted, but encouraged in general.
This small overview will also be part of workshops which will be held during the TYPO3 Developer Days 2014, presented by the OSS Watch guys. Worth visiting, for sure!
User Experience Week 2014 (UXW)
We were in contact with Jens Hoffmann who is still willing to organize this event during this year. The real "kick off" for the organization will start in May and we expect the event to take place around November. Felix Kopp and Ernesto offered their help. Expect more news about this starting next month.
Thomas Maroschik then presented the concept of rolling release, as a way of tackling some of the problems we identified in the past. The idea is not new and already being applied in several open source software systems.
A release cycle consists of the phases "development", "release" and "feedback". The feedback round is the most crucial and currently it comes way too late. In order to achieve a shorter release cycle, the testing, deployment, updating, migration and eventual rollback have to be fully automated.
To achieve shorter release cycles, the core needs to have upgrade and downgrade (rollback) automatisms. The processes have to be changed from former implicitly defined (manual steps) to explicitly defined (automatic steps). This could also lead to a standardization of this workflows across the users/agencies. These could also contribute back and improve the processes for all stakeholders again.
The most interesting and possibly problematic parts are custom applications and extensions - thus API versions or legacy layers need to be in place.
Helmut points out that in the end, the requirements for this kind of rolling releases are goals that we set for our product development anyway: complete documentation about what has been changed, and complete automated upgrade and rollback services. The testing coverage needs to grow and we need more automation around the whole testing scenario. Our master branch should never be broken.
Future TYPO3 CMS Release Cycles
The discussion lead to a very fruitful discussion about the release cycle for the future TYPO3 CMS in general. Ernesto presented an idea which would also fit with the concept of rolling releases, but also appeal to the agencies demanding LTS versions.
We agreed upon several criterias, played around with the ideas and possible solutions. The team reached a very good coverage of tackling several problems we have identified in the first day with the change. We decided that a Blueprint will be written on this concept by a team (Mattes, Christian, Helmut, Tom, Ernesto), presented to the community and then agreed upon. The team is already pretty confident that it will be very well received by all stakeholders.
Future of CMS
After lunch we split up in five (randomly chosen) groups to separately talk about certain aspects of the product, to try to shape some topic specific visions. The participants then presented their results:
Group: Georg, Olly, Nicole and Felix Oertel. Vision: "Improve workflow for every-day users"
First and most important aspect to be considered was to get more feedback and input from end-users and editors. Olly and Nicole mentioned that they could gather some information within their customer base. A broader approach would be required, but this needs to be tackled separately.
- More user friendly backend design
- Better usability (streamline, identify and remove UX gremlins, etc)
- New or improved features, like:
- Improved search/auto-suggest
- Improved clipboard (Drag and drop)
- Frontend Editing
- Backend Page module (toolbar, localization handling, functionality like in GridElements / MultiColumn, visualization of other records)
- Task center module (more role-oriented, required tasks, interaction with a „dash board“)
- Versioning for file contents (binary data)
- Consistent rendering and functionality for trees (Element browser, category tree, file tree, page tree)
Content & Structures
Group: Stefan Neufeind, Felix Kopp, Martin, Stefano. Vision: "TYPO3 is a data management system"
Focus and improve the "record management", i.e. List Module and use this as a flexible framework with more features:
- configurable and customizable
- allowing mass manipulation
- predefined filters
- facettes & types
- needs to be able to handle huge amount of records
This kind of framework could be used already in the core for multiple situations, i.e. syslog viewer, beuser admin, scheduler task list, and many more. Extensions or integrators building new "list of records" would be very easy then.
On this area of interest, Fabien Udriot showed the team the current status of the Vidi Extension, which provides some of these features already.
Compatibility & "Rolling Releases"
Group: Anja, Alex, Fabien, Ernesto.
Basically the group discussed the topic, which has already been briefly discussed on Monday, in more detail and identified the main tasks to be tackled to achieve the goal:
- Communication: very easy to grasp explanation about the past, present, future. Considering target groups, benefits, risks and side effects.
- Technical aspects: increase test framework coverage, add pre-merge checks (running tests before the actual merge). Mid-term: add some layer of behaviour tests (behat) and calculate metrics around "risk of changes" / identifying hot spots in the code to automatically evaluate risks.
- Infrastructure: provide an upgrade-script and/or a scheduler task to perform the upgrade, execute migrations, test the involved installation, report on problems, roll back if necessary.
- Extension handling: TER should support branches / composer to allow extensions supporting this rolling release as a natural process. Extension Manager needs to be adapted to filter available extensions and deal with the snapshot releases
Group: Thomas, Marc-Bastian, Mattes, Christian. Vision: "Be a good web application framework for your web application"
Important topics in this area include:
- improve scalability
- being "ready for the cloud"
- potentially use a CQRS approach
- detach from the current SQL storage
- support alternative PHP runtimes
- being able to provide a really good mobile UX
Community, Teams and Communication
Group: Philip, Ben, Helmut, Patrick. This group identified the strengths, weaknesses and future steps to be taken:
- Strengths: social skills, enormous knowledge inside the community, having the ability to discuss thing on a technical level (i.e Gerrit)
- Weaknesses: language barriers, communication between teams could be improved, communication style within the teams sometimes can get rough, communication to the outside could improve
- Future tasks:
- Create more awareness of communication issues to foster changes (i.e. workshops during events).
- Use less rules, but more "guidelines" and be open for any input
- Remind ourselves that all of our actions should be inclusive
- Use or improve the code of conduct.
- Improve communication between teams which need to work closed together
- Maybe have a community contact for TYPO3 CMS (like Christian Müller does for TYPO3 Flow/Neos)
- Having a TYPO3 Planet (blogging platform) to collect contents from the community and present that knowledge to others
- Work on a "Values" for the CMS team, or adapt the ones from the Neos team.
Stefano was working for his bachelor thesis on integrating the Doctrine DBAL into TYPO3 as a replacement for our own "old school" DBAL implementation. Stefano basically finshed the practical part and is currently writing the thesis - which will be done by end of May 2014 and will be available for the public then.
During the presentation the topic of general Doctrine integration and general integration of TYPO3 Flow popped up and has been shifted to some other date for further discussion.
Fabien shows us the current state of the Media extension and also the generic Vidi record visualization, searching and filtering. He explained how the extension was created and that most new feature are funded by requirements from his clients. More help with manpower and even sponsoring is always very welcome.
Coding and Discussion
Second day ended with pizza, nice talks, some beer, coding and reviewing.
Security Team and Workflow
Helmut explained the workflow of the security team. The team already has the help of several active contributors to help fixing, reviewing and discussing open issues. Helmut invited more active contributors to help on these tasks. Stefan Neufeind and Thomas volunteered and will be invited to the internal areas where the team works.
Christian started a round of discussion about the team spirit, the different moods, reasons for frustration, modes of our working. We reflected and discussed several reasons why there are frictions within the group and by doing that started working on possible solutions.
It was a good, sincere and open talk involving most participants. We settled for rising trust in the form of "groups of interests". Another aspect was also to join these groups of interest in case of doubts or questions. Value each other's contribution and be ready to mentor new contributors in a particular area of expertise. Lower the pressure in general and discuss more often on non-technical matters. Document the plans and visions of each area of interest in order to keep everybody on the same track.
To finalize, we collected topics of interest that we aim to tackle during the next six months. Everybody wrote four main concerns. From the topics we chose four main "must have" topics which were thus not up to voting:
- Marketing / Communication / Roadmap
- Clean up Forge & solve / review known Issues
- Technical Documentation
The remaining concerns were clustered around major groups again. Note that this is just a huge list that came up on the mind of every individual participant, so this was just to get an idea of the main topics of interest:
Usability, User Experience, User Interface
- Page tree
- Page model
- Structured content
- Task oriented user interface
- Record editing and manipulation ("data management")
- Drag and drop clipboard
- Drag and drop to create content elements
- Frontend editing
- Backend for mobile devices
- Video content element
- New backend skin ("flat design")
- Improve "end user happiness"
- Language handling in the backend
- Composer support
- Improve UX
- Add features
Extension Developer / Lower Entry Level
- Closed User Area (Registration, User Profile, User Registration, Forgot Password, ...)
- Strong & Simple Configuration, replace CSC
- Create Content Elements without creating an extension
- Easier Modelling
- Doctrine DBAL Integration ("Enterprise" label to TYPO3 CMS)
- Extbase / TYPO3 Flow
- Flow integration, integrate more Flow code into Extbase
- More intelligent Extbase Persistence
- Web Services: RESTfull API for Core API
- Get Logging Done, use Logging API
- Decoupling of Classes
- Shell API
- Core Bootstrap Cleanups
- Prepare for the rolling release
- Integration Framework / Travis
- Test Environment improvement
- better coverage (i.e. Extbase)
- Functional Tests
Topics of Interest
The voting turned out that the most important aspects were:
- An "admin topic": Extension Management / Composer
- A "user topic": User Interface / Usability
- A "developer topic": Architecture
Olly put it well in his ending words: "We are relieved of the outcome, but not relaxed. There are many tasks to be done.".
The next events for the TYPO3 CMS team is the Code Sprint in Bremen from 2nd until 4th May, 2014. Then the Code Sprint in Zürich from 10th until 12th June followed by the TYPO3 Developer Days where plenty of us will be present and most probably also gather for some kind of meeting.
We would like to thank (again) the sponsors of last year's ACME (2013), which we didn't forget:
- meeting place sponsored by Nürnberger Initiative für die Kommunikationswirtschaft
- dinner is sponsored by jweiland.net
- another dinner is sponsored by dkd Internet Service
- snacks during the meetings are sponsored by Patrick Broens
- beer and beverages is sponsored by lufed IT
- milk and coffee is sponsored by DFAU
For this year's event (2014), our thanks go to:
- dinner sponsored by dkd Internet Service, wmdb and Anja Leichsenring
- pizza sponsored by in2code
- beer, coffee and milk by Thomas Maroschik
It is still possible to sponsor our meals or other minor expenses. Contact me.