Professional Content Management

Free and open source, TYPO3 CMS is the most widely used enterprise-level CMS.

Test TYPO3 now:

TYPO3 live demo

Inspire people to share

Offer your skills and contribute to the project. The community is growing and does more than just coding. 

A Community Effort

TYPO3 CMS is an Open Source project managed by the TYPO3 Association.

The Project

Do you have a question?

Ask the community or a professional partner.

TYPO3 QA Code Sprint Linz 2012

Categories: Development Created by Andy Grunwald
From September 20 - 23 the last sprint in this series took place in Linz, Austria. The main topic of this sprint was Quality Assurance (QA) and Continuous Integration (CI).
In June Helmut Hummel, release manager of the upcoming TYPO3 version 6.0, announced <link http: typo3.org news article code-sprints-for-typo3-60>five codesprints with five different topics. The sprint was a success and a big step in the direction of further improving the quality of the TYPO3 core.

Participants

  • Philipp Gampe
  • Andy Grunwald
  • Helmut Hummel
  • Markus Klein
  • Stefano Kowalke
  • Christian Kuhn
  • Susanne Moog
  • Georg Ringer

Topics

Quality Assurance and Continuous Integration are hot topics. Overall goals of the QA sprint were to raise the quality of the TYPO3 code, to stabilize, to extend the QA / CI infrastructure and to improve our automated Quality Assurance workflow. Furthermore the participants and the community collected goals and ideas at the <link http: wiki.typo3.org>wiki page.

Goals and results

New Git commit message hook for review.typo3.org

If you want to contribute to the TYPO3 core, you have to create a patch and your commit message has to follow special formatting rules. One of these rules is the limit of chars per line (74 character), another one specifies the categorisation (e.g. BUGFIX, FEATURE, TASK, ...). This check is executed by <link https: ci.typo3.org>ci.typo3.org after you push a change to our review system Gerrit. We extended the existing commit-message hook at the client side. The hook will now check for maximum line-length, commit-type annotation, Release-line and Resolves-line and will issue a warning if one of these rules is violated. We think it is more convenient if git reports violations during the commit before pushing it to Gerrit. One advantage of this change is that you no longer have to wait for Mr. Jenkin's (negative) vote, due to, for example, an exceeded line-length. To activate the new commit hook, you have to re-download the hook. See the wiki page "<link http: wiki.typo3.org contribution_walkthrough_with_commandline>Contribution Walkthrough with CommandLine" for further instructions. Status: 95% TODO:
  • Move this new hook to an official location and integrate this in Gerrit like the old one. The QA team takes care of this with the help of the Server team (<link http: forge.typo3.org issues>Forge #41457)<link http: forge.typo3.org issues>.
Information:
  • Wiki: <link http: wiki.typo3.org contribution_walkthrough_with_commandline>Contribution Walkthrough with CommandLine
  • Wiki: <link http: wiki.typo3.org>Commit Message
  • <link https: ci.typo3.org>Jenkins-Server
  • <link http: wiki.typo3.org wiki images commit-msg.txt>Sourcecode of Commit Hook
  • <link http: forge.typo3.org issues>Feature #41457: Integrate new commit hook into Gerrit

Introduction into the “TYPO3 => TravisCI” process 

<link https: travis-ci.org>TravisCI is an open source service that creates an easy way to execute different tests for your project. For every test a virtual machine is fired up. It is possible to initiate different databases, different PHP versions and so on. After these tests your virtual machine will be removed. This service can only be used with source code hosted on <link https: github.com>github. TYPO3 is using this service since T3DD12 in Munich. Thanks to <link https: twitter.com ctrabold>Christian Trabold who made this happen for TYPO3. Currently TYPO3 is not hosted on github, but every change which merge into the core will be checked by this service. This process is quite complex. Helmut Hummel and Christian Kuhn gave the team a introduction how the process is currently working. The knowledge about this process is important to improve this even further. One long term goal is to improve this process, to enable all available unit tests, to test all supported database systems and other subsystems. Status: 100% Information:
  • <link https: travis-ci.org typo3 typo3v4-core>TYPO3 at TravisCI
  • <link https: github.com typo3 typo3v4-core>TYPO3 at Github
  • <link https: github.com typo3-ci typo3-travis-integration>TYPO3-Travis-Integration

Running unit tests for every patch submission by TravisCI

Unit tests are very useful to test a single functionality. At the moment the TYPO3 core contains 4390 unit tests which are executed for every merge into the main repository. If one test fails the developer will be immediately be notified by email informing that one or more functionalities are currently broken and need fixing. These tests are executed by an external service called TravisCI. Unit tests are useful when executed every time a change is applied. At the moment the tests are executed by TravisCI AFTER a change is merged. This makes sense, but thinking about the process, the developer needs to know whether the current change will break existing functionality. As mentioned above TravisCI can only execute the tests if the source code is hosted at Github. The same goes for TravisCI supports <link http: about.travis-ci.org blog announcing-pull-request-support>Pull <link http: about.travis-ci.org blog pull-request-testing-for-everyone>request, too. During the sprint the team created a concept and developed some small tools which introduce patch set support into our Gerrit review system. After a developer pushed a new patch set into Gerrit, TravisCI will check this patch against the current core. This opens up the possibility to check if a new patch breaks functionality BEFORE merging. This sounds great, right? Yes it is! Status: 75% TODO: 
  • More testing and finetuning
  • Get it live with the support of the server team
Information:
  • <link http: about.travis-ci.org blog announcing-pull-request-support>TravisCI Pull Request
  • <link http: about.travis-ci.org blog pull-request-testing-for-everyone>TravisCI Pull Request activation
  • <link https: review.typo3.org c>First Change which got tested by <link https: github.com typo3 typo3v4-core pull>Travis

Release a new version of TYPO3 Codesniffer standard for PHP_CodeSniffer

<link http: pear.php.net package php_codesniffer>PHP_CodeSniffer is a tool to check existing source code against defined syntax formatting rules. Those rules are mainly defined in our coding guidelines. The TYPO3 codesniffer standard aims to build such a ruleset for the TYPO3 coding guidelines. The last release of the TYPO3 ruleset was over a year now. During this time we fixed a lot of bugs, include new sniffs and at the TYPO3 Developer Days 2012 in Munich we kickstarted a new standard for FLOW3. One goal at the Code Sprint was to release a new version of the standards. To ease the release process for pear packages we wrote a wrapper class for PEAR_PackageFileManager2 which can be called from command line. By using this we released version 0.0.4 of TYPO3v4 standard and 0.0.1 for FLOW3 standard on saturday. Due to a misconfiguration the pear packages did not install in the correct path by updating through pear. After fixing this issue, we released version 0.0.5/0.0.2. Have a look at <link http: pear.typo3.org>pear.typo3.org for installing the pear packages.  We plan to write some detailed articles about how to use this to improve the TYPO3 core and your extensions. As third way to install the standards and its dependencies without using pear, we plan <link http: getcomposer.org>Composer support for the next release. Big Kudos goes to Andrej Kolos and <link http: www.dkd.de>d.k.d. for helping hands with the CI and PEAR server. Status: 100% Information:
  • <link http: pear.php.net package php_codesniffer>PHP_CodeSniffer
  • <link http: pear.typo3.org>PEAR-Installer
  • <link http: getcomposer.org>Composer
  • <link http: forge.typo3.org projects team-php_codesniffer>TYPO3 CodeSniffer

Synchronize TYPO3 Codesniffer standard to an official location

The <link http: forge.typo3.org projects team-php_codesniffer>TYPO3 Codesniffer standard is developed at Forge, <link https: review.typo3.org q project:teams forge.typo3.org hudson-helpers tools>Gerrit and <link http: git.typo3.org teams forge.typo3.org hudson-helpers tools php_codesniffer.git>Git. To use the full service of the open source world we wrote a <link https: github.com andygrunwald github-sync>script which syncs the code from git.typo3.org to github.com. You can grab our source code also from <link https: github.com typo3-ci typo3-codesniffer>Github. With this sync process from Git.typo3.org to Github.com we are able to execute our unit and coding guideline tests through TravisCI after each merge. This will notice us immediately if an error will occur. Status: 100% Information:

Activation of new TYPO3 coding guideline sniffs in ci.typo3.org

Since some time the TYPO3 core is checked by the defined coding guildelines (<link extensions repository view doc_core_cgl>typo3.org/extensions/repository/view/doc_core_cgl link "coding guidelines"). Every new patch will run against various tests for syntax formatting. E.g. is the opening brace of a new method in the same line? Are there any excess whitespaces? And so on. The TYPO3 core contains many lines of source code. Some contributers are working hard to get code shaped up to comply to the TYPO3 coding guidelines. We activate more tests as time progresses. One requirement to activate a new test is that this test does not fail at the current TYPO3 core. All violations must be fixed before a new test will be activated. During the code sprint we activated <link http: forge.typo3.org issues>seven new tests. Thanks to all code contributors to fixing these coding guideline violations! Status: 100% Information:

Fixing CGL violations automatically

For the core we made a big effort to get to Coding GuideLines (CGL) compliance. Community members (like Wouter Wolters and Roland Waldner) fixed violations manually, other violations were fixed by the automatic namespace change script by Thomas Maroschik (e.g. the visibility modifier for PHP methods). Thank you Thomas, Wouter and Roland and all the other CGL contributors for doing this awesome work. The next beer is for you! Fixing CGL violations is both time-consuming and boring, so there is the need to find a tool that does the job for us. There are some around like PHP_Beautfier (<link http: pear.php.net package php_beautifier>pear.php.net/package/PHP_Beautifier/ link "PHP_Beautfier") or PHP-CS-Fixer (<link https: github.com fabpot php-cs-fixer>github.com/fabpot/PHP-CS-Fixer link "PHP-CS-Fixer") which operates on Regular Expressions and/or PHP_Tokens. We experienced PHP_Beautfier as having strange default behavior that is not practical for our purposes. Arvid Jakobsson came up with a tool fixing the CGL issues called lex-pass (<link https: github.com facebook lex-pass>github.com/facebook/lex-pass/ link "lex-pass") from Facebook. Lex-Pass is written in Haskell (<link http: www.haskell.org>www.haskell.org link haskell) and creates a so called Abstract Syntax Tree (AST) like a DOM tree of an PHP file on which you can make changes. It is shipped with some ready to use transformations and it is possible to create your own ones. Arvid published his transformations at (<link https: github.com facebook lex-pass wiki transformers>github.com/facebook/lex-pass/wiki/Transformers link "transformations"). Stefano dug into Haskell and tired to figure out how to write Transformations. A weekend, however, is not enough to finish this task. Work on this continues. Our goal is to have a Lex-Pass transformation for every CGL violation. Big thanks goes out to Arvid for introducing lex-pass, sharing his transformations with us and providing Changesets. Status: 10% Information:

Create one PHP Unit Base class for TYPO3 core

A unit test is a single and independent test to verify a single functionaly. For example to test if the logger API properly functions. Those unit tests are executed by the Extension „<link http: typo3.org extensions repository view phpunit>phpunit“. The long term goal is to create a set-up to run all unit tests without an external extension or without a full installed TYPO3 system (which is currently needed). The first step to reach this goal was done at the code sprint. We standardized the way to create a new unit test. Before there were many different base classes, but now the core contains one single base class which needs to be extended for your unit tests. This change provides a much more simplified approach. Status: 100% Information:

TYPO3 CMS 6.0 must haves

The new TYPO3 CMS 6.0 is just right around the corner. Many bugfixes, some new features and big changes were made for the last release. There are some open tasks, however, which need to be tackled and solved before the new version will be released. Helmut Hummel, Christian Kuhn und Susanne Moog created a list of open tasks. We always need helping hands from the community. Please, if you got some time take on one of the tasks of the list. We need reviewing, testing and programming! Status: 100% Information:

Improving the TYPO3 core

Besides the quality topics we addressed during the weekend we used the time to discuss and tests some patches. Here is a small list of interesting merges:
  • [TASK] Fix CGL violations against OpeningFunctionBraceKernighanRitchie (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] Cleanup character after PHP closing tag (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] Fix CGL violations against UpperCaseConstant (<link https: review.typo3.org c>review.typo3.org )
  • [TASK] Fix UnnecessaryStringConcatSniff (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] CGL Cleanup: Fix Squiz.Scope.MethodScope-Sniff (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] Fix TYPO3.ControlStructures.DisallowEachInLoopCondition (<link https: review.typo3.org c>review.typo3.org) 
  • [TASK] add leading backslashes to Date* (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] add leading backslashes to Runtime* (<link https: review.typo3.org c>review.typo3.org) 
  • [TASK] Use namespaced class name in adodb flexform (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] Fix class naming and alias map after namespace change (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] Provide a file for IDEs to recognize old class names (<link https: review.typo3.org c>review.typo3.org) 
  • [TASK] Remove content element "script" (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] Remove content element "textbox"  (<link https: review.typo3.org c>review.typo3.org)
  • [TASK] Make all classes non final (<link https: review.typo3.org c>review.typo3.org)
  • [BUGFIX] Set permission module for admins only (<link https: review.typo3.org c>review.typo3.org)
Status: 100%

Visualizing TYPO3 Core activity

Tolleiv Nietsch started <link http: blog.tolleiv.de visualizing-typo3-core-activity>Visualizing TYPO3 Core activity. Susanne Moog announced a new link with <link http: psychomieze.de stats stat.html>current updated statistics. This is not a full „task“ / „progress“ / „result“ of this sprint, but we just want to point you to it, because we know how important statistics visualizing activity ? Status: 100% Information:

Sponsoring

We want to send a really big „THANK YOU“ to our sponsors!
  • At first <link http: www.cyberhouse.at>Cyberhouse who sponsored the location, coffee, beer, food and worktime of Georg Ringer. Thanks for your support!
  • Further more thanks to <link http: opendo.at>opendo for pizza. The food was delicious!
  • Thanks to Steffen Gebert who sponsored some cocktails to celebrate the impressive reduction of CGL violations.
  • A big thanks to <link http: infomesh.at>Andreas Ofner and Georg Ringer who sponsored the hotel rooms for all attendees.
  • Last but not least thanks to <link http: www.wmdb.de>wmdb and <link http: www.bitmotion.de>bitmotion who sponsored time for Andy and Helmut to attend this sprint..
Without you such an event could no take place. Thanks for your efforts!

terms

  • QA - Quality Assurance
  • CI - Continuous Integration
  • Git - Distributed revision control and source code management
  • Gerrit - Web based code review and project management for Git based  projects
  • CGL - Coding GuideLines
  • Mr. Jenkins - An extendable open source continuous integration server (<link http: jenkins-ci.org>jenkins-ci.org)

Latest news

Show more