Documentation Screenshots—Puppeteer Makes Life Easy

Categories: Community, Documentation Created by Lina Wolf
This year, a special interest group was formed to update screenshots in the official TYPO3 documentation. Updating screenshots manually is tedious and boring. This article explains the really cool automatic solution we have devised, and we’re looking for people to help us make it even better.

The Background

The initial idea was quite simple: Let someone manually update all screenshots in the official documentation. Screenshots in the official manual of TYPO3 v10 range from different LTS (long term support) versions, so they need to be updated. At the same time as removing outdated screenshots, we thought we should also make visual additions like arrows, boxes and numbers consistent.

So the idea for this budget was created: Update Outdated Screenshots in the Official Documentation. The impact summary focuses on newcomers, and how the documentation represents the community and project. 

"Showing up-to-date and consistent screenshots is important to people learning TYPO3 as they are able to relate what they are seeing to their own installations. It lowers frustration potential while learning TYPO3. Consistency in style is relevant to present a professional image and assures readers, that the documentation (and the product) is a reliable source of information."

What we figured out when we scoped the work is that:

  • Screenshots in documentation are good. 
  • Maintaining screenshots is hard. 
  • Keeping screenshots current takes a lot of work and effort.

Scoping Out the Challenges

It became clear quickly, that having all screenshots up to date for the release of LTS 11 would be almost impossible to do manually—even if we had the capacity, because of several challenges. 

Documentation Includes More Than The TYPO3 Core Install. 

There are five references, about 10 guides and many system extensions. The range of screenshots these need differ. Almost none of the screenshots can be taken with a fresh TYPO3 install. Some depend on the Introduction Package, some on special extensions like Styleguide.

We Want To Build It For The Next Major Release Which Is Under Development. 

At the time of our kickoff the first official sprint release of TYPO3 v11 was not yet available. But it was already clear that there would be important visual changes well after the release of Version 11.0 with the introduction of Bootstrap 5. So simply working with Version 11.0 would be no solution.

The Contributed Extensions Aren’t Up To Date Yet Either. 

Another challenge is that the extensions we need to update the screenshots were not yet available for v11. So we would have to squeeze the production of hundreds of screenshots into a short time before the release of 11 LTS. And then we would already have to think about screenshots for Version 12.

To handle all these challenges we would need a reproducible (preferably automatic) way to update the screenshots as needed.

Automating the Screenshots

During the kickoff meeting we developed some great ideas to tackle the problem. 

Alexander Nitsche, who had worked on the acceptance tests in the Core development, got inspired by the screenshot feature of Codeception, a PHP testing framework. We considered if the production of the screenshots could be integrated into the automatic acceptance tests of the Core.

However the kind of screenshots we need for the guides and references depend on extensions and data that is not available during Core development. And while Codeception is a great tool for automated acceptance tests, it doesn’t provide much functionality to support sophisticated screenshots.

Then Georg Ringer threw the idea of using Puppeteer into the room. Puppeteer is a node.js library to automate anything you can do in the browser. 

Georg even gave us a quick demonstration of how he uses it and some ideas how we could circumvent TYPO3 security features. For example, we wouldn’t have backend tokens to take a screenshot on a webpage that isn’t publicly accessible. 

This motivated us to get the automatic screenshots set up. With a flurry of excitement came the first proof-of-concept and the team quickly embraced the idea and got it up and running.

About Puppeteer 

Puppeteer is really interesting. Internally, it uses a real Chrome browser. It can interfere with the browser like a real human: Hover here, click there, type something. It can also execute JavaScript or CSS changes on the page as it takes screenshots. For example, it’s possible to open select boxes to show options or even hide distracting elements. And now, we’re even thinking about how we can integrate the visual elements to mark up screenshots (like orange boxes, numbers and arrows) with JavaScript before even capturing the screenshots.

We also got the idea to automatically compare the generated screenshots. This should help to identify errors in the picture generation and see where there are significant changes, making it necessary to update the written documentation.

Benefits Beyond Documentation

The great thing is that the community will be able to use the screenshot generation tool not just for the official documentation. The tool can be used to take screenshots for third party extensions for technical and backend user documentation of sites created with TYPO3.

This will also come in really useful for the TYPO3 Microcopy project, to improve UI text in the application. Changes made to text on screen will necessitate updating screenshots wherever they are used, especially in official docs, so this is a big win for everyone!

Want to see it in action? Check out the 20 minute Show and Tell on YouTube for updating the screenshots in the TYPO3 TCA Reference.

How it Works: Automating Screenshots with Puppeteer

Try it out for yourself!

Clone the project. The complete screenshot project is a DDEV project dependent on the TYPO3 Core and some Core-close third party extensions. 

Run the setup script. The automatic setup script takes care of installing both TYPO3 and Puppeteer. The project runs on a wide variety of systems. We tried it on Linux, Windows and the Mac. It could also be run automatically in a Docker container in the future.

Install dependencies. We use ImageMagick to compare screenshots so that is required too. 

Define what screenshots you want to take in the corresponding manual’s Git in the directory ‘scripts/GenerateScreenshots’. 

Come and Join Us

Has this story sparked any ideas? If this technology interests you, we’d love to welcome new people to the team. 

  • We need people to use our automatic screenshot tool to update the official TYPO3 manuals, tutorials, guides and references.
  • We also need a Frontend Developer with a good eye and experience in DOM manipulations (CSS / JavaScript) to help us improve the optic output of the screenshots. 

This technology can be used to produce the screenshots for third party extensions or customer projects. If anyone would like to do a proof of concept for their own project, please contact us.

Find us in TYPO3 Slack at #t3docs-sig-screenshots or contact Lina Wolf.

Additional contributors for this article
  • Content Publisher : Mathias Bolt Lesniak