Community Budget Idea Report: Image Rendering Improvements

A Look at Other Systems and Their Solutions

During our exploration of potential solutions for integration into the TYPO3 core, we engaged in several internal discussions. These discussions covered both simple and advanced approaches to the topic, including a review of existing solutions. 

We decided to examine how other popular CMSs and applications, such as WordPress, Drupal, Contentful, and Magento 2, handle image processing in their core. Notably, Neos CMS has a solution for deferred image processing that is very similar to TYPO3's approach for the backend. 

After exploring how other products deal with image processing and after several discussions, we got a feel for what could work well or what things to avoid. Armed with this knowledge, we began to outline a potential solution for asynchronous image processing in TYPO3's frontend.

Defining a Solution to Prevent Image Processing from Blocking Page Rendering

During our research, which involved examining concepts from previous initiatives and considering the best strategy to speed up page rendering when there are many images in the content that need to be processed, we identified five potential solutions. These range from a straightforward approach, which might not accommodate more complex setups, to more sophisticated concepts designed to meet the needs of complex projects.

Here we present the five solution options. You can read more about each of these approaches, including details pros and cons, in our technical document: Strategies and Solutions for Enhancing Frontend Image Processing in TYPO3. 

Option 1: Trigger Page Rendering After Each Content Change

This strategy involves re-rendering TYPO3 page content upon backend edits, with an emphasis on managing frontend cache behavior and ensuring content updates in the chosen language. While simple to implement, it faces limitations with complex cases, edits outside the Page module, and content dynamically updated on the frontend. Additionally, integrating workspaces into TYPO3's Core could streamline content management by allowing pre-live editing and selective publishing, similar to Neos CMS or Drupal, enhancing efficiency in content re-rendering and image processing.

Option 2: Generate and Return Temporary URLs for Deferred Image Processing

This strategy uses a dedicated processor to generate temporary URLs for unprocessed images, allowing pages to load without delay. When a temporary image URL gets called, early processing by dedicated middleware is key for efficiency, and security measures must ensure only valid images are processed to avoid misuse and mitigate DDoS risks. Temporary URLs within page content should not be cached until final URLs are used (to avoid SEO risk if search engines encounter these first). An alternative is making pages with pending images uncached, but this might reduce performance on busy sites.

Option 3: Generate Final URLs for Images Without Processing During Page Rendering

This method ensures that even when image processing is deferred, the final URL for a processed image is generated, addressing caching and SEO concerns. It involves a specialized processor storing instructions in the database without immediately processing images, allowing the final URL to be retrieved and included in page content. When an image is requested, the server checks if the file exists; if not, it redirects the request to be handled by middleware, requiring server configuration.

Option 4: Delegate Image Processing to External Service

This method bypasses TYPO3 for direct file processing, instead routing requests through an Image Proxy server. Entry points like view helpers and TypoScript objects are provided with URLs to this server, detailing required image processing. The processing happens externally, not affecting TYPO3's content rendering time. We're considering an Image Proxy service, potentially self-managed or as a paid third-party service, offering different levels of control and maintenance.

Option 5: Use Placeholder Images and Process Images Through Scheduler Task

This approach involves using a processor to assign URLs to placeholder images and a scheduled task for background image processing. When processing starts, the processor returns a placeholder image URL if the final image isn't ready, ensuring these placeholders aren't cached so that processed images can replace them upon subsequent requests. Users can also customize placeholder images. This method, mirroring Drupal's backend thumbnail rendering system, allows for queued processing of selected media types.

Each of these solutions has its pros and cons (detailed in our technical document), which need to be carefully considered before deciding which approach to implement in TYPO3. From our perspective, at least two of them could be implemented. Solutions 3 and 4 appear to be the best approaches, as they work well for both small and large projects and have the fewest cons among the others. Moreover, these solutions differ in their operation: the third one processes images locally, while the fourth delegates image processing and serving to a separate service, such as an image proxy. 

The final decision should come after reaching an agreement with the TYPO3 Core developers, which we found challenging due to difficulties in scheduling in-depth consultations. However, I would like to extend my gratitude to everyone involved in our project for their help and support. We had an extensive discussion with Simon Praetorius after forming our proposals, during which Simon provided valuable feedback and highlighted some drawbacks, especially for solution number 1. Thank you all for your assistance so far.

What Comes Next: The Implementation

The next step involves more in-depth consultations with the TYPO3 Core team members to decide which of the proposed solutions should be adopted. It needs to be determined whether these should be integrated into the Core code or offered as a separate extension, and whether they will be activated by default or remain optional. For everyone interested in this topic, we encourage you to review our technical document and share any suggestions you might have. 

We aim to propose the implementation phase as one of the Community Budget Ideas for Q3 2024. This proposal will outline the selected approaches for asynchronous image rendering in TYPO3, detailing their implementation and distribution methods.