Datahandler & Persistence Initiative

Summary

The main goal of the first part of the persistence initiative is cleaning up the existing data handling layers in TYPO3 and providing a unified persistence layer for use in all contexts and by extension developers and the core alike.

Background

DataHandling in terms of combined localization and workspace versioning is complex and requires a good understanding of the data structures used in the TYPO3 internals. Besides that these topic lead to similar bugs (regressions) again and again when new features get introduced.

Working with workspace versions from an editor standpoint is not an easy task in terms of reviewing changes for many pages, content elements and nested data structures like file/image references.

In order to overcome these obstacles and to introduce a unified persistence layer for all TYPO3 components (frontend, backend and Extbase) this initiative has been created.

Goals

Short-Term

As an extension developer I want to be able to read or write data in TYPO3 (without knowing specifics about internals like workspaces, persistence or permissions)
As a core developer I want to be able to write new core code without taking care of all aspects every time (like workspaces, persistence or permissions)
As a core developer I want to be able to handle workspaces, translations and relations of any kind transparently.
As an extension developer I want to easily be able to read complete data sets with relations and dependent data.
As an extension developer I want to use the same API to read or write data independent of the current request context (frontend, backend, cli …).
As an administrator I want to be able to install TYPO3 with different storage mechanisms depending on the current hosting environment (for example Microsoft partners using SqlServer, company policies requiring Oracle …).

Mid-Term

As a frontend application developer I want to be able to read data via REST to build modern frontend applications.
As a third party app developer I want to be able to read data via REST to integrate TYPO3 content in my application.

Long-Term

As a third party app developer I want to be able to write data via REST to add/enrich TYPO3 with content from my app.
As an editor I want to be able to preview changes before publishing them.
As an editor I want my content to be automatically saved so I don’t risk losing data.
As an editor I want to be able to share my current version of content with others to get comments and feedback.
As an editor I want to be able to specify a publishing date for one version of my content while simultaneously being able to manage more than one version of the same data set.
As an editor I want to get a fast overview of the translation state of my website and easily find changed content to translate.
As an editor I want to have a workflow module guiding me through my current tasks (like translating stuff, preparing a landing page …).
As an administrator I want to be able to use different storage mechanisms depending on the data use case (for example redis / sqlite etc. // for example when having mostly read-only data aka “materialized views”).

Milestones

Precursor: DataHandler separation
Not fully, but remove concerns.
- Extract validation & sanitization (‘eval’ => ‘datetime’)
- Extract permission handling (not BE_USER specific)
- Extract configuration resolver (interpretation of $TCA)
- Extract relation resolver (TCA group, select, inline)
- Extract aggregate resolver (e.g. tt_content & images)
- Remove global scope dependencies (BE_USER for logging for example)
- Refactor logging
Define Read/Write API
1. Introduce new API for read access
2. Introduce new API for write access
3. Introduce new API for relation resolving
4. Use new API everywhere (frontend, backend, extbase)