Recommended for all new TYPO3 websites is to set the global character set to UTF-8. This is done by putting a line like this into the localconf.php file of the site:

$TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';

Explanation of “forceCharset”

The reason is that without a value for “forceCharset” the backend will use a charset depending on the backend language of the user. So lets say User A has a backend in Russian and User B a backend in Danish, then User A will enter data in the charset “windows-1251” and User B in “iso-8859-1”. Since data stored in TYPO3 is not tagged with a charset this will lead to an inconsistent database.

If $TYPO3_CONF_VARS['BE']['forceCharset'] is set, the same charset is used for all users in the backend regardless of their backend language and you will get consistent data in your database. But since “iso-8859-1” and “windows-1251” are “incompatible” charsets (using the same byte values for different glyphs) you will have to choose “utf-8” as charset since that can contain both danish and russian chars - and all other charsets of the world.

Conclusion: Using utf-8 means you have a consistent data storage and can store any glyph from any language without thinking more about charsets.

Note: $TYPO3_CONF_VARS['BE']['forceCharset'] is not set by default for backwards compatibility reasons but is recommended for all new TYPO3 projects.

Charset in frontend (advanced)

The “forceCharset” value will also be used in the frontend automatically. But it is possible to change charset settings in TypoScript. There are two settings, “config.renderCharset” (don't change unless you know what you are doing) and “config.metaCharset”.

config.metaCharset defines the character set of the HTML output. If this is set to another value than renderCharset / forceCharset, all content is converted before output although internally processed in “renderCharset”. This is useful for special cases like japanese websites where they like “shift-jis” used for content delivery.

If config.renderCharset != config.metaCharset there are a few things to be aware of:

Database field lengths

If you choose to use utf-8 as charset you might face the problem that the database field lengths must be extended. For example, each chinese glyph takes three bytes. So if a field is a varchar(10) and an author enters 10 chinese glyphs only the first 3 glyphs will be stored (since they take up 9 bytes). utf-8 is tricky in this respect because all ascii chars take only 1 byte while european special chars typically take up 2 and asian charsets take up 3 - but some special glyphs could take even 5-6 bytes!

If you are using MySQL 4.1 or above you can set the database to use UTF-8. Please read more here:

http://dev.mysql.com/doc/refman/4.1/en/charset-unicode.html

If you are not using MySQL (or any other DB) that supports UTF-8 you have to consider this workaround:

As long as a database does not support utf-8 so varchar(10) means “store 10 utf-8 characters regardless of byte length” we offer a setting in TYPO3 which takes care of it:

$TYPO3_CONF_VARS['SYS']['multiplyDBfieldSize'] = '3';

This setting will automatically change all varchar(10) fields to “varchar(30)”. After changing this setting you must use the Install Tool, “Database Analyzer” to alter all fields in tables.

Next, you must set up which languages you want the site localized to. These languages are additional to the “default” language of the site. What the default language is will be up to you to define.

The system languages are added on the root level of the page tree. In this guide I have added two languages, Danish and Russian.

The record for the Danish language record looks like this:

When you want to localize pages and content in TYPO3 you will now have the option between “Default”, “Danish” and “Russian”. Here is a view from the Web>List module:

If you want to specify the name of the default language and a flag icon you can do so for a branch of the page tree by setting this Page TSconfig for the root page of your website:

This setting is for example used by the Web>Page module, Web>List and TemplaVoila Page module.

When you want a translation of a page in the page tree you create an “Alternative Page Language” record on that page. This contains fields similar to that of the page record which you fill in with translated content:

When such a record exists you can translate the content of the page to the language it defines. There are various methods to translate the content of pages and I will get back to that. The Alternative Page Language record is important to understand first since these define which pages has a translation and which not.

In the Web>List module they look like this:

You can get a complete overview of the page tree and according translations by using the Web>Info module, choosing the function menu item “Localization Overview”:

All the green entries show which languages each page is available in. The gray areas means no translation is available but access to the page specifying this language will be tolerated. It is also very quick to create new translations from this module (using the check boxes).

Hiding pages if no translation exist

If a translation of a page does not exist it will by default still appear in the website menu when using that language. Here the “Contact” page is still shown with default, English content:

If you check the “Hide page if no translation for current language exists” checkbox for the “Contact” page in “page properties” you can avoid this:

This will be reflected in the Localization Overview:

From this it is clear that when viewing the website in Russian the “Contact” page is not accessible. And the menu will reflect this since now the “Contact” page is not shown.

Trying to access the “Contact” page with the id will even show an error.

Hiding default translation of pages

If you want pages in only the alternative languages you must still create a default language page in the page tree which simply acts as a placeholder. Setting this status is done by selecting the checkbox “Hide default translation of page”:

This is reflected like this in the Localization Overview:

On the website it will look like this:

Danish:

English:

Trying to access the page “Visioner” in default language will yield an error.

Hiding pages in the default language is probably a rare thing to do, but it is possible to imagine cases where the topic of a page or section is only relevant in one of the alternative languages. Especially if a language of the site is not only a translation but may serve subsidiaries of a company in a local context.

Inverse control of hidden translations

If you prefer that all pages are hidden by default (reverse action of the “Localization Settings” check boxes) you can set a TYPO3_CONF_VARS option:

$TYPO3_CONF_VARS['FE']['hidePagesIfNotTranslatedByDefault'] = '1';

The Localization Overview will reflect this immediately:

Notice, that the Localization Setting checkbox is reversed for the “Contact” page (the background above is gray, not red):

Final notes

Summary of colors in “Localization Overview”:

Notice about possibly false menu items when “Localization settings” are used:

When running in default fallback mode (config.sys_language_mode = 0/blank/not set) menu generation may create false items because sys_language_uid will be different from the value set by the &L parameter (which the menu will just pass through not knowing what it does) when a page translation is not found. Menu items not available in the default language will not be shown even if they should and items which are not available in the translation will be shown regardless of this.

Solution is to configure in TypoScript  "config.sys_language_mode = content_fallback" because it will keep the “sys_language_uid” value according to that defined by &L regardless of whether a translation exists or not.

If you wish not to use “config.sys_language_mode = content_fallback” you can also choose to set the property “protectLvar” for the HMENU objects; this will correct the L-variable for those menu items which are not available. See TSref for details about that option.

In the Web>List module you can enable a localization view of the list which will provide you with buttons for localization of records from tables supporting localization:

The result is a view like this:

When a localization does not exist there will be an icon (“Localize to”) to click which will localize the element.

The elements can be localized only to the alternative languages on the page and since the page has only a Danish “Alternative Page Language” record (named “Ikke-cached”) you see only Danish flags. Adding a Russian translation will add Russian icons:

Technical note about “Localizations”

The “Localize to” links presented by the Web>List module, Web>Page module etc. uses the core “localize” command. This command creates a copy of the default language record, changes the language setting to the requested language and sets a reference back to the default language original. Thus the localized version is bound to the original giving the possibility of tracking translation precisely.

Behind this method there is an assumption that when localized elements are selected and presented in the frontend of a website it is done by selecting the default language element subsequently being overlaid with the alternative language record if found. This means that all queries and references between records should be done with the default language records and localizations are just dumb overlays automatically selected and shown if available. Not only is this a more sound technical practice, it is also possible to take advantage of inheritance of content from the default record; images, links, references etc.  - data that you might not like to enter a second time for the localized record!

Taking advantage of overlays and inheritance is enabled by TypoScript config options like “config.sys_language_overlay” and “config.sys_language_softMergeIfNotBlank”

In your TypoScript template you will want to have a configuration of localization which looks something like this:

   0: # Localization:

   1: config {

   2:   linkVars = L

   3: }

   4: [globalVar = GP:L=1]

   5: config {

   6:   sys_language_uid = 1

   7:   language = dk

   8:   locale_all = da_DK

   9:   metaCharset = iso-8859-1

  10: }

  11: [globalVar = GP:L=2]

  12: config {

  13:   sys_language_uid = 2

  14:   language = ru

  15: }

  16: [global]

The “&L” variable

Although it is not a requirement to use the L-variable for localization it is highly recommended because it has been hardcoded a large number of places to hold the uid of the sys_language record (menus / realurl / TemplaVoila / Page module / Preview links / etc.).

The lines 1-3 configures that if the GET request variable “&L=X” is found it should be passed on to all generated links.

Lines 4 and 11 are conditions which tell that if the GET variable “L” is 1 or 2 the system language (config.sys_language_uid) is set accordingly (plus all other TypoScript template related settings you might want to change for a localization). Also those conditions are the reason that caching will support and recognize a change in the L-parameter.

Setting up the L-variable like this is the minimum requirement to make language switching work.

config.language

(line 7 and 14) Setting this configures the system “language key” to be used for the language. This is used to select labels from llXML (locallang-XML) files. Setting this should make frontend plugins respond by showing labels from the correct language (requires installation of the corresponding language packs).

config.locale_all

(line 8) Configures the locale which influences how some values from PHP is formated in the output, eg. dates formatted using strftime() which will then output month/day names in that language.

config.metaCharset

(line 9) Setting the output charset of the page to “iso-8859-1”. Means that content internally rendered in utf-8 (due to “forceCharset” setting) is converted to “iso-8859-1” before output:

For default and Russian language:

Even if “config.metaCharset” is set to “iso-8859-1” for Russian the output will be correct since HTML-entities will be used for glyphs which cannot be represented in the chosen language:

utf-8 (default):

iso-8859-1:

The localization mode defines how the system behaves if there is no translation of a page to the language requested (ie. there is no Alternative Page Language record). For instance: Would you like a “page not found” error? Or should the default language be shown? Or should another of the available languages be tried first? And if so, would you like the menu to stay in the selected language? These are the preferences you can obtain using this setting.

Overview of combinations

Localization mode is defined by the TypoScript configuration “config.sys_language_mode”.

The table below provides an overview of the combinations of language modes (config.sys_language_mode) and the language uid (config.sys_language_uid, which is essentially defined by the input “&L=” variable in a typical set-up).

This is how the localization overview is set for the page tested:

A few additional technical notes:

Best-practice localization mode

It is recommended to use

config.sys_language_mode = content_fallback

This is so, because the compatibility with other localization functions are greatest in this case since the “TSFE->sys_language_uid” value is set to that of the requested language.

Using llXML labels in TypoScript structures

llXML files are XML files containing labels that the system can fetch in a localized version if a language pack is installed. If you want to retrieve values from llXML files in TypoScript you can do it like this:

page.20 = TEXT

page.20.data = LLL:EXT:indexed_search/pi/locallang.xml:submit_button_label

This looks for the label “submit_button_label” in the file “pi/locallang.xml” from the extension “indexed_search”.

If the “config.language” value is set to “dk” and the Danish language pack is installed in “typo3conf/l10n/dk/” the output should be “Søg” (and not “Search” which is what you will get if the label is retrieved from the default language).

Using llXML labels in frontend plugins

How to use llXML labels in your PHP code is not covered in this guide (see “Inside TYPO3”), but properly made frontend plugins will use llXML files with the plugins to generate translated labels for their output.

For instance, if “config.language = dk” and the Danish language pack is installed you will see this page for “indexed search”:

You can use TypoScript to override with custom values (supported by most extensions made by kickstarter and using the piBase API). Here is an example for how the search button label can be overridden:

plugin.tx_indexedsearch._LOCAL_LANG.dk.submit_button_label = SØØG!

This yields this output:

You must look into the file “indexed_search/pi/locallang.xml” to know that the label key was “submit_button_label” and of course “dk” is the language key. The prefix “plugin.tx_indexedsearch._LOCAL_LANG” is exposed using the TypoScript Object Browser:

Providing a translation to a frontend plugin for a language key not found in the system?

There is an unfortunate binding between the system languages of TYPO3 (those you can select for the backend users) and the translations of frontend extensions. As long as you are making a website in a language supported in the backend you just install the language pack and you are done. But what if we wanted to make a translation of the site to “Marsian”?

Well, we can actually do that using the “_LOCAL_LANG” overriding feature!

So a setup like this will translate the search button label to “Marsian”:

config.language = marsian

plugin.tx_indexedsearch._LOCAL_LANG.marsian {

  submit_button_label = !"#€%&/(

}

You probably want to have a website language selector menu somewhere on your website. This menu should show an entry for each possible language and since you might not have a translation for each page you might like the menu items to appear in 4 variants:

The way to make such a menu is to use the HMENU cObject from TypoScript and use the special type “language”.

Here is example configurations:

GMENU example

## Localization menu:

lib.langMenu = HMENU

lib.langMenu {

special = language

special.value = 0,1,2

special.normalWhenNoLanguage = 0

1 = GMENU

1.NO {

XY = [5.w]+4, [5.h]+4

backColor = white

5 = IMAGE

5.file = EXT:cms/tslib/media/flags/flag_uk.gif  || EXT:cms/tslib/media/flags/flag_dk.gif  || EXT:cms/tslib/media/flags/flag_fr.gif

5.offset = 2,2

}

1.ACT < lib.langMenu.1.NO

1.ACT=1

1.ACT.backColor = black

1.USERDEF1 < lib.langMenu.1.NO

1.USERDEF1=1

1.USERDEF1.5.file = EXT:cms/tslib/media/flags/flag_uk_d.gif  || EXT:cms/tslib/media/flags/flag_dk_d.gif  || EXT:cms/tslib/media/flags/flag_fr_d.gif

1.USERDEF1.noLink = 0

1.USERDEF2 < lib.langMenu.1.USERDEF1

1.USERDEF2.backColor = green

}  

TMENU example

lib.langMenu = HMENU

lib.langMenu {

special = language

special.value = 0,1,2

special.normalWhenNoLanguage = 0

1 = TMENU

1 {

        # Normal link to language that exists:

NO = 1

NO.allWrap = |*| | *  |*| |

NO.linkWrap = <b style="background-color : grey"> | </b>

NO.stdWrap.setCurrent = English || Danish || Russian

NO.stdWrap.current = 1

        # Current language selected:

ACT < .NO

ACT.linkWrap = <b style="background-color : red"> | </b>

        # Language that is NOT available:

USERDEF1 < .NO

USERDEF1.linkWrap = <span style="background-color : yellow"> | </span>

USERDEF1.doNotLinkIt = 1

}

}