As with any other extension, download the extension rs_userimp from TER (as local or global extension) and load it in the Extension Manager. The Generic User Import Tool relies on two additional database tables (tx_rsuserimp_presets, needed for storing import presets and tx_rsuserimp_session, used to store import session data) which will be created while loading the extension.
You should reload the BE after installation (or logout and then login again). Doing so, you will see the new module icon right under Tools
After loading, the extension is ready to work but you may configure some safety settings for the rollback feature and general file handling directives.
If you want to keep your import files, you may set Use recycler for this purpose. Of course you have to configure the TYPO3 recycler folder first - if there is no recycler available, import files will be deleted right away.
If enabled, the Create drop file feature creates a CSV file with all dropped out (skipped) users: users which were skipped for whatever reason are written to a new CSV file which you could download, edit and re-import again.
The Rollback Safety Timespan defines the timespan (in minutes) in which you are allowed to rollback any of your import sessions. This timespan is meant as a safety measure because it seems not advisable to allow rollbacks forever.
The Rollback Data Preview Rows is the amount of preview datasets shown in the rollback info box.
With the Rollback Delete DB Records setting you may specify the record deletion characteristics of rollbacks: you may either choose the native TYPO3 way (where deleted database records get marked “deleted” but are actually kept in the DB) or the extension's own way of removing rolled back records from the DB. So, if you used the tool to import several hundreds or even thousands of user accounts and encounter errors afterwards, it might be wise to physically delete those user datasets during rollback. Also, if you have chosen NOT to delete the datasets, those usernames are no longer available, these usernames exists even if they are not active!
The Garbage Collection Trigger Timer determines when general “housekeeping” processes are triggered. Aged out import sessions, old import and drop files are deleted after the time specified here. Set this value to 0 if you want to keep these items forever.
Finally, with the Unique Identifier - fe_users and Unique Identifier - tt_address settings you may define fields of the DB tables fe_users and tt_address which will serve as a identifier when doing user updates. Imported users are checked for this identifier and if they are already present in the corresponding DB table, the datasets are updated instead of simply inserted. Make sure that your identifier is a really unique value, otherwise you will screw up your database!
Perhaps you have wondered what all these timer settings are really good for, so here is a final warning note:
Once a particular import session has elapsed the Rollback Safety Timespan, the session changes its internal status from “active” to “inactive”. Inactive sessions are no longer recoverable via the tool's rollback feature, even if you set the Rollback Safety Timespan to a higher value afterwards! Next, the Garbage Collection Trigger Timer determines when inactive sessions finally get marked “deleted”. Once an inactive session has elapsed this timer, it won't show up in the list of available sessions, even if you increase this timer afterwards!
BOTH STATES CANNOT BE UNSET VIA BUILTIN FUNCTIONS!
If after all you are still not sure what to do: set both timers to 0 to totally disable the built-in security settings – BUT THIS IS NOT RECOMMENDED AT ALL!!!
Currently, the extension is available to administrators only! This is because of a basic security consideration: administrators should have full control of all system users. Logically, administrators are the only ones allowed to import users.
You start the CSV Generic User Import Tool by clicking its module icon in the backend.
The module implements a single, streamlined workflow (the task of importing users from file) where all the necessary steps to complete this workflow are organized in a so-called DynTab menu (Dynamic Tabs). These dynamic menus react on conditions set by the programmer in that they show their content only if certain criteria are met - an ideal method for our purpose.
The CSV file format
First of all you will of course need a suited import file. For example, the CSV Generic User Import Tool has successfully been tested against address export files from
Microsoft Outlook
Microsoft Outlook Express
Microsoft Excel
Lotus Notes
Palm Desktop's Contacts
But in general, every file with the following characteristics should work:
data field must be separated by a either comma, semicolon, colon or TAB
data fields may be encapsulated with single or double quotes
data rows must be terminated with common line ending characters such as newline, carriage return or line feed
data rows should not exceed 10000 characters per line (will be changed in a future release)
Note: For some strange reason, Outlook Express handles field encapsulation inconsistently amongst the different data fields. You should use double quotes as the encapsulation character.
Tab CSV file
Once you have a suited import file, you may upload it from your local computer to the TYPO3 server. Files get uploaded to the first available temp folder for that specific user, so normally they end up in fileadmin/_temp_. You don't really have to care about filenames: the files get renamed to a valid filename during upload.
You also may choose the overwrite function to replace previously uploaded files.
As soon as the file is in place, some basic file information is shown: the filename on the server and its size.
Tab Import Settings
Here, you have to define all basic settings for the import process. First of all, you have to select the user type you are going to import. Currently, FE users and tt_address datasets are supported.
Then, you have to choose a storage folder for your new users. The import tool queries the database for all possible storage places and displays them in this drop-down selector. FE users may be imported to any folder that has the fe_users plugin installed and tt_address datasets to any sysfolder.
Next, for fe_users you have to define one or more usergroups which all the imported users are assigned to. Again, the database is queried for all possible values and displays them in this dropdown selector (select multiple values by pressing CTRL). This option is disabled if you are importing tt_address datasets.
If you want to update existing users while importing new ones, you have to activate the following option and have to choose a unique identifier. You may specify values for the dropdown box during the extension setup. Take great care that this value really determines unique users! The list of available values for the dropdown box can be defined during the configuration of the extension.
Finally, you have to define how the data fields in your import file are organized (see section CSV file format for some general file format information).
First row of import file has field names: Normally, export files carry the field names in the first row, so you should tick this option. The field names will be shown during the mapping process and simplify this task enormously. If your file has no such descriptive row then this field names will be generated automatically (Field1, Field2 and so on).
Enable auto-rename: Probably you already experienced this: if you create a username which already exists in TYPO3, a zero is appended to the original username you entered. This option enables this feature. The renaming process is applied recursively to each username until it is unique (you could end up with something like newuser000). If you choose not to enable this option, those users will not be imported at all! The auto-rename feature is not available if you have choosen to update existing users.
Enable auto-values: auto-values are auto generated values which may be defined with a special syntax. Choose this option if you need to manipulate datasets during import. Read more about this feature in section Tab Field Mapping.
Attention: The performance of the import process might be greatly decreased by the auto-rename and auto-values setting, especially when importing hundreds or thousands of users at a time.
Field delimiter:Choose the field delimiter.
Field encapsulation character: Choose the field encapsulation character.
Preview lines: Choose how many sample lines of your CSV you want to be displayed for the mapping process. You should keep this value to something below 5.
Additional Mandatory Fields: The fields you enter here are appended to the system-defined mandatory fields (username and password for fe_users, empty for tt_address). Select multiple values by pressing CTRL. If for example you choose name and email here, all imported users are checked for username, password, name and email entries. If an import dataset misses any of this fields, the user is skipped.
Update settings: Please always press the update button once you have changed any of the settings in this configuration menu!
Presets: Probably you already imported some users in a previous session and have chosen to save your settings and field mappings as a preset. Here, you may select or delete such a preset.
After the following mapping process you could also return to this tab and save your settings as a new preset. Choose a name for the preset and save it.
Tab Field Mapping
Please note: This is a very important step and you should always carefully review what you did here! Errors in the mapping may result in many wrongly setup user accounts or address entries.
The mapping form itself is nearly self-explanatory. Each row displays one data field of your import file. If you have fieldnames in your CSV they are used as the description for that field. You have a dropdown selector which shows all available fields for TYPO3 user data information. The last column shows examples as they were read from the CSV.
The screenshot below shows a mapping session without the “Enable auto-value” option being set. We already have mapped 3 fields but still need to provide mappings for email and gender fields.
This screenshot below shows a mapping session with “Enable auto-value” option set. As you can see, things get a bit more complicated. The AV tickbox enables auto-values for that data field. You still have to choose a mapping value from the dropdown selector but the actual value of that field has to be specified with an expression of a special syntax. The actual values get then computed during import.
In the example above, the username value is set to {l(0)}. You might have probably guessed it: “l” stands for “lowercase” and the “0” is the argument passed to the lowercase function and represents the value of datafield #0. So the username is the lowercased christian name.
The password value is set to {m(1)}. “m” stands for the MD5 hash function, “1” stands for the value of datafield #1. So the password is computed as the MD5 hash of the family name and voilá, the Generic Import Tool is compatible with EXT:danp_md5fepassword and EXT:kb_md5fepw.
The value for module_sys_dmail_html is set to “{s(1)}”. “s” stands for string value, “1” is the string value itself. So all imported/updated users are configured to receive direct mail newsletters as HTML.
As you can see: this is a rather generic approach and gives you full control during the import process! For example, consider this mapping:
You are right: multiple auto-values are allowed for each mapping field and in this case, the username is now computed as <christian name>_<family name> (all lowercase values, for example “peter_pan”). Now, have a look at this:
The comments field is set to “{f(3)}{s( - CSV)}”: take the unaltered data value from datafield #3 and append the string “- CSV” to it (would result in something like “Peter Pan – CSV”).
And finally, this mapping
creates a mapping for direct mail categories. The thing is that these direct mail categories must be entered as a bit mask, so with “{b(3,4,5,6,7,8,9)}” we create this mask from CSV field values 3 to 9 (from lowest to highest bit). Please note: the bitmask option cannot be combined with any other auto-value.
Summary of allowed auto-value expressions:
{f(n)}: unaltered value of data field n
{l(n)}: lowercased value of data field n
{m(n)}: MD5 hash of data field n
{s(text)}: plain text
{b(n[,m...])}: creates bit mask of data values n to m (from lowest to highest bit)
It should be noted that mandatory fields are still scanned for illegal values (spaces, uppercase characters, ...) but that no automatic substitution takes place since the auto-rename feature is not available in this mode. Therefor, make sure that your mapping values meet the corresponding criteria!
Tab CSV Import
After correcting and updating the settings we are almost finished with the user import.
This tab gives you the chance to review your settings or to save them as a preset. You end the import workflow by clicking the import button.
Please note: After clicking the import button, all tab functions are deactivated and you may only choose to view the errors and messages tab!
Tab Errors and Messages
All messages and errors are displayed in this last tab. Here, we were about to import some users with the auto-rename feature enabled. In the midst of the file, the import tool detected some empty mandatory fields. Those users are skipped as well. Since the drop file feature is enabled as well, we are presented a link to download the generated CSV file which we can download, edit and import again.
Monitoring user import actions
Administrators may monitor the import tool's actions by inspecting the administration log. All successfully imported or updated users are fully logged.
Handling of temporary import files
After the actual import, the CSV file gets either deleted or is moved to the recycler. This behavior is determined by the “Use recycler” setting from the extension configuration (for details read section “Administration and Configuration”).
Rollback
When it comes to mass import of data, there may always occur errors which are detected just too late. In our case, this might happen mainly because of wrongly mapped data fields or auto-value expressions. It would be a lot of work to manually correct or delete this afterwards by hand.
Fortunately, the CSV Generic User Import Tool offers you an automated way to revert your changes: every import session is written to the database and with that information, you may completely roll back a session.
Rollback in this sense means a deletion of imported users, the tool does not yet offer a way restore previous values of updated datasets!
To roll back a session, choose the corresponding menu item in the main menu. What you get is a list of all import sessions along with some basic information. The most important part here is the so-called Rollback Status.
According to the configuration of the extension (see section “Administration and Configuration” for details), the rollback feature of an import session may have already expired (those session are marked red) or the session was already rolled back.
Sessions marked green may be rolled back. To choose a session, move the mouse over the table row and click.
