A step into the future
TYPO3 4.0 provided a 'hidden/undocumented' feature that lets you define your own 404 page as a file or URL using configuration settings ([FE][pageNotFound_handling] = /404/). One option is to create a “static” HTML file, set the path to it in the configuration, prepending it by “READFILE:”, and TYPO3 will show it as 404 page. However static files are not good. You cannot display, for example, an automatically updated site map with it. Here URLs come to play. If you set a URL (relative or absolute) for pageNotFound_handling, TYPO3 will fetch it and display it as the 404 page. The problem with this solution is that TYPO3 (3.x and 4.0) redirected you to this 404 page. Technically this means that a search engine like Google was not able to recognize this as a 404. So it would never remove this page from it's database. Also the HTTP error codes section from your statistics program like AWStats would never indicate that you have missing pages on your website.
Finally not found
Since TYPO3 4.0.1 the same as before is possible, but a real 404 is thrown at the client. NOW Google knows what is going on and will remove the page from it's database. Nice to know is maybe that during development a bug caused the error handling to start a loop in trying to find a 404 page that was not there and brought the test server (actually production) down to it's knees several times. This is all solved now and this error is shown.
Unfortunately, the error message is not complete yet, but it gives a good indication of what is going on. Of course you should provide your 404 page with useful information and I will show you later how to do this.
My statistics program also gives me some correct feedback now.
In this case you can also click this 404 and see what pages were attempted to visit.
