Handling Entourage 2008 database corruption, lost email
John Underhill is having frequent corruption in Entourage databases, resulting in losses of thousands of email messages. He found he could recover them from Exchange Server using Microsoft's PFDAVAdmin:
Our Entourage mail databases routinely become corrupt, resulting in bizarre behavior in Entourage, or warnings from Entourage that its database is corrupt. Rebuilding the database sometimes helps, sometimes not.
Some of our databases had become too large, and we've now set reasonable limits for users, but the problems occur with databases of all sizes. We currently have databases ranging in size from 2GB to 12GB. 4GB is our current recommended maximum size.
In one case, a corrupt database seems to have caused 20,000 messages to be deleted from a user's Inbox. The next day, 8,600 contacts disappeared from the same user's account, and 10,000 or so messages disappeared from another user's account. The users did not delete these items. The items were found on the Exchange Server, soft deleted, in their original locations (not in the Deleted Items folder). They were recovered using Microsoft's PFDAVAdmin.
Management of corrupt Entourage mail databases: Delete the user's account in Entourage, then kill Entourage (it hangs when you delete the account), delete the mail database completely (buried in the user's home folder; sorry, don't have the exact path handy), reboot (or else Entourage hangs again), then reconfigure Entourage and re-sync all email. This has had to be done at least once for all users of Entourage 2008 during the past six months, and has been done several times for some users. We wish Microsoft would pay more attention to this product.
If you've seen this problem
More reports of Entourage losing messages
Two readers respond Monday's report of Entourage losing email due to database corruption, and how to deal with it. Joshua Bierman reports an odd twist to the problem:
Same problem as reported, except that we have seen where it replaces the messages with empty folders of the same name. The answer (and it is not much of an answer) is to use OWA or Outlook (or the server if you have direct access) to undelete the mail, then do as the other fellow said: delete the Entourage account, clean out Entourage, reboot and rebuild.
Jane MacPherson sees this only on one Mac:
This just started happening to me a few days ago. I haven't changed any settings and it's not happening to the other Mac in the office. Only to mine. Very, very frustrating!!
If you've seen this issue
Reader loses email, calander when rebuilding Entourage database |Top of Page |
Stephen Chiang says that rebuilding the Entourage database erases data:
When I rebuild my Entourage 2008 database (was originally around 4 GB), it ERASES everything in my whole inbox, including any folders I have created inside. The other folders in my mailbox appear to be fine though. It also deletes calendar and contacts as well. Something is seriously screwed up with the Database Utility. I am using Exchange Server 2003 and I haven't tried replicating with today's Office Update yet either, although I plan to at some point. I'm running Mac OS X 10.5.7.
We've previously published reports and workarounds for Entourage dabase corruption. If you've seen this
TIP: Alternatives to using Entourage Database Utility can avoid corruption |Top of Page |
Two readers recommend against rebuilding an Entourage Database with the Entourage Database Utility. They wrote in response to Reader loses email, calendar when rebuilding Entourage database from last week. Robert Liebsch sent an alternative procedure that doesn't corrupt data:
I have 2 cents to offer regarding Entourage Database corruption: First, never, under any circumstance use the Microsoft Entourage Database Utility with any version of Entourage.
We have databases from 10MB through 20GB. The larger the database the more likely you will have the corruption notification. The best way, best that I've found - with my team, is to:
- Quit Entourage.
- Open Activity Monitor (for older versions of Entourage) and kill the Database Daemon.
- Delete the MUD folder ~/Documents/Microsoft User Data/Office 2008 Identities/Main Identity/ if you are on 2004, is Office 2004 Identities
- *Note Rules, Signatures, Labels, etc. are lost.
- Empty the Trash.
- Open Entourage and reconfigure.
This process downloads a new clean copy of everything on the Exchange store with no loss of data. Again, Rules and Signatures have to be replaced. Also, to save a call back to the desk, confirm that you have set up the Contact: On My Computer: Me contact to the users proper identification. Most important if you use CCC/NetBoot/Scripting to build systems.
This has been happening significantly more lately as we have added more delegates to our executive staff. Another new variable is we have an office in NY and another in SF. This happens more frequently for the users in NY. This is much more common for users with delegates than those without.
Another note: the empty cache is great for resolving issues in a specific folder or calendar; no need to download the new database.
Final note: with very large databases and companies who lean heavily on Entourage and Exchange for business operations, you will see a significant increase in HDD failures on the Mac. Of our 80, we've sent over 30 back in a years time. This is about our fail rate for HDDs. I've conferred with other Mac shops, and they have nothing near this failure rate in HDDs.
Incidentally, we have a few SSDs and they experience the database corruption issues with the same frequency as those with HDDs.
Jeff Spears rebuilds the database by creating a new Identity:
I'm an admin for an Exchange and Entourage 2004/2008 environment, and I see all the same issues with bizarre behavior when the Entourage Database goes corrupt and is rebuilt. They key to this is to never rebuild your database using the Database Utility unless absolutely necessary. The ONLY reason you should ever use the Database Utility is to try and recover email that is not backed up and cannot be found anywhere else. Once you rebuild the database and open it, immediately drag the email out to save the email and trash that database. NEVER USE A REBUILT DATABASE! This can cause issues which may propagate to the server-side Exchange mailbox and cause your Entourage database to get corrupted much more frequently.
Whenever I have a user with a corrupt database the procedure is to trash that database and create a new one by creating a new Identity.
If you've tried these approaches
Readers rebut advice on Entourage database rebuilds |Top of Page |
Two readers disagree with advice from two other readers about Entourage database corruption. On Monday we posted reports from two readers who recommended against using Entourage Database Utility to rebuild Entourage databases. They suggested that this can lead to database corruption, as did other reports. ( (Alternatives to using Entourage Database Utility can avoid corruption.) Today, two readers disagreed with this advice.
Oliver Block sent this:
Regarding the post from Robert Liebsch regarding rebuilding Entourage databases. Here are two few points to consider:
- You should explicitly warn users that if they use POP accounts, deleting the Entourage identity will delete all their mail. That may seem obvious in a conversation that started about Exchange, but the first few paragraphs of that post do not even mention Exchange.
- There is no evidence to suggest that Entourage use is in any wayrelated to hard drive failures. This is speculation at best and really doesn't make any sense.
Luis Antezana sent in a rebuttal:
I'd just like to rebut the statements made today about not using the Entourage Database Utility to rebuild databases.
I occasionally have to rebuild a database here and there at our firm, and I always use the Utility and it is usually just fine. Once in a while it does something strange and then I rebuild it again, from the original, and it_s fine. To make such a broad statement as to advocate never using the Database Utility is total overkill.
If you're having this many problems you have something seriously wrong going on systematically. You just don_t get that many database problems under normal circumstances.
Also, in all my years working with Macs, I've had maybe three bad hard drives. If your failure rate is this high, it is another sign you have something else going on that_s wrong. Who knows what that is but I do not see anything near what you report. Could be your building_s power, who knows.
Importantly, everyone should take caution when following the advice to just dump a faulty Entourage database and rebuild from the Exchange store. Yes, the new data coming from Exchange is fresh and good, but if you have anything stored in Notes, Tasks, Address Book (On My Computer), or in Folders on My Computer, they will be totally lost, not to mention any categories, project designations, and Entourage links. A successful repair allows you to keep all this intact, and moves it into a new file anyway.
The one anomaly I will report is that sometimes the message will pop up that a database is damaged and should be repaired. Do go through the simple Verify process in Database Utility, as I have seen it return a No Repair Needed result, meaning the initial warning was in error, so you don_t have to worry about any if this Utility or no Utility business. And do try to keep those databases small, preferably under 4-ish if possible, to avoid database issues.
Lastly, I do notice the Database Utility doesn't always clean up after itself well. It occasionally leaves the old database in your Identity folder, or it creates a backup folder but that ends up creating a new Identity. Because I like to keep my Identities clean and lean, I create a backups folder within the Microsoft User Documents folder, inside which I store any recent database backup.
Have an opinion?
More on Entourage 2008 database problems
Eric Gatten echoes previous reports and fixes about frequent corruption in Entourage databases:
I have also seen the Entourage problem that John Underhill described on May 18, 2009. The client site is using an Exchange server. With the exception of a couple of clients, all the rest of my clients are not using an Exchange server and they are not experiencing this problem.
I have also had to delete the user's database and/or create a new user. The database sizes range from 1.8GB to 9.4GB. It appears that size does not matter.
TIP: Recovering Entourage messages from corrupt database
J. D. West expanded on a previous suggestion that works in recovering Entourage email that is deleted with a database rebuild:
Came across this and have just experienced the EXACT same thing. User got a "Database corrupt, need to rebuild" type message. Ran the rebuild, took several hours. Next morning I came in, all mail and contacts were DELETED from the Exchange mail server.
As suggested by Joshua Bierman on your page, we went into Outlook->Tools Menu->Recover Deleted Items, and they were there. Something in the sync process clearly triggered a delete command on the Exchange server.
The local Entourage database also had the emails, but something very strange was also there. Under the Inbox, at the very top, was a Brand New Folder, but the name was completely Blank. In this Blank Folder, were emails and subfolders, seemingly a copy of the Inbox, inside it's own folder. But, the subfolders, did not have any data in them. They were blank.
Very strange, not sure what happened but thanks for the website and this tip was a lifesaver!
Upate on Entourage 2008 Database Rebuild deleting emails, contacts, and calendar
George Zervakos commented on the story Handling Entourage 2008 Database Corruption, sharing his experience with the problem:
I came across this page on your site and read it with great interest. I had a user whose Entourage database was rebuilt. He had a roughly 4GB database, one IMAP account and one Exchange account. The database rebuild took a few hours, after which everything seemed to be working fine. A few days later, someone pointed out that his contacts went missing along with his calendar. Then I had a closer look and realized that the majority of the inbox and sent items were also missing. I traced back the deletions to the date of the database rebuild.
I had to have my email provider restore a point in time backup from before the database rebuild, luckily we caught this in time because the backups went back only 7 days. I had the user quit Entourage and use a browser for any new email. In the meantime, I had to reconcile the restored data with the current data for the calendar, contacts, and entire mailbox which went back three years. This I did by connecting to the mailbox with Outlook in NON-cached mode as the mailbox size was now about ~8GB (the restore pretty much doubled it in size). I decided I would archive the majority of things older than 2010 and dumped that data into a pst. I managed to get the entire mailbox down to 700MB.
Once that was done, I took Entourage and moved out the current Identities and completely reconfigured Entourage from the start, IMAP and Exchange. The resync took a few hours but it worked and now everything is fine.
I can't help but wonder though how and why a database rebuild would start deleting things off the server. It's not very reassuring to know that one of my users is out there with the potential to do severe damage to his mailbox and not really even know about it.
Aside from backing up the Entourage database, is there any other sensible way to prevent this other than telling my user to never, ever rebuild a database and to let me simply reconfigure Entourage should this happen again?
||Snow Leopard Server for Dummies
By John Rizzo
A 432-page book that simplifies the installation, configuration, and management of Apple's Mac OS X 10.6 Server software. Support Mac and Windows clients for file sharing, email, and directory services; Incorporate a Mac subnet into a Windows Active Directory domain, manage Mac and Windows clients, and configure security options, and more. Click here for more.
On this Page: