I first read about this last Friday. Beloved cloud CRM solution had a data breach, and fixed it by removing all security features.
It seems the US tech giant granted Pardot customers access privileges they should not have, which is to say access to everything and alter any data. As was observed on Reddit, "One of our projects had all its profiles modified to enable modify all, allowing all users access to all data."
To deal with the mess, Salesforce's IT team has denied all access to more than 100 cloud instances that host Pardot users, shutting out everyone else using those same systems, whether or not they were using Pardot.
And the outage continued on Monday. Over the weekend, Salesforce staff developed, tested, and ran a script that attempted to restore user permissions from backups, though this was not always successful. In some cases, it even went backwards, and regranted full read-write permissions to users.
Timeline
- A dodgy database script on Friday gave all current and past Pardot users sysadmin-level create-read-write-delete access to all data, which is a huge security and privacy blunder.
- Instances running Pardot were disabled to avoid any data theft or tampering, which kicked any customers using those instances – Pardot and non-Pardot – off Salesforce.
- Salesforce removed all access permissions from affected users and restored full permissions to customers' administrator accounts, allowing them to repair users' permissions. Some instances were fired back up so folks could log back in, others remained offline. Admins were encouraged to reassign their users' access permissions by hand to allow users to continue working.
- On Saturday, Salesforce was able to restore previous user permissions from a backup using a script on one instance of 105 affected.
- By Sunday, the permission restoration script had been run on the majority of instances, repairing user permissions with an "89 per cent" success rate – meaning about one in ten organizations still had their user rights wiped. Admins were told, if they still had permission issues, to fix it themselves or contact Salesforce support.
- On Monday, Salesforce staff are still working to restore sandboxed instances. We're told GovCloud was not affected by the snafu.
So if you are contemplating moving to the cloud, now will be…a good time ?
