A client that uses AccountEdge accounting software along with Shopify recently had a bad case of data loss that took some clever thinking to recover from.
Their primary product database exists in AccountEdge – product names, SKUs, costs & pricing, inventory – but metadata such as descriptions, images, SEO content, and Shopify metafields all only existed at Shopify. We use metafields to store labels and descriptions about the product options, without which customers can't make a purchase.
A typo in the name of a product variant within AccountEdge caused it to suddenly see all Shopify products as unknown products. The next time the app was launched and attempted to sync with Shopify, it deleted every product and recreated it with the new typo-variant. The recreated products had no images, metafields, etc., rendering the entire ecommerce platform unusable. This client had subscribed to a Shopify backup app, but after running a restore we discovered 2 big issues:
After several back-and-forths with the backup service, they were ultimately unable to provide any metafield data and we had to form a new plan of attack.
Our new strategy:
collectssaved to a local MongoDB.
Once the details of the process were worked out and the proof-of-concept was prototyped that utilized Shopify's API, the rollout was relatively quick. It took only seconds to grab all collections and thousands of collects, and about 20 minutes to grab several thousand products and copy all of their images. Restoring took a little bit longer, as each product PATCH takes anywhere from 200-2,000ms and Shopify's rate limits aren't negligible. The site was back up and running, and the client suffered a minimal amount of downtime.
As a bonus, we now have an in-house Shopify backup solution that includes all content (theme, pages, products, collections, orders) along with critical product data (images, metafields, other metadata). We can store this data in a JSON file, SQLite, MongoDB, or anything else that a client's use case requires. More importantly, the data is accessible and can be easily browsed or restored to guarantee its integrity.
This was a relatively elegant and interesting solution to a catastrophic problem. I don't want to share all of the code here since it's tailored specifically to this client, but if you're in a position where you need some assistance with something like this feel free to contact me.
Lesson learned: a backup is only as good as a test restore has proven it to be.