tinyapps.org / blog


Backing up a large Gmail / G Suite account #

is theoretically possible via Google Takeout, but there are issues:

Got Your Back (GYB) does not suffer from these issues; messages are downloaded as individual EML files, sorted into folders by year, month, and day:

$ gyb --email name@example.com --action estimate

Select the actions you wish GYB to be able to perform for name@example.com

[ ]  0)  Gmail Backup And Restore - read/write mailbox access
[ ]  1)  Gmail Backup Only - read-only mailbox access
[ ]  2)  Gmail Restore Only - write-only mailbox access and label management
[*]  3)  Gmail Full Access - read/write mailbox access and message purge
[ ]  4)  No Gmail Access

[ ]  5)  Groups Restore - write to G Suite Groups Archive
[*]  6)  Storage Quota - Drive app config scope used for --action quota

      7)  Continue
7

Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth...

If your browser is on a different machine then press CTRL+C and
create a file called nobrowser.txt in the same folder as GYB.

Authentication successful.
Got 67100 Message IDs                                                           
GYB needs to examine 67100 messages
GYB already has a backup of 0 messages
GYB needs to estimate 67100 messages
Estimated size 26.47gb 67100/67100 messages 

$ gyb --email name@example.com --action backup

Using backup folder GYB-GMail-Backup-name@example.com
Got 67100 Message IDs                                                           
GYB needs to examine 67100 messages
GYB already has a backup of 0 messages
GYB needs to backup 67100 messages
backed up 67100 of 67100 messages                                               

GYB needs to refresh 0 messages

If desired, import the collection into Thunderbird via ImportExportTools.

If you need to copy the massive collection of EML files, tar it without compression to save a LOT of time (pv is of course optional, but very handy when tarring tens of thousands of files):

$ tar -c ./GYB-GMail-Backup-user\@example.com/ | pv -s $(gdu -sb ./GYB-GMail-Backup-user\@example.com/ | awk '{print $1}') > user\@example.com.tar
4.03GiB 0:00:23 [ 177MiB/s] [====>                             ] 15% ETA 0:02:08

* Attempting to split a ~26GB MBOX file by label via mbox_split.py or WB Optimum's script on which it is based did not avail.

UPDATE: Royce Williams kindly informs us of outtake, which "sync[s] Gmail to maildir...quickly", "uses the Gmail API to efficiently sync only deltas", and "can also be limited to a specific label". Crafted in Go by Google software engineer, Daniel Margolis. (On a related note, Evolution is swell for browsing and searching Maildir directories; add via Edit > Preferences > Mail Accounts > Add > Next > enter required info and deselect "Look up mail server details based on the entered e-mail address" > Next > for "Server Type:" select "Maildir-format mail directories" > select the "Mail Directory:" location, etc.)

/misc | Sep 25, 2018


Subscribe or visit the archives