What does HackerNews think of data-transfer-project?

Pretty much everybody agrees that you need to backup your cloud storage as well as your local computer, and Apple even backs up your i-devices to the cloud, and yet, there is no automated way of backing up your iCloud storage.

About a decade ago, Google initiated the Data Transfer Framework[1] that allows you to transfer data from one cloud provider to another, directly from provider to provider instead of downloading it first. It sadly appears to not have gotten enough traction to be of any use.

[1]: https://github.com/google/data-transfer-project

I've moved everything to the cloud. I have nothing running at home except networking gear, and a small "server" that pulls nightly backups from the clouds to a local USB drive.

In theory i could probably do without the local server if looking at Apple/Microsoft/Google data redundancy (Microsoft is multi geo, i can't figure out what Apple is).

Sadly i need to guarantee that some random account closure doesn't remove all my data, so the backup server stays for now. The way cloud prices are going, it will only be a question of months/years before it's cheaper/easier to just utilize two cloud services, one for main storage and one for backup storage, and with projects like the data transfer project [1], you don't even need to download them first.

[1] https://github.com/google/data-transfer-project

https://github.com/google/data-transfer-project

https://datatransferproject.dev/

The Data Transfer Project: https://news.ycombinator.com/item?id=17574707

137 points l2dy 4 days ago 50 comments

https://datatransferproject.dev/

--

The Data Transfer Project: https://news.ycombinator.com/item?id=17580502

97 points cududa 3 days ago 47 comments

https://github.com/google/data-transfer-project

I wanted to provide my thinking on some of these very valid wories,

Re: Copy vs. Move: This was a conscious choice that I think has a solid backing in two things: 1) In our user research for Takeout, the majority of users who user Takeout don't do it to leave Google. We suspect that the same will be true for DTP, users will want to try out a new service, or user a complementary service, instead of a replacement. 2) Users should absolutely be able to delete their data once they copy it. However we think that separating the two is better for the user. For instance you want to make sure the user has a chance to verify the fidelity of the data at the destination. It would be terrible if a user ported their photos to a new provider and the new provider down-sampled them and originals were automatically deleted.

Re: Scraping Its true that DTP can use API of companies that are 'participating' in DTP. But we don't do it by scraping their UIs. We do it like any other app developer, asking for an API key, which that service is free to decline to give. One of the foundational principals we cover in the white paper is that the source service maintain control over who, how, and and when to give the data out via their API. So if they aren't interesteed in their data being used via DTP, that is absolutely their choice.

Re: Economics As with all future looking statements we'll have to wait and see how it works out. But I'll give one antidote on why I don't think this will happen. Google Takeout (which I also work on) allows users to export their data to OneDrive, DropBox, and Box (as well as Google Drive). One of the reasons we wanted to make DTP is we were tired of dealing with other peoples APIs, as it doesn't scale well. Google should build adapters for Google, and Microsoft should build adapters for Microsoft. So with Takeout we tried the specialized transport method, but it was a lot of work, so we went with the DTP approach specifically to try to avoid having specialized transports.

DTP is still in the early phases, and I would encourage you, and everyone else, to get involved in the project (https://github.com/google/data-transfer-project) and help shape the direction of the project.