What does HackerNews think of VFSForGit?

Virtual File System for Git: Enable Git at Enterprise Scale

Language: C#

We put half a million files in one Git repository (2022) | Aug 2023

> Are there some fundamental aspects of git that would make it either very difficult to improve that, or that would sacrifice some important benefits if they were made?

I can't speak to improving git, but I think some light on this area can be shed by Linus' tech talk at Google in 2007.

1. Linus says there's a specific focus on full history and content, not files ... so it's a deliberate, different axis of focus than file count:

https://youtu.be/4XpnKHJAok8?t=2586

... AND it's a specific pitfall to avoid when using Git:

https://youtu.be/4XpnKHJAok8?t=4047

2. As Linus tells it, Git appears to be designed specifically for project maintenance while not getting in the way of individual commits and collaboration. But the global history and more expensive operations on things like "who touched this line" are deliberate so lines of a function are tracked across all moves of the content itself.

Maintainer tool enablement: https://youtu.be/4XpnKHJAok8?t=3815

Content tracking slower than file-based "who touched this": https://youtu.be/4XpnKHJAok8?t=4071

===

I have no answer, but ...

Practically, I've used lazy filesystems both for Windows-on-Git via GVFS [1][2] and Google's monorepo jacked into a mercurial client (I think that's what it is?). Both companies have made this work, but as Linus says, a lot of the stuff just doesn't work well with either system.

Windows-on-Git still takes a lot of time overall, and stacking > 10 patches of an exploratory refactor with the monorepo on hg starts slowing WAY WAY down to the point where any source control operations just get in the way.

[1] https://devblogs.microsoft.com/devops/announcing-gvfs-git-vi...

[2] https://github.com/microsoft/VFSForGit

We put half a million files in one Git repository (2022) | Aug 2023

Expand Context ↕

VFS for Git is still Open Source: https://github.com/microsoft/VFSForGit

Microsoft's blog posts have indicated a move to use something as close to off-the-shelf git as possible, though. They say they've stopped using VFS much and are instead more often relying on sparse checkouts. They've upstreamed a lot of patches into git itself, and maintain their own git fork but the fork distance is generally shrinking as those patches upstream.

Why SQLite does not use Git (2018) | Jul 2023

Expand Context ↕

https://github.com/microsoft/VFSForGit

better than it used to, with the caveat that git in particular is not and has never claimed to be good at versioning blobs.

Committing changes to a 130GB Git repository without full checkouts [video] | May 2023

Looks like you've just reinvented GVFS (https://github.com/microsoft/VFSForGit) for a specific use case? Or is this just a partial clone? Or a shallow clone? Or both? It's unclear from the video if this is 130 GB of current state at the branch head or 130 GB of commit history.

FYI: LLVM-project repo has exceeded GitHub upload size limit (2022) | Jan 2023

Expand Context ↕

This is where something like VFSForGit [0] helps out. Instead of cloning the entire repo, it creates a virtual file system and fetches objects on demand. MSFT uses it internally for the Windows source tree (which now exceeds 300GB).

[0]: https://github.com/microsoft/VFSForGit

WebKit on GitHub | Aug 2022

Expand Context ↕

I was just looking at Microsoft's git VFS (https://github.com/microsoft/VFSForGit), which is deprecated and now points to Scalar (https://github.com/microsoft/scalar), which is also deprecated I think? What's Microsoft's story with git for large repos now? Is there still a virtual file system involved at all?

Improve Git monorepo performance with a file system monitor | Jun 2022

Expand Context ↕

Have you seen https://github.com/microsoft/VFSForGit? It's used by the Windows team to manage a monorepo containing most Windows source code.

Unfortunately that approach was put in maintenance mode since it didn't seem like it would be supportable on macOS.

We Put Half a Million Files in One Git Repository, Here’s What We Learned | Jun 2022

Did you consider VFS For Git and Scalar from Microsoft? What was the result? https://github.com/microsoft/VFSForGit

https://github.com/microsoft/scalar

Eden | Apr 2022

Expand Context ↕

First it was https://github.com/microsoft/VFSForGit then replaced with https://github.com/microsoft/scalar and then into their git fork https://github.com/microsoft/git

Eden | Apr 2022

Expand Context ↕

Source Depot is what it was called. IIRC, Windows was moving to Git (through some virtual file system [0]) and had some major teams actively using it, but that was a while ago too.

[0] https://github.com/microsoft/VFSForGit

Eden | Apr 2022

Expand Context ↕

https://github.com/microsoft/VFSForGit

Which seems to have been superseded by https://github.com/microsoft/scalar according to the README.

Gitfs: Version Controlled File System | Aug 2021

How does this compare with Microsoft's "VFS for Git"?

https://github.com/microsoft/VFSForGit

Improving large monorepo performance on GitHub | Mar 2021

When is GitHub going to finally add support for Microsoft’s VFSforGit?

https://github.com/microsoft/VFSForGit

https://vfsforgit.org/