About ProGet Legacy Feeds

KB#1094: Last Updated Nov 24, 2014

ProGet 3.3 changes the way NuGet feeds are stored and indexed. If you have upgraded from an earlier version, you will see a notice on existing feeds about migrating to the new type. For instructions on performing this migration, please see KB#1092. This article describes the changes that were made and the reason for the changes.

How Legacy Feeds Work

NuGet feeds in ProGet were originally designed to just be as simple as possible - just a directory on the file system that users could copy packages to so they would appear in a feed. Unfortunately this simplicity can start to be a problem when there are large numbers of packages in a feed (especially if there are also very large packages in the feed), since in order to maintain consistency, ProGet has to periodically verify that all of the packages on disk match up with the indexed data in the database. In order to limit the number of full scans that need to be performed, the ProGet service used a file system watcher to be notified of changes to feed directories, but these watchers also become somewhat erratic with large numbers of files.

In short, this design was perfectly adequate for ProGet's beginnings as a very simple package repository, but its usage profile today requires a different approach.

Changes in New Feeds

Starting in ProGet 3.3, feed package stores (directories, by default), are meant to be managed only by ProGet; users should no longer be copying packages directly into a feed. This enables the service to perform consistency checks much less frequently.

Feeds are now also referenced on disk using the internal feed index instead of the feed name. This means that feeds can now be renamed without having to manually move all of the packages and reindex everything. Beyond this, we also tried to make the directory structure a little more sensible. Directories are created for each package ID, so everything isn't stored in one giant folder.

Bulk Imports

In practice, we've found that very few of our users were even aware that you could just copy a bunch of new packages directly into your feed to add them, but we felt that this was an important usage pattern to support. It's one of those features that may be rarely used, but is very nice to have when it is needed.

You will notice a new field on the Edit Feed page called bulk import drop path. This defines a path either somewhere on the ProGet server (or a file share of course) that ProGet's service will scan for .nupkg files. Any NuGet packages which it finds will be moved into the feed, just as if they had been added via the NuGet client or ProGet web UI. Note that the ProGet service account must have read/write access to this directory.

Feed Cleanup

For newer feeds there is now a notion of a feed cleanup task. This is similar to the reindex/rescan operations in legacy feeds, but is performed less frequently and has better exposure in the UI. By default this will occur once a day, and you can change the next scheduled cleanup time on the Edit Feed page. Because a cleanup can reduce performance while it is occurring, there is also a message displayed on the Browse Feed page if there is a cleanup in progress, as well as a link to a page showing cleanup progress and detailed logging information.