Various Documentation

Universal Feeds & Packages

Universal Package Feeds were designed to be provide immediate, out-of-the-box functionality for most packaging needs, and also to serve as a platform that you can easily extend or interact with.

Universal Package Format

There's not much to a universal package file; at a minimum it:

  • is a ZIP archive format with a .upack file extension
  • has a metadata file at the root level of archive named upack.json
  • contains any number of files in the /package directory; these files are the files that will get "unpackaged" by the client or server

You can add any number of files or directories outside of these minimal requirements - and in fact, that's exactly what it's designed for!

In the extremely unlikely event that an future version of this specification will require or uses additional files (we can think of absolutely no reason this would ever be required), then these will be opt-in and specified using the metadata file, or potentially use a .upack2 extension.

Package Metadata Specification

The upack.json file is a JSON object with the following properties:

Property Format
group A string of zero to fifty characters: numbers (0-9), upper- and lower-case letters (a-Z), dashes (-), periods (.), forward-slashes (/), and underscores (_); may not start or end with a forward-slash; if not specified, the group name will be considered an empty string.
nameR A string of one to fifty characters: numbers (0-9), upper- and lower-case letters (a-Z), dashes (-), periods (.), and underscores (_).
versionR A string representing a Semantic Version; this is a three-part, dot- specification.
title A string of no more than fifty characters
icon A string of an absolute url pointing to an image to be displayed in the ProGet UI (at both 64px and 128px); if package:// is used as the protocol, ProGet will search within the package and serve that image instead.
description A string containing any number of charters; these will be formatted as Markdown in the ProGet UI.
dependencies An array of strings, each consisting of a package identification string; this string is formatted as follows:
  • «group»:«package-name»
  • «group»/«package-name»:«version»
When the version is not specified, the latest is used.

An R denotes a required property, and the object may contain additional properties as needed. However, if you need to add additional metadata, it's strongly recommended that you prefix these properties with an underscore (_), or using a separate file altogether... just on the off-chance that a property you add will exist in a future version of the specification, or is returned in other metadata queries.

  • Example: Minimal
  • Example: All Fields
  • Example: Extended File
{
 "name": "HDARS",
 "version": "1.3.9"
}
{
 "group": "initrode/vendors/abl",
 "name": "ABLast",
 "version": "2.2.1",
 "title": "ast distribution files for ABL",
 "icon": "package://ablast.svg",
 "description": "This contains [ast distro](http://initrode-net.local/ast) files specific to ABL",
 "dependencies": [ "initrode/vendors-common:ast-common:2.0.0" ]
}
{
 "group": "virtudyne/simdesk",
 "name": "var-index-service",
 "version": "5.3.9",
 "_sourceRoot": "$/global/vindex/branches/v5-hotfix"
 "_deployTarget": "/var/vsimdesk/vindex"
}

Note that a package is uniquely identified by (group, name, version); package groups are an optional feature, and if you don't specify or use them, then all of your packages will be considered in an empty group.

Universal Feed API

Like the package format, this was designed with simplicity in mind. Because universal packages are designed to be consumed by any language or platform, the API offers several different ways to do the same thing, as some operations are much more difficult in some languages than others.

The API consists of a few different URL endpoints, accessible over HTTP/S. If configured, they can be secured using Integrated Windows Authentication or Basic Authentication using whatever granular feed- or system-level privileges needed.

In ProGet, all endpoints are prefixed with /upack/«feed-name»; note that the endpoint prefix itself is not a valid API endpoint, and may return some sort of "this is not the endpoint you are looking for" Star Wars memepic, or simply redirect you to this page.

Currently, the API only returns results in JSON format with a standard, 200 (success) status code unless there's an error; future versions (if anyone requests it) may add additional formats, such as XML, which would be specified using a Content-Type request headers and/or an alternate querystring parameter.

All searching and matching is case insensitive. This will most certainly never change, as it's either a mistake or a bad practice to have different packages named SomeThing and someThing.

List Packages Endpoint

GET /upack/«feed-name»/packages?group=«group»&name=«name»&count=«count»

Returns either a JSON object (if name is specified) or a JSON array of objects of package metadata, mostly from the latest package version.

ParameterDescription
groupOptional. If specified without name, returns an array of packages with a matching group name or an empty array.
nameOptional. If specified, returns an object with a matching name and group or a 404 status with an error message in the body.
countOptional. If specified, returns an array with at most as many entries as specified; otherwise, at most 1000 packages are returned. This is ignored if name is specified.

Following are some example request/responses:

  • Single Package
  • All Packages
  • Packages in a Group

GET /upack/dev-feed/packages?name=hdars

{
 "name": "HDARS",
 "downloads": "1",
 "versions": ["0.0.22", "1.3.9", "1.3.10"]
}

GET /upack/dev-feed/packages?count=1000

[
  {
   "name": "HDARS",
   "latestVersion": "1.3.9",
   "downloads": "1",
   "versions": ["0.0.22", "1.3.7", "1.3.9"]
  },
  {
   "group": "initrode/vendors/abl",
   "name": "ABLast",
   "latestVersion": "2.2.1",
   "title": "ast distribution files for ABL",
   "icon": "package://ablast.svg",
   "description": "This contains [ast distro](http://initrode-net.local/ast) files specific to ABL",
   "downloads":"55",
   "versions": ["2.2.1"]
  },
  {
   "group": "virtudyne/simdesk",
   "name": "var-index-service",
   "latestVersion": "5.3.10",
   "_sourceRoot": "$/global/vindex/branches/v5-hotfix"
   "_deployTarget": "/var/vsimdesk/vindex",
   "downloads":"17",
   "versions": ["5.0.0","5.2.1","5.3.10"]
 }
]

GET /upack/dev-feed/packages?group=initrode/vendors/abl

[
  {
   "group": "initrode/vendors/abl",
   "name": "ABLast",
   "latestVersion": "2.2.1",
   "title": "ast distribution files for ABL",
   "icon": "package://ablast.svg",
   "description": "This contains [ast distro](http://initrode-net.local/ast) files specific to ABL",
   "downloads":"55",
   "versions": ["2.2.1"]
  }
]

To return the packages in the empty group, specify the group parameter without a value (e.g. /upack/dev-feed/packages?group=)

Note that specifying the name parameter will cause an object to be returned instead of an array; if you don't specify a group, then packages in the empty group will be searched

Note that the group parameter must be a full match; future versions of this endpoint may allow for a sub-group searching (such as groupName* or something).

List Versions Endpoint

GET /upack/«feed-name»/versions?group=«group»&name=«name» &version=«version»&includeFileList=«includeFileList»&count=«count»

Returns either a JSON object or a JSON array of objects containing metadata about specific package versions.

ParameterDescription
groupOptional. If specified without version, returns an array of packages with a matching group name or an empty array.
nameOptional. If specified without version, returns an array of packages with a matching group name or an empty array.
versionOptional. If specified, returns an object with a matching group, name, and version, or a 404 status with an error message in the body.
includeFileListOptional. If true, then inspects each package returned and includes a list of files in the body.
countOptional. If specified, returns an array with at most as many entries as specified; otherwise, at most 1000 packages are returned. This is ignored if name is specified.

Following are some example request/responses:

  • Single Version
  • All Versions
  • Single Version with Files

GET /upack/dev-feed/versions?name=hdars&version=1.3.9

{
 "name": "HDARS",
 "version": "1.3.9",
 "downloads": "1",
 "published": "2016-01-07T06:51:51.403Z",
 "isLocal": true,
 "isCached": false
}

GET /upack/dev-feed/versions?count=1000

[
  {
   "name": "HDARS",
   "version": "0.0.22",
   "downloads": "133",
   "published": "2016-01-01T06:51:51.403Z",
   "isLocal": true,
   "isCached": false
  },
  {
   "name": "HDARS",
   "version": "1.3.7",
   "downloads": "21",
   "published": "2016-01-05T06:51:51.403Z",
   "isLocal": true,
   "isCached": false
  },
  {
   "name": "HDARS",
   "version": "1.3.9",
   "downloads": "1",
   "published": "2016-01-07T06:51:51.403Z",
   "isLocal": true,
   "isCached": false
  },
  {
   "group": "initrode/vendors/abl",
   "name": "ABLast",
   "version": "2.2.1",
   "title": "ast distribution files for ABL",
   "icon": "package://ablast.svg",
   "description": "This contains [ast distro](http://initrode-net.local/ast) files specific to ABL",
   "dependencies": [ "initrode/vendors-common:ast-common:2.0.0" ]
   "downloads":"55",
   "isLocal": false,
   "isCached": true
  },
  {
   "group": "virtudyne/simdesk",
   "name": "var-index-service",
   "version": "5.0.10",
   "_sourceRoot": "$/global/vindex/root"
   "_deployTarget": "/var/vsimdesk/vindex/newv5",
   "downloads":"18",
   "isLocal": true,
   "isCached": false
  },
  {
   "group": "virtudyne/simdesk",
   "name": "var-index-service",
   "version": "5.2.1",
   "_sourceRoot": "$/global/vindex/branches/5.2"
   "_deployTarget": "/var/vsimdesk/vindex/2tmp",
   "downloads":"18",
   "isLocal": true,
   "isCached": false
  },
  {
   "group": "virtudyne/simdesk",
   "name": "var-index-service",
   "version": "5.3.10",
   "_sourceRoot": "$/global/vindex/branches/v5-hotfix"
   "_deployTarget": "/var/vsimdesk/vindex",
   "downloads":"17",
   "isLocal": true,
   "isCached": false
  }
]

GET /upack/dev-feed/versions?name=hdars&version=1.3.9&includeFileList=true

{
 "name": "HDARS",
 "version": "1.3.9",
 "downloads": "1",
 "published": "2016-01-07T06:51:51.403Z",
 "isLocal": true,
 "isCached": false,
 "fileList": [
    { "name": "assets/muth.png", 
      "date": "2015-01-01T01:52:51.403Z", 
      "size": 68012 },
    { "name": "assets/styles.css", 
      "date": "2015-01-01T01:52:51.403Z", 
      "size": 3021 },
    { "name": "index.htm", 
      "date": "2015-01-01T01:52:51.403Z", 
      "size": 1028 }
 ]
}

Note that the group parameter must be a full match; future versions of this endpoint may allow for a sub-group searching (such as groupName* or something), if anyone requests.

Download Package Endpoint

Specific Version

GET /upack/«feed-name»/download/«group-name»/«package-name»/«package-version»?contentOnly=«zip|tgz»

Latest Version

GET /upack/«feed-name»/download/«group-name»/«package-name»?contentOnly=«zip|tgz»&latest

Returns either a package file, the contents of a package, or an error.

ParameterDescription
group-nameOptional. If not specified, the empty group will be searched.
package-nameRequired.
package-versionOptional. If a specific version is not specified, "latest" must be supplied as a query string argument, or a 400 will be returned.
contentOnlyOptional. If specified, the contents of /package directory are returned as either a ZIP archive (default if no value is specified for the parameter) or TGZ archive.

Following are some example request/responses:

  • Specific Package
  • Latest Contents as TGZ

GET /upack/dev-feed/download/hdars/1.3.9

Response:

  • Header: Content-Type: application/zip
  • Header: Content-Disposition: attachment; filename=hdars.1.3.9.upack
  • Body: a universal package file for HDARS 1.3.9

GET /upack/dev-feed/download/initrode/vendors/abl/var-index-service?contentOnly=tgz&latest

Response:

  • Header: Content-Type: application/x-compressed
  • Header: Content-Disposition: attachment; filename=var-index-service.4.2.0.tgz
  • Body: a GZipped TAR containing the contents of the /package directory of the latest var-index-service package

Upload Package Endpoint

PUT or POST /upack/«feed-name»/upload«...»

There are quite a few ways to access this endpoint, but the end result is the same: it adds or replaces a package in a feed. Because there are so many permutations of how you can use this, it's easiest to specify the various options and behaviors instead.

First and foremost, consider that a complete package consists of required metadata and content (arbitrary files and directories). This endpoint is designed to allow you to upload a complete, pre-built package, or upload a partial package with content and metadata you specified using path, query, form-encoded, and/or JSON parameters.

No duplicate parameters. That's the second thing to consider. For example, if you specify a different package name in both the query and path... you'll get an error (400).

Content Type

The Content-Type header can be any of the following:

  • application/json - properties on the JSON object will be used for content and metadata parameters
  • application/x-www-form-urlencoded - the key/value pairs will be used for content and metadata parameters
  • application/zip - the request body will treated either as content or a partial package

Using application/zip Content-Type

You must send the raw bytes of a ZIP file as the body of your request. If the archive doesn't conform to the universal package format, ProGet will convert it for you, if you supply the required metadata via query string parameters.

If the archive is already in the .upack format, you can specify additional metadata paramters via the querystring.

Metadata Parameters

Any of the following parameters fields may be specified through querystring or content; the format must follow a valid metadata format specification.

Parameter Description
content-b64 A string representing the contents as a base64-encoded ZIP archive; this is not valid with application/zip Content-Type, and will be considered duplicative if content-url is specified
content-url A url where content can be downloaded from as a ZIP archive; this is not valid with application/zip Content-Type, and will be considered duplicative if content-b64 is specified
group This may also be specified as the first path following the endpoint
nameR This may also be specified as either the last or second-to-last path
versionR This may also be specified as either the last path
dependencies When specified in JSON, it should be an array; otherwise (querystring or form format), it should be a comma-separated string of package identifiers
anything else If any other parameter is specified (including the well-defined title, icon, description), it will be added as a package metadata property.

A 201 is returned for all valid requests; following are some example requests

  • PUT
  • cUrl
  • OtterScript
  • PowerShell

PUT /upack/dev-feed/upload/initrode/vendors/abl/var-index-service?version=5.3.9

Content-Type: application/json

{
 "content-url": "http://sdbuildsv1/latest-stable?project=vindex&branch=v5-hotfix",
 "_sourceRoot": "$/global/vindex/branches/v5-hotfix",
 "_deployTarget": "/var/vsimdesk/vindex"
}

curl PUT http://proget.server/upack/dev-feed/upload --upload-file hdars.upack

# This uses the regular Upload-Http operation
Upload-Http content.zip
(
   Url: http://progetsv/upack/Extensions-Dev/upload?name=$UrlEncode($ExtensionName)&group=${ProductName}&version=$UrlEncode($ReleaseNumber)
);

# There is also an operation specifically for this that allows you to securely specify credentials
ProGet::Push-Package content.upack
(
  Credentials: PGExtensionDev,
  Version: $ReleaseNumber
)
$bytes = [Text.Encoding]::UTF8.GetBytes('USERNAME:PASSWORD')
$creds = 'Basic ' + [Convert]::ToBase64String($bytes)

$query = (`
  '?name = 'PACKAGE_NAME' + `
  '&version = $Version.ToString() + `
  '&title = 'PACKAGE_TITLE' + `
  '&description = [uri]::EscapeDataString('PACKAGE_DESCRIPTION') );


Invoke-RestMethod -Method Put `
   -Uri ('http://PROGET_HOST/upack/FEED_NAME/upload' + $query) `
   -ContentType 'application/zip' `
   -Body ([IO.File]::ReadAllBytes($pathToZipFile)) `
   -Headers @{ Authorization = $creds }

Delete Package Endpoint

DELETE or POST /upack/«feed-name»/delete/«group-name»/«package-name»/«package-version»

Deletes the specified package (if group-name is omitted, then the empty group is used), returning a 200 on success or 404 if not found.

We don't know what else to document here, because this seems quite straightforward of a request, but just let us know if you think we should add anything else.

GET /upack/«feed-name»/search?query=«search-query»&count=«max-items»

This is the what the ProGet Web Application uses to show results to users, and returns a list of packages similar to the packages endpoint.

You're welcome to use it, but it's unsupported because its behavior is not at all documented, and we really can't think of a use-case outside of the ProGet Web Application. Don't hesitate to contact us if this is an endpoint you would find helpful, and we can either document it better or make it more useful outside of the ProGet UI.

Download Package File Endpoint

The download-file endpoint is scheduled for ProGet v4.9, and may become available in a maintenance release of v4.8

Specific Version

GET /upack/«feed-name»/download-file/«group-name»/«package-name»/«package-version»&path=«path»

Latest Version

GET /upack/«feed-name»/download-file/«group-name»/«package-name»?latest&path=«path»

Returns either a package file or an error.

ParameterDescription
group-nameOptional. If not specified, the empty group will be searched.
package-nameRequired. If not found, a 404 is returned.
package-versionOptional. If not specified, "latest" must be specified as a query string argument. If no version is specified and "latest" is not supplied, a 400 is returned. If no matching versions are found, a 404 is returned.
pathRequired. Relative path to the file within the package. Note, package contents are contained within the "package" directory of the package, so most request paths should start with "package" unless files in the package root are desired, such as the upack.json file. If the file is not found within the package, a 404 is returned. If a directory name is specified, a 403 is returned.

Following are some example request/responses:

  • Package File
  • Latest upack.json File

GET /upack/dev-feed/download-file/hdars/1.3.9?path=package/example.txt

Response:

  • Header: Content-Type: text/plain
  • Header: Content-Disposition: attachment; filename=example.txt
  • Body: the contents of the example.txt file in the package contents

GET /upack/dev-feed/download-file/hdars?latest&path=upack.json

Response:

  • Header: Content-Type: application/json
  • Header: Content-Disposition: attachment; filename=upack.json
  • Body: the contents of the "hdars" package's upack.json file