Various Documentation

Universal Package Format

There's not much to a universal package file; at a minimum it:

  • is a ZIP archive format with a .upack file extension
  • has a manifest file at the root level of archive named upack.json with a few required field
  • contains any number of files in the /package directory; these files are the files that will get "unpackaged" by the client or server

You can add any number of files or directories outside of these minimal requirements - and in fact, that's exactly what universal packages are designed for.

In the extremely unlikely event that a future version of this specification will require or uses additional files (we can think of absolutely no reason this would ever be required), then these will be opt-in and specified using the manifest, or potentially use a .upack2 extension.

Package Identification

A universal package can be uniquely identified it's group, name, and version. These are different properties in the manifest file.

In some cases, you will need to refer to a package with a single string. In this case, the group and name are combined into a single string, separated by a forward-slash, and the version is followed (separated by a colon). If a package is not in a group, then it's considered in the empty group, and that slash is not used. For example:

  • HDARS:1.3.9 is version 1.3.9 of the HDARS package (which is in the empty group)
  • initrode/vendors/abl/ABLast:2.2.1-rc.1 is rc.1 prerelease of v2.2.1 of ABLast in the "initrode/vendors/abl" group

You should avoid really generic names for your packages, like "Utils" or "Tool". Instead, consider InitechUtils or InitrodeTool.

You may also append a colon and then a hash string to the end of a package. This will be used for verification purposes when possible. For example:

  • HDARS:1.3.9:fca66ce2a8ceea2d651eecf2369d4072d1871aec
  • initrode/vendors/abl/ABLast:2.2.1-rc.1:5b31eaa26d0c6e7bb985f740dbceed854293c369

Validation & Security

A universal package is meant to be read-only and once created, it's contents and metacontents sealed within the package, untampered. But it's a zip file, which means anyone can tamper with the file using nothing more than a zip editor.

This is where cryptographic hashing comes in. It is a small string of text that acts as a "thumbprint" of a file and lets you verify that, after you've downloaded "Accounts/HDars v1.3.4" from a package source, you can be certain it's the file you expect.

Because a package's hash is calculated from the bytes of the package file, it means that it's impossible to store a package's hash inside of that package, since changing the package would change it's hash. This is why you must also have a trusted package source tell you the hash of the package.

However, a package manifest file may reference other packages' hashes in the dependencies and repackageHistory properties.

Package Hash Strings

You can use the following rules to determine how to validate a package hash string against a file, or generate a hash of your own package.

  • 40 characters - sha-1
  • 64 characters - sha-256
  • 128 characters - sha-512
  • 73 characters and beginning with "SHA3-256:" - SHA3-256, beginning after the colon
  • 137 characters and beginning with "SHA3-512:" - SHA3-512, beginning after the colon
  • other - invalid or unknown hash

All Inedo-managed UPack tools (including ProGet and Hedgehog) currently only support sha-1.

Repackaging & Auditing

Packages work at their best when they are read-only and cryptographically verified, but this means that you can't change any of the package metadata (including the version number) without "tampering" with the package.

For example, if you have a pre-release package (HDars.Web:1.4.5-rc.2), but just want to "change" the version, then you will have created a new package. If you change a different piece of metadata (such as the author name), then you've created a very confusing situation where two different files are apparently the same package.

This is where repackaging comes in: it's an operation that involves changing a small part of package metadata (such as the version number) without altering the contents, while retaining a "pointer" to the original package inside the newly created package. Of course, this operation can only be performed by a trusted person or service to ensure that the repackaging is securely performed.

The universal packaging manifest file allows for storing a chain of repackaging events that allow you to verify each preceding package.

Package Metacontent & Guidance

Files and directories stored outside of the ZIP archive's /package directory are referred to as metacontent, and the upack.json manifest file is the only required metacontent in a package.

Unlike actual content, there is no "general purpose" usecase for metacontent. Instead, think of it as a tool to help you build a specification for a package format that is based on a universal package. For example, romp packages define several metacontent files and folders that are used to help install the /package contents.

Manifest Specification

The upack.json file is a JSON object with the following properties.

Required Properties

Property Format
group A string of zero to 250 characters: numbers (0-9), upper- and lower-case letters (a-Z), dashes (-), periods (.), forward-slashes (/), and underscores (_); may not start or end with a forward-slash; if not specified, the group name will be considered an empty string.
nameR A string of one to fifty characters: numbers (0-9), upper- and lower-case letters (a-Z), dashes (-), periods (.), and underscores (_).
versionR A string representing a SemVer2 Semantic Version; this is a three-part, dot-separated specification.

Additional Descriptive Properties (Optional)

Property Format
title A string of no more than fifty characters.
projectUrl A string of an absolute url where more about the package can be found
icon A string of an absolute url pointing to an image to be displayed in a UI (at both 64px and 128px); if package:// is used as the protocol, it references a package in image within the package instead
description A string containing any number of characters; these will be formatted as Markdown in the UI.
tags An array of strings, each being one to fifty characters: numbers (0-9), upper- and lower-case letters (a-Z), dashes (-), periods (.), and underscores (_). Tags may not start with a number and must be unique within the array.
dependencies An array of strings, each consisting of a package identification string; this string is formatted as follows:
  • «group»:«package-name»
  • «group»/«package-name»:«version»
  • «group»/«package-name»:«version»:«sha-hash»
When the version is not specified, the latest is used. If a hash is specified, the client may use it to verify a downloaded package; see package identification.

Additional Audit-based Properties (Optional)

Property Format
createdDate A string representing the UTC date when the package was first created, in ISO 8601 format (yyyy-MM-ddThh:mm:ssZ)
creatednReason A string describing the reason or purpose of the creation

For example, BuildMaster uses {Application Name} v{Release Number} #{Package Number} (ID{Execution-Number})
createdUsing A string describing the mechanism the package was created with; there are no format restrictions, but we recommend treating it like a User Agent string and including the tool name and version

For example, BuildMaster uses BuildMaster/5.6.11
createdBy A string describing the person or service that performed the installation

For example, BuildMaster uses the user who triggered the deployment or SYSTEM if it was a triggered/scheduled deployment
repackageHistory An array containing package identification strings or an objects with the following properties:
  • idR - a package identifications string
  • date - A string representing the UTC date when the package was repackaged, in ISO 8601 format (yyyy-MM-ddThh:mm:ssZ)
  • reason - A string describing the reason or purpose of the repackaging
  • using - A string describing the mechanism the package was repackaged with; there are no format restrictions, but we recommend treating it like a User Agent string and including the tool name and version
  • by - A string describing the person or service that performed the repackage
  • url - A string where more information about the repackaging can be found, typically to logs within the tool that performed it
You may include other properties in the object, but it is recommended to prefix the names with an underscore (_).

An R denotes a required property.

Additional Metadata

This object may contain additional properties as needed. However, if you need to add additional metadata, it's strongly recommended that you prefix these properties with an underscore (_)... just on the off-chance that a property you add will exist in a future version of the specification, or is returned in other metadata queries.

Example upack.json Files

  • Example: Minimal
  • Example: All Fields
  • Example: Extended File
{
 "name": "HDARS",
 "version": "1.3.9"
}
{
 "group": "initrode/vendors/abl",
 "name": "ABLast",
 "version": "2.2.1",
 "title": "ast distribution files for ABL",
 "icon": "package://ablast.svg",
 "description": "This contains [ast distro](http://initrode-net.local/ast) files specific to ABL",
 "dependencies": [ "initrode/vendors-common:ast-common:2.0.0" ],
 "createdDate": "2017-11-09T04:03:01Z",
 "createdReason": "ABLast v2.2.1 #18 (ID8843) ",
 "createdUsing": "BuildMaster/5.6.11",
 "createdBy": "THoven",
 "repackageHistory": [
    "initrode/vendors/abl/ABlast:2.2.1-rc.3:2660bf74fc8147ca41bd53bdb1defc3aae35bc91",
    "initrode/vendors/abl/ABlast:2.2.1-ci.22322:ec702530cf73dbec0fc8667f7235fd3f17e24205"
  ]
}
{
 "group": "virtudyne/simdesk",
 "name": "var-index-service",
 "version": "5.3.9",
 "_sourceRoot": "$/global/vindex/branches/v5-hotfix"
 "_deployTarget": "/var/vsimdesk/vindex"
}