Repository anatomy

Record in sources.list may look like this:

deb trusty main

It consists of repository url, distribution and one or more components.

deb <repository> <distribution> <component(s)>


Distribution name consists of Linux distribution version codename (eg. trusty) and our internal stability identifier (stable or testing)

It can also have a prefix that can identify source.

  • trusty-stable
  • trusty-testing
  • upstream/trusty-testing


Components are subsets of each distribution repository. They can be used independently by choice or all together.

  • main - Ubuntu packages
  • universe - Ubuntu universe packages

Local repository

Custom packages build by us. Published as cloudlab component.

aptly repo create -component=cloudlab -architectures=amd64 cloudlab


Direct publish

It’s possible to publish repository directly with no snapshots. Unfortunately this is not possible with mirrors so if it’s needed to combine components including custom repository and mirrorred ones, it has to be via snapshots. See Multi component publishing

New publish
aptly publish repo -distribution=trusty-testing cloudlab
Update publish
aptly publish update trusty-testing

Snapshot publish

To create new snapshot and create or update existing publish.

New publish
aptly snapshot create cloudlab-`date +%s` from repo cloudlab
 aptly publish snapshot -distribution=trusty-stable cloudlab-1433941328
Update publish
aptly snapshot create cloudlab-`date +%s` from repo cloudlab
 aptly publish switch -component=cloudlab trusty-stable cloudlab-1433941328

Repository mirror

First you need to import distributor’s GPG key to be able to verify packages.

gpg --no-tty --no-default-keyring --keyring trustedkeys.gpg --keyserver --recv-keys 0E27C0A6
Create mirror
aptly mirror create -architectures=amd64 salt trusty main
Update mirror

This will fetch newest packages from upstream. Should be run periodically to be always up to date.

aptly mirror update salt

Real-world example

Let’s try to design some real-world example.

Software sources

Example status
aptly@aptly01:/home/filip$ aptly repo list
List of local repos:
 - [cloudlab]: TCP Cloud custom components (packages: 0)

aptly@aptly01:/home/filip$ aptly mirror list
List of mirrors:
 - [salt-trusty]: trusty
 - [trusty-main]: trusty
 - [trusty-security]: trusty-security
 - [trusty-updates-juno]: trusty-updates/juno

Sync mirrors

It’s good idea to keep mirrors up to date and sync them periodically. You can use script bellow, place it eg. into /usr/local/bin/ and create crontab record for aptly user (suppose that aptly repo is not managed by root).


SCRIPT=$(basename $0)

log_info() {
    logger -p -t ${SCRIPT} "$*"
    [ $VERBOSE -eq 1 ] && echo "[INFO] $*"

log_error() {
    logger -p user.error -t ${SCRIPT} "$*"
    echo "[ERROR] $*" >&2

if [[ "$*" == *--verbose* || "$*" == *-v* ]]; then

if [[ "$*" == *--snapshot* || "$*" == *-s* ]]; then

MIRRORS=$(aptly mirror list --raw 2>&1)
if [[ $? -ne 0 ]]; then
    log_error "$MIRRORS"
    exit 1

for mirror in $MIRRORS; do

    while [[ $retval -ne 0 && $try -lt $MAXTRIES ]]; do
        log_info "Starting update of mirror ${mirror}"
        if [[ $VERBOSE -eq 0 ]]; then
            out=$(aptly mirror update -force=true ${mirror} >/dev/null 2>&1)
            aptly mirror update -force=true ${mirror}


        if [[ $retval -ne 0 ]]; then
            try=$[ $try + 1 ]
            log_error "Failed to update mirror ${mirror}, try=${try}"
            [ ! -z "$out" ] && log_error "${out}"
            log_info "Synced mirror ${mirror}"
            if [[ $SNAPSHOT -eq 1 ]]; then
                snapshot_name="${mirror}-$(date +%s)"
                if [[ $VERBOSE -eq 0 ]]; then
                    out=$(aptly snapshot create ${snapshot_name} from mirror ${mirror} >/dev/null 2>&1)
                    aptly snapshot create ${snapshot_name} from mirror ${mirror}

                if [[ $retval -ne 0 ]]; then
                    log_error "Failed to create snapshot ${snapshot_name} from mirror ${mirror}"
                    [ ! -z "$out" ] && log_error "$out"
                    log_info "Created snapshot ${snapshot_name} from mirror ${mirror}"
$ crontab -e -u aptly
# Sync aptly mirrors every 6 hours
0 */6 * * * -v -s

First run may take a long time so it’s good idea to execute it in terminal multiplexer (Tmux, Screen, Byobu, etc.) in verbose mode. Parameter --snapshot will automatically create snapshots of newly synced mirrors. --verbose --snapshot


First we need to decide how our published repository will look like.


We want to support only Ubuntu Trusty in three branches:
  • nightly with up-to-date packages with mirrors
  • testing for stabilization
  • stable for rock-solid distribution

So final distributions will be:

  • trusty-nightly
  • trusty-testing
  • trusty-stable


Each OS distribution has different approach to components. For our workflow, component means:

  • set of independently versioned packages (sub-distribution) where customer may choose - eg. Openstack juno, kilo - or Opencontrail 21, 22, etc.
  • set of packages with different or faster release workflow - because you can update publish per-component - eg. security from trusty-security distribution - or custom packages in our repository

For our example, components will be:

  • main - merged upstream + ppa repositories with newer software versions
  • security - security updates
    • because we want to push security updates between publishes separately
  • cloudlab - our packages
    • because our packages can have different release cycle than rest of the distribution
  • juno - packages of Openstack Juno
    • because this component is optional and one can use different openstack version than juno

Create publish

This way you will create snapshots, merge some of them into one component snapshot (main) and publish using multi-component publishing.

Create snapshots for each mirror and repo
STAMP=$(date +%s)
aptly mirror list --raw|while read i;do aptly snapshot create $i-${STAMP} from mirror $i;done
aptly repo list --raw|while read i;do aptly snapshot create $i-${STAMP} from repo $i;done
Merge snapshots into desired final components
aptly snapshot merge main-${STAMP} salt-trusty-${STAMP} trusty-main-${STAMP}
Create multi-component publish of snapshots
aptly publish snapshot -component=main,cloudlab,juno,security -distribution=trusty-nightly main-${STAMP} cloudlab-${STAMP} trusty-updates-juno-${STAMP} trusty-security-${STAMP}
When new snapshots are created, use similar command as the one above to update publish:
aptly publish switch -component=main,cloudlab,juno,security trusty-nightly main-${STAMP} cloudlab-${STAMP} trusty-updates-juno-${STAMP} trusty-security-${STAMP}

For more information, read multi-component publishing

Naming conventions


There has to be one mirror per each repository distribution, so distribution has to be reflected in the name. Aptly doesn’t work with components on this level so each component has to be mirrored separately, otherwise it will be merged into main component.

Source repository can be ommited from mirror name if it’s upstream repository. Distribution name can me ommited if there’s only one distribution per mirrored repository and naming it explicitly would be confusing.

Patterns and examples
  • <source>-<distribution>(-<component>)
  • salt-trusty (<salt_ppa> trusty main)
  • trusty-main (<ubuntu_upstream> trusty main)
  • trusty-updates-juno (<ubuntucloud_repository> trusty-updates/juno main)


It’s always good idea to use Unix timestamp for snapshot versioning and same name as for source mirror or repository.

For snapshots created by merging other snapshots, naming by published distribution is a good idea.

Patterns and examples
  • cloudlab-$(date +%s)
  • trusty-updates-juno-1434444631
  • main-1434444631