6 Developing Packages with Git

When a Git repository is specified as a package source, then a copy of the repository content is installed as the package implementation. That installation mode is designed for package consumers, who normally use a package without modifying it. The installed copy of the package is unsuitable for development by the package author, however, since the installation is not a full clone of the Git repository. The Racket package manager provides different installation modes to support package authors who work with Git repository clones.

6.1 Linking a Git Checkout as a Directory

Since a Git repository checkout is a directory, it can be linked as a package as described in Linking and Developing New Packages. In that case, any modifications made locally take effect immediately for the package installation, including any updates from a git pull. The developer must explicitly pull any remote updates to the repository, however, including when the updates are needed to satisfy the requirements of dependent packages.

In the following section, we describe an alternative that makes raco pkg update aware of the checkout directory’s status as a repository clone. Furthermore, a directory-linked package can be promoted to a clone-linked package with raco pkg update.

6.2 Linking a Git Checkout as a Clone

When a package is installed with

  raco pkg install --clone ‹dir› ‹git-pkg-source›

then instead of installing the package as a mere copy of the repository source, the package is installed by creating a Git clone of ‹git-pkg-source› as ‹dir›. The clone’s checkout is linked in the same way as a directory, but unlike a plain directory link, the Racket package manager keeps track of the repository connection. The ‹git-pkg-source› must be a Git or GitHub package source, or it must be a package name that the catalog maps to a Git or GitHub package source; if the source URL includes a fragment, it must name a branch or tag (as opposed to a raw commit). If ‹git-pkg-source› refers to a repository over HTTPS but has no .git suffix, use git+https:// to refer to the repository.

When the repository at ‹git-pkg-source› is changed so that the source has a new checksum, then raco pkg update for the package pulls commits from the repository to the local clone. In other words, raco pkg update works as an alternative to git pull --ff-only to pull updates for the package. Furthermore, raco pkg update can pull updates to local package repositories when checking dependencies. For example, raco pkg update --all pulls updates for all linked package repositories.

A package source provided with --clone can include a branch and/or path into the repository. The branch specification affects the branch used for the initial checkout, while a non-empty path causes a subdirectory of the checkout to be linked for the package.

Suppose that a developer works with a large number of packages and develops only a few of them. The intended workflow is as follows:

• Install all the relevant packages with raco pkg install.

• For each package to be developed out of a particular Git repository named by ‹pkg-name›, update the installation with

    raco pkg update --clone ‹dir› ‹pkg-name›

  which discards the original installation of the package and replaces it with a local clone as ‹dir›.

  As a convenience, when ‹git-pkg-source› and the last element of ‹dir› are the same, then ‹pkg-name› can be omitted. Put another way, the argument to --clone can be a path to ‹pkg-name›:

    raco pkg update --clone ‹path-to›/‹pkg-name›

  As a further convenience, when building from scratch from the main Racket source repository, the Git configuration ignores a top-level "extra-pkgs" directory. The directory is intended to be used as a target for --clone:

    raco pkg update --clone extra-pkgs/‹pkg-name›

  which creates the "extra-pkgs" subdirectory if it doesn’t exist.

• If a package’s current installation is not drawn from a Git repository (e.g., it’s drawn from a catalog of built packages for a distribution or snapshot), but ‹catalog› maps the package name to the right Git repository, then combine --clone with --lookup and --catalog:

    raco pkg update --lookup --catalog ‹catalog› --clone ‹path-to›/‹pkg-name›

  A suitable ‹catalog› might be https://pkgs.racket-lang.org.

• A newly cloned package will have the specified (or existing installation’s) repository as its Git origin. If you want to push and pull updates from a different repository—for instance, your own fork of the package source—then use git commands to add or change the origin of your clone to the other repository. For example, the command

    git remote set-url origin ‹url-of-other-repo›

  in the clone’s directory causes git pull and git push to pull and push to the given ‹url-of-other-repo›.

  You can preserve the clone’s connection to its central repository by setting an upstream remote, e.g. git remote add upstream ‹url-of-central-repo›. This gives you the option to periodically pull in commits from the central repository with git pull --ff-only upstream.

  Alternatively, use git to clone the target ‹url› first, and then supply the local clone’s path as ‹dir› in

    raco pkg update --clone ‹dir› ‹pkg-name›

  Either way, when raco pkg update pulls updates to the clone, it will still pull them from the repository corresponding to ‹pkg-name›’s old source, and not from the git remote ‹url›. Usually, that’s what package developers want; when they’re not actively modifying a package, other developers’ updates should be pulled from the package’s main repository. In case where ‹url› is the preferred source of updates for raco pkg update, use ‹url› in

    raco pkg update --clone ‹dir› ‹url›

  Beware, however, that raco pkg update may be less able to detect repository sharing among multiple packages (and keep the package installations consistently associated with a particular clone) when an alternative ‹url› is provided.

• Manage changes to each of the developed packages in the usual way with git tools, but raco pkg update is also available for updates, including mass updates.

6.3 Interactions Between git and raco pkg

The git and raco pkg tools interact in specific ways:

• With the link-establishing

    raco pkg install --clone ‹dir› ‹git-pkg-source›

  or the same for raco pkg update, if a local repository exists already as ‹dir›, then it is left in place and any new commits are fetched from ‹git-pkg-source›. The package manager does not attempt to check whether a pre-existing ‹dir› is consistent with ‹git-pkg-source›; it simply starts fetching new commits to ‹dir›, and a later git pull --ff-only will detect any mismatch.

  Multiple ‹git-pkg-source›s can be provided to raco pkg install, which makes sense when multiple packages are sourced from the same repository and can therefore share ‹dir›. Whether through a single raco pkg use or multiple uses with the same --clone ‹dir›, packages from the same repository should be linked from the same local clone (assuming that they are in the same repository because they should be modified together). The package system does not inherently require clone sharing among the packages, but since non-sharing or inconsistent installation modes could be confusing, raco pkg install and raco pkg update report non-sharing or inconsistent installations. In typical cases, the default --multi-clone ask mode can automatically fix inconsistencies.

• When pulling changes to repositories that have local copies, raco pkg update pulls changes with the equivalent of git pull --ff-only by default. Supplying --pull rebase pulls changes with the equivalent of git pull --rebase, instead. Supplying --pull try attempts to pull with git pull --ff-only, but failure is ignored.

• When raco pkg update is given a specific commit as the target of the update, it uses the equivalent of git merge --ff-only ‹checksum› or git merge --rebase ‹checksum›. This approach is intended to preserve any changes to the package made locally, but it implies that the package cannot be “downgraded” to a older commit simply by specifying the commit for raco pkg update; any newer commits that are already in the local repository will be preserved.

• The installed-package database records the most recent commit pulled from the source repository after each installation or update. The current commit in the repository checkout is consulted only for the purposes of merging onto pulled commits. Thus, after pushing repository changes with git push, a raco pkg update makes sense to synchronize the package-installation database with the remote repository state (which is then the same as the local repository state).

• When checking a raco pkg install or raco pkg update request for dependencies and collisions, the clone directory’s content is used directly only if the current checkout includes the target commit.

  Otherwise, commits are first fetched with git fetch, and an additional local clone is created in a temporary directory. If the overall installation or update is deemed to be successful with respect to remote commits (not necessarily the current commit in each local repository) in that copy, then an update to the linked repository checkout proceeds. Finally, after all checkouts succeed, other package installations and updates are completed and recorded. If a checkout fails (e.g., due to a conflict or uncommitted change), then the repository checkout is left in a failed state, but all package actions are otherwise canceled.

• Removing a package with raco pkg remove leaves the repository checkout intact while removing the package link.