Add a new package#
The following document demonstrates how to create a new Python package for
Software Heritage, hereafter named swh-foo
.
We will need to create a project, initialize the new repository, reference the project in the Continuous Integration system and finally add the project to the documentation.
Create a project#
Creating the project should be done using the gitlab
command-line tool
provided by the python-gitlab module.
Make sure to have the configuration working and an access token with api
as
scope.
To create the project:
PROJECT_NAME=swh-foo
DESCRIPTION="Software Heritage Foo management library"
NAMESPACE_ID="$(gitlab --output json namespace get --id 'swh/devel' | jq .id)"
gitlab project create \
--name "$PROJECT_NAME" \
--path "$PROJECT_NAME" \
--namespace "$NAMESPACE_ID" \
--description "$DESCRIPTION" \
--issues-access-level enabled \
--auto-devops-enabled false \
--wiki-access-level disabled \
--requirements-access-level disabled \
--pages-access-level disabled \
--operations-access-level disabled \
--container-registry-access-level disabled \
--visibility public
Initialize the repository with our template#
The following commands need to run from the base directory
swh-environment
.
Clone the new repository:
git clone https://gitlab.softwareheritage.org/swh/devel/swh-foo.git
Use
bin/init-py-repo
to initialize the repository with a project template:bin/init-py-repo swh-foo
Install the pre-commit hook:
pre-commit install
Customize the template#
Now look for the string foo
in all files and file names and replace it with
the name of the new package. Push these commits directly to the repository as
initial content.
For an example, you can see what was done for swh-counters.
Add the repo on the swh-environment project#
Declare the repository on the mr configuration:
Edit the
.mrconfig
file and declare the new repository. For an example, look at the addition of swh-graphql.Create a merge request with the changes.
Note
Adding the repository in .mrconfig
will break swh-docs
builds until
the new module is registered in the documentation as explained below.
Install CI jobs#
In swh-jenkins-jobs, open jobs/swh-packages.yaml and add a section like the others for the new repository.
Note
Jobs will automatically be recreated when changes are pushed to the
swh-jenkins-jobs
repository. See Jenkins documentation
for details.
Make an initial release#
Releases are made automatically by Jenkins when a tag is pushed to a module repository. Making an initial release is thus done by doing:
git tag v0.0.0
git push origin --tags v0.0.0
Note
Before adding a new module to the documentation, at least one release must have been made. Otherwise, the documentation will not build as it won’t be able to fetch the Python package from PyPI nor determine the version number. This is why we need to make an initial release before moving forward.
Update the documentation#
The documentation is in the swh-docs project. Each Python module get a section of the documentation automatically generated from its source code.
To add a new module to the documentation:
Add the package to the dependencies in
requirements-swh.txt
(publication build) andrequirements-swh-dev.txt
(documentation development build).Reference the package in the
toctree
located indocs/devel/api-reference.rst
Add the package with a concise description to the index of the development part, located in
docs/devel/index.rst
.:ref:`swh.foo <swh-foo>` short description of the repository
Ensure this builds fine locally (run
tox run
andtox run -e sphinx-dev
)Open a merge request with the above changes.