Contents

Make a skeleton spec file

We are going to create a package for memory_allocator. You don’t have to understand what memory-allocator does. The point is to show the steps needed to create an RPM package for a Python extension project.

First, change to the directory where you want the RPM spec file to be created.

cd ~/rpmbuild/SPECS

I ordinarily recommend the use of rpmdev-newspec from the rpmdevtools package to create a skeleton spec file. However, rpmdevtools tries to stay compatible with all Linux distributions that use RPM. The result is that its python template contains a lot of stuff that Fedora packages do not need. Instead, start with this empty spec file. Modify the Name: line to read python-memory-allocator. Save it with the name python-memory-allocator.spec.

Determine the version

Let’s start filling in the spec file. First, what version are we packaging? At the time of this writing, the latest released version is 0.1.0, so add that to the Version: line. While most software, including memory-allocator, now carries version numbers of the form number.number[.number], that style is not universal. If you are making a package for an upstream that has an unusual version numbering style, see the versioning guidelines for help on determining what to put in the Version: field.

Use autochangelog

Autochangelog is a recent addition to Fedora. It lets you avoid dealing directly with the Release: field and the %changelog section. Change the Release: field to read %autorelease. At the end of the spec file, add %autochangelog on the line after %changelog.

Write a summary

Next we need a summary. What does this package do? The github page says:

An extension class to allocate memory easily with cython.

Let’s shorten that to “Allocate memory easily with cython”. Notice that there is no period in the summary. It is not a complete sentence, but rather a short descriptive phrase.

Determine the license

If you haven’t yet downloaded the tarball for the version you are packaging, do so now. Unpack the sources, and run licensecheck on the source files, as described here. In the case of memory-allocator 0.1.0, the output looks like this:

./AUTHORS: *No copyright* UNKNOWN
./LICENSE: UNKNOWN
./MANIFEST.in: *No copyright* UNKNOWN
./README.md: *No copyright* UNKNOWN
./cydoctest.py: *No copyright* UNKNOWN
./pyproject.toml: *No copyright* UNKNOWN
./setup.py: *No copyright* GNU General Public License, Version 3
./test.py: *No copyright* UNKNOWN
./memory_allocator/__init__.pxd: *No copyright* UNKNOWN
./memory_allocator/memory.pxd: GNU Lesser General Public License v3.0 or later
./memory_allocator/memory_allocator.pxd: *No copyright* UNKNOWN
./memory_allocator/memory_allocator.pyx: *No copyright* UNKNOWN
./memory_allocator/signals.pxd: *No copyright* UNKNOWN
./memory_allocator/test.pyx: *No copyright* UNKNOWN
./.github/workflows/flake.yml: *No copyright* UNKNOWN
./.github/workflows/main.yml: *No copyright* UNKNOWN

The LICENSE file contains the text of GPL version 3, and setup.py contains the line “license=’GPLv3’”. We need to determine whether the license is version 3 only, or if later versions are included. Run this:

grep -Fr 'any later'

That only turns up the example language in the LICENSE file, and memory_allocator/memory.pxd, which licensecheck identified as carrying a license of LGPv3+. Sure enough, that file appears to have been borrowed from a project called cysignals (which we have in Fedora, by the way). It appears that the final license is a combination of GPLv3 and LGPLv3+. Since GPLv3 subsumes LGPLv3, the entire project can be distributed under the terms of GPLv3, so set the License: field accordingly.

The URL field

Next, put the project URL into the URL: field of the spec file. This should be some sort of home page for the project. In the memory-allocator case, the URL https://sagemath.github.io/memory_allocator returns a 404, so the value of the field is https://github.com/sagemath/memory_allocator.

The Source field

The Source fields in the spec file identify those files that must be present in the build root in order to build the software. See the Source URL guidelines for help determining the appropriate URL for this field.

For projects on pypi, we can use the %pypi_source macro with the project name, like this:

Source0: %{pypi_source memory_allocator}

BuildArch

Pure Python modules can be declared noarch, but this is an extension module, meaning it has a compiled component. Remove BuildArch: noarch / , leaving BuildRequires: gcc behind.

Description

The %description section is typically filled in with text from a README file. Failing that, do your best to write a paragraph describing what this package does. You want a person who has never heard of the software before to be able to get some idea of the purpose of the package from reading the description. As noted in the packaging guidelines, keep lines to 80 characters or less. Longer lines can be displayed in strange ways in the various graphical software managers. I usually limit description lines to 72 characters to be on the safe side.

In the case of Python packages, we typically have a main package named “python-[project]” and a subpackage named “python3-[project]”. Both have %description sections, but the contents should typically be identical. The Python template accounts for this by defining a variable named _description, which is then used for both packages. Paste the contents of README.md into that field on the line after %{expand:, replacing the triple dots.

Python3 subpackage

Find the line that reads %package -n python3-... and replace the triple dots with memory-allocator. Do the same for the %description line just below it.

Prep

The %prep section is where we do anything that must be done prior to building the software. Typically, the first step is to unpack a tarball or zip file. The %autosetup macro is often sufficient. It knows how to unpack a variety of files and compression types. This macro assumes that, after unpacking, it will find the source files in a directory named %{name}-%{version}. If that is not the case, we need to tell it so. In the memory-allocator case, the name of the directory is memory_allocator-0.1.0. The package name is python-memory-allocator, however, so the names do not match. Change the %autosetup macro to read:

%autosetup -p1 -n memory_allocator-%{version}

Generate BuildRequires

Next we find ourselves looking at this:

%generate_buildrequires
%pyproject_buildrequires -rx... / -t

This is a relatively new feature. For python projects, the build system can (mostly) figure out what packages are needed as BuildRequires! We do need to figure out what flags, if any, to pass to %pyproject_buildrequires. If we pass -t, then the system will attempt to install everything needed to run tests. We should start with that, and back off only if we discover that the tests cannot be run for some reason. Remove -rx... / for now.

Save files

There are more triple dots in the %install section, after %pyproject_save_files. Those dots should be replaced with the names of all modules this package installs. That’s just memory_allocator, so replace that now.

Check

The %check section lists a selection of testing macros. Which one do we want? There is no tox.ini file and grepping for pytest turns up nothing. There is a file named test.py, though. Maybe we should just run that?

At this point, we’re not sure. Let’s gamble on %tox, and revisit this once we have a build to play with.

Files

On the %files line we see another set of triple dots. Replace that with the package name; i.e., it should read python3-memory-allocator.

On the %doc line, replace README.* with a selection of files that look useful for documentation. In this case, I recommend using %doc AUTHORS README.md. Similarly, the * on the %license line is not needed, since the license file is named just LICENSE.

This is an extension module, so there shoudn’t be anything in %{_bindir}. Erase that line.

At this point, the spec file should look like this:

Name:           python-memory-allocator
Version:        0.1.0
Release:        %autorelease
Summary:        Allocate memory easily with cython

License:        GPLv3
URL:            https://github.com/sagemath/memory_allocator
Source0:        %{pypi_source memory_allocator}

BuildRequires:  gcc
BuildRequires:  python3-devel

%global _description %{expand:
This package contains an extension class to allocate memory easily with
Cython.  This extension class started as part of sagemath.  It provides
a single extension class `MemoryAllocator` with `cdef` methods:

- `malloc`,
- `calloc`,
- `alloarray`,
- `realloc`,
- `reallocarray`,
- `aligned_malloc`,
- `aligned_calloc`,
- `aligned_allocarray`.

Memory is freed when the instance of `MemoryAllocator` is deallocated.
On failure to allocate the memory, a proper error is raised.}

%description %_description

%package -n python3-memory-allocator
Summary:        %{summary}

%description -n python3-memory-allocator %_description

%prep
%autosetup -p1 -n memory_allocator-%{version}

%generate_buildrequires
%pyproject_buildrequires -t

%build
%pyproject_wheel

%install
%pyproject_install
%pyproject_save_files memory_allocator

%check
%tox

%files -n python3-memory-allocator -f %{pyproject_files}
%doc AUTHORS README.md
%license LICENSE

%changelog
%autochangelog

First test build

Make sure that your spec file is in ~/rpmbuild/SPECS, and that memory_allocator-0.1.0.tar.gz is in ~/rpmbuild/SOURCES. While in the ~/rpmbuild/SPECS directory, run rpmbuild -bs python-memory-allocator.spec. That command generates a source rpm in ~/rpmbuild/SRPMS.

Next, we will attempt to build the package with mock. Refer to building with mock for details. Briefly, run this command:

mock -r fedora-rawhide-x86_64 --rebuild ~/rpmbuild/SRPMS/python-memory-allocator-0.1.0-1.fc35.src.rpm

You may have to replace fc35 in that filename. Make it match the name of file file in your ~/rpmbuild/SRPMS directory.

The test build fails:

ERROR: tox config file (either pyproject.toml, tox.ini, setup.cfg) not found
Traceback (most recent call last):
  File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 416, in main
    generate_requires(
  File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 353, in generate_requires
    generate_tox_requirements(toxenv, requirements)
  File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 303, in generate_tox_requirements
    r.check_returncode()
  File "/usr/lib64/python3.10/subprocess.py", line 456, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'tox', '--print-deps-to', '/tmp/tmp4z5l2eoq', '--print-extras-to', '/tmp/tmpjnlmpu4q', '--no-provision', '/tmp/tmpk83z08_9', '-qre', 'py310']' returned non-zero exit status 1.

What does that mean? Both pyproject.toml and setup.cfg exist! What it probably means is that we can’t pass the -t flag to %pyproject_buildrequires, since the project does not use tox for testing. Let’s remove that flag and rerun the build.

Run tests

That looks better, but the %tox macro in %check failed, which we should have expected at this point. So how are the tests supposed to be run? Maybe setup.py knows what to do. Replace %tox with python3 setup.py test and try again. That works! Since there is no significant documentation in the package, and the final Requires and Provides look sane, things look good.

Run rpmlint

Let’s run an automated tool over the binary packages that can detect many common packaging problems.

mock -r fedora-rawhide-x86_64 --install /var/lib/mock/fedora-rawhide-x86_64/result/*.x86_64.rpm rpmlint python3-enchant

Afterwards, we enter a mock shell and run rpmlint:

mock --enable-network -r fedora-rawhide-x86_64 --shell

The --enable-network argument allows rpmlint to access the network in order to do some checks, such as URL validity. In the shell, run this command:

rpmlint -i python3-memory-allocator

That generates a long list of undefined-non-weak-symbol warnings. No, that is normal for a Python extension. Those symbols are defined in libpython, which will be loaded in memory long before the extension is loaded. As long as all of the symbols start with Py or _Py, we can ignore the warnings. There are no other warnings, so everything looks fine.

Conclusion

I hope you learned something from this case study. Please send suggestions for improvements to Jerry James.