Python conventions¶

Modules¶

Add all public members to __all__, declared immediately after the imports. (Note that mkdocstrings requires __all__.)

Use mainpkg/__init__.py to import the most important classes. Do not set __author__ or similar fields, but do set mainpkg/__version__.

Formatting¶

The Ruff formatter – which is equivalent to Black – should be used, so don’t worry much about formatting. Avoid add trailing commas so that Black can decide whether to keep code on one line or to chop it.

Sometimes Ruff/Black wraps lines in awkwardly, by prioritizing argument lists over call chains. If this happens, either shorten the lines or break the code into multiple statements. For example:

data = my_long_named_function_that_makes_the_line_too_long(
    data
).my_other_long_named_function_being_chained(1)

data = my_shorter_function(data).my_other_shorter_function(1)

data = my_long_named_function_that_makes_the_line_too_long(data)
data = data.my_other_long_named_function_being_chained(1)

Pydantic and dataclasses¶

Use pydantic or dataclasses. Most libraries should use only dataclasses to avoid a dependency on pydantic. Use immutable types unless there’s a compelling reason otherwise.

from dataclasses import dataclass
from typing import Self


@dataclass(slots=True, frozen=True, order=True)
class Cat:
    breed: str | None
    age: int
    names: frozenset[str]

from pydantic import BaseModel


class Cat(BaseModel):
    breed: str | None
    age: int
    names: frozenset[str]

    class Config:
        frozen = True

import orjson
from pydantic import BaseModel


def to_json(v) -> str:
    return orjson.dumps(v).decode(encoding="utf-8")


def from_json(v: str):
    return orjson.loads(v).encode(encoding="utf-8")


class Cat(BaseModel):
    breed: str | None
    age: int
    names: frozenset[str]

    class Config:
        frozen = True
        json_loads = from_json
        json_dumps = to_json

Abstract base classes¶

Inherit directly from abstract base classes in collections.abc rather than using metaclass=.

from collections.abc import Sequence, MutableSequence
from dataclasses import dataclass
from typing import Literal, Self


@dataclass(slots=True, frozen=True)
class AxisTicks(Sequence[int]):

    orientation: Literal["x"] | Literal["y"]
    items: Sequence[int]

    def __getitem__(self: Self, i: int) -> int:
        return self.items[i]

    def __len__(self: Self) -> int:
        return len(self.items)


# Error!
# dataclasses.FrozenInstanceError: cannot assign to field 'orientation'
AxisTicks(orientation="x", items=[1, 3, 5])
a.orientation = "y"

Class members¶

IoC¶

Apply inversion of control, and do so ruthlessly.

@staticmethod and @abstractmethod¶

Use @staticmethod and @abstractmethod only for utilities that are specific to their class. Replace static factory @abstractclass methods with property factories, to separate the creation of an object from its API.

Never create utility classes with static (@staticmethod or @abstractmethod) methods; use valid OOP instead.

class Utils:
    @abstractmethod
    def method(cls):
        ...

class Utils:
    def method(self):
        ...
utils = Utils()  # wherever it's needed

Ordering members¶

Sort class members in the following order. (These rules are copied from semi-official sources.)

1. ClassVar
2. attributes
3. @staticmethod
4. @classmethod
5. magic methods
6. @property methods, getters, and setters
7. regular methods
8. inner classes

Within each of the 8 types, order by, in order of decreasing importance:

1. Pairing getters and setters together, with the getter first.
2. Listing public, then private (_xxx), then dunder (__xxx).

OS compatibility¶

Use pathlib instead of os wherever possible. Always read and write text as UTF-8, and pass encoding="utf-8" (i.e. not utf8 or UTF-8).

from pathlib import Path

directory = Path.cwd()
(directory / "myfile.txt").write_text("hi", encoding="utf-8")

Typing¶

Use typing annotations for both public APIs and internal components. Annotate all module-level variables, class attributes, and functions. Annotate both return types and parameters. Annotate self, cls, *args, and **kwargs parameters.

from dataclasses import dataclass
from typing import Any, Self, Unpack


@dataclass(slots=True, frozen=True)
class A(SomeAbstractType):
    value: int

    @classmethod
    def new_zero(cls: type[Self]) -> Self:
        return cls(0)

    def __add__(self: Self, other: Self) -> Self:
        return self.__class__(self.value + other.value)

    def add_sum(self: Self, *args: int) -> Self:
        return self.__class(self.value + sum(args))

    def delegate(self: Self, *args: Any, **kwargs: Unpack[tuple[str, Any]]) -> None:
        ...  # first do something special
        super().delegate(*args, **kwargs)

Collection types¶

In Python 3.9, PEP 585 enabled type parameterization for the concrete types list, set, frozenset, dict, tuple, and type and deprecated their aliases in typing‘.

It also enabled parameterization on types from collections.abc like Set, and made corresponding types in typing deprecated aliases. For example, typing.AbstractSet is a deprecated alias to collections.abc.Set, typing.MutableSet is a deprecated alias to collections.abc.MutableSet, and typing.Set is a deprecated alias to set. Similarly, typing.Sequence is a deprecated alias to collections.abc.Sequence, and typing.List is a deprecated alias to list. typing.Mapping is a deprecated alias to collections.abc.Mapping, and typing.Dict is a deprecated alias to dict.

• To annotate a parameter, Sequence is usually better than the concrete type list, but list is so ubiquitous that it’s acceptable.
• Using Mapping, at least for parameters, is usually better than using dict.
• For parameters, collections.abc.Set is almost always better than set. Although set[str] | frozenset[str] is comparable to Set[str], the latter is more general, as well as shorter and more obvious.

from collections.abc import MutableSet, Set

isinstance(set(), set)  # True
isinstance(set(), MutableSet)  # True
isinstance(set(), Set)  # True

isinstance(frozenset(), set)  # False
isinstance(frozenset(), MutableSet)  # False
isinstance(frozenset(), Set)  # True

Collection hierarchy¶

Refer to this diagram. Types like KeysView and defaultdict are rarely useful for typing.

stateDiagram
  direction RL
  classDef abc font-style:italic,stroke:#999,stroke-width:2px,stroke-dasharray: 5;
classDef concrete stroke:#999,stroke-width:2px;
Container --> Iterable:::abc
Sized:::abc --> Iterable:::abc
Collection:::abc --> Iterable:::abc
Iterator:::abc --> Iterable:::abc
Reversible:::abc --> Iterable:::abc
Sequence:::abc --> Reversible:::abc
Collection:::abc --> Container:::abc
Callable:::abc --> Container:::abc
Collection:::abc --> Sized:::abc
Set:::abc --> Collection:::abc
Mapping:::abc --> Collection:::abc
Sequence:::abc --> Collection:::abc
ByteString:::abc --> Collection:::abc
MappingView:::abc --> Collection:::abc
MutableMapping:::abc --> Mapping:::abc
MutableSequence:::abc --> Sequence:::abc
MutableSet:::abc --> Set:::abc
ItemsView:::abc --> MappingView:::abc
ValuesView:::abc --> MappingView:::abc
KeysView:::abc --> MappingView:::abc
MutableSequence:::abc --> Sequence:::abc
str:::concrete --> Sequence
tuple:::concrete --> Sequence
range:::concrete --> Sequence
list:::concrete --> MutableSequence
bytearray:::concrete --> MutableSequence
deque:::concrete --> MutableSequence
frozenset:::concrete --> Set
set:::concrete --> MutableSet
bytes:::concrete --> ByteString
memoryview:::concrete --> ByteString
dict:::concrete --> MutableMapping
defaultdict:::concrete --> MutableMapping
Counter:::concrete --> MutableMapping

Docstrings¶

Use Google-style docstrings as mkdocstrings supports.

Ruff rules¶

Use Ruff to catch potential problems and bad practices. Use at least the rules enabled in the cicd pyproject.toml. To disable counting a line or block in test coverage, use # nocov (not # pragma: nocov, etc.).