swh.counters.kafka_client module

class swh.counters.kafka_client.KeyOrientedJournalClient(brokers: Union[str, List[str]], group_id: str, prefix: Optional[str] = None, object_types: Optional[List[str]] = None, privileged: bool = False, stop_after_objects: Optional[int] = None, batch_size: int = 200, process_timeout: Optional[float] = None, auto_offset_reset: str = 'earliest', stop_on_eof: Optional[bool] = None, on_eof: Optional[Union[EofBehavior, str]] = None, value_deserializer: Optional[Callable[[str, bytes], Any]] = None, **kwargs)

Bases: JournalClient

Journal Client implementation which only decodes the message keys. This does not need to bother with the message deserialization (contrary to swh.journal.client.JournalClient) Message values are still passed unparsed to worker_fn so it can deserialize and use it if needed.

handle_messages(messages, worker_fn)