swh.loader.mercurial.chunked_reader module

class swh.loader.mercurial.chunked_reader.ChunkedFileReader(file, size_unpack_fmt='>I')[source]

Bases: object

A binary stream reader that gives seamless read access to Mercurial’s bundle2 HG20 format which is partitioned for some reason at the file level into chunks of [4Bytes:<length>, <length>Bytes:<data>] as if it were encoding transport packets.

  • file – rb file handle pre-aligned to the start of the chunked portion

  • size_unpack_fmt – struct format string for unpacking the next chunk size


Returns the file size in bytes.


Return N bytes from the file as a single block.


bytes_to_read – int number of bytes of content


Return a generator that yields N bytes from the file one file chunk at a time.


bytes_to_read – int number of bytes of content

seek(new_pos=None, from_current=False)[source]

Wraps the underlying file seek, additionally updating the chunk_bytes_left counter appropriately so that we can start reading from the new location.

  • new_pos – new cursor byte position

  • from_current – if True, it treats new_pos as an offset from the current cursor position, bypassing any chunk boundaries as if they weren’t there. This should give the same end position as a read except without the reading data part.