jina.types.arrays.memmap.bpm module

class jina.types.arrays.memmap.bpm.BufferPoolManager(pool_size=1000)[source]

Bases: object

Create a buffer pool manager that maps hot Document s of a DocumentArrayMemmap to a memory buffer.

This helps keep access to memory-loaded Document instances synced with DocumentArrayMemmap values. The memory buffer has a fixed size and uses an LRU strategy to empty spots when full.

add_or_update(idx, doc)[source]

Adds a document to the buffer pool or updates it if it already exists

Parameters
  • idx (str) – index

  • doc (Document) – document

Return type

Optional[Tuple[str, ForwardRef]]

Returns

returns a couple of ID and Document if there’s a document to persist

delete_if_exists(key)[source]

Adds a document to the buffer pool or updates it if it already exists

Parameters

key – document key

docs_to_flush()[source]

Persists the updated documents in disk

Return type

List[Tuple[str, ForwardRef]]

Returns

returns a list of documents to be flushed

clear()[source]

Clears the memory buffer