Storage

Delta Storage Handler

DeltaStorageHandler(
    root: str,
    options: dict[str, str] | None = None,
    known_sizes: dict[str, int] | None = None,
)

Bases: DeltaFileSystemHandler, FileSystemHandler

DeltaStorageHandler is a concrete implementations of a PyArrow FileSystemHandler.

get_file_info_selector(
    selector: FileSelector,
) -> List[FileInfo]

Get info for the files defined by FileSelector.

Parameters:

Name	Type	Description	Default
`selector`	`FileSelector`	FileSelector object	required

Returns:

Type	Description
`List[FileInfo]`	list of file info objects

open_input_file(path: str) -> pa.PythonFile

Open an input file for random access reading.

Parameters:

Name	Type	Description	Default
`path`	`str`	The source to open for reading.	required

Returns:

Type	Description
`PythonFile`	NativeFile

open_input_stream(path: str) -> pa.PythonFile

Open an input stream for sequential reading.

Parameters:

Name	Type	Description	Default
`path`	`str`	The source to open for reading.	required

Returns:

Type	Description
`PythonFile`	NativeFile

open_output_stream(
    path: str, metadata: Optional[Dict[str, str]] = None
) -> pa.PythonFile

Open an output stream for sequential writing.

If the target already exists, existing data is truncated.

Parameters:

Name	Type	Description	Default
`path`	`str`	The source to open for writing.	required
`metadata`	`Optional[Dict[str, str]]`	If not None, a mapping of string keys to string values.	`None`

Returns:

Type	Description
`PythonFile`	NativeFile