FileStream¶
iotoolz.FileStream
is a stream interface implemented with python's built-in open
function.
iotoolz.file.FileStream
¶
FileStream is the stream interface to the local file system with python's "open" method.
supported_schemas: Set[str]
¶
Methods¶
__init__(self, uri, mode='r', buffering=-1, encoding=None, newline=None, content_type='', inmem_size=None, delimiter=None, chunk_size=8192, etag='', **kwargs)
special
¶
exists(self)
¶
Whether the stream points to an existing resource.
is_dir(self)
¶
Whether stream points to a existing dir.
is_file(self)
¶
Whether stream points to a existing file.
iter_dir_(self)
¶
If the current stream is a directory, this method should yield StreamInfo in the directory. Otherwise, it should yield other StreamInfo in the same directory (or level) as the current stream.
mkdir(self, mode=511, parents=False, exist_ok=False)
¶
Create a new directory at this given path. If mode is given, it is combined with the process’ umask value to determine the file mode and access flags. If the path already exists, FileExistsError is raised.
If parents is true, any missing parents of this path are created as needed; they are created with the default permissions without taking mode into account (mimicking the POSIX mkdir -p command).
If parents is false (the default), a missing parent raises FileNotFoundError.
If exist_ok is false (the default), FileExistsError is raised if the target directory already exists.
If exist_ok is true, FileExistsError exceptions will be ignored (same behavior as the POSIX mkdir -p command), but only if the last path component is not an existing non-directory file.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
mode |
int |
mask mode. Defaults to 0o777. |
511 |
parents |
bool |
If true, creates any parents if required. Defaults to False. |
False |
exist_ok |
bool |
If true, will not raise exception if dir already exists. Defaults to False. |
False |
read_to_iterable_(self, uri, chunk_size, fileobj, **kwargs)
¶
read_to_iterable_ is an abstract method to implement the reading of the source resource into the a binary iterator - i.e. you will need to encode your data appropriately if it is a string.
Alternatively, 'fileobj' argument is also provided where you can write to the stream file buffer directly. In this case, you should return an empty iterable. However, this will be less efficient as the actual read will only start after all the data have been read into the buffer.
The method should return a tuple of the bytes iterator and StreamInfo object which ideally should provide the following info:
-
content_type: (see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type)
-
encoding: (see https://docs.python.org/2.4/lib/standard-encodings.html)
-
etag: (see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag)
An optional dict "extras" is also provided for any other information.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
uri |
str |
source uri to the resource |
required |
chunk_size |
int |
size for each chunk |
required |
fileobj |
IO[bytes] |
(IO[bytes]): temp fileobj for the stream |
required |
Exceptions:
Type | Description |
---|---|
NotImplementedError |
[description] |
Returns:
Type | Description |
---|---|
Tuple[Iterable[bytes], iotoolz._abc.StreamInfo] |
Tuple[Iterable[bytes], StreamInfo]: tuple of the iterable and StreamInfo object describing the data |
rmdir(self, ignore_errors=False, **kwargs)
¶
Remove the entire directory.
stats_(self)
¶
Retrieve the StreamInfo.
unlink(self, missing_ok=True, **kwargs)
¶
Delete and remove the resource.
write_from_fileobj_(self, uri, fileobj, size, **kwargs)
¶
Abstract method to implement the write to the destination resource from the provided file-like object.
This file-like object only provides binary outputs - i.e. if the resources only accepts text input, you will need to decode the output inside this method.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
uri |
str |
destination uri of the resource |
required |
fileobj |
IO[bytes] |
file-like object to read from |
required |
size |
int |
size of the data inside the file-like object |
required |
Exceptions:
Type | Description |
---|---|
NotImplementedError |
[description] |
Returns:
Type | Description |
---|---|
StreamInfo |
StreamInfo: StreamInfo object describing the data |