Skip to content

Unable to use compress.write_zip() with s3fs source #556

Description

@james-emerton

I believe this ticket should be for this package, but the issue is caused by implementation details of the fs-s3fs package. Attempting to create an archive of files from s3 using write_zip() fails with the following error:

Traceback (most recent call last):
  File "/Users/james/dev/spire/mount-s3-archive/test.py", line 7, in <module>
    write_zip(source, sys.argv[2])
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/compress.py", line 78, in write_zip
    mt = info.modified or datetime.utcnow()
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/info.py", line 322, in modified
    self._require_namespace("details")
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/info.py", line 128, in _require_namespace
    raise MissingInfoNamespace(namespace)
fs.errors.MissingInfoNamespace: namespace 'details' is required for this attribute

Code:

import sys

import fs
from fs.compress import write_zip

source = fs.open_fs(sys.argv[1])
write_zip(source, sys.argv[2])

Executing the above code as test.py s3://some-bucket/ test.zip will fail with the above error while a local source directory will compress nicely.

The issue seems to be that S3FS directory entries don't expose a details namespace. It's unclear if that's correct or not, but the code in write_zip() seems to assume that details is always available (but makes allowance for Info.modified to be a falsy value).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions