Skip to content

recursive option of fs.readdir results in blocking synchronous I/O instead of async readdir #51749

Description

@Rob--W

Affected URL(s)

https://nodejs.org/api/fs.html#fspromisesreaddirpath-options

Description of the problem

The readdir method specifies a recursive option that "reads the contents of a directory recursively". This can be a potentially expensive/long-running filesystem query.

The current implementation is synchronous (see below), which means that passing recursive:true would block the calling thread until the I/O completes. Ideally, the implementation would not block the calling thread, but until that is implemented (I didn't find any open issue), the least that can be done is to update the documentation to warn about this risk.

Another aspect worth documenting is to explicitly call out that symlinks are NOT followed. If symlinks were to be followed, then that could potentially result in security issues, such as DoS (infinite readdir loop) or directory traversal outside of the specified directory.

Relevant issues / PRs that introduced this feature:

Relevant source:

node/lib/fs.js

Lines 1458 to 1467 in 544cfc5

function readdir(path, options, callback) {
callback = makeCallback(typeof options === 'function' ? options : callback);
options = getOptions(options);
path = getValidatedPath(path);
if (options.recursive != null) {
validateBoolean(options.recursive, 'options.recursive');
}
if (options.recursive) {
callback(null, readdirSyncRecursive(path, options));

Activity

  1. added
    docIssues and PRs related to Node.js documentation.
    on Feb 13, 2024
  2. MoLow commented on Feb 13, 2024

    @MoLow
    Member

    the recursive watching is done synchronously to avoid timing conditions.
    see #51406
    feel free to open a PR for adjusting the docs

  3. added
    fsIssues and PRs related to file-system APIs and the fs module.
    on Feb 13, 2024
  4. Rob--W commented on Feb 14, 2024

    @Rob--W
    ContributorAuthor

    the recursive watching is done synchronously to avoid timing conditions.
    see #51406

    That issue is about fs.watch, which is independent of fs.readdir. If someone really cares about the state synchronously, then they could call fs.readdirSync instead.

    feel free to open a PR for adjusting the docs

    I filed this issue to request documentation of the fact that there is unbounded sync I/O operation in a supposedly async API. Secondly, I am asking for the symlink behavior to be documented because of security implications when the behavior is undefined.

    What is the purpose of the documentation issue template if you close such issues before resolution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    docIssues and PRs related to Node.js documentation.fsIssues and PRs related to file-system APIs and the fs module.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions