Skip to content

tools/doc/versions.js should cache version data #32512

Description

@rvagg

make doc forces a version fetch which fetches the CHANGELOG.md from this repo. This is done for each of the doc files that requires the versions list (because they are run one at a time). We should have caching in place so that the first one saves the data somewhere and subsequent runs just pick that it up. For speed, and also to save our CI hammering GitHub each time this is run.

Ref: #32511

Activity

  1. hassaanp commented on Mar 27, 2020

    @hassaanp
    Contributor

    Hey @rvagg,

    Do you think that the following flow makes sense:

    1. Initialize and store a timestamp value at the start of the run
    2. Fetch latest CHANGELOG.md and store it in .cache/timestamp-CHANGELOG.md
    3. Check if local .cache/timestamp-CHANGELOG.md exists locally and use that otherwise repeat step 2
  2. rvagg commented on Mar 27, 2020

    @rvagg
    MemberAuthor

    Maybe simpler than that, this will be in a clean repo (in the git clean -fdx sense) and this file can be added to .gitignore so it could be in the top level, .master_CHANGELOG.md or something like that. If it exists, use it, if it doesn't exist, fetch it. I don't think there's a good reason to do any timestamping on this.

    Unless someone thinks otherwise?

    Using https://nodejs.org/download/release/index.json is another option, which would be easy to process, but I guess the reason we use CHANGELOG.md is that we always have a local copy if this needs to be done in offline mode. We just don't have a local master copy unless we're actually running on master, which will lead to a truncated version list.

  3. hassaanp commented on Mar 27, 2020

    @hassaanp
    Contributor

    Acknowledged.

    Here is my approach in versions.js

        ...
        const masterChangelog = path.join(srcRoot, '.master-CHANGELOG.md');
        if (kNoInternet) {
          changelog = readFileSync(file, { encoding: 'utf8' });
        } else if (existsSync(masterChangelog)) {
          changelog = readFileSync(masterChangelog, { encoding: 'utf8' });
        } else {
          try {
            changelog = await getUrl(url);
            createMasterChangelog(masterChangelog, changelog);
          }
        ...
    

    the createMasterChangelog uses writeFileSync to create the .master-CHANGELOG.md

    What do you think

  4. richardlau commented on Mar 27, 2020

    @richardlau
    Member

    I've had some half finished code to address this for awhile (started since #31849 (review)). Tidied up today (using up work vacation today so had time) and submitted as #32518.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueIssues that are suitable for first-time contributors.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions