Skip to content

consider making html5lib.tokenizer public #532

Description

@mgrandi

Hello,

In version https://github.com/html5lib/html5lib-python/releases/tag/0.999999999 , html5lib.tokenizer was made private

The wpull project (https://github.com/ArchiveTeam/wpull ) uses this library, and if we were to ever migrate to using the 1.X versions, it would negatively impact the application, because instead of just tokenizing a webpage (see https://github.com/ArchiveTeam/wpull/blob/a4ff4a93f613ce18ad3c515aa3d4f5848a88b98c/wpull/document/htmlparse/html5lib_.py ), we would have to use the full tree parsing which is slower and uses more ram

is there any reason this was made private when the 1.x branch was released?

Activity

  1. theRealProHacker commented on Feb 20, 2023

    @theRealProHacker
    Contributor

    I don't understand what you mean with private. How can something be made private in Python?

  2. mgrandi commented on Feb 20, 2023

    @mgrandi
    Author

    This project seems abandoned....but by private I mean that obviously yes you can't make it private it actually in python, but I do mean that it changes location, and usually denoted with an underscore, and that means there is no guarantee that it will be in the same place / renamed / etc in future releases

    Making it public = making it part of the public API so that way even if the underlying implementation changes , the API stays the same

  3. theRealProHacker commented on Feb 21, 2023

    @theRealProHacker
    Contributor

    Yeah, it's really annoying. If you are a maintainer of a project, then at least answer some issues, every month. Even if you don't want to write any new code.

    I now understand what you mean. A simple solution would be to pin the version and then just use the undocumented, or to use your words, private part of the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions