Skip to content

TypeError: cannot use a string pattern on a bytes-like object #1405

Description

@jumpnett

Description

When launching pgcli, I get the following error:

Exception in thread completion_refresh:
Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.11/threading.py", line 975, in run-mode  [F5] Explain: OFF      Refreshing completions...                                                                            
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3/dist-packages/pgcli/completion_refresher.py", line 68, in _bg_refresh
    refresher(completer, executor)
  File "/usr/lib/python3/dist-packages/pgcli/completion_refresher.py", line 109, in refresh_schemata
    completer.set_search_path(executor.search_path())
  File "/usr/lib/python3/dist-packages/pgcli/pgcompleter.py", line 306, in set_search_path
    self.search_path = self.escaped_names(search_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pgcli/pgcompleter.py", line 150, in escaped_names
    return [self.escape_name(name) for name in names]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pgcli/pgcompleter.py", line 150, in <listcomp>
    return [self.escape_name(name) for name in names]
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pgcli/pgcompleter.py", line 131, in escape_name
    (not self.name_pattern.match(name))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot use a string pattern on a bytes-like object

This error occurs when connecting to a PostgreSQL 9.3.4, but not a PostgreSQL 9.3.20 server.

Your environment

OS: Ubuntu 23.04 x86_64
pgcli Version: 3.5.0
pip freeze:

asn1crypto==1.5.1
attrs==22.2.0
Babel==2.10.3
blinker==1.5
Brlapi==0.8.4
certifi==2022.9.24
chardet==5.1.0
cli-helpers==2.3.0
click==8.1.3
cloud-init==23.1.2
colorama==0.4.6
command-not-found==0.3
configobj==5.0.8
cryptography==38.0.4
cupshelpers==1.0
dbus-python==1.3.2
defer==1.0.6
distro==1.8.0
distro-info==1.5
html5lib==1.1
httplib2==0.20.4
idna==3.3
importlib-metadata==4.12.0
jaraco.classes==3.2.1
jeepney==0.8.0
Jinja2==3.1.2
jsonpatch==1.32
jsonpointer==2.0
jsonschema==4.6.0
keyring==23.9.3
language-selector==0.1
launchpadlib==1.11.0
lazr.restfulclient==0.14.5
lazr.uri==1.0.6
louis==3.24.0
Mako==1.2.4.dev0
markdown-it-py==2.1.0
MarkupSafe==2.1.2
mdurl==0.1.2
mechanize==0.4.8
more-itertools==8.10.0
netifaces==0.11.0
oauthlib==3.2.2
olefile==0.46
pendulum==2.1.2
pexpect==4.8.0
pgcli==3.5.0
pgspecial==2.0.1
Pillow==9.4.0
prompt-toolkit==3.0.36
psycopg==3.1.7
psycopg-pool==3.1.5
psycopg2==2.9.5
ptyprocess==0.7.0
pycairo==1.20.1
pycups==2.0.1
Pygments==2.14.0
PyGObject==3.44.1
PyJWT==2.6.0
pyparsing==3.0.9
pyrsistent==0.18.1
pyserial==3.5
python-apt==2.5.3+ubuntu1
python-dateutil==2.8.2
python-debian==0.1.49+ubuntu2
pytz==2022.7.1
pytzdata==2020.1
pyxdg==0.28
PyYAML==6.0
reportlab==3.6.12
requests==2.28.1
rich==13.3.1
SecretStorage==3.3.3
setproctitle==1.3.1
six==1.16.0
sqlparse==0.4.2
systemd-python==235
tabulate==0.8.9
terminaltables==3.1.10
typing_extensions==4.4.0
ubuntu-advantage-tools==8001
ubuntu-drivers-common==0.0.0
ufw==0.36.1
unattended-upgrades==0.1
urllib3==1.26.12
wadllib==1.3.6
wcwidth==0.2.5
webencodings==0.5.1
xdg==5
xkit==0.0.0
zipp==1.0.0

Activity

  1. j-bennet commented on May 22, 2023

    @j-bennet
    Contributor

    Thanks for reporting @jumpnett. If you're able to submit a PR, it will be very welcome.

  2. pomo-mondreganto commented on May 26, 2023

    @pomo-mondreganto

    I'm experiencing the same error. Adding a few logs, I can see that pg_catalog name is bytes in my Postgres (v15) installation.

  3. hakalyanpur commented on Jun 1, 2023

    @hakalyanpur

    I can replicate the error on my setup too. Thank you. @j-bennet

    Details:
    Server: PostgreSQL 12.11 (Ubuntu 12.11-0ubuntu0.20.04.1)
    PgCli version: 3.5.0

  4. NasaGeek commented on May 22, 2024

    @NasaGeek

    This may be occurring because your database/client's encoding is set to SQL_ASCII (which will prevent the underlying postgres library, psycopg, from decoding strings to Python str objects). Quick example:

    import psycopg
    
    # Assuming you have PG* env vars set
    with psycopg.connect() as conn:
        result = conn.execute("SHOW SERVER_ENCODING").fetchone()
        print(result)
        result = conn.execute("SELECT 'str'").fetchone()[0]
        print(conn.info.encoding, result, type(result))
    
    with psycopg.connect("client_encoding=utf8") as conn:
        result = conn.execute("SHOW SERVER_ENCODING").fetchone()
        print(result)
        result = conn.execute("SELECT 'str'").fetchone()[0]
        print(conn.info.encoding, result, type(result))
    (b'SQL_ASCII',)
    ascii b'str' <class 'bytes'>
    ('SQL_ASCII',)
    utf-8 str <class 'str'>
    
  5. NasaGeek commented on May 22, 2024

    @NasaGeek

    A workaround for the issue was to force the client encoding to utf8 via an environment variable when launching pgcli like so

    PGCLIENTENCODING=utf8 pgcli
    

    This isn't really a permanent solution, as it could result in trying to decode undecodable text from your database (since SQL_ASCII means it could contain anything, that's why psycopg won't decode it by default).

    As a full fix, It's probably reasonable for pgcli to force its own decoding of returned bytes in specific cases like this where it's essentially fetching postgres identifiers (in this case, the elements of the search path) which I would always expect to be decodable. Instead of doing its own decoding, pgcli could also temporarily set the client encoding to utf8 or similar (e.g. SET client_encoding TO utf8) so that psycopg decodes the results

  6. added a commit that references this issue on Aug 3, 2026
    4beae47
  7. added a commit that references this issue on Aug 3, 2026
    d195baa
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions