# PostgreSQL

From ArchWiki
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Related articles

- PhpPgAdmin

PostgreSQL is an open source, community driven, standard compliant object-relational database system.

## Installation

This article or section needs language, wiki syntax or style improvements. See Help:Style for reference.

Reason: Do not duplicate sudo and su. (Discuss in Talk:PostgreSQL#)

Install the postgresql package. It will also create a system user called postgres.

Warning: See #Upgrading PostgreSQL for necessary steps before installing new versions of the PostgreSQL packages.
Note: Commands that should be run as the postgres user are prefixed by `[postgres]$` in this article.

You can switch to the PostgreSQL user by executing the following command:

- If you have sudo and are in sudoers:

      $ sudo -iu postgres

- Otherwise using su:

      $ su
      # su -l postgres

See sudo(8) or su(1) for their usage.

## Initial configuration

Before PostgreSQL can function correctly, the database cluster must be initialized:

    [postgres]$ initdb -D /var/lib/postgres/data

Where `-D` is the default location where the database cluster must be stored (see #Change default data directory if you want to use a different one).

Note that by default, the locale and the encoding for the database cluster are derived from your current environment (using $LANG value). [1] However, depending on your settings and use cases this might not be what you want, and you can override the defaults using:

- `--locale=locale`, where locale is to be chosen amongst the system's available locales;
- `-E encoding` for the encoding (which must match the chosen locale);

Example:

    [postgres]$ initdb --locale=en_US.UTF-8 -E UTF8 -D /var/lib/postgres/data

Many lines should now appear on the screen with several ending by `... ok`:

    The files belonging to this database system will be owned by user "postgres".
    This user must also own the server process.

    The database cluster will be initialized with locale "en_US.UTF-8".
    The default database encoding has accordingly been set to "UTF8".
    The default text search configuration will be set to "english".

    Data page checksums are disabled.

    fixing permissions on existing directory /var/lib/postgres/data ... ok
    creating subdirectories ... ok
    selecting default max_connections ... 100
    selecting default shared_buffers ... 128MB
    selecting dynamic shared memory implementation ... posix
    creating configuration files ... ok
    running bootstrap script ... ok
    performing post-bootstrap initialization ... ok
    syncing data to disk ... ok

    WARNING: enabling "trust" authentication for local connections
    You can change this by editing pg_hba.conf or using the option -A, or
    --auth-local and --auth-host, the next time you run initdb.

    Success. You can now start the database server using:

        pg_ctl -D /var/lib/postgres/ -l logfile start

If these are the kind of lines you see, then the process succeeded. Return to the regular user using `exit`.

Note: To read more about this `WARNING`, see #Restricts access rights to the database superuser by default.
Tip: If you change the root to something other than `/var/lib/postgres`, you will have to edit the service file. If the root is under `home`, make sure to set `ProtectHome` to false.
Warning:

- If the database resides on a Btrfs file system, you should consider disabling Copy-on-Write for the directory before creating any database.
- If the database resides on a ZFS file system, you should consult ZFS#Databases before creating any database.

Finally, start and enable the `postgresql.service`.

## Create your first database/user

Tip: If you create a PostgreSQL user with the same name as your Linux username, it allows you to access the PostgreSQL database shell without having to specify a user to login (which makes it quite convenient).

Become the postgres user. Add a new database user using the createuser command:

    [postgres]$ createuser --interactive

Create a new database over which the above user has read/write privileges using the createdb command (execute this command from your login shell if the database user has the same name as your Linux user, otherwise add `-O database-username` to the following command):

    $ createdb myDatabaseName

Tip: If you did not grant your new user database creation privileges, add `-U postgres` to the previous command.

## Familiarize with PostgreSQL

### Access the database shell

Become the postgres user. Start the primary database shell, psql, where you can do all your creation of databases/tables, deletion, set permissions, and run raw SQL commands. Use the `-d` option to connect to the database you created (without specifying a database, `psql` will try to access a database that matches your username).

    [postgres]$ psql -d myDatabaseName

Some helpful commands:

Get help:

    => \help

Connect to a particular database:

    => \c <database>

List all users and their permission levels:

    => \du

Show summary information about all tables in the current database:

    => \dt

Exit/quit the `psql` shell:

    => \q or CTRL+d

There are of course many more meta-commands, but these should help you get started. To see all meta-commands run:

    => \?

## Optional configuration

The PostgreSQL database server configuration file is `postgresql.conf`. This file is located in the data directory of the server, typically `/var/lib/postgres/data`. This folder also houses the other main configuration files, including the `pg_hba.conf` which defines authentication settings, for both local users and other hosts ones.

Note: By default, this folder will not be browsable or searchable by a regular user. This is why `find` and `locate` are not finding the configuration files.

### Restricts access rights to the database superuser by default

The defaults `pg_hba.conf` allow any local user to connect as any database user, including the database superuser. This is likely not what you want, so in order to restrict global access to the postgres user, change the following line:

    /var/lib/postgres/data/pg_hba.conf

    # TYPE  DATABASE        USER            ADDRESS                 METHOD

    # "local" is for Unix domain socket connections only
    local   all             all                                     trust

To:

    /var/lib/postgres/data/pg_hba.conf

    # TYPE  DATABASE        USER            ADDRESS                 METHOD

    # "local" is for Unix domain socket connections only
    local   all             postgres                                peer

You might later add additional lines depending on your needs or software ones.

### Require password for login

Edit `/var/lib/postgres/data/pg_hba.conf` and set the authentication method for each user (or "all" to affect all users) to `scram-sha-256` (preferred), or `md5` (less secure; should be avoided if possible):

    /var/lib/postgres/data/pg_hba.conf

    # TYPE  DATABASE        USER            ADDRESS                 METHOD

    # "local" is for Unix domain socket connections only
    local   all             user                                    scram-sha-256

If you choose `scram-sha-256`, you must also edit `/var/lib/postgres/data/postgresql.conf` and set:

    /var/lib/postgres/data/postgresql.conf

    password_encryption = scram-sha-256

Restart `postgresql.service`, and then re-add each user's password using `ALTER USER user WITH ENCRYPTED PASSWORD 'password';`.

### Configure PostgreSQL to be accessible exclusively through UNIX Sockets

In the connections and authentications section of your configuration, set:

    /var/lib/postgres/data/postgresql.conf

    listen_addresses = ''

This will disable network listening completely. After this you should restart `postgresql.service` for the changes to take effect.

### Configure PostgreSQL to be accessible from remote hosts

In the connections and authentications section, set the `listen_addresses` line to your needs:

    /var/lib/postgres/data/postgresql.conf

    listen_addresses = 'localhost,my_local_ip_address'

You can use `'*'` to listen on all available addresses.

Note: PostgreSQL uses TCP port `5432` by default for remote connections. Make sure this port is open in your firewall and able to receive incoming connections. You can also change it in the configuration file, right below `listen_addresses`

Then add a line like the following to the authentication config:

    /var/lib/postgres/data/pg_hba.conf

    # TYPE  DATABASE        USER            ADDRESS                 METHOD
    # IPv4 local connections:
    host    all             all             ip_address/32   md5

where `ip_address` is the IP address of the remote client.

See the documentation for pg_hba.conf.

Note: Neither sending your plain password nor the md5 hash (used in the example above) over the Internet is secure if it is not done over an SSL-secured connection. See Secure TCP/IP Connections with SSL for how to configure PostgreSQL with SSL.

After this you should restart `postgresql.service` for the changes to take effect.

For troubleshooting take a look in the server log file:

    # journalctl -u postgresql.service

### Configure PostgreSQL authenticate against PAM

PostgreSQL offers a number of authentication methods. If you would like to allow users to authenticate with their system password, additional steps are necessary. First you need to enable PAM for the connection.

For example, the same configuration as above, but with PAM enabled:

    /var/lib/postgres/data/pg_hba.conf

    # IPv4 local connections:
    host   all   all   my_remote_client_ip_address/32   pam

The PostgreSQL server is however running without root privileges and will not be able to access `/etc/shadow`. We can work around that by allowing the postgres group to access this file:

    # setfacl -m g:postgres:r /etc/shadow

### Change default data directory

The default directory where all your newly created databases will be stored is `/var/lib/postgres/data`. To change this, follow these steps:

Create the new directory and make the postgres user its owner:

    # mkdir -p /pathto/pgroot/data
    # chown -R postgres:postgres /pathto/pgroot

Become the postgres user, and initialize the new cluster:

    [postgres]$ initdb -D /pathto/pgroot/data

Edit `postgresql.service` to create a drop-in file and override the `Environment` and `PIDFile` settings. For example:

    /etc/systemd/system/postgresql.service.d/PGROOT.conf

    [Service]
    Environment=PGROOT=/pathto/pgroot
    PIDFile=/pathto/pgroot/data/postmaster.pid

If you want to use `/home` directory for default directory or for tablespaces, add one more line in this file:

    ProtectHome=false

### Change default encoding of new databases to UTF-8

Note: If you ran `initdb` with `-E UTF8` or while using an UTF-8 locale, these steps are not required.

When creating a new database (e.g. with `createdb blog`) PostgreSQL actually copies a template database. There are two predefined templates: `template0` is vanilla, while `template1` is meant as an on-site template changeable by the administrator and is used by default. In order to change the encoding of a new database, one of the options is to change on-site `template1`. To do this, log into PostgreSQL shell (`psql`) and execute the following:

First, we need to drop `template1`. Templates cannot be dropped, so we first modify it so it is an ordinary database:

    UPDATE pg_database SET datistemplate = FALSE WHERE datname = 'template1';

Now we can drop it:

    DROP DATABASE template1;

The next step is to create a new database from `template0`, with a new default encoding:

    CREATE DATABASE template1 WITH TEMPLATE = template0 ENCODING = 'UNICODE';

Now modify `template1` so it is actually a template:

    UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template1';

Optionally, if you do not want anyone connecting to this template, set `datallowconn` to `FALSE`:

    UPDATE pg_database SET datallowconn = FALSE WHERE datname = 'template1';

Note: This last step can create problems when upgrading via `pg_upgrade`.

Now you can create a new database:

    [postgres]$ createdb blog

If you log back in to `psql` and check the databases, you should see the proper encoding of your new database:

    \l

                                  List of databases
      Name    |  Owner   | Encoding  | Collation | Ctype |   Access privileges
    -----------+----------+-----------+-----------+-------+----------------------
    blog      | postgres | UTF8      | C         | C     |
    postgres  | postgres | SQL_ASCII | C         | C     |
    template0 | postgres | SQL_ASCII | C         | C     | =c/postgres
                                                         : postgres=CTc/postgres
    template1 | postgres | UTF8      | C         | C     |

## Graphical tools

- phpPgAdmin — Web-based administration tool for PostgreSQL.

  https://github.com/phppgadmin/phppgadmin || phppgadmin

- pgAdmin — Comprehensive design and management GUI for PostgreSQL.

  https://www.pgadmin.org/ || pgadmin3AUR or pgadmin4

- pgModeler — Graphical schema designer for PostgreSQL.

  https://pgmodeler.io/ || pgmodelerAUR

For tools supporting multiple DBMSs, see List of applications/Documents#Database tools.

## Upgrading PostgreSQL

This article or section needs language, wiki syntax or style improvements. See Help:Style for reference.

Reason: Do not show basic systemctl commands, etc. (Discuss in Talk:PostgreSQL#)

This article or section needs expansion.

Reason: How to upgrade when using third party extensions? (Discuss in Talk:PostgreSQL#pg_upgrade problem if extensions (like postgis) are used)

Upgrading major PostgreSQL versions requires some extra maintenance.

Note:

- Official PostgreSQL upgrade documentation should be followed.
- From version `10.0` onwards PostgreSQL changed its versioning scheme. Earlier upgrade from version `9.x` to `9.y` was considered as major upgrade. Now upgrade from version `10.x` to `10.y` is considered as minor upgrade and upgrade from version `10.x` to `11.y` is considered as major upgrade.

Warning: The following instructions could cause data loss. Do not run the commands below blindly, without understanding what they do. Backup database first.

Get the currently used database version via

    # cat /var/lib/postgres/data/PG_VERSION

To ensure you do not accidentally upgrade the database to an incompatible version, it is recommended to skip updates to the PostgreSQL packages:

    /etc/pacman.conf

    ...
    IgnorePkg = postgresql postgresql-libs
    ...

Minor version upgrades are safe to perform. However, if you do an accidental upgrade to a different major version, you might not be able to access any of your data. Always check the PostgreSQL home page to be sure of what steps are required for each upgrade. For a bit about why this is the case, see the versioning policy.

There are two main ways to upgrade your PostgreSQL database. Read the official documentation for details.

### pg_upgrade

For those wishing to use `pg_upgrade`, a postgresql-old-upgrade package is available that will always run one major version behind the real PostgreSQL package. This can be installed side-by-side with the new version of PostgreSQL. To upgrade from older versions of PostgreSQL there are AUR packages available: postgresql-96-upgradeAUR, postgresql-95-upgradeAUR, postgresql-94-upgradeAUR, postgresql-93-upgradeAUR, postgresql-92-upgradeAUR. Read the pg_upgrade(1) man page to understand what actions it performs.

Note that the databases cluster directory does not change from version to version, so before running `pg_upgrade`, it is necessary to rename your existing data directory and migrate into a new directory. The new databases cluster must be initialized, as described in the #Installation section.

This article or section needs language, wiki syntax or style improvements. See Help:Style for reference.

Reason: The description of the initdb command is getting way too long and starts duplicating #Initial configuration. Why not just say that the same options which were used for the initial creation should be used here? (Discuss in Talk:PostgreSQL#)

While the database is still accessible, one may take the opportunity to check the locale and encoding used, and whether data checksums are used:

    [postgres]$ psql -l

                                      List of databases
       Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
    -----------+----------+----------+-------------+-------------+-----------------------
     postgres  | postgres | UTF8     | fr_FR.UTF-8 | fr_FR.UTF-8 |
     template0 | postgres | UTF8     | fr_FR.UTF-8 | fr_FR.UTF-8 | =c/postgres          +
               |          |          |             |             | postgres=CTc/postgres
     template1 | postgres | UTF8     | fr_FR.UTF-8 | fr_FR.UTF-8 | =c/postgres          +
               |          |          |             |             | postgres=CTc/postgres
    (3 rows)

    [postgres]$ psql -c "SHOW data_checksums"

     data_checksums
    ----------------
     off
    (1 row)

When you are ready, stop the postgresql service, upgrade the following packages: postgresql, postgresql-libs, and postgresql-old-upgrade. Finally upgrade the databases cluster.

Stop and make sure PostgreSQL is stopped:

    # systemctl stop postgresql.service
    # systemctl status postgresql.service

Make sure that PostgresSQL was stopped correctly. If it failed, `pg_upgrade` will fail too.

Upgrade the packages:

    # pacman -S postgresql postgresql-libs postgresql-old-upgrade

Rename the databases cluster directory, and create an empty one:

    # mv /var/lib/postgres/data /var/lib/postgres/olddata
    # mkdir /var/lib/postgres/data /var/lib/postgres/tmp
    # chown postgres:postgres /var/lib/postgres/data /var/lib/postgres/tmp
    [postgres]$ cd /var/lib/postgres/tmp
    [postgres]$ initdb -D /var/lib/postgres/data

Should you have a conflicting system locale and encoding, add `--locale=xy_XY.UTF-8` `--encoding=UTF8` options (where xx_YY matches your need). Should you have data-checksums enabled, add `--data-checksums` option.

Upgrade the cluster, replacing `PG_VERSION` below, with the old PostgreSQL version number (e.g. `12`):

    [postgres]$ pg_upgrade -b /opt/pgsql-PG_VERSION/bin -B /usr/bin -d /var/lib/postgres/olddata -D /var/lib/postgres/data

Note: If necessary, adjust the configuration files of new cluster (e.g. `pg_hba.conf` and `postgresql.conf`) to match the old cluster.

Start the cluster:

    # systemctl start postgresql.service

`pg_upgrade` will have created the scripts `analyze_new_cluster.sh` and `delete_old_cluster.sh` in `/var/lib/postgres/tmp/` and will have output some instructions about running these.

- `analyze_new_cluster.sh` generates optimizer statistics for the new cluster and should be run as user `postgres`. It requires the `postgresql` service to have been started.
- `delete_old_cluster.sh` simply deletes the directory `/var/lib/postgres/olddata` and should be run as a user with write privileges for `/var/lib/postgres` (e.g. as `root`).

You may delete the `/var/lib/postgres/tmp` directory once the upgrade is completely over.

### Manual dump and reload

You could also do something like this (after the upgrade and install of postgresql-old-upgrade).

Note:

- Below are the commands for upgrading from PostgreSQL 12. You can find similar commands in `/opt/` for your version of PostgreSQL cluster, provided you have matching version of postgresql-old-upgrade package installed.
- If you had customized your `pg_hba.conf` file, you may have to temporarily modify it to allow full access to old database cluster from local system. After upgrade is complete set your customization to new database cluster as well and restart `postgresql.service`.

    # systemctl stop postgresql.service
    # mv /var/lib/postgres/data /var/lib/postgres/olddata
    # mkdir /var/lib/postgres/data
    # chown postgres:postgres /var/lib/postgres/data
    [postgres]$ initdb -D /var/lib/postgres/data
    [postgres]$ /opt/pgsql-12/bin/pg_ctl -D /var/lib/postgres/olddata/ start
    # cp /usr/lib/postgresql/postgis-3.so /opt/pgsql-12/lib/ # Only if postgis installed
    [postgres]$ pg_dumpall -h /tmp -f /tmp/old_backup.sql
    [postgres]$ /opt/pgsql-12/bin/pg_ctl -D /var/lib/postgres/olddata/ stop
    # systemctl start postgresql.service
    [postgres]$ psql -f /tmp/old_backup.sql postgres

## Troubleshooting

### Improve performance of small transactions

If you are using PostgresSQL on a local machine for development and it seems slow, you could try turning synchronous_commit off in the configuration. Beware of the caveats, however.

    /var/lib/postgres/data/postgresql.conf

    synchronous_commit = off

### Prevent disk writes when idle

PostgreSQL periodically updates its internal "statistics" file. By default, this file is stored on disk, which prevents disks from spinning down on laptops and causes hard drive seek noise. It is simple and safe to relocate this file to a memory-only file system with the following configuration option:

    /var/lib/postgres/data/postgresql.conf

    stats_temp_directory = '/run/postgresql'

### pgAdmin 4 issues after upgrade to PostgreSQL 12

If you see errors about `string indices must be integers` when navigating the tree on the left, or about `column rel.relhasoids does not exist` when viewing the data, remove the server from the connection list in pgAdmin and add a fresh server instance. pgAdmin will otherwise continue to treat the server as a PostgreSQL 11 server resulting in these issues.

Category:

- Relational DBMSs

Hidden categories:

- Pages or sections flagged with Template:Style
- Pages or sections flagged with Template:Expansion