postgresql学习笔记--基础篇 -psql工具

--创建用户
CREATE ROLE pguser WITH ENCRYPTED PASSWORD 'pguser';
--创建表空间目录
mkdir -p /database/pg10/pg_tbs/tbs_mydb
--创建表空间
CREATE TABLESPACE tbs_mydb OWNER pguser LOCATION '/database/pg10/pg_tbs/tbs_mydb';
--创建数据库
CREATE DATABSE mydb WITH OWNER = pguser TEMPLATE=template0 ENCODING='UTF8' TABLESPACE=tbs_mydb;
--赋权
GRANT ALL ON DATABASE mydb TO pguser WITH GRANT OPTION;
GRANT ALL ON TABLESPACE tbs_mydb TO pguser;

1. psql 元命令介绍

1.1 \l 列出所有数据库列表

postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |  Collation  |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 ambari    | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres
                                                             : postgres=CTc/postgres
                                                             : ambari=CTc/postgres
 ambarirca | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres
                                                             : postgres=CTc/postgres
                                                             : mapred=CTc/postgres
 mydb      | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres
                                                             : postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres
                                                             : postgres=CTc/postgres
(6 rows)

1.2 、\db 查看表空间列表

postgres=# \db
       List of tablespaces
    Name    |  Owner   | Location
------------+----------+----------
 pg_default | postgres |
 pg_global  | postgres |
(2 rows)

1.3 \d查看表定义

postgres=# \dt
                  List of relations
 Schema |           Name           | Type  |  Owner
--------+--------------------------+-------+----------
 public | cloud_sql_static         | table | postgres
 public | cloud_subscriber_devices | table | postgres
 public | cloud_subscribers        | table | postgres
 public | cloud_table_static       | table | postgres
 public | sxacc-device-types       | table | postgres
 public | sxacc-devices            | table | postgres
(6 rows)

postgres=# \d cloud_table_static
             Table "public.cloud_table_static"
       Column        |           Type           | Modifiers
---------------------+--------------------------+-----------
 relname             | character varying(255)   |
 seq_scan            | bigint                   |
 seq_tup_read        | bigint                   |
 idx_scan            | bigint                   |
 idx_tup_fetch       | bigint                   |
 n_tup_ins           | bigint                   |
 n_tup_upd           | bigint                   |
 n_tup_del           | bigint                   |
 n_tup_hot_upd       | bigint                   |
 n_live_tup          | bigint                   |
 n_dead_tup          | bigint                   |
 n_mod_since_analyze | bigint                   |
 last_vacuum         | timestamp with time zone |
 last_autovacuum     | timestamp with time zone |
 last_analyze        | timestamp with time zone |
 last_autoanalyze    | timestamp with time zone |
 vacuum_count        | bigint                   |
 autovacuum_count    | bigint                   |
 analyze_count       | bigint                   |
 autoanalyze_count   | bigint                   |
 import_date         | timestamp with time zone |

postgres=#

 1.4 查看表/索引占用空间大小

给测试表test插入500万数据：

postgres=# create table test(id int primary key, name varchar(100));
CREATE TABLE
postgres=# insert into test(id,name) select n,n||'_francs' from generate_series(1,5000000) n;
INSERT 0 5000000
postgres=# \di+ test_pkey
                          List of relations
 Schema |   Name    | Type  |  Owner   | Table |  Size  | Description
--------+-----------+-------+----------+-------+--------+-------------
 public | test_pkey | index | postgres | test  | 107 MB |
(1 row)

postgres=# \dt+ test
                    List of relations
 Schema | Name | Type  |  Owner   |  Size  | Description
--------+------+-------+----------+--------+-------------
 public | test | table | postgres | 249 MB |
(1 row)

1.5 \sf 查看函数代码

mydb#\sf random_range
CREATE OR REPLACE FUNCTION pguser.random_range(integer,integer)
     RETURN integer
     LANGUAGE sql
AS $function$
     SELECT ($1 + FLOOR(($2-$1 +1 )* random()))::int4;
     $function$

上述\sf命令后面可以只接函数的名称，或者函数名称及输入参数类型，例如random_range(integer,integer)，Postgres支持名称相同但输入参数类型不同的函数，如果有同名函数，\sf 必须指定参数类型。 

1.6 \x 设置查询结果输出

postgres=# \x
Expanded display is on.
postgres=# select * from test limit 1;
-[ RECORD 1 ]--
id   | 1
name | 1_francs

 

1.7 -E 获取元命令对应的SQL代码

postgres:~$ psql -E
psql (9.5.14)
Type "help" for help.

postgres=# \db
********* QUERY **********
SELECT spcname AS "Name",
  pg_catalog.pg_get_userbyid(spcowner) AS "Owner", pg_catalog.pg_tablespace_location(oid) AS "Location" FROM pg_catalog.pg_tablespace ORDER BY 1; ************************** List of tablespaces Name | Owner | Location ------------+----------+---------- pg_default | postgres | pg_global | postgres | (2 rows) postgres=#

 

1.8 -A 设置非对齐输出模式

psql执行SQL输出默认是对齐模式，例如

postgres@nancloud-onprem-06:~$ psql -c "select datname,dattablespace from pg_database limit 1;"
  datname  | dattablespace
-----------+---------------
 template1 |          1663
(1 row)

--注意返回结果，这里有空行

以上输出，格式是对齐的，psql加上-A选项如下所示：

postgres@nancloud-onprem-06:~$ psql -A -c "select datname,dattablespace from pg_database limit 1;"
datname|dattablespace
template1|1663
(1 row) --注意返回结果，没有空行

加上-A选项以后输出的格式变成不对齐的了，并且返回结果中没有空行。

1.9 -t 只显示记录数据

-t 参数设置输出只显示数据，而不显示字段名称和返回的结果集行数。

postgres@nancloud-onprem-06:~$ psql -t -c "select datname,dattablespace from pg_database limit 1;"
 template1 |          1663
--注意返回结果，这里有空行

 

postgres@nancloud-onprem-06:~$ psql -At -c "select datname,dattablespace from pg_database limit 1;"
template1|1663    --> 注意这里没有空行
postgres@nancloud-onprem-06:~$

以上在写shell 脚本非常有效。 

1.10 -q 不显示输出信息

默认情况下，使用psql执行sql命令会返回多种消息，使用-q参数后将不再显示这些信息。

postgres@nancloud-onprem-06:~$ psql -f test.sql
DROP TABLE
CREATE TABLE 
INSERT 0 1
postgres@nancloud-onprem-06:~$ psql  -q -f test.sql
--这里不显示输出信息

 

注意：psql 的-single-transaction 或 -l 选项支持在一个事物中执行脚本，要么脚本中所有SQL执行成功，如果其中有SQL执行失败，则文件中的所有SQL回滚。 

 

2. psql 如何传递变量到SQL

2.1 \set 元命令方式传递变量

\set 元子命令可以设置变量，格式如下所示，name表示变量名称，value表示变量值，如果不填写value，变量值为空。

\set name value

mydb=>\set v_id 2
mydb=>select * from test where id:v_id;
id  | name
----+------
2   |   b
(1 row)

如果想取消之前变量设置的值，\set 命令后接参数名称即可：

mydb=>\set v_id

通过\set 元命令设置变量的一个典型应用场景是使用pgbench进行压力测试。

 

2.2 psql 的-v 参数传递变量

样例：写一个select_1.sql 脚本

select * from test where id:v_id;

通过psql接-v传递变量，并执行脚本

postgre# psql -v v_id=1 mydb pguser -f select_1.sql
id   | name
-----+-------
1    | a
(1 row)

 

2.3 使用psql定制日常维护脚本

先介绍 .psqlrc文件，如果psql没有带-X 选项，psql尝试读取和执行用户~/.psqlrc 启动文件中的命令，结合这个文件能够方便地预先定制维护脚本。

2.3.1.查询活动会话

cloud=# select pid,usename,datname,application_name,client_addr,age(clock_timestamp(), query_start),query from pg_stat_activity where pid<>pg_backend_pid() and state='active' order by query_start desc;
  pid  |   usename   | datname |    application_name    | client_addr |       age       |          query
-------+-------------+---------+------------------------+-------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  6634 | replication |         | walreceiver            | 10.2.38.108 |                 |
  6636 | replication |         | walreceiver            | 10.2.39.66  |                 |
 30315 | calixcloud  | cloud   | PostgreSQL JDBC Driver | 10.2.42.249 | 00:00:00.419892 | SELECT subscriber_id,service_addr,street,city,state,country,postcode FROM cloud_subscribers WHERE lat IS NULL and lon IS NULL AND service_addr IS NOTNULL LIMIT 1
  7791 | calixcloud  | cloud   | PostgreSQL JDBC Driver | 10.2.35.36  | 00:00:00.602242 | UPDATE "sxacc-net-perf-test" t SET info = jsonb_set(t.info,'{worker}','"gcs4-us.calix.com.8339"',true) FROM (SELECT t.info FROM "sxacc-net-perf-test"t WHERE (t.info->>'nextRunTime')::jsonb < '{"$date":1576820536640}' AND (t.info->'worker')::jsonb IS NULL LIMIT 1)  t1 WHERE t.info->>'_id' =  t1.info->>'_id' RETURNING  t1.*
  9747 | calixcloud  | cloud   | PostgreSQL JDBC Driver | 10.2.36.242 | 00:00:00.979992 | UPDATE "sxacc-net-perf-test" t SET info = jsonb_set(t.info,'{worker}','"gcs3-us.calix.com.32325"',true) FROM (SELECT t.info FROM "sxacc-net-perf-test t WHERE (t.info->>'nextRunTime')::jsonb < '{"$date":1576820536261}' AND (t.info->'worker')::jsonb IS NULL LIMIT 1)  t1WHERE t.info->>'_id' =  t1.info->>'_id' RETURNING  t1.*
  8615 | calixcloud  | cloud   | PostgreSQL JDBC Driver | 10.2.35.127 | 00:00:01.071608 | UPDATE "sxacc-net-perf-test" t SET info = jsonb_set(t.info,'{worker}','"gcs2-us.calix.com.1277"',true) FROM (SELECT t.info FROM "sxacc-net-perf-test"t WHERE (t.info->>'nextRunTime')::jsonb < '{"$date":1576820536171}' AND (t.info->'worker')::jsonb IS NULL LIMIT 1)  t1 WHERE t.info->>'_id' =  t1.info->>'_id' RETURNING  t1.*
 14164 | postgres    | cloud   | psql                   |             | 00:05:52.667716 | VACUUM (VERBOSE, ANALYZE) "sxacc-files";
(7 rows)

• active： 任务正在执行
• idle：后台进程为空闲状态，等待后续客户端发出命令
• idle in transaction：后台进程正在事务中，并不是指正在执行SQL
• idle in transaction（aborted）：和idle in transaction 类似，只是事务中的部分SQL异常

vim ~/.psqlrc
-- 设置活动会话
\set active_session 'select pid,usename,datname,application_name,client_addr,age(clock_timestamp(), query_start),query from pg_stat_activity where pid<>pg_backend_pid() and state=\'active\' order by query_start desc;'

之后重新连接数据库，执行:active_session即可。 

postgres#:active_session
  pid  |   usename   | datname |    application_name    | client_addr |       age       |          query
-------+-------------+---------+------------------------+-------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  6634 | replication |         | walreceiver            | 10.2.38.108 |                 |
  6636 | replication |         | walreceiver            | 10.2.39.66  |                 |
 30315 | calixcloud  | cloud   | PostgreSQL JDBC Driver | 10.2.42.249 | 00:00:00.419892 | SELECT subscriber_id,service_addr,street,city,state,country,postcode FROM cloud_subscribers WHERE lat IS NULL and lon IS NULL AND service_addr IS NOTNULL LIMIT 1
 7791 | calixcloud | cloud | PostgreSQL JDBC Driver | 10.2.35.36 | 00:00:00.602242 | UPDATE "sxacc-net-perf-test" t SET info = jsonb_set(t.info,'{worker}','"gcs4-us.calix.com.8339"',true) FROM (SELECT t.info FROM "sxacc-net-perf-test"t WHERE (t.info->>'nextRunTime')::jsonb < '{"$date":1576820536640}' AND (t.info->'worker')::jsonb IS NULL LIMIT 1) t1 WHERE t.info->>'_id' = t1.info->>'_id' RETURNING t1.*
 9747 | calixcloud | cloud | PostgreSQL JDBC Driver | 10.2.36.242 | 00:00:00.979992 | UPDATE "sxacc-net-perf-test" t SET info = jsonb_set(t.info,'{worker}','"gcs3-us.calix.com.32325"',true) FROM (SELECT t.info FROM "sxacc-net-perf-test t WHERE (t.info->>'nextRunTime')::jsonb < '{"$date":1576820536261}' AND (t.info->'worker')::jsonb IS NULL LIMIT 1) t1WHERE t.info->>'_id' = t1.info->>'_id' RETURNING t1.*
 8615 | calixcloud | cloud | PostgreSQL JDBC Driver | 10.2.35.127 | 00:00:01.071608 | UPDATE "sxacc-net-perf-test" t SET info = jsonb_set(t.info,'{worker}','"gcs2-us.calix.com.1277"',true) FROM (SELECT t.info FROM "sxacc-net-perf-test"t WHERE (t.info->>'nextRunTime')::jsonb < '{"$date":1576820536171}' AND (t.info->'worker')::jsonb IS NULL LIMIT 1) t1 WHERE t.info->>'_id' = t1.info->>'_id' RETURNING t1.*
 14164 | postgres | cloud | psql | | 00:05:52.667716 | VACUUM (VERBOSE, ANALYZE) "sxacc-files";
(7 rows)

 

2.3.2.查询等待事件

-- check wait events
\set wait_event 'select pid,application_name,client_addr,age(clock_timestamp(),query_start),state,wait_event_type,wait_event from pg_stat_activity where pid<>pg_backend_pid() and wait_event is not null order by wait_event_type;'

cloud=# :wait_event
  pid  |    application_name    |  client_addr   |       age       | state  | wait_event_type |     wait_event
-------+------------------------+----------------+-----------------+--------+-----------------+---------------------
  6576 |                        |                |                 |        | Activity        | CheckpointerMain
  6636 | walreceiver            | 10.2.39.66     |                 | active | Activity        | WalSenderMain
  6634 | walreceiver            | 10.2.38.108    |                 | active | Activity        | WalSenderMain
  6581 |                        |                |                 |        | Activity        | LogicalLauncherMain 6577 | | | | | Activity | BgWriterMain 6578 | | | | | Activity | WalWriterMain 8466 | PostgreSQL JDBC Driver | 10.2.41.193 | 00:08:38.596477 | idle | Client | ClientRead 18770 | PostgreSQL JDBC Driver | 10.2.42.249 | 00:28:02.041479 | idle | Client | ClientRead 9401 | PostgreSQL JDBC Driver | 10.2.42.219 | 00:07:35.883319 | idle | Client | ClientRead 5690 | PostgreSQL JDBC Driver | 10.2.41.193 | 00:10:09.736458 | idle | Client | ClientRead

 

3. PSQL 其他功能

 3.1 \timing 显示SQL执行时间

cloud=# \timing
Timing is on.
cloud=# select tablename,tableowner from pg_tables limit 10;
             tablename             | tableowner
-----------------------------------+------------
 pg_statistic                      | postgres
 cloud_subscriber_services_bak     | calixcloud
 sxacc-es-index                    | calixcloud
 pg_foreign_table                  | postgres
 pg_authid                         | postgres
 cloud_subscriber_devices_12670060 | postgres
 cloud_subscribers_12670060        | postgres
 pg_user_mapping                   | postgres
 pg_subscription                   | postgres
 pg_largeobject                    | postgres
(10 rows)

Time: 64.629 ms
cloud=# \timing
Timing is off.

 

3.2 \watch 反复执行当前SQL

\watch 元命令会反复执行当前查询缓冲区的SQL命令，直到SQL被终止或执行失败。语法如下：

\watch [seconds]

seconds表示两次执行的间隔时间，以秒为单位，默认为2s，例如

postgres=# select now()
postgres-# ;
              now
-------------------------------
 2019-12-23 11:06:17.857293+08
(1 row)

postgres=# \watch 3
Watch every 3s  Mon Dec 23 11:06:21 2019

              now
-------------------------------
 2019-12-23 11:06:21.457454+08
(1 row)

Watch every 3s  Mon Dec 23 11:06:24 2019

              now
-------------------------------
 2019-12-23 11:06:24.461225+08
(1 row)

Watch every 3s  Mon Dec 23 11:06:27 2019

              now
-------------------------------
 2019-12-23 11:06:27.465306+08
(1 row)

 

3.3 psql 客户端提示符

用户可以根据喜好设置psql客户端提示符，常用选项如下：

• %M：数据库服务器别名，不是指主机名，显示的是psql -h 参数设置的值；
• %> ：数据库服务器的端口号
• %n ：数据库会话的用户名，在数据库会话期间，这个值可能会因为命令SET SESSION AUTHORIZATION的结果而改变
• %/  ：当前数据库名称
• %# ：如果是超级用户则显示#，其他用户显示>，这个值可能会因为命令SET SESSION AUTHORIZATION的结果而改变
• %p ：当前数据库连接的后台进程号
• %R：在PROMPT1中通常显示＂＝＂,如果进程被断开则显示！

 

postgres=# \echo :PROMPT1
%/%R%#

  postgres=# \set PROMPT1 '%/@%M:%>%R%#'
  postgres@[local]:5432=#

同样可以编辑~/.psqlrc

vim ~/.psqlrc
 \set PROMPT1 '%/@%M:%>%R%#'