postgresql 对 lz4 压缩算法的支持

背景

PG 从 14 开始支持 lz4 压缩算法，但是内核默认没有直接支持， 如果要使用， 需要手动编译（configure的时候使用 –with-lz4 选项）

编译选项

./configure '--prefix=/usr/local/pgsql' '--with-python' '--with-openssl' '--with-libxml' '--enable-debug' '--enable-dtrace' 'PYTHON=/usr/bin/python3' '--with-readline' --with-lz4 && make -j8 world  && make install-world

编译过程中遇到报错， 直接 GPT 搜一下，就知道应该安装哪些开发包

使用实例

https://www.rockdata.net/zh-cn/tutorial/toast-lz4-compression/

示例

postgres=# create table test(info text);
CREATE TABLE
postgres=# create table test2(info text compression lz4);
CREATE TABLE


postgres=# \timing 
Timing is on.
postgres=# insert into test2 SELECT lpad('a',1000000,'a') FROM generate_series(1,10000);
INSERT 0 10000
Time: 2088.799 ms (00:02.089)
postgres=# insert into test SELECT lpad('a',1000000,'a') FROM generate_series(1,10000);
INSERT 0 10000
Time: 44354.305 ms (00:44.354)

可以看到速度快了 20 倍

修改默认压缩算法

postgres=# alter system set default_toast_compression='lz4';
ALTER SYSTEM
Time: 2.409 ms
postgres=# select pg_reload_conf();

postgres=# create table test3(info text);
CREATE TABLE
Time: 3.241 ms
postgres=# \d+ test3
                                         Table "public.test3"
 Column | Type | Collation | Nullable | Default | Storage  | Compression | Stats target | Description 
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
 info   | text |           |          |         | extended |             |              | 
Access method: heap

postgres=# insert into test3 SELECT lpad('a',1000000,'a') FROM generate_series(1,10000);
INSERT 0 10000
Time: 2110.107 ms (00:02.110)
postgres=# 

通过速度, 可以看到默认压缩算法生效了

查看表空间占用

postgres=# \d+ test3
                                         Table "public.test3"
 Column | Type | Collation | Nullable | Default | Storage  | Compression | Stats target | Description 
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
 info   | text |           |          |         | extended |             |              | 
Access method: heap

postgres=#  select pg_size_pretty(pg_total_relation_size('test'));
 pg_size_pretty 
----------------
 119 MB
(1 row)

Time: 0.224 ms
postgres=#  select pg_size_pretty(pg_total_relation_size('test2'));
 pg_size_pretty 
----------------
 40 MB
(1 row)

Time: 0.271 ms
postgres=#  select pg_size_pretty(pg_total_relation_size('test3'));
 pg_size_pretty 
----------------
 40 MB
(1 row)

可以看到使用 lz4 算法, 存储空间也有明显的优化