标签:img ble opera git nta get har option ssi
cstore_fdw 是citus 团队开源的pg 列式存储扩展,可以加速我们的数据分析,关于列式存储以及行式存储的比较
可以参考下边连接的动图(来自clickhouse 官方网站)
https://clickhouse.tech/docs/en/
以下是关于cstore fdw 的简单使用
version: ‘3‘
services:
pgspider-cstore:
image: dalongrong/pgspider:cstore
ports:
- "5433:5432"
environment:
- "POSTGRES_PASSWORD=dalong"
启动docker-compose 服务
docker-compose up -d
下载demo 数据
wget http://examples.citusdata.com/customer_reviews_1998.csv.gz
wget http://examples.citusdata.com/customer_reviews_1999.csv.gz
加压文件:
gzip -d customer_reviews_1998.csv.gz
gzip -d customer_reviews_1999.csv.gz
copy 数据到容器,使用docker cp 命令
CREATE EXTENSION cstore_fdw;
CREATE SERVER cstore_server FOREIGN DATA WRAPPER cstore_fdw;
CREATE FOREIGN TABLE customer_reviews
(
customer_id TEXT,
review_date DATE,
review_rating INTEGER,
review_votes INTEGER,
review_helpful_votes INTEGER,
product_id CHAR(10),
product_title TEXT,
product_sales_rank BIGINT,
product_group TEXT,
product_category TEXT,
product_subcategory TEXT,
similar_product_ids CHAR(10)[]
)
SERVER cstore_server
OPTIONS(compression ‘pglz‘);
容器内部
导入数据:
\COPY customer_reviews FROM ‘customer_reviews_1998.csv‘ WITH CSV;
\COPY customer_reviews FROM ‘customer_reviews_1999.csv‘ WITH CSV;
SELECT
customer_id, review_date, review_rating, product_id, product_title
FROM
customer_reviews
WHERE
customer_id =‘A27T7HVDXA3K2A‘ AND
product_title LIKE ‘%Dune%‘ AND
review_date >= ‘1998-01-01‘ AND
review_date <= ‘1998-12-31‘;
实际如果为了生成cstore的列式数据,我们可以通过pg_cron 扩展,集成起来都是很不错的
https://github.com/citusdata/cstore_fdw
https://github.com/citusdata/pg_cron
https://github.com/rongfengliang/pgspider-docker
pgspider cstore fdw docker 镜像试用
标签:img ble opera git nta get har option ssi
原文地址:https://www.cnblogs.com/rongfengliang/p/12355547.html