使用的工具
top - 16:01:06 up 2 days, 3:44, 1 user, load average: 0.09, 0.38, 0.71
Tasks: 117 total, 1 running, 116 sleeping, 0 stopped, 0 zombie
%Cpu0 : 0.0 us, 1.4 sy, 0.0 ni, 98.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 1970.9 total, 354.0 free, 861.4 used, 755.5 buff/cache
MiB Swap: 2045.0 total, 2044.5 free, 0.5 used. 786.5 avail Mem
MiWiFi-R4A-srv :) select count(),name from t10 group by name order by count() desc limit 10;
SELECT
count(),
name
FROM t10
GROUP BY name
ORDER BY count() DESC
LIMIT 10
Query id: 79b52b2e-8fd0-4321-b4d1-270d3800164b
┌─count()─┬─name─────────────────┐
│ 13856 │ 异邦 │
│ 13850 │ 长江三峡水利枢纽工程 │
│ 13849 │ 龙旗 │
│ 13842 │ 王揖唐 │
│ 13832 │ 用费 │
│ 13830 │ 兴办 │
│ 13820 │ 榨油 │
│ 13814 │ 增值 │
│ 13805 │ 宗谱 │
│ 13803 │ 膏粱 │
└─────────┴──────────────────────┘
10 rows in set. Elapsed: 23.574 sec. Processed 1.35 billion rows, 22.66 GB (57.35 million rows/s., 961.18 MB/s.)
$f = fopen(__DIR__ . '/../dict.txt.small.txt', 'r');
$names = [];
while ($str = fgets($f)) {
$arr = explode(' ', trim($str));
if (strlen($arr[0]) > 5) {
$names[] = $arr[0];
}
}
$ck = new Client('tcp://192.168.23.129:9091', 'default', '123456', 'test1');
$data['drop table'] = $ck->query('DROP TABLE IF EXISTS t10');
$table = [
'CREATE TABLE t10 (',
'`name` String',
') ENGINE = MergeTree() ORDER BY name SETTINGS index_granularity = 8192'
];
$data['create table'] = $ck->query(implode("\n", $table));
for ($nj = 0; $nj < 150; $nj++) {
$ck->writeStart('t10', ['name']);
for ($i = 0; $i < 1000; $i++) {
$da = [];
for ($j = 0; $j < 10000; $j++) {
$da[] = [
'name' => $names[mt_rand(1, 101048)],
];
}
$ck->writeBlock($da);
}
$ck->writeEnd();
echo 'nj : ' . $nj . PHP_EOL;
}
本打算写 15 亿的,感觉没必要 中途暂停了 一共写入了 1352100000 条记录。
估计服务器 cpu 核心 8 以上的应该能达到 1s 以内。
1
magua 2021-01-25 16:05:13 +08:00
好狠
|
3
so1n 2021-01-25 16:14:40 +08:00
clickhouse 真香
|
4
anonyp 2021-01-25 16:16:49 +08:00 via iPhone
试下 summingmergetree 存储引擎看看,感觉这个会比较适合聚合的场景
|
6
Yelp 2021-01-25 16:19:22 +08:00
没进来就想到 ClickHouse 了
|
8
Leonard 2021-01-25 16:43:54 +08:00 2
我点进来只是想知道 13 亿人口使用人数最多的十个姓名是哪十个
|