0
私はここ数週間問題を診断しようとしています。私たちの生産データベースはQA DB（120ms）で非常に迅速に実行されるクエリを実行するのに非常に長い時間（84秒）を要しています。異なるpostgresデータベース（同じデータを持つ）で同じクエリが700倍長くなります
- 生産DBは、はるかに強力な私たちのQA DB
- のProd Postgresのバージョンよりもです：
PostgreSQL 9.3.16 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4, 64-bit - QA DB Postgresのバージョン：
PostgreSQL 9.6.4 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4, 64-bit - QA DBは、最新のデータでバックアップされます。機密情報の一部スクラブは、バックアップ・プロセスであり
EXPLAIN (ANALYZE, BUFFERS)を実行 - は2つのDBSのクエリ計画が大幅に異なっていることを示しているが、私はあなたがするために行うことができます違い
何かを理解し、トラブルを抱えています私が本当に感謝する正しい方向に私を向ける。
はあなたに
プロダクションクエリプランをありがとう：
Limit (cost=5411.63..5411.63 rows=1 width=47) (actual time=84719.786..84719.788 rows=10 loops=1)
Buffers: shared hit=109988
-> Sort (cost=5411.63..5411.63 rows=1 width=47) (actual time=84719.785..84719.787 rows=10 loops=1)
Sort Key: events.updated_at
Sort Method: top-N heapsort Memory: 25kB
Buffers: shared hit=109988
-> Nested Loop Left Join (cost=5371.22..5411.63 rows=1 width=47) (actual time=273.194..84652.985 rows=25846 loops=1)
Join Filter: (events.id = filters_2.target_id)
Rows Removed by Join Filter: 134062366
Filter: ((filters_2.target_id IS NULL) OR (filters_1.target_id IS NOT NULL))
Rows Removed by Filter: 60
Buffers: shared hit=109988
-> Nested Loop Left Join (cost=2468.49..2481.16 rows=1 width=51) (actual time=119.443..15665.354 rows=25906 loops=1)
Join Filter: (events.id = filters_1.target_id)
Rows Removed by Join Filter: 29713058
Buffers: shared hit=108373
-> Nested Loop (cost=1241.39..1253.61 rows=1 width=47) (actual time=94.514..692.546 rows=25906 loops=1)
Buffers: shared hit=107956
-> Merge Anti Join (cost=1241.31..1241.34 rows=3 width=47) (actual time=94.497..151.266 rows=26171 loops=1)
Merge Cond: (events.id = filters.target_id)
Buffers: shared hit=2439
-> Sort (cost=14.16..14.17 rows=3 width=47) (actual time=64.147..84.460 rows=26171 loops=1)
Sort Key: events.id
Sort Method: quicksort Memory: 3131kB
Buffers: shared hit=2022
-> Bitmap Heap Scan on events (cost=4.18..14.16 rows=3 width=47) (actual time=3.687..46.416 rows=26171 loops=1)
Recheck Cond: (((brand_id = 218) AND (brand_id = 218)) OR ((brand_id IS NULL) AND (brand_id = 218)))
Filter: ((eventable_type)::text = ANY ('{Achievement,Blog,Kudo,Story,User}'::text[]))
Buffers: shared hit=2022
-> BitmapOr (cost=4.18..4.18 rows=5 width=0) (actual time=3.369..3.369 rows=0 loops=1)
Buffers: shared hit=96
-> Bitmap Index Scan on index_events_on_brand_id (cost=0.00..2.10 rows=5 width=0) (actual time=3.366..3.366 rows=26171 loops=1)
Index Cond: ((brand_id = 218) AND (brand_id = 218))
Buffers: shared hit=96
-> Bitmap Index Scan on index_events_on_brand_id (cost=0.00..2.09 rows=1 width=0) (actual time=0.001..0.001 rows=0 loops=1)
Index Cond: ((brand_id IS NULL) AND (brand_id = 218))
-> Sort (cost=1227.15..1227.16 rows=19 width=4) (actual time=30.345..30.345 rows=0 loops=1)
Sort Key: filters.target_id
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=417
-> HashAggregate (cost=1226.95..1227.01 rows=19 width=4) (actual time=30.336..30.336 rows=0 loops=1)
Buffers: shared hit=417
-> Bitmap Heap Scan on filters (cost=381.47..1226.89 rows=127 width=4) (actual time=30.334..30.334 rows=0 loops=1)
Recheck Cond: (((constraint_id = 218) OR ((constraint_id = 5229) OR (constraint_id = 5183) OR (constraint_id = 5148)) OR ((constraint_type)::text = 'Country'::text)) AND ((target_type)::text = 'Event'::text))
Filter: ((NOT visible) AND ((((constraint_type)::text = 'Brand'::text) AND (constraint_id = 218)) OR (((constraint_type)::text = 'Team'::text) AND ((constraint_id = 5229) OR (constraint_id = 5183) OR (constraint_id = 5148))) OR (((constraint_type)::text = 'Country'::text) AND (constraint_id IS NULL))))
Rows Removed by Filter: 1147
Buffers: shared hit=417
-> BitmapAnd (cost=381.47..381.47 rows=634 width=0) (actual time=5.591..5.591 rows=0 loops=1)
Buffers: shared hit=149
-> BitmapOr (cost=32.60..32.60 rows=2733 width=0) (actual time=0.464..0.464 rows=0 loops=1)
Buffers: shared hit=24
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.16 rows=53 width=0) (actual time=0.010..0.010 rows=0 loops=1)
Index Cond: (constraint_id = 218)
Buffers: shared hit=3
-> BitmapOr (cost=28.29..28.29 rows=2680 width=0) (actual time=0.384..0.384 rows=0 loops=1)
Buffers: shared hit=18
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.16 rows=53 width=0) (actual time=0.005..0.005 rows=0 loops=1)
Index Cond: (constraint_id = 5229)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.16 rows=53 width=0) (actual time=0.004..0.004 rows=0 loops=1)
Index Cond: (constraint_id = 5183)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..23.95 rows=2575 width=0) (actual time=0.373..0.373 rows=2979 loops=1)
Index Cond: (constraint_id = 5148)
Buffers: shared hit=12
-> Bitmap Index Scan on index_filters_on_constraint_type (cost=0.00..2.09 rows=1 width=0) (actual time=0.022..0.022 rows=0 loops=1)
Index Cond: ((constraint_type)::text = 'Country'::text)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_target_type (cost=0.00..348.81 rows=35153 width=0) (actual time=5.088..5.088 rows=35092 loops=1)
Index Cond: ((target_type)::text = 'Event'::text)
Buffers: shared hit=125
-> Index Scan using users_pkey on users (cost=0.08..4.09 rows=1 width=6) (actual time=0.016..0.018 rows=1 loops=26171)
Index Cond: (id = events.user_id)
Filter: ((active AND visible) OR ((events.eventable_type)::text = 'Blog'::text))
Rows Removed by Filter: 0
Buffers: shared hit=105517
-> HashAggregate (cost=1227.10..1227.24 rows=47 width=4) (actual time=0.001..0.309 rows=1147 loops=25906)
Buffers: shared hit=417
-> Bitmap Heap Scan on filters filters_1 (cost=381.53..1226.94 rows=311 width=4) (actual time=5.647..6.835 rows=1147 loops=1)
Recheck Cond: (((constraint_id = 218) OR ((constraint_id = 5229) OR (constraint_id = 5183) OR (constraint_id = 5148)) OR ((constraint_type)::text = 'Country'::text)) AND ((target_type)::text = 'Event'::text))
Filter: (visible AND ((((constraint_type)::text = 'Brand'::text) AND (constraint_id = 218)) OR (((constraint_type)::text = 'Team'::text) AND ((constraint_id = 5229) OR (constraint_id = 5183) OR (constraint_id = 5148))) OR (((constraint_type)::text = 'Country'::text) AND (constraint_id IS NULL))))
Buffers: shared hit=417
-> BitmapAnd (cost=381.53..381.53 rows=634 width=0) (actual time=5.595..5.595 rows=0 loops=1)
Buffers: shared hit=149
-> BitmapOr (cost=32.65..32.65 rows=2733 width=0) (actual time=0.449..0.449 rows=0 loops=1)
Buffers: shared hit=24
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.16 rows=53 width=0) (actual time=0.007..0.007 rows=0 loops=1)
Index Cond: (constraint_id = 218)
Buffers: shared hit=3
-> BitmapOr (cost=28.32..28.32 rows=2680 width=0) (actual time=0.378..0.378 rows=0 loops=1)
Buffers: shared hit=18
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.16 rows=53 width=0) (actual time=0.005..0.005 rows=0 loops=1)
Index Cond: (constraint_id = 5229)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.16 rows=53 width=0) (actual time=0.005..0.005 rows=0 loops=1)
Index Cond: (constraint_id = 5183)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..23.95 rows=2575 width=0) (actual time=0.366..0.366 rows=2979 loops=1)
Index Cond: (constraint_id = 5148)
Buffers: shared hit=12
-> Bitmap Index Scan on index_filters_on_constraint_type (cost=0.00..2.09 rows=1 width=0) (actual time=0.014..0.014 rows=0 loops=1)
Index Cond: ((constraint_type)::text = 'Country'::text)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_target_type (cost=0.00..348.81 rows=35153 width=0) (actual time=5.110..5.110 rows=35092 loops=1)
Index Cond: ((target_type)::text = 'Event'::text)
Buffers: shared hit=125
-> HashAggregate (cost=2902.73..2911.49 rows=2920 width=4) (actual time=0.006..1.438 rows=5175 loops=25906)
Buffers: shared hit=1615
-> Bitmap Heap Scan on filters filters_2 (cost=349.80..2892.87 rows=19720 width=4) (actual time=19.215..124.613 rows=34923 loops=1)
Recheck Cond: ((target_type)::text = 'Event'::text)
Filter: (visible AND ((((constraint_type)::text = 'Brand'::text) AND (constraint_id <> 218)) OR (((constraint_type)::text = 'Team'::text) AND ((constraint_id <> 5229) OR (constraint_id <> 5183) OR (constraint_id <> 5148))) OR (((constraint_type)::text = 'Country'::text) AND (constraint_id IS NOT NULL))))
Buffers: shared hit=1615
-> Bitmap Index Scan on index_filters_on_target_type (cost=0.00..348.81 rows=35153 width=0) (actual time=18.952..18.952 rows=35092 loops=1)
Index Cond: ((target_type)::text = 'Event'::text)
Buffers: shared hit=125
Total runtime: 84720.000 ms
(116 rows)
QAクエリプラン：
Limit (cost=5474.52..5474.52 rows=10 width=47) (actual time=117.227..117.229 rows=10 loops=1)
Buffers: shared hit=97438
-> Sort (cost=5474.52..5474.56 rows=97 width=47) (actual time=117.227..117.227 rows=10 loops=1)
Sort Key: events.updated_at DESC
Sort Method: top-N heapsort Memory: 25kB
Buffers: shared hit=97438
-> Hash Left Join (cost=4013.62..5474.10 rows=97 width=47) (actual time=29.635..110.750 rows=23404 loops=1)
Hash Cond: (events.id = filters_2.target_id)
Filter: ((filters_2.target_id IS NULL) OR (filters_1.target_id IS NOT NULL))
Rows Removed by Filter: 60
Buffers: shared hit=97438
-> Hash Left Join (cost=2023.52..3483.92 rows=97 width=51) (actual time=8.641..83.892 rows=23464 loops=1)
Hash Cond: (events.id = filters_1.target_id)
Buffers: shared hit=96393
-> Hash Anti Join (cost=1012.67..2472.99 rows=97 width=47) (actual time=4.906..75.057 rows=23464 loops=1)
Hash Cond: (events.id = filters.target_id)
Buffers: shared hit=96119
-> Nested Loop (cost=5.00..1464.98 rows=97 width=47) (actual time=1.750..66.678 rows=23464 loops=1)
Buffers: shared hit=95845
-> Bitmap Heap Scan on events (cost=4.92..667.30 rows=214 width=47) (actual time=1.738..9.563 rows=23720 loops=1)
Recheck Cond: (((brand_id = 218) AND (brand_id = 218)) OR ((brand_id IS NULL) AND (brand_id = 218)))
Filter: ((eventable_type)::text = ANY ('{Achievement,Blog,Kudo,Story,User}'::text[]))
Heap Blocks: exact=801
Buffers: shared hit=869
-> BitmapOr (cost=4.92..4.92 rows=361 width=0) (actual time=1.630..1.630 rows=0 loops=1)
Buffers: shared hit=68
-> Bitmap Index Scan on index_events_on_brand_id (cost=0.00..2.81 rows=361 width=0) (actual time=1.629..1.629 rows=23720 loops=1)
Index Cond: ((brand_id = 218) AND (brand_id = 218))
Buffers: shared hit=68
-> Bitmap Index Scan on index_events_on_brand_id (cost=0.00..2.09 rows=1 width=0) (actual time=0.001..0.001 rows=0 loops=1)
Index Cond: ((brand_id IS NULL) AND (brand_id = 218))
-> Index Scan using users_pkey on users (cost=0.08..3.72 rows=1 width=6) (actual time=0.002..0.002 rows=1 loops=23720)
Index Cond: (id = events.user_id)
Filter: ((active AND visible) OR ((events.eventable_type)::text = 'Blog'::text))
Rows Removed by Filter: 0
Buffers: shared hit=94976
-> Hash (cost=1007.22..1007.22 rows=128 width=4) (actual time=3.145..3.145 rows=0 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 8kB
Buffers: shared hit=274
-> Unique (cost=1006.71..1006.84 rows=128 width=4) (actual time=3.145..3.145 rows=0 loops=1)
Buffers: shared hit=274
-> Sort (cost=1006.71..1006.77 rows=128 width=4) (actual time=3.145..3.145 rows=0 loops=1)
Sort Key: filters.target_id
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=274
-> Bitmap Heap Scan on filters (cost=283.82..1005.81 rows=128 width=4) (actual time=3.137..3.137 rows=0 loops=1)
Recheck Cond: (((constraint_id = 218) OR ((constraint_id = 5229) OR (constraint_id = 5183) OR (constraint_id = 5148)) OR ((constraint_type)::text = 'Country'::text)) AND ((target_type)::text = 'Event'::text))
Filter: ((NOT visible) AND ((((constraint_type)::text = 'Brand'::text) AND (constraint_id = 218)) OR (((constraint_type)::text = 'Team'::text) AND ((constraint_id = 5229) OR (constraint_id = 5183) OR (constraint_id = 5148))) OR (((constraint_type)::text = 'Country'::text) AND (constraint_id IS NULL))))
Rows Removed by Filter: 974
Heap Blocks: exact=156
Buffers: shared hit=274
-> BitmapAnd (cost=283.82..283.82 rows=641 width=0) (actual time=2.809..2.809 rows=0 loops=1)
Buffers: shared hit=118
-> BitmapOr (cost=28.65..28.65 rows=2762 width=0) (actual time=0.237..0.237 rows=0 loops=1)
Buffers: shared hit=22
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.17 rows=56 width=0) (actual time=0.005..0.005 rows=0 loops=1)
Index Cond: (constraint_id = 218)
Buffers: shared hit=3
-> BitmapOr (cost=24.33..24.33 rows=2707 width=0) (actual time=0.189..0.189 rows=0 loops=1)
Buffers: shared hit=16
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.17 rows=56 width=0) (actual time=0.003..0.003 rows=0 loops=1)
Index Cond: (constraint_id = 5229)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.17 rows=56 width=0) (actual time=0.003..0.003 rows=0 loops=1)
Index Cond: (constraint_id = 5183)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..19.98 rows=2596 width=0) (actual time=0.183..0.183 rows=2502 loops=1)
Index Cond: (constraint_id = 5148)
Buffers: shared hit=10
-> Bitmap Index Scan on index_filters_on_constraint_type (cost=0.00..2.09 rows=1 width=0) (actual time=0.010..0.010 rows=0 loops=1)
Index Cond: ((constraint_type)::text = 'Country'::text)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_target_type (cost=0.00..255.12 rows=34021 width=0) (actual time=2.555..2.555 rows=33832 loops=1)
Index Cond: ((target_type)::text = 'Event'::text)
Buffers: shared hit=96
-> Hash (cost=1009.75..1009.75 rows=314 width=4) (actual time=3.727..3.727 rows=974 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 43kB
Buffers: shared hit=274
-> Unique (cost=1008.49..1008.81 rows=314 width=4) (actual time=3.383..3.596 rows=974 loops=1)
Buffers: shared hit=274
-> Sort (cost=1008.49..1008.65 rows=316 width=4) (actual time=3.381..3.453 rows=974 loops=1)
Sort Key: filters_1.target_id
Sort Method: quicksort Memory: 70kB
Buffers: shared hit=274
-> Bitmap Heap Scan on filters filters_1 (cost=283.88..1005.87 rows=316 width=4) (actual time=2.765..3.224 rows=974 loops=1)
Recheck Cond: (((constraint_id = 218) OR ((constraint_id = 5229) OR (constraint_id = 5183) OR (constraint_id = 5148)) OR ((constraint_type)::text = 'Country'::text)) AND ((target_type)::text = 'Event'::text))
Filter: (visible AND ((((constraint_type)::text = 'Brand'::text) AND (constraint_id = 218)) OR (((constraint_type)::text = 'Team'::text) AND ((constraint_id = 5229) OR (constraint_id = 5183) OR (constraint_id = 5148))) OR (((constraint_type)::text = 'Country'::text) AND (constraint_id IS NULL))))
Heap Blocks: exact=156
Buffers: shared hit=274
-> BitmapAnd (cost=283.88..283.88 rows=641 width=0) (actual time=2.744..2.744 rows=0 loops=1)
Buffers: shared hit=118
-> BitmapOr (cost=28.70..28.70 rows=2762 width=0) (actual time=0.202..0.202 rows=0 loops=1)
Buffers: shared hit=22
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.17 rows=56 width=0) (actual time=0.003..0.003 rows=0 loops=1)
Index Cond: (constraint_id = 218)
Buffers: shared hit=3
-> BitmapOr (cost=24.36..24.36 rows=2707 width=0) (actual time=0.169..0.169 rows=0 loops=1)
Buffers: shared hit=16
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.17 rows=56 width=0) (actual time=0.002..0.002 rows=0 loops=1)
Index Cond: (constraint_id = 5229)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..2.17 rows=56 width=0) (actual time=0.001..0.001 rows=0 loops=1)
Index Cond: (constraint_id = 5183)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_constraint_id (cost=0.00..19.98 rows=2596 width=0) (actual time=0.165..0.165 rows=2502 loops=1)
Index Cond: (constraint_id = 5148)
Buffers: shared hit=10
-> Bitmap Index Scan on index_filters_on_constraint_type (cost=0.00..2.09 rows=1 width=0) (actual time=0.006..0.006 rows=0 loops=1)
Index Cond: ((constraint_type)::text = 'Country'::text)
Buffers: shared hit=3
-> Bitmap Index Scan on index_filters_on_target_type (cost=0.00..255.12 rows=34021 width=0) (actual time=2.525..2.525 rows=33832 loops=1)
Index Cond: ((target_type)::text = 'Event'::text)
Buffers: shared hit=96
-> Hash (cost=1943.94..1943.94 rows=13189 width=4) (actual time=20.931..20.931 rows=4918 loops=1)
Buckets: 16384 Batches: 1 Memory Usage: 301kB
Buffers: shared hit=1045
-> HashAggregate (cost=1864.80..1904.37 rows=13189 width=4) (actual time=19.485..20.163 rows=4918 loops=1)
Group Key: filters_2.target_id
Buffers: shared hit=1045
-> Bitmap Heap Scan on filters filters_2 (cost=256.07..1855.22 rows=19167 width=4) (actual time=2.605..13.438 rows=33832 loops=1)
Recheck Cond: ((target_type)::text = 'Event'::text)
Filter: (visible AND ((((constraint_type)::text = 'Brand'::text) AND (constraint_id <> 218)) OR (((constraint_type)::text = 'Team'::text) AND ((constraint_id <> 5229) OR (constraint_id <> 5183) OR (constraint_id <> 5148))) OR (((constraint_type)::text = 'Country'::text) AND (constraint_id IS NOT NULL))))
Heap Blocks: exact=949
Buffers: shared hit=1045
-> Bitmap Index Scan on index_filters_on_target_type (cost=0.00..255.12 rows=34021 width=0) (actual time=2.482..2.482 rows=33832 loops=1)
Index Cond: ((target_type)::text = 'Event'::text)
Buffers: shared hit=96
Planning time: 1.244 ms
Execution time: 117.581 ms
(129 rows)
統計年齢、effective_cache、すべての両方のマシンの設定をデフォルト設定ではない非常に歓迎されている –
も関連するテーブルのディスク上のサイズの比較は素晴らしいでしょう。あなたは時に生産DBをバキュームしますか？ btw、QAのPostgreSQLをプロダクション用よりずっと新しいバージョンにするのは良い考えです。 – moooeeeep
vacuum analyze？ – Teejay