Understanding DB2 Bufferpool Tuning 2005 Final

Understanding Buffer Pool Performance and Tuning in DB2 UDB v8.2

Jeff Brokaw Product Manager Quest Quest Centr Central® al® for DB2 DB2 November 2, 2005

You Can Expect More

Agenda • • • •

Buffer po pool ov overview Buffer pool management Colllecting per Co performa man nce met metrics Eval Evalua uati ting ng pe perf rfor orma manc nce e me metr triics – Physical vs. logical logical IO – Asynchronous Asynchronous vs. synchronous synchronous IO – Page cleaner cleaner tuning

• Q& A

You Can Expect More

Introduction Q: With so many tuning areas to focus on why should we pay special attention to the buffer pool? A: DB2 uses the buffer pool to offset the performance disparity between CPU and disk, so they are vital for optimizing transaction throughput. From the 8.2 Admin Guide: Performance (emphasis mine): “Because most data manipulation takes place in buffer pools, configuring buffer pools is the single most important tuning area.”

You Can Expect More

You Are Her Here e – DB2 Pro Proces cess s Model Model

Client Application Prefetchers

Listener

Coordinator

Buffer Pool Subagents

Logger

External Storage

Page Cleaners

Log

You Can Expect More

Buffer Pool Overview

Basics and Terminology •

•

Area Area of me memo morry int into whi which ch da dattab abas ase e pag pages es are are re read ad,, modified, and held during processing –

A hit occurs when required page is found in buffer pool

–

A miss occurs when required page not found in buffer pool

Pag Pages in bu bufffe ferr poo pooll exis existt in on one e of three hree sta tattes: es: 1. In use 2. Dirty 3. Clean

You Can Expect More


Basics and Terminology (cont’d) • Page Pages s read read into into bu buff ffer er po pool ol from from disk disk by: by: – Agents, using using synchronous synchronous I/O – Prefetchers, Prefetchers, using asynchronou asynchronous s I/O

• Page Pages s writ writte ten n to to dis disk k fro from m buf buffe ferr poo pooll by: by: – Agents, using using synchronous synchronous I/O – Page cleaners, cleaners, using asynchrono asynchronous us I/O

You Can Expect More


Tuning Goals • Recovery vs. Performance – – – –

Recovery wants wants everything everything on disk Database manager manager wants wants everything everything in memory memory Which goal is more important important – largely depends depends on data data Which combination combination works works best for both goals goals

• High High hit hit rat ratio io me mean ans s log logic ical al read reads su usu sual ally ly succ succes essf sful ul,, avoiding disk access • Norma Normalllly yw wan antt to to max maxim imiz ize e asy async nchr hron onou ous s II/O /O to mini minimi mize ze waits

You Can Expect More


Prefet Prefetche chers rs and Page Page Clean Cleaners ers • DB2 DB2 use uses sp pre refe fetc tch h and and pa page ge clea cleane nerr pro proce cess sses es to redu reduce ce the time that agents have to wait for IO operations to complete • New in 8. 8.1. 1.4: 4: Proa Proact ctiv ive e Page Page Clea Cleani ning ng,, AKA AKA Alte Altern rnat ate e Page Cleaning – DB2_USE_ALTERNATE_PAGE_C DB2_USE_ALTERNATE_PAGE_CLEANING=ON LEANING=ON

You Can Expect More


Prefetch • Pref Prefet etch cher ers s try try to en ensu sure re th that at ag agen ents ts do doin ing g ta tabl bles espa pace ce scans never wait for disk I/O • Agen Agents ts send send asyn asynch chro rono nous us readread-ah ahea ead d req reque uest st to commo common n prefetch queue • Pref Prefet etch cher ers s serv servic ice e the the requ reques estt by by b bri ring ngin ing g the the requ reques este ted d pages into the buffer pool • Shou Should ld resu result lt in logi logica call rea read, d, driv drive e hit hit rate rate up • Two types: – List prefetch, prefetch, efficiently efficiently reads a set of non-consec non-consecutive utive data pages – Sequential prefetch, prefetch, reads reads consecutive consecutive pages

You Can Expect More


Page Cleaners • Two main goals – Ensure that agents reading a page into the buffer pool never need to first flush a dirty page to disk to free up a slot – Speed recovery time by reducing # log files needed

• Configuration pa parameters – num_iocleaners num_iocleaners – number of of page cleaners cleaners for a database – chngpgs_thresh chngpgs_thresh – percentage percentage of changed changed pages at at which the page cleaners will be started, if not currently active

• Alte Altern rnat ate e pa page ge clea cleani ning ng (8.1 (8.1.4 .4 an and d late later) r) – List of “good “good victim pages” (dirty pages pages just written written out) kept, kept, prevents searching in many cases – LSN gaps anticipated, anticipated, prevents I/O I/O spikes – chngpgs_thresh chngpgs_thresh is ignored

You Can Expect More

You Are Her Here e – DB2 Pro Proces cess s Model Model

Client Application Prefetchers

Listener

Coordinator

Buffer Pool Subagents

Logger

External Storage

Page Cleaners

Log

You Can Expect More

Buffer Pool Management

Buffer pools not used with all data types • LONG LONG VARCH VARCHAR AR,, BLOB BLOB an and d CLO CLOB B ret retri riev eved ed dire direct ctly ly from from disk using non-buffered (direct) IO • Keep Keepin ing g lar large ge ob obje ject ct pa page ges so out ut of bu buff ffer er po pool ols sh hel elps ps DB DB2 2 maintain effective caching for regular data types Without use of direct IO, large data types would force DB2 to flush all resident pages to disk – ruining the hit rate

1 GB BLOB

1 GB BP

Disk

You Can Expect More


Creating/Altering Buffer Pools • IBMD IBMDEF EFAU AULT LTBP BP au auto toma mati tica calllly y cre creat ated ed with with ea each ch da data taba base se • Addi Additi tion onal al bu buff ffer er po pool ols s add added ed with with th the e CRE CREAT ATE E BUFFERPOOL statement • SYSC SYSCTR TRL L or or SYS SYSAD ADM M pri privi vile lege ges s are are req requi uire red d to to wor work k wit with h buffer pools CREATE BUFFERPOOL bufferpool_name SIZE number of pages [IMMEDIATE | DEFERRED] [PAGESIZE integer [K]] [[NOT] EXTENDED STORAGE | NUMBLOCKPAGES number of pages [BLOCKSIZE number of pages ]] [ALL DBPARTITIONNUMS | DATABASE PARTITION GROUP db_partition_group_name [,…]] [EXCEPT ON DBPARTITIONNUM[S] ( db_partition_number1 [TO db_partition_number2] SIZE number of pages [,…])]

You Can Expect More


Memory Usage • Vers Versio ion n 8 ma make kes sD DB2 B2 ea easi sier er to ma mana nage ge by usin using g dat datab abas ase e global memory (DATABASE_MEMORY) instead of database heap (DBHEAP) • Prev Preven ents ts resi resizi zing ng of DB DBHE HEAP AP with with bu buff ffer erpo pool ol resi resizi zing ng

CREATE BUFFERPOOL bufferpool_name SIZE number of pages [IMMEDIATE | DEFERRED] [PAGESIZE integer [K]] [[NOT] EXTENDED STORAGE | NUMBLOCKPAGES number of pages [BLOCKSIZE number of pages ]] [ALL DBPARTITIONNUMS | DATABASE PARTITION GROUP db_partition_group_name [,…]] [EXCEPT ON DBPARTITIONNUM[S] ( db_partition_number1 [TO db_partition_number2] SIZE number of pages [,…])]

You Can Expect More


Extended Storage • 32-b 32-bit it DB DB2 2 can can’t ’t supp suppor ortt cre creat atin ing g buf buffe ferr poo pools ls larg larger er th than an 4G due to addressable storage limits • Exte Extend nded ed stor storag age e all allow ows s DB2 DB2 to flus flush h dir dirty ty pa page ges s to memory rather than disk, so future access to the pages is much faster • Re Requ quir ires es NUM NUM_E _EST STOR ORE_ E_SE SEGS GS and and EST ESTOR ORE_ E_SE SEG_ G_SI SIZE ZE configuration parameters CREATE BUFFERPOOL bufferpool_name SIZE number of pages [IMMEDIATE | DEFERRED] [PAGESIZE integer [K]] [[NOT] EXTENDED STORAGE | NUMBLOCKPAGES number of pages [BLOCKSIZE number of pages ]] [ALL DBPARTITIONNUMS | DATABASE PARTITION GROUP db_partition_group_name [,…]] [EXCEPT ON DBPARTITIONNUM[S] ( db_partition_number1 [TO db_partition_number2] SIZE number of pages [,…])]

You Can Expect More


Block-Based Buffer Pools • Conti Contigu guou ous sb blo lock cks so off pag pages es to be mo move ved d int into o con conti tigu guou ous s portions of memory • Bloc Blockk-ba base sed d buf buffe ferr poo pools ls will will be was waste ted d if if you yourr appl applic icat atio ion n doesn’t perform sequential prefetch

CREATE BUFFERPOOL bufferpool_name SIZE number of pages [IMMEDIATE | DEFERRED] [PAGESIZE integer [K]] [[NOT] EXTENDED STORAGE | NUMBLOCKPAGES number of pages [BLOCKSIZE number of pages ]] [ALL DBPARTITIONNUMS | DATABASE PARTITION GROUP db_partition_group_name [,…]] [EXCEPT ON DBPARTITIONNUM[S] ( db_partition_number1 [TO db_partition_number2] SIZE number of pages [,…])]

You Can Expect More


Hidden Buffer Pools • For im improved av availability o on nly • DB DB2 2 cre creat ates es smal smalll (16 (16 pa page ge)) buff buffer er poo pools ls of ea each ch pa page ge size size at database activation (4, 8, 16, 32k) • Hidd Hidden en from from th the e use user, r, an and d use used d onl only yw whe hen n ord ordin inar ary yb buf uffe ferr pools cannot be allocated • Perf Perfor orma manc nce e wil willl be be not notic icea eabl bly y impa impact cted ed,, and and DB DB2 2 will will writ write e a warning message to the administration notification log

You Can Expect More


Related Objects • CREAT CREATE E TAB TABLE LESP SPAC ACE E spe speci cifi fies es wh whic ich h buf buffe ferr poo pooll to to use use • Page Page size sizes s of of buf buffe ferr poo pooll and and ta tabl bles espa pace ce mu must st ma matc tch h – valid values are 4, 8, 16, and 32 kilobytes • CRE CREAT ATE E BU BUFF FFER ERPO POOL OL stat statem emen entt can can spec specif ify y pa part rtit itio ion n group, to control partitions in which a given buffer pool is created

You Can Expect More

Collecting Performance Metrics

Tunin Tuning g – An Ite Iterat rativ ive e Proce Process ss • Collect performance data – Snapshot monitoring monitoring – Event monitoring monitoring

• Review the results – Understand SQL workload workload – Evaluate performance performance (calculations)

• Take action – Resize buffer pools – Define more page cleaners cleaners – Break out tables tables with “like” “like” workloads

• Start th the pr process ag again

You Can Expect More

Collecting Performance Metrics Snapshot monitor

Event Monitor

“Poi “Point nt iin n tim time” e” repr repres esen enta tati tion on of dat data a “Eve “Event nt”” ba base sed d rep repre rese sent ntat atio ion n of of data data Less overhead (~5%)

More overhead (~10-20%)

Need to reissue over time

“Create” it once and it runs until it’s told to stop

Monitor Switches need to be turned on to collect data

“Create” “Create” takes takes care of of everythi everything ng

“Real “Real time” time” balance balance of of applicat application ion and database performance

“Histori “Historical” cal” main focus focus on Application statistics

You Can Expect More


DB2 Snapshot Switches • Seven Switches – Buffer Pool (dft_mon_bufpool) (dft_mon_bufpool) – Lock (dft_mon_lock) – Sort (dft_mon_sort) – Statement (dft_mon_stmt) (dft_mon_stmt) – Unit of Work (dft_mon_uow) – Table (dft_mon_table) – Timestamp (dft_mon_timestamp) (dft_mon_timestamp)

You Can Expect More


DB2 Snapshot Switches (cont’d) • Turn urning ing swi swittches ches on at the the ins insta tanc nce e – take ake car care e her here e UPDATE DATABASE MANAGER CONFIGURATION USING dft_ df t_mo mon_ n_bu bufp fpoo ool l ON dft_ df t_mo mon_ n_lo lock ck ON dft_ df t_mo mon_ n_so sort rt ON dft_ df t_mo mon_ n_st stmt mt ON dft df t_m _mon on_ _ta tabl ble e ON dft df t_m _mon on_u _uow ow ON IMMEDIATE

• Turning sw switches on on fo for se session UPDATE UPD ATE MONIT MONITOR OR SWITCH SWITCHES ES USING USING buffe bufferpo rpool ol ON

You Can Expect More

Collecting Performance Metrics • Issuing a SNAPSHOT re request – Instance level (requires “ATTACH”) • For For all all acti active ve da data taba base ses s GET SNAPSHOT FOR ALL BUFFERPOOLS

• For For a spec specif ific ic da data taba base se GET SNAPSHOT FOR BUFFERPOOLS ON database_name

• For For a spec specif ific ic dat datab abas ase e pa part rtit itio ion n GET SNAPSHOT FOR BUFFERPOOLS ON database_name AT DBPARTITIONNUM DBPARTITIONNUM db_partition_number

• BUFF BUFFER ERPO POOL OL an and d DAT DATAB ABAS ASE E sna snaps psho hott cal calls ls are are essential for tuning buffer pools

You Can Expect More


• BUFFERPOOL Snapshot Buffer Buff erpo pool ol na name me Database name Database path C:\DB2\NODE0000\SQL00005\ Input database alias Buffer pool data logical reads Buffer pool data physical reads Buffer pool data writes Buffer pool index logical reads Buffer pool index physical reads . . . Node number Tablespaces using bufferpool Alter bufferpool information: Pages left to remove Current size Post-alter size

= BA BASE SEBP BP1 1 = BASEBALL = = = = = = =

BASEBALL 300 3490 3050 67 34

= 0 = 4 = 0 = 1000 = 1000

You Can Expect More


• DATABASE Snapshot Database name Database path . . . Buffer pool data logical reads Buffer pool data physical reads Asynchronous pool data page reads Buffer pool data writes Asynchronous pool data page writes Buffer pool index logical reads Buffer pool index physical reads Asynchronous pool index page reads Buffer pool index writes Asynchronous pool index page writes Total buffer pool read time (ms) Total buffer pool write time (ms) Total elapsed asynchronous read time Total elapsed asynchronous write time Asynchronous read requests LSN Gap cleaner triggers Dirty page steal cleaner triggers Dirty page threshold cleaner triggers Time Tim e wai waited ted for pre prefet fetch ch (ms (ms) )

= AUTO_PRD = C:\DB2\NODE0000\SQL00009\

= = = = = = = = = = = = = = = = = = =

165079 36264 10546 3991 3873 4014 558 332 26 26 8329 121852 5353 121292 1681 24 63 33 626

You Can Expect More

Evaluating Performance Metrics

Know your workload • Is the SQ SQL wo workload pr primarily: – Update/Insert/Delete Update/Insert/Delete (OLTP) (OLTP) – Select (ad hoc, data warehouse warehouse))

• Even Eventt Mon Monit itor orin ing g ca can hel help p ifif uns unsur ure e

You Can Expect More

Evaluating Performance Metrics Logical Logical IO vs. vs. physical physical IO – minimize minimize number number of times times DB2 must access physical disks Key I/O Metrics: – Overall hit rate – Data hit rate – Index hit rate – Physical IO read rate

You Can Expect More


Overall Hit Rate – Includes all data and index reads into buffer buffer pool ⎡ ⎤ ⎢ ⎥ (bp.data.logical.reads + bp.index.logical.reads) ⎥ BuffePoolOHR = ⎢ BufferpoolOHR ⎢ ⎛ bp.index.logical.reads + bp.index. physical.reads + ⎞ ⎥ *100 ⎟⎟ ⎥ ⎢ ⎜⎜ ⎣ ⎝ bp.data.logical.reads + bp.data. physical.reads ⎠ ⎦

– Low values = more physical IO – High values = more logical logical IO – 85 – 90% is a good starting starting point

You Can Expect More


Data Hit Rate – Includes all data reads reads into buffer pool ⎛ ⎞ bp.data .logical .reads ⎟⎟ *100 Bufferpool DHR = ⎜⎜ ⎝ bp.data .logical .reads + bp.data . physical .reads ⎠


You Can Expect More


Index Hit Rate – Includes all index reads reads by buffer pool ⎛ ⎞ bp .index .logical .reads ⎟⎟ * 100 Bufferpool IHR = ⎜⎜ ⎝ bp .index .logical .reads + bp .index . physical .reads ⎠


You Can Expect More


Physical IO Read Rate • Tota Totall num numbe berr of of phy physi sica call rea reads ds by bu buff ffer er po pool ol over over time time ⎛ bufferpool.data.logical.reads + bufferpool.data. physical.reads ⎞ ⎟ total collection time . . ⎝ ⎠

PhysicalIO Reads = ⎜

• Use: GET SNAPSHOT FOR TABLESPACES ON database_name

• Could indicate the need for… – Additional containers containers – Buffer pool pool breakout breakout

You Can Expect More


Prefetch – Asynchronous vs. Synchronous I/O – Waiting is always bad – Make DB2 read ahead for you as much as possible, possible, given application mix

Approach: – – – –

Evaluate prefetch ratio Know the workload Evaluate prefetcher configuration Use block I/O

You Can Expect More


Pref Prefet etch ch Ra Rati tio o ⎛ async.pool .data.read s + async.pool .index.rea ds ⎞ ⎟⎟ *100 ⎝ bp.logical .reads + bp.index.l ogical.rea ds ⎠

PrefetchRa tio = ⎜⎜

• Lower values = more synchronous • High Higher er value alues s = mo morre asy asynchr nchron onou ous s • Are there enough defined?

You Can Expect More


Block-based Buffer Pools – If using sequential sequential prefetch, enable buffer pools to utilize block IO ALTER BUFFERPOOL BUFFERPOOL basebp2 basebp2 NUMBLOCKPAGES NUMBLOCKPAGES 30; ALTER BUFFERPOOL BUFFERPOOL basebp2 basebp2 BLOCKSIZE 10;

– NUMBLOCKPAGES should be a multiple of block size

You Can Expect More


Evaluating Page Cleaner Performance: – LSN Gap Triggers – Dirty Page Threshold Triggers – Dirty Page Steal Triggers Triggers

Approach: – – – – –

Evaluate workload and recovery recovery goals Evaluate each metric against all others Modify parameters influencing influencing page cleaner activity Enough page cleaners defined? More automated in V8.1.4 and above

You Can Expect More


Log Cleans Ratio ⎛ ⎞ ⎜ ⎟ lsn.gap.cleaner .triggers ⎜ ⎟ LCR = ⎜ *100 ⎟ ⎛ dirt . page.steal .cleaner .triggers + lsn.gap.cleaner .triggers ⎞ ⎟ ⎜⎜ ⎟ ⎜ ⎟ ⎜ ⎟ + dirty . page.threshold .cleaners .triggers ⎠ ⎠ ⎝ ⎝ – Has an affect on on recovery times – Too small or too large could be detrimental to buffer buffer pool performance – Affected by SOFTMAX DB CONFIG CONFIG parameter – 0 if ALTERNATE_PAGE_CLEANING=ON ALTERNATE_PAGE_CLEANING=ON

You Can Expect More


Threshold Cleans Ratio ⎞ ⎛ ⎜ ⎟ dirty . page.threshold .cleaner .triggers ⎜ ⎟ TCR = ⎜ *100 ⎟ ⎛ dirt . page.steal .cleaner .triggers + lsn.gap.cleaner .triggers ⎞ ⎜⎜ ⎟ ⎟ ⎜ ⎟ ⎜ ⎟ + dirty . page.threshold .cleaner .triggers ⎠ ⎠ ⎝ ⎝ – CHNGPGS_THRESH DB CONFIG parameter parameter – Finding a “happy medium” is key – 0 if ALTERNATE_PAGE_CLEANING=ON ALTERNATE_PAGE_CLEANING=ON

You Can Expect More


Victim Cleans Ratio ⎛ ⎞ ⎜ ⎟ dirty . page.steal .cleaner .triggers ⎜ ⎟ VCR = ⎜ *100 ⎟ ⎛ dirt . page.steal .cleaner .triggers + lsn.gap.cleaner .triggers ⎞ ⎜⎜ ⎟ ⎟ ⎜ ⎟ ⎜ ⎟ dirty page threshold cleaner triggers + . . . . ⎝ ⎠ ⎝ ⎠ – Above 40%, typically typically means buffer pool needs to be larger – Could also mean SOFTMAX parameter is too high (not enough page turnover) – 0 if ALTERNATE_PAGE_CLEANING=ON ALTERNATE_PAGE_CLEANING=ON

You Can Expect More

Conclusion

•

Focus on three main areas 1. BP hit hit rat ratio ios s (lo (logi gica call vs. vs. ph phys ysic ical al)) 2. Pref Prefet etch cher er tuni tuning ng (asy (async nchr hron onou ous s vs. sync synchr hron onou ous) s) 3. Page Page clean cleaner er tun tuning ing (if no nott using using ne newer wer alter alternat nate e clea cleanin ning) g)

•

Collect pe performance da data ov over titime –

•

Review the results – – –

•

Sampling methodology methodology – snapshots snapshots every 5-15 5-15 minutes over the course of the day, or during a batch run Decipher data data and plug plug into equations equations Know the workload workload Chart performanc performance e over time time

Take action –

Various solutions solutions depending depending on findings

You Can Expect More

Quest Software: You Can Expect More

http://www.quest.com/db2/

Understanding DB2 Bufferpool Tuning 2005 Final

Recommend Documents