DBMSguy.com

DB2 LUW V9.7 CUR_COMMIT database configuration option – buyer beware

Paul Mayhew — Wed, 19 Jan 2011 23:58:46 +0000

While working on a deadlocking problem, I came across the cur_commit optoin that is now available in version 9.7. This option is part of the ongoing efforts to make DB2 more Oracle compliant.

After reading about the cur_commit database option, it seemed to be a fair solution to the problem. We installed/upgraded our databases to v9.7 and enabled the option - our deadlocking problems were over. Case closed.

Well, maybe not. Here’s the rub. If you read about what little information there is about cur_commit, you will find that the option works by reading logs to get the currently committed value if an update has been done. This works great in theory and even in practice, but I found that during our performance tests, we suffered greatly. Reading the logs does not seem to be especially efficient for these cases and our system suffered greatly.

In analyzing system counters, I noted that ‘Log pages read’ (db snapshot) values were very high. In response to this, I increased LOGBUFSZ value to something very large. I started off at 4096 and this make no difference what-so-ever. I am up to 20480 which is 80Mb and things have mostly stabilized, but I am still seeing a lot of ‘log pages read’ from time to time and when that value goes up, our performance goes down.

So the use of ‘CUR_COMMIT’ option in 9.7 is not a cure-all for deadlocks and should be used with care. I don’t think it should be turned on without some forethought and a good amount of performance testing. The CUR_COMMIT option is really a bandaid option. Most deadlocking problems are caused by bad application design and should be fix from that end if possible.

http://publib.boulder.ibm.com/infocenter/db2luw/v9r7/index.jsp?topic=/com.ibm.db2.luw.admin.config.doc/doc/r0053556.html

db2pd command – get info about db2

Paul Mayhew — Mon, 20 Sep 2010 15:42:08 +0000

The db2pd command is a relatively new command (released with version 8.2) that will tell you a huge amount of information about your db2 system. I first made myself familiar with it when looking for a way to get the status of HADR without going through a snapshot output.

db2pd -db sample -hadr - will tell you all you need to know about your hadr configuration status

The db2pd command can be run in two ways; you can run it interactively by entering db2pd and then whatever commands you want or you can simply enter the db2pd command with whatever parameters you want. I’m not sure what the advantage of running it interactively is, but it’s there for you.

I find the output from some of the parameters very heavy indeed (some are just memory dumps for the most part). You will find useful intormation however.

a few of the handier commands in my opinion are:
db2pd -db sample -logs
db2pd -db sample -locks
db2pd -db sample -dbcfg
db2pd -db sample -hadr

here is a really good tutorial on how to use db2pd for locks and the entire manual can be found in the command reference.

sql for sampling data from tables

Paul Mayhew — Mon, 06 Sep 2010 16:15:23 +0000

Some time ago, I had the requirement to sample data from a table to do validation work. Basically, we wanted to validate syncronized data with a base table, but could not afford to compare every row. We had to come up with a way to ‘sample’ the data. Each table we had to sample had a different number of rows and in addition to that were growing. The mechanizm had to be simple, yet take into account table size. I did not want to do a custom query for every table. I wanted something somewhat generic – something that I could say check ‘x’ rows from this table, not check every ‘nth’ row which would mean that the result would grow over time.

The obvious first consideration was to number the rows and do it from that, but considering we were doing this in a program, I needed something a little more elegant. Here is the sql I came up with.

SELECT E.* FROM
(SELECT ROWNUMBER() OVER (ORDER BY EMPNO) AS ROWNBR, EMPNO FROM EMPLOYEE ) TEMP,
(SELECT INTEGER(ROUND((COUNT(*)/23.0),0)) AS HOWMANY FROM EMPLOYEE) TEMP2,
EMPLOYEE E
WHERE MOD(ROWNBR,HOWMANY)=0 AND E.EMPNO=TEMP.EMPNO

So here is what we do,
TEMP is an ordered list of the table key – we number each row and order by the empno (or whatever key)
TEMP2 is a cartesian join of the count of the table and how many rows you want. In this case it is 23 rows
E is the table we want and join to the above
The where clause is the key – we only take rows that have a modulus of 0 when you devide the rownumber by how many rows you want

Notes:

This was written for the sample database employee table, however I did have to modify it to have more than 42 rows
Small tables will have inaccurate results just because of the math so this works best on larger tables
The number of rows returned is not exactly what was requested. Once again because of the math, but it is usally close. I found the larger the table, the more accurate the result. You can add a ‘FETCH FIRST n ROWS ONLY’ (where ‘n’ is your value) so you don’t get more than you want.
How often do you find a legitimate use for a cartesian join?
Don’t ask for more rows than are in the table – this can cause errors with the mod function

Inline database shell scripts using awk

Paul Mayhew — Wed, 01 Sep 2010 21:01:37 +0000

One of the things I find very powerful in my day to day work as a DBA, is the use of inline shell/awk scripts. I use this technique when I need to do the same thing to many objects. I often use it when I want something quick and dirty or create a base script to work from.

An example might be when I want to do row counts on every table, or maybe set up a reorg on all the tables in a database.

Here is the command string you can use to do such a thing:

db2 connect to sample

db2 -x “select tabschema, tabname from syscat.tables where tabschema=’DB2INST1′ and type=’T'” | awk ‘BEGIN{print “db2 connect to sample”} {print “db2 -v \” select count(*) from “$1″.”$2″\”"}’ | ksh

The above inline script will select all the tables and return tablenames, then piping that into awk, we first print a connect statement and then print all the count statements. All of this is then piped into a ksh. The connect must be put there because a new instance of ksh is generated.

The base query can, of course, be customize to whatever suits your needs.

The pipe in to ksh is optional as you could just direct it to an output file and use it later and or make further modifications.

db2 batch scripts – tips and tricks – part 2

Paul Mayhew — Mon, 30 Aug 2010 00:18:42 +0000

I’ve always been a little frustrated with db2 when running scripts and command lines with fact that when a command would run like insert, update or delete, you never knew how many rows you affected. Some GUI products will tell you, but with command line it would just come back as being successful (or not) and it was up to you to validate what you really did with a subsequent query.

Being a DBA, I always feel the need to validate what I do especially with ad hoc statements to ensure that I have done what I expected.

Enter the m option on the db2 command line. This is a relatively new switch that will cause db2 to return to you how many rows were affected. The m option was introduced with verison 9.

To do this from the command line:

db2 -m “delete from test” will return

Number of rows affected : 18
DB20000I The SQL command completed sucessfully.

To do this from a script:

db2 -tvmf test.db2

By doing this, you will have the number of rows affected at your finger tips, not more hoping or assuming or doing count before/count after!

Happy DBAing

db2 batch scripts – tips and tricks – part 1

Paul Mayhew — Fri, 30 Jul 2010 14:18:15 +0000

There are a lot of things you can do with db2 batch scripts that can make your life easy. In this set of posts I will talk about just a few things that might make your life easier.

-s switch- I find the -s switch the handiest of all. This will allow you to stop the script execution if there is an error in the script. To do this execute your scripts with “db2 -stvf filename”. By doing this, you will ensure the successful execution of your script. If the script encounters an error, it will just stop at that point. Checking becomes a little easier because you need only go to the bottom of the script for basic checking.

One additional trick – you can turn off the -s off and on in the script stream. This allows you to disable checking for certain parts of your script you might expect to fail. Things such as dropping a table that might or might not exist. To do this your script would look like this:

update table ……;
update command options using s off;
drop table test.table;
update command options using s on;
update table ……;

you would then execute with db2 -stvf filename

I guess of course you could just leave out the -s and use the “update command options” at the top of the script to turn it on, but I usually don’t do that because I want it on more than off.

Windows 7 — First Impressions

Paul Mayhew — Thu, 14 Jan 2010 18:43:57 +0000

A long time XP user, I was very nervous about getting a new system with windows 7. Would anything work? Would it be slower? All kinds of things ran through my head. I read all the reviews and for the most part, it seemed like a sucessful release, but then?……..

I finally bought a new system which had Windows 7…. I got myself a HP P6240F with

8 Gb of memory — can’t have too much memory can you?
Intel Q8300 quad core 2.5 GHz
Windows 7 – 64bit
5200 rpm, 750Gb SATA hard drive

I’m no gamer, so the graphics is no biggy for me, pretty much us it for business. I do us DB2, so the extra memory will come in handy.

This is a replacement system, not an upgrade, so I didn’t have to worry about data much as it still resides on the old system. Same with my applications, if it didn’t work on W7, then I could always leave it.

First Impressions -Happy, I think. Got the system home, plugged it in and it went through a little setup process to install Windows 7. HP seems to do it nicely, although there are some minor annoyances like bloatware type stuff. It’s fast, but nothing is installed, so no big surprise there. The ‘areo’ stuff is kinda kool.

Installing my software – Next step is to install all of my varous software. Here is where the problems start. My printer is fairly new – HP C6380 and that auto installed fine.

One of the things about Windows 7 is that it does not come with a mail client. No more outlook express. At first I could not find my windows office 2003 disk, so I installed my old office 2000 apps. Outlook did not work from office 2000. I tried a few others – Thunderbird and Windows live mail. Didn’t like the very much but they worked fine. I finally found my windows office 2003 and installed that and outlook was fine.

My biggest problem was with my video/photo editing software.
-Ulead Video Studio would not even install. That was disappointing as I really like it.
-Photo Exposion Deluxe 3 does not work very well either. You can install it, but it has problems finding libraries, so in the end does not work.

Those are the two most notable.

My other problem was my accounting software (MYOB 11) does not work as well. Had to leave that installed on XP. Pretty small annoyance, but notable if you do not have a backup system.

My other big problem is that my backup software does not work. Acronis True Image Home works somewhat, but hangs when I am doing backups… it has been relegated to the useless pile now. The restore function seems to work ok, but there are occasional corruptions.

Sometimes my printer gets ‘forgotten’ — I go to print something and the printer is gone. Only way to get it back is to reboot. An annoyance, but not that big a deal – it doesn’t happen that often.

I have not had any problems with bluescreen crashes or freezing issues. It has run nicely since I got it. With all my software installed I run at about 1.5 Gb in use minimum. Goes up from there, so my advice is to get at least 4Gb of memory – stay away from those 2Gb bargan machines.

Last Impressions – In the end, I’m pretty hapy with my new Windows 7 machine. I like that it is 64Bit as that is the way of the new world. As more software is upgraded to 64bit, I’ll be better positioned to take advantage of it. The boot/shutdown times are good. Boots up much quicker than XP as well as recovery from sleep. Other than the photo/video software problems, I think I’ll be happy with my investment.

Getting records based on the first and last days of a month

Ivan Beg — Fri, 18 Dec 2009 01:06:50 +0000

Have you ever had a situation where you needed to select rows from a table for a specific month? This example demonstrates how to retrieve all of the records belonging to the previous month by setting the first and last days in the WHERE clause of a SELECT statement. But with some tweaking, you can make this query retrieve data from any month you wish.

For this example, we’ll assume that we have a table in a data warehouse called DW.TRANSACTION with two columns (TRANSACTION_ID and EFFECTIVE_TS) that is populated and up to date.

To retrieve the previous month’s records, we would create a query of the form:

SELECT
       TRANSACTION_ID
      ,EFFECTIVE_TS
FROM
       DW.TRANSACTION
WHERE
       EFFECTIVE_TS >= TIMESTAMP_ISO(DATE(CURRENT DATE) - (DAY(CURRENT DATE)-1) DAYS - 1 MONTH)
AND    EFFECTIVE_TS <= TIMESTAMP_ISO(DATE(CURRENT DATE) - (DAY(CURRENT DATE)-1) DAYS) - 1 MICROSECOND
;

If you would like to retrieve data from two months ago, the WHERE clause would be changed to:

       EFFECTIVE_TS >= TIMESTAMP_ISO(DATE(CURRENT DATE) - (DAY(CURRENT DATE)-1) DAYS - 2 MONTHS)
AND    EFFECTIVE_TS <= TIMESTAMP_ISO(DATE(CURRENT DATE) - (DAY(CURRENT DATE)-1) DAYS - 1 MONTH) - 1 MICROSECOND

At this point, you can keep increasing the values for the MONTHS to keep going into the past.

DB2 HADR for dummies – Part 3

Paul Mayhew — Wed, 16 Dec 2009 19:47:47 +0000

This is the last post about HADR, I think I covered most of the basics of HADR finishing off with how it switches primary and standby. What I have gone over is the basics of HADR. There are many other topics within HADR. One important thing to remember is to NEVER do unlogged operations (careful with the load command). Not only will it ‘break’ HADR standby, but it will not tell you — you’ll not find out until you switch over to the standby to find it unusable. I believe that in version 9.7, we are being allowed access to the standby. This will allow you to validate the usability of the standby as well as other things like reporting.

This entry is about how to use HADR. Once again, just the basics.

The command to switch the primary/standby pair is:

from the standby database:
db2 takeover hadr on database sample

simple as that, the standby is now the primary and the old primary is now the standby.

There is also a feature – ACR (automatic client re-route) which can automatically automatically re-route your client to the new primary database when it can not access the current primary.

DB2 HADR for dummies – Part 2

Paul Mayhew — Thu, 26 Nov 2009 22:54:06 +0000

Okay, so here we go …. setting up an hadr environment…. HADR in it’s lowest form is very straight forward to set up. Here are the steps to start HADR

You need to have either a second machine with DB2 installed, or alternately you can use a second instance. I’m sure I don’t have to tell you that using a second instance of DB2 on the same machine defeats the whole purpose of this, but it is fine for testing
Make sure you have Log Retain set on either explicitly or implicitly
Make sure you have db cfg LOGINDEXBUILD set to On
you need to set the following db config parameters –
HADR_LOCAL_HOST set to the IP address or host name of the local host
HADR_LOCAL_SVC set to the port you are going to assign
HADR_REMOTE_HOST set to the IP address of standby instance – same as LOCAL if using an alternate instance
HADR_REMOTE_SVC set to the standby port should be at least LOCAL +2
HADR_REMOTE_INST set to standby instance name
Make a backup of your database – can be either online or offline
Send the backup to the standby machine. If you are using two instances, then you can skip this
restore from the backup on the standby instance — I’m assuming you are using the IBM SAMPLE database for this excerise.
On the standby database, you need to set the same db config values except you need to flip them around so they are correct – LOCAL values become REMOTE values.
On the standby machine/instance “db2 start hadr on database sample as standby”
On the primary machine/instance “db2 start hadr on database sample as primary”
Check if HADR is working – Do “db2 get snapshot for database on sample” — in the output look for:
HADR Status
Role                   = Primary
State                  = Peer
Synchronization mode   = Nearsync
Connection status      = Connected , 11/26/2009 17:47:40.447523
Heartbeats missed      = 0
Local host             = 192.168.0.105
Local service          = 10000
Remote host            = 192.168.0.101
Remote service         = 11000
Remote instance        = DB2
timeout(seconds)       = 120
Primary log position(file, page, LSN) = S0000036.LOG, 0, 000000000B3B0000
Standby log position(file, page, LSN) = S0000036.LOG, 0, 000000000B3B0000
Log gap running average(bytes) = 0

if it says anything other than ‘peer’ then there is a problem. One common problem is a firewall… if you are doing this on a windows box with a firewall, then you will have to open up the HADR ports both ways.

You now have a HADR pair up and running — anything you do on the primary will be done on the standby as well. Be sure to review things that don’t work.

Standby for Part 3 where we will go over how to switch back and forth from primary to standby.