Master Yavor from Tryavna

My presentation “The adventure of upgrading to 11.2” from BGOUG seminar in Hisar can be downloaded – here

I’d like to thank once again to Milena, SPI and all other people that invest lots of time and effort to make BGOUG seminars come true. You’ve done a great job, again!

Well, this is something new. Usually every new version of Oracle Database comes with fewer blocking scenarios than the previous. But this is a new behavior, introduced with Oracle 11g.

Some people would say this is a problem with application design. Indeed, having a non-indexed foreign keys is a well known no-no. But with 10g it would lead to slower deletes from parent table, or updates of primary key on parent, in order to have the child scanned. And now we have a whole new story – enqueue wait, which can last as long as you want. A simple INSERT on parent blocks any UPDATEs and DELETEs from other sessions, until the INSERTing session commits. To make things worse, this have hit us with 724-row child table, which was full scanned in an instant before, so the “performance” penalty was unnoticed.

* * *

Here is the whole story. Note 1343365.1 introduces the following

There is change in lock modes behavior introduced intentionally by fix 5909305 in 11.1 onward.

Change has been made to DML (TM) lock modes for foreign key constraints (Doc ID 5909305.8).

The change in behavior mentioned in note 5909305.8 , introduces higher level of lock on parent table while running DML against Child table. ( SX vs SS )

There are cases unrelated to this bug which lack of indexes on foreign key columns increase the time locks take place. This reflect as wait on TM-enq.

Sounds quite innocent. The behavior of SS and SX locks is almost the same. There are two differences – SX is not “compatible” with S (Share) and SSX (Share Sub-eXclusive). But those two are not seen with DML. Usually.

Here comes the unindexed foreign key. This is the only DML I know, that puts Share lock on whole table. And with the following scenario I managed to get locked every time on 11.2.0.2:

1. Prepare data

drop table t2 purge;
drop table t1 purge;

create table t1 (pk number primary key);

create table t2 (pk number primary key, fk number references t1(pk));

insert into t1 values (1);
insert into t1 values (2);
insert into t1 values (3);
insert into t1 values (4);

insert into t2 values (11, 1);
insert into t2 values (21, 2);
insert into t2 values (22, 2);
insert into t2 values (31, 3);
insert into t2 values (32, 3);
insert into t2 values (33, 3);

commit;

2. Put the trap form Session 1

insert into t1 values (5);

If you look at v$lock, you will see Session 1 holds 3 locks (and a few more in 11g, but they are not interesting)
– mode 6 (eXclusive) TX lock on the newly inserted row
– mode 3 (Sub eXclusive) TM lock on table t1. This is our parent table.
– mode 2 (Sub Share) on 10g or Mode 3 (Sub eXclusive) on 11g, TM lock on table t2. This is our child table. Note the lock is on the table itself, and it doesn’t matter if there are any rows affected on the child. Also note there is one such lock on every table that has a foreign key, pointing to our parent.
(if there were child records affected by cascade delete, for example, we would have TX locks on them, which is expected)

3. Try to delete or update one row on parent from Session 2:

delete from t1 where pk=4;

or

 update t1 set pk=2 where pk=2;

In 10g the statements work just fine. But on 11g both statements and hanging forever, waiting to acquire Mode 4 (Share) lock on child table – because the foreign key column is not indexed.

How to diagnose this issue:
First of all, we see the sessions waiting for TM locks in Grid Control. The color is red, so they are easily noticed. We can also get the session information form V$SESSION:

 SQL> select sid, final_blocking_session
  2    from v$session
  3   where event = 'enq: TM - contention';

       SID FINAL_BLOCKING_SESSION
---------- ----------------------
       132                    332

In order to find the resource, causing the problem (on other words – the unindexed table), I use the following query:

 SQL> select SID, type, id1, lmode, request, block
  2    from v$lock
  3   where sid in (132, 332)
  4     and type = 'TM';

       SID TYPE        ID1      LMODE    REQUEST      BLOCK
---------- ---- ---------- ---------- ---------- ----------
       132 TM       950744          3          0          0
       132 TM       950746          0          4          0
       332 TM       950744          3          0          0
       332 TM       950746          3          0          1

On line number 4 we see BLOCK=1, which indicates this is a blocker. On line number 2 we have REQUEST=4 and LMODE=0 – this means we want mode 4 lock, but we cannot get it. The column ID1 in this case shows the object ID (for TM locks):

 SQL> select owner, object_name, object_type
  2    from dba_objects
  3   where object_id = 950746;

OWNER      OBJECT_NAM OBJECT_TYPE
---------- ---------- -------------------
YAVOR      T2         TABLE

The solution is easy, once you know the problem. Simply create an index on child table’s foreign key column

 create index t2_fk on t2(fk);

When we create the index, the UPDATE and DELETE statement do not need mode 4 (S) lock on the child table. Instead they use the index to acquire mode 3 (SX) lock on the child, which is compatible with the other SX on the same object.

Now, this is kind of hard to explain to developers. To be honest, even I do not understand how the index helps in converting the need of S lock to SX. We do not affect any rows on child here. But this is how it works.