This is an old story. It started in 2015 when I found a problem with TOP-N query performance. I wrote a blog post about it and later an update. Here I want to show the full testcase and some updates after a twitter discussion.
I had a bad night last Thursday.
After patching two test RAC databases and one production RAC with 180417 DB Bundle Patch (and some one-offs), I got to the point where it was time to update the most critical RAC system.
We were really looking forward to this as we had hit a few bugs that this DBBP and one-offs should fix. But boy, did that go wrong…
Lately I started patching a client’s database (220.127.116.11) to the latest PSU (180417). This is a RAC environment with streams and all kind of other features, so over the time we hit quite a lot of different bugs. When we planned this PSU (we installed the bundle patch version), we added about 7 one-off patches (some are recommended by Oracle and some we had to add because the bugs affected us quite badly).
This post is following a question I found on LinkedIn. A DBA pasted a strange test case in 18.104.22.168 and I managed to reproduce it in 22.214.171.124 (non-multitenant) and 12.2 (multitenant). But that’s not where the story ends, I wanted to understand what’s going on, so I did some research about it and the result is this post.
A few months ago we hit an Oracle bug related to streams replication crash after creating an index (bug 21320182). There is a patch so we installed the patch in test and it seemed to solve the problem, but we never patched the production.
Today we hit this issue in production after creating an index we needed. It’s important to say that we wanted to patch the prod a while ago, but we didn’t get approval for maintenance window.
This week I worked on a messy patch. I have a RAC environment with 126.96.36.199 and an old PSU and all kind of one-off patches and I wanted to install the latest PSU (180417). It sounds simple, but it’s not so simple. The thing is that I have 4 database running from the same ORACLE_HOME, all of them are RAC, while some of them are stand-alone, some are primary for standby located on a different RAC and some are standby for a primary located on a different RAC. And the problem with that is that you cannot install a PSU on the primary first, either together or patch the standby first.
In our first BCOUG Tech Day conference, I presented my session “Look Inside the Locking Mechanism”. I presented this topics before a few times and prepared a few demos to show different locking scenarios.
During the BCOUG Tech Day I did the same, while the only difference was that for the demo I used Oracle 12.2 PDB (I think in previous times I always used 11.2). During one of the demos I noticed something strange.