Configure MFA for OCI / IDCS - how to make sure you have a plan B

When you want to configure MFA on oracle Cloud Infrastructure (OCI), using ICDS, there is a good starting point in the OCI documentation. This document contains a warning (prerequisite number 5), but could have been a little more emphasized: 

"If you don’t register this client application and a Sign-On Policy configuration restricts access to everyone, then all users are locked out of the identity domain until you contact Oracle Support."

I recently had myself locked out of OCI and IDCS (and everyone else along with me), and this prerequisite step helped me through it. There were two challenges, though. The document does not tell you how to actually solve the problem of locking yourself out and it doesn't really explain why I locked myself out in the first place. Here, I will focus on the first part: how to solve the problem once you locked yourself out. It is nice to have this Emergency Access, but how do you use it?

Please note: if you skipped "prerequisite 5" (and did not create an OAuth2 application), this blog does not in fact really help. You will probably need Oracle Support in that case...

Prerequisite - Register a Client Application

So, very important step. Can't repeat it enough: if you want to enable MFA on IDCS, make sure this prerequisite is met. As mentioned in the documentation, use this link to Oracle by Example to set up your Emergency Access application. That's what I call it, as that is what it is for me. Should you lock yourself (and everyone else) out of OCI/IDCS, use this setup to get back in.

The Oracle by Example instructions are pretty clear on how to configure and use Postman. Due to some newer versions, some instructions may be a bit outdated, but all in all it works pretty well. Step 1 is done in IDCS and works as described, that is where you configure the Emergency Access application. Note the client ID and secret and store them in a safe location. 

Step 2 and 3 are about getting Postman configured with a (full) Environment and Collection to work with IDCS. Step 4 is all about getting an actual OAuth2 token. Very important, make sure that that part works. In my version of Postman config, I needed to use "On the Collections tab, expand OAuth, then Tokens and select Obtain access_token (client credentials)" and then press "Send" to get the actual token.

Even more important: make sure that with this token, you can access your IDCS environment. I like to do that with "On the Collections tab, expand Users, then Search and select List all users" and after pressing "Send", you should see all your users as a result. If not: don't proceed, but troubleshoot first! Make sure you have the access working, as this might be your last resort at some point.

How does this save you?

Now we know we can access IDCS, even if our MFA policy would happen to shut us out with the message: "Sign-on policy denies access". If that happens, the steps to solve this are quite easy if you know what to do. First of all: I assume you will have created a separate policy in IDCS for this, and not re-use the default one. As a matter of fact: I highly recommend not messing with the default one.

Even if you are locked out, you can retrieve the list of policies in IDCS with Postman. The value for this policy can be obtained from "Collections tab, expand Policies, then Search and select List all policies". All policies have an id. Find the one for your MFA policy and either note it down somewhere, or better yet: store it in a new custom variable in your Postman Environment. I used "MFA_Policy_id" for this.

With the Policy ID, you can go to "On the Collections tab, expand Policies, then Policy, Modify and choose to duplicate Patch Update a Policy". Rename the duplicate to (for example) "Disable MFA Policy". 

Replace the "{{id}}" in the URI with "{{MFA_Policy_id}}" (or just paste the id itself, if you didn't create a variable).

In the body of this new entry, paste:

{
  "schemas": [
    "urn:ietf:params:scim:api:messages:2.0:PatchOp"
  ],
  "Operations": [
    {
      "op": "replace",
      "path": "active",
      "value": false    }
  ]
}

Executing this request disables your new policy (literally setting "active" to "false"), allowing you to log in once again. That is why I leave the default policy as it is. It will now be the only one in effect and allows access to verified users.

Now, reconfigure the MFA policy, enable it again and see if it works this time. If not: disable the policy and try again. And remember: as long as you stay signed in to IDCS, you can disable the policy from the console as long as your connections stays valid. And should you get locked out (again), you will always have a plan B!

TYPE becomes INVALID after export/import

A very specific case: at a customer site, we had to refresh a database using expdp/impdp. After the refresh, some TYPEs got INVALID and couldn't be recompiled. The types themselves were not that complex, and not depending on any other objects.

To spoil the surprise, it turns out that the expdp for the database version we were using (12c) messed up the TYPE definition, because it expanded an object name in the definition itself. This in turn was caused by the use of a keyword "ALTER" in the TYPE. Funny thing is: we didn't actually use the keyword as such, but it was used as part of an attribute name.

For example:

CREATE TYPE testuser.test_type_with_keyword
as object
(
   name           varchar2(20),
   alternate_name varchar2(20),
   constructor function test_type_with_keyword

     (self in out nocopy test_type_with_keyword) 

     return self as result
)

not final
/

CREATE TYPE BODY testuser.test_type_with_keyword as

  constructor function test_type_with_keyword

    (self in out nocopy test_type_with_keyword)
    return self as result is
  begin
    return;
  end;
end;
/

This snippet contains ALTER as part of the "alternate_name". The export thinks it recognizes a keyword and expands the name in the definition. That way, the name in the TYPE does no longer match the name in the TYPE BODY and recompilation fails.

This can be reproduced using DBMS_METADATA, as that will result in the same name expansion, thus invalidating the TYPE. This is the result after import:

CREATE TYPE testuser.test_type_with_keyword
as object
(
   name           varchar2(20),
   alternate_name varchar2(20),
   constructor function test_type_with_keyword

     (self in out nocopy "TESTUSER"."TEST_TYPE_WITH_KEYWORD") 

     return self as result
)
not final

CREATE TYPE BODY testuser.test_type_with_keyword as
  constructor function test_type_with_keyword

    (self in out nocopy test_type_with_keyword)
    return self as result is
  begin
    return;
  end;
end;

Two possible workarounds:

• prefix the object reference in specification and body yourself
• don't use the keyword ALTER as part of any TYPE to avoid this expansion (there might be other keywords that are treated the same, but I haven't found any yet)

The good news is: this seems to be a 12c problem, it doesn't reproduce on 18c or 19c.

Complete example:

--

-- Create a normal type first

-- without any keyword, so this will be fine

--

set long 10000

CREATE OR REPLACE TYPE testuser.test_type_without_keyword
as object
(
   name     varchar2(20),
   alt_name varchar2(20),
   constructor function test_type_without_keyword

     (self in out nocopy test_type_without_keyword)

     return self as result
)
not final
/

CREATE OR REPLACE TYPE BODY testuser.test_type_without_keyword as
  constructor function test_type_without_keyword

    (self in out nocopy test_type_without_keyword)
    return self as result is
  begin
    return;
  end;
end;
/

select DBMS_METADATA.GET_DDL('TYPE','TEST_TYPE_WITHOUT_KEYWORD','TESTUSER') from dual;

/*

-- DBMS_METADATA results
CREATE OR REPLACE EDITIONABLE TYPE "TESTUSER"."TEST_TYPE_WITHOUT_KEYWORD"
as object
(
   name     varchar2(20),
   alt_name varchar2(20),
   constructor function test_type_without_keyword

     (self in out nocopy test_type_without_keyword)

     return self as result
)
not final
CREATE OR REPLACE EDITIONABLE TYPE BODY "TESTUSER"."TEST_TYPE_WITHOUT_KEYWORD" as
  constructor function test_type_without_keyword

    (self in out nocopy test_type_without_keyword)
    return self as result is
  begin
    return;
  end;
end;
*/

DROP TYPE testuser.test_type_without_keyword;

--
-- Test with the same type, 

-- but then with the ALTER keyword hidden in the code
--

CREATE OR REPLACE TYPE 

testuser.test_type_with_keyword
as object
(
   name           varchar2(20),
   alternate_name varchar2(20),
   constructor function test_type_with_keyword

     (self in out nocopy test_type_with_keyword)

     return self as result
)
not final
/

CREATE OR REPLACE TYPE BODY 

testuser.test_type_with_keyword as
  constructor function test_type_with_keyword

    (self in out nocopy test_type_with_keyword)
    return self as result is
  begin
    return;
  end;
end;
/

select DBMS_METADATA.GET_DDL('TYPE','TEST_TYPE_WITH_KEYWORD','TESTUSER') from dual;               

/*
-- DBMS_METADATA results
CREATE OR REPLACE EDITIONABLE TYPE 

"TESTUSER"."TEST_TYPE_WITH_KEYWORD"
as object
(
   name           varchar2(20),
   alternate_name varchar2(20),
   constructor function test_type_with_keyword

     (self in out nocopy "TESTUSER"."TEST_TYPE_WITH_KEYWORD" )

     return self as result
)
not final
CREATE OR REPLACE EDITIONABLE TYPE BODY

"TESTUSER"."TEST_TYPE_WITH_KEYWORD" as
  constructor function test_type_with_keyword

    (self in out nocopy test_type_with_keyword)
    return self as result is
  begin
    return;
  end;
end;
*/

DROP TYPE testuser.test_type_with_keyword;

Downloading files from an Oracle Directory on Amazon RDS

On Amazon RDS, you can upload dumpfiles with a Perl script as described in This Amazon RDS document (see pages 24/25). This works after a little rewrite for some linebreaks and syntax inconsistencies. The script enables you to upload and subsequently import your dumpfile using dbms_datapump on your Amazon RDS Oracle database.

But what if you want to create a datapump export and download the resulting dumpfile(s)? So far I have not found a working example that fits my needs, so I decided to create a new script. As I have no real Perl experience, it will most probably not be an award winning script, but it does the job.

# INPUT parameters:
# 1 – the full name of the RDS server
# 2 - the filename to be downloaded from RDS
# 3 – the password for the DBA user
use strict;
use DBI qw(:sql_types);
use DBD::Oracle qw(:ora_types);
# turn off screen buffering (for displaying progress)
$|++;
# RDS instance info
my $RDS_PORT=1521;
my $RDS_HOST=$ARGV[0];
my $RDS_LOGIN_USER="dba_user";
my $RDS_LOGIN_PW=$ARGV[2];
my $RDS_SID="ORCL";
my $dirname = "DATA_PUMP_DIR";
# Timing variable - start time
my $sttime = time;
my $loopcount = 0;
my $totalcount = 0;
my $totalsize = 0;
# Rest of the variables
my $fname = $ARGV[1];
my $data;
my $chunk = 32767; #max READ size
my $sql_open = "BEGIN perl_global.fh := utl_file.fopen(:dirname, :fname, 'rb', :chunk); END;";
my $sql_read = "BEGIN utl_file.get_raw(perl_global.fh, :data, NULL); EXCEPTION WHEN NO_DATA_FOUND THEN :datalength := 0; WHEN OTHERS THEN RAISE; END;";
my $sql_close = "BEGIN utl_file.fclose(perl_global.fh); END;";
my $sql_global = "create or replace package perl_global as fh utl_file.file_type; end;";
my $conn = DBI->connect('dbi:Oracle:host='.$RDS_HOST.';sid='.$RDS_SID.';port='.$RDS_PORT, $RDS_LOGIN_USER, $RDS_LOGIN_PW) || die ( $DBI::errstr . "\n") ;
my $updated=$conn->do($sql_global);
my $stmt = $conn->prepare ($sql_open);
# Processing the file
$stmt->bind_param_inout(":dirname", \$dirname, 12);
$stmt->bind_param_inout(":fname", \$fname, 12);
$stmt->bind_param_inout(":chunk", \$chunk, 4);
$stmt->execute() || die ( $DBI::errstr . "\n");
open (INF, '>', $fname) || die "\nCan't open $fname for writing: $!\n";
binmode(INF);
$stmt = $conn->prepare ($sql_read);
my %attrib = ('ora_type','24');
my $datalength=1;
print "\nTransferring file: \n";
while ($datalength > 0) {
  $stmt->bind_param_inout(":data", \$data , $chunk, \%attrib);
  $stmt->bind_param_inout(":datalength", \$datalength , 4);
  $stmt->execute() || die ( $DBI::errstr . "\n");
  if ($datalength > 0) {
    print INF $data;
    $loopcount++;
    $totalcount++;
    # refresh display after 320 chunks of 32k bytes = 10 Mb
    if ($loopcount == 320) {
      $totalsize = $totalcount*32/1024;
      print "\r# $totalsize Mb written";
      $loopcount = 0;
    };
  };
};
# Wrap up
$totalsize = $totalcount*32/1024;
print "\r# $totalsize Mb written (done)\n";
close INF || die "Can't close $fname: $!\n";
$stmt = $conn->prepare ($sql_close);
$stmt->execute() || die ( $DBI::errstr . "\n");
#
my $entime = time;
my $elapse = $entime - $sttime;
print "\nElapsed time : $elapse seconds\n\n";


The database/instance name, the DBA database user and port are hardcoded in the script, and I added some parameters. This was sufficient for my situation, and can of course be modified as needed. Just modify the section "RDS Instance Info" or add additional parameters.

I added some "features", like a progress tracker that updates every 10 Mb downloaded content. Not really necessary, but for larger files I like to know what he is doing and far along the download is. All in all this enables me to directly download the dumpfiles (and other files of course) without having to use the dbms_file_transfer (to get it to another database) or the limited "SELECT text FROM table(rdsadmin.rds_file_util.read_text_file('DIR','file.txt'))" for reading text files.

Now, just call it from the commandline, for example:
perl /home/oracle/scripts/download_from_RDS.pl rds_server_name filename

Resolve ORA-01996 (password file full) without losing your password file entries

As I was playing around with one of my 12c databases, I ran into the ORA-01996 error "GRANT failed: password file '/<>/orapwORCL' is full". The documentation and various other sources are clear about this: you will have to re-create the password file and add all existing entries manually afterwards.

Just when I was about to create a new password file and re-create the entries in there, it occurred to me that 12c has a new feature: the input_file parameter for orapwd. As far as the documentation is concerned, this is to migrate older versions of the password file to 12c.

But, what if we can use this to "migrate" our current 12c password file to a new 12c password file? What if we backup this file and use that as an input for our new password file? I created a quick test setup and found out that this seems to be exactly what I needed. I re-created the password file using the input_file parameter and I could add new entries, but at the same time I kept all the entries that were already there.

The testcase

First of all, I created a new passwordfile with entries=1. Actually, this will hold up to 4 entries for me, because of the fact that it is rounded up to the number of entries per block (in the same documentation I mentioned above).

[oracle@DBServer1 ~]$ cd $ORACLE_HOME/dbs
[oracle@DBServer1 dbs]$ orapwd file=orapwORCL entries=1 force=y
Enter password for SYS:

Using SQL*Plus, I verified that all I had was 1 entry:

SQL> select * from v$pwfile_users order by username, con_id;

USERNAME                       SYSDB SYSOP SYSAS SYSBA SYSDG SYSKM     CON_ID
------------------------------ ----- ----- ----- ----- ----- ----- ----------
SYS                            TRUE  TRUE  FALSE FALSE FALSE FALSE          0

So, now we can start adding users to the passwordfile, until it is full:

SQL> grant sysdba to c##erik container=all;
Grant succeeded.

SQL> grant sysdg to sysdg;
Grant succeeded.

SQL> grant sysbackup to sysbackup;
grant sysbackup to sysbackup
*
ERROR at line 1:
ORA-01996: GRANT failed: password file
'/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwORCL' is full

What do we have in the v$pwfile_users at this point?

SQL> select * from v$pwfile_users order by username, con_id;

USERNAME                       SYSDB SYSOP SYSAS SYSBA SYSDG SYSKM     CON_ID
------------------------------ ----- ----- ----- ----- ----- ----- ----------
C##ERIK                        TRUE  FALSE FALSE FALSE FALSE FALSE          0
SYS                            TRUE  TRUE  FALSE FALSE FALSE FALSE          0
SYSDG                          FALSE FALSE FALSE FALSE TRUE  FALSE          0
SYSDG                          FALSE FALSE FALSE FALSE TRUE  FALSE          1

Now, we will go back to $ORACLE_HOME/dbs and re-create the password file, using the input_file parameter. Make sure you backup the current password file, as you can never be too careful...

[oracle@DBServer1 dbs]$ cp orapwORCL orapwORCL_backup
[oracle@DBServer1 dbs]$ orapwd file=orapwORCL entries=10 input_file=orapwORCL_backup force=y

Immediately after that, check the contents of the password file with SQL*Plus:

SQL> select * from v$pwfile_users order by username, con_id;

USERNAME                       SYSDB SYSOP SYSAS SYSBA SYSDG SYSKM     CON_ID
------------------------------ ----- ----- ----- ----- ----- ----- ----------
C##ERIK                        TRUE  FALSE FALSE FALSE FALSE FALSE          0
SYS                            TRUE  TRUE  FALSE FALSE FALSE FALSE          0
SYSDG                          FALSE FALSE FALSE FALSE TRUE  FALSE          0
SYSDG                          FALSE FALSE FALSE FALSE TRUE  FALSE          1

And all entries are still there! Just one last step to prove we have successfully re-created the password file with more entries than we had:

SQL> grant sysbackup to sysbackup;
Grant succeeded.

SQL> select * from v$pwfile_users order by username, con_id;

USERNAME                       SYSDB SYSOP SYSAS SYSBA SYSDG SYSKM     CON_ID
------------------------------ ----- ----- ----- ----- ----- ----- ----------
C##ERIK                        TRUE  FALSE FALSE FALSE FALSE FALSE          0
SYS                            TRUE  TRUE  FALSE FALSE FALSE FALSE          0
SYSBACKUP                      FALSE FALSE FALSE TRUE  FALSE FALSE          0
SYSBACKUP                      FALSE FALSE FALSE TRUE  FALSE FALSE          1
SYSDG                          FALSE FALSE FALSE FALSE TRUE  FALSE          0
SYSDG                          FALSE FALSE FALSE FALSE TRUE  FALSE          1

And I didn't even have to reconnect to or restart the database :-)



Edited: Thanks to Pom for pointing out a small but essential typo

TNS-12154 on Database Link When Connected Through Listener

I ran into a problem when creating a database link that uses a new TNS entry. The database link seemed to work fine when I created it, but the developers complained about getting TNS-12154 errors when they tried to use it. Checked it again on my side, and the database link worked just fine.

 

After some discussion with the development team, we noticed that the database did work when I connected locally from the database server, but it threw the TNS-12154 when connecting through the listener. So I focussed on why the listener prevented the database link from recognizing my newly created TNS entry. Turns out it has a lot to do with setting a TNS_ADMIN environment variable, the dynamic registration to the listener and the way the listener uses environment variables.

 

So, to paint a picture of the situation:

·        We have a database on Linux (probably any Linux or UNIX variant will have this problem)

·        We have set the TNS_ADMIN environment variable

·        The listener uses both static and dynamic service registration (because we are using OEM)

 

Solutions proposed so far

 

This problem is described in a lot of places, but I could not find a satisfactory solution until recently. The answers I found so far are not satisfactory, some more so than others. They include:

·        Restart the database. When creating a new TNS entry, this will not been seen by the dynamic service registration to the listener. Restarting the database somehow resets this and then the database link also “magically” works.

o   My problem with this: Restarting a (Production) database is not something I want to do when I just want to add a new TNS entry for a database link.

·        Use a different naming method when creating the database link. Instead of the USING clause with a TNS entry, you can specify an EZConnect string or the full description of the TNS entry. Both imply that server and database specific information is to be used in the database link itself.

o   But: I use the tnsnames.ora to have a central location for all descriptions and use that to point to any database/service I need. I don’t want to repeat sevrer/instance information in every database link I create.

·        Set the ENVS clause of the static registration (SID_LIST_LISTENER) to include TNS_ADMIN.

o   This has several drawbacks: You need to use static registration and static registration only. Dynamic takes precedence over Static, and Dynamic cannot be used to set TNS_ADMIN. Even when you decide to use only Static, then you have to maintain the TNS_ADMIN in the environment variable itself and in the listener.ora which means extra maintenance.

·        Don’t use TNS_ADMIN. When not using TNS_ADMIN, all works fine. As the database and the listener will then both use the default location ($ORACLE_HOME/network/admin), they can always find the tnsnames.ora and will pick up new entries right away.

o   The problem is obvious: we use TNS_ADMIN to create a centralized location for our tnsnames.ora file. Not using the TNS_ADMIN would defeat this initial setup.

 

My solution

 

The last “solution”, however, opens up a final solution. When the listener (the dynamic service registration) fails to pick up (or refresh, or whatever you like to call it) changes to tnsnames.ora in the TNS_ADMIN location, it will look at the default location. Creating an OS link there will allow the listener to find the file anyway and use the new TNS entry without having to restart the database. And yes, there is a drawback to this solution as well: you need to create this link in every RDBMS home you created and point it to the TNS_ADMIN location. I think this is still slightly better than not using TNS_ADMIN at all, because we will still have a centralized tnsnames.ora file, and only use links to that file in the RDBMS home directory.

 

Technical “proof”

 

The example sketched below is from an AIX machine, using an 11gR2 database. I also tested this on a 12c database on Linux. Both have the exact same results when it comes to recognizing the new TNS entry.

 

Using the original setup, I modified the tnsnames.ora file in the TNS_ADMIN location:

 

dbserver::../home/oracle:> cd $TNS_ADMIN

dbserver::../oracle/tns:> ls -l

total 16

-rw-r--r--    1 oracle   dba             483 Dec 16 16:37 listener.ora

-rw-r--r--    1 oracle   dba            1581 Dec 16 16:14 tnsnames.ora

dbserver::../oracle/tns:> vi tnsnames.ora

 

I added an “orcldb_self_ref.world” entry, pointing toward my original database. After this, without restarting the database or reloading the listener, I created the database link.

 

dbserver::../oracle/tns:> sqlplus system@orcldb

 

SQL*Plus: Release 11.2.0.2.0 Production on Thu Dec 17 09:41:34 2015

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Enter password:

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

With the Partitioning, OLAP, Data Mining and Real Application Testing options

 

SQL> create database link dblink_test_erik using 'orcldb_self_ref.world';

 

Database link created.

 

SQL> select * from dual@dblink_test_erik;

select * from dual@dblink_test_erik

                   *

ERROR at line 1:

ORA-12154: TNS:could not resolve the connect identifier specified

 

And there you have it, the TNS-12154. Normally, you would think that we did something wrong defining the TNS entry or something like that. However, when not connecting through the listener, the same database link does work:

 

dbserver::../oracle/tns:> sqlplus system

 

SQL*Plus: Release 11.2.0.2.0 Production on Thu Dec 17 09:41:34 2015

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Enter password:

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

With the Partitioning, OLAP, Data Mining and Real Application Testing options

 

SQL> select * from dual@dblink_test_erik;

 

D

-

X

 

So, on to the next part, creating an OS link:

 

dbserver::../home/oracle:> cd $ORACLE_HOME/network/admin

dbserver::../network/admin:> ln -s $TNS_ADMIN/tnsnames.ora ./tnsnames.ora

dbserver::../network/admin:> ls -l

total 8

drwxr-xr-x  2 oracle dba 256 Oct 23 09:49 samples

-rw-r--r--  1 oracle dba 187 May 07 2007  shrept.lst

lrwxrwxrwx  1 oracle dba  28 Dec 17 09:45 tnsnames.ora -> /u01/oracle/tns/tnsnames.ora

 

Directly after creating the link, our database link works as it should. Just to prove it works right away, I dropped and re-created it.

 

dbserver::../oracle/tns:> sqlplus system@orcldb

 

SQL*Plus: Release 11.2.0.2.0 Production on Thu Dec 17 09:49:10 2015

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Enter password:

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

With the Partitioning, OLAP, Data Mining and Real Application Testing options

 

SQL> drop database link dblink_test_erik;

 

Database link dropped.

 

SQL> create database link dblink_test_erik using 'orcldb_self_ref.world';

 

Database link created.

 

SQL> select * from dual@dblink_test_erik;

 

D

-

X

 

Note that when you are already connected, it might be necessary to re-connect to pick up the changes. But that beats restarting the entire database...

 

Final remarks

 

Though this is my preferred solution, it doesn’t mean that it is the only solution. If any of the other options suits your needs better, then by all means: use them ;-)

 

This solution uses an existing setup with TNS_ADMIN set and dynamic service registrations. Both good things in my opinion. Then, by creating a simple link, you can make sure that any changes to your TNS entries are useable right away, instead of having to configure other files or restarting the database.

Testing all Oracle database links for all schema's

I had to validate quite a lot of database links this week, and a lot of different schema's were involved. A database link is deemed valid/usable when the statement "select * from dual@db_link" succeeds, so simply executing this code on every database link would suffice.

However, when connected as one user, you can validate the database links for that user and any public links. Validating the database links for other users doesn't seem to be possible from that single connection. You can't prefix the database link name with the schema, as database links themselves can contain dots. Thus, the schema prefix will be treated as part of the link name, causing the statement to fail (because the link cannot be found under that name).

Then I figured that using a stored procedure created in the owner schema could validate the database link in that schema, when created with definer's rights (the default). And because you can use dynamic SQL ("execute immediate" in this case), you can create the procedure in the other schema, invoke it to test the database link in that same schema, and drop it afterwards.

-- Execute as SYS user (because of ORA-01031)

CREATE OR REPLACE PROCEDURE test_any_db_link ( p_owner IN varchar2
                                             , p_link  IN varchar2
                                             )
AS
   l_owner  varchar2(100);
BEGIN
   IF p_owner = 'PUBLIC' THEN
      l_owner := 'SYSTEM';
   ELSE
      l_owner := p_owner;
   END IF;
   --
   BEGIN
     EXECUTE IMMEDIATE 'create procedure '||l_owner||'.test_any_db_link_temp '||
                       'as '||
                       '  l_dummy varchar2(1); '||
                       'begin '||
                       '  select * into l_dummy from dual@'||p_link||'; '||
                       '  dbms_output.put_line(''OK : '||p_owner||'.'||
                                                         p_link||'''); '||
                       'exception '||
                       ' when others then '||
                       '   dbms_output.put_line(''NOK: '||p_owner||'.'||
                                                          p_link||'''||'' ''||
                                                          sqlcode); '||
                       'end;';
     EXECUTE IMMEDIATE 'begin '||l_owner||'.test_any_db_link_temp; end;';
   EXCEPTION
     WHEN OTHERS THEN
       dbms_output.put_line('ERR: '||p_owner||'.'||p_link||' '||sqlcode);
   END;
   EXECUTE IMMEDIATE 'drop procedure '||l_owner||'.test_any_db_link_temp';
END;
/

set serveroutput on size 1000000;

spool test_db_links

BEGIN
  FOR r_stmt IN (select 'begin test_any_db_link('''||owner||''','''||
                                                     db_link||'''); end;' stmt 
                   from dba_db_links 
                  order by owner, db_link
                )
  LOOP
    execute immediate r_stmt.stmt;
  END LOOP;
END;
/

spool off

drop procedure test_any_db_link;

-- done

Main points about this code:

• You execute it as SYS, because of an ORA-01031 (insufficient privileges) otherwise
• The script creates a procedure for every test, and drops it afterwards
• The status of a link is either OK (everything checks out), or ERR (the procedure could not be created, because the link is not valid). In theory, the status could be NOK, when the link is valid, but something else prevents the execution of "select * from dual" from succeeding. So far, I never saw an NOK code
• This script was used on database versions ranging from 9.2 to 12.1
• The error output is very short, you could enhance this with dbms_utility.format_error_stack or something similar
• When a database link is marked as "ERR", investigate further. I have some "ExtProc" call type database links, and those were marked as "ERR", because there is no dual at the other end
• For PUBLIC links, I translated the user for the procedure to SYSTEM, as you cannot create a procedure under the "schema PUBLIC"

Application Express - using the sort arrows in a tabular form - Updated for Apex 4.x

In a previous post (already a few years back) I demonstrated how to get the Apex sort arrows as seen in Apex builder in your own applications.



This was done in Apex 3.x, and in the meanwhile someone pointed out that in Apex 4.0 this was not working anymore. Not having the time to go into that in detail, I left it at that. Recently, someone else reported errors with this approach, again for the Apex 4.x versions. So I sat down and figured out the new way to do this for Apex 4.x

Again, we will be using a simple Tabular Form:



First step: create the arrows

This step is different from what you did in Apex 3.x because there is an additional DIV section and an extra TABLE around Tabular Forms now. The surrouding section carries the report region as an ID, the Tabular Form itself has no ID to work with. In order to get an ID, we are going to modify the template on which the Tabular Form is based.
In this case, I modify the template directly, you should always choose to create a new template (copy it from this one) and assign that template to the region... Otherwise, all other reports using this template will get sort-arrows and that might not be what you wanted.

In the Apex Builder, go to "Shared Components - User Interface / Templates - Report / Standard". Normally, the Tabular Forms use this template. If you use another template modify that one (or rather: copy that one for modifying). In the "Before Rows" section, modify the last line (modify the class and add an ID):

• from <td><table cellpadding="0" border="0" cellspacing="0" summary="" class="report-standard">
• to <td><table cellpadding="0" border="0" cellspacing="0" summary="" class="report-standard-sort" ID="report_#REGION_ID#_sortable">

In the "After Rows" section, add this block right after the </table> tag (and before any other code already there):

<script src="/i/libraries/apex/minified/builder.min.js" type="text/javascript"></script>
<script type="text/javascript">
  var g_rpreview_global = 'report_#REGION_ID#_sortable';
  var g_#REGION_ID#_sortable;
  function f_#REGION_ID#_sortable(){
    g_#REGION_ID#_sortable = new apex.tabular.sort(g_rpreview_global);
    g_#REGION_ID#_sortable.row.after_move = function(){rpreview()};
  }
  addLoadEvent(f_#REGION_ID#_sortable);
</script>

Do this for all templates that you will be using for sortable reports. When switching themes, you will need to do this again!
Note the extra JS library included: the builder.min.js contains the necessary Javascript do implement the sorting.

After this, the form has the up/down arrows next to every line.



Second step: hide the order column and make it orderable

To enable ordering and make the order column hidden, just take these steps:

• Edit the form properties and set the "order" item property "show" unchecked
• Edit the column properties for "order" and set "Element Attributes" to class="orderby"

Now we have a simple form which we can order using the arrows.


Third step: adjusting the style

The last step we need to take is to make the background and header to look like the template we use. Regrettably, I found no really easy (configurable) way to do this. So, we'll do this the hard way.

First of all, you'll have to get the style you're using in the application. The stylesheet is referenced in your application and viewable by just showing the source of your page in your browser. In this example I ran the application, selected "show source" after right-clicking and searched for the stylesheet. This shouldn't be too hard to find (was on line 34 for me, first mention of themes).

When you look at this stylesheet (by either downloading it from the application server, or looking it up in an APEX software download), you should be able to find the section of interest by searching for "report-standard th.header". The section you'll find and the section for "report-standard th.data" are to be used.
Depending on the template you chose, the numbers and settings will be somewhat different. These sections will be used to create .report-standard-sort qualifiers (that is why we modified the class in step 1).

We must set the background color for the header, which is not directly mentioned in the section we just found. There is an url pointing to the background image, but that is relative to the stylesheet itself. Modify it to reflect the theme (theme_2 for me) and make the path relative to the apex page itself. Place the style tags around this block. We now have (after reformatting):

<style>
.report-standard-sort th {color: #ffffff; background-color: #cccccc; padding: 2px 10px; border-bottom: 1px solid #cccccc; background-image: url(../../i/themes/theme_2/images/report_bg.gif); background-repeat: repeat-x;}
.report-standard-sort td { background-color: #f0f0f0; padding: 4px 10px; border-bottom: 1px solid #cccccc;}
</style>

By placing this as a style tag in the HTML header of the page, our tabular form is now ready to go.



Keep in mind:

• When using a new template, you should also (copy and) modify that template for the region
• When using a new theme, all templates should be (copied and) modified again
• When creating a new Tabular Form, change the report region template to the sorting template and copy the HTML header section for the style