There is a lot of change that release 12.1 or 12c has brought up. And it’s not in the more advanced new features that I am talking about but even in some areas about which, we have memorized a particular concept by heart from last many years. Once such change is what I am going to mention in this post-introduction of the LREG(Listener Registration) process.
Basics of Dynamic Registration
In the earlier releases,to register an instance with the listener, dba’s were required to manage the lsitener configuration file-Listener.ora. One would need to mention in this file the specifications of database whose instance we would want listener to listen for incoming connection requests. Not that it was very hard to do but it was still one of those tasks which if could be made automatic, it would make life a little simpler. And that’s what happened when from 9i onwards. The requirement to use a listener.ora file to register an instance with the listener was removed from this release and this task was delegated to the PMON(Process Monitor) process. At the time of the instance startup, PMON process would register the instance with the listener. If the listener is not up, it would keep on looking for the listener and once it’s up, the registration would happen with a delay of about 60 seconds. And this continued till 11.2.
Now, from 12.1, the same is handled by a process dedicated for this work, LREG(Listener Registration)process.
If I had stopped here, it would be a very simple post and probably, a not-so-interesting one either. So I thought to explain a little more. If you are interested to understand how does the whole thing works, keep on reading it.
How does Dynamic Registration works?
There is a lot that goes on behind the scene when the database and its related processes are functional. And all of it goes back to how OS works. In the dynamic registration, two important concepts-File Descriptors and Sockets come into the play.
When the listener process starts, it creates two socket files under the location /var/tmp/.oracle. I haven’t started my listener right now. So let’s see what’s the output from the folder,
|
1 2 3 4 5 6 7 8 9 10 11 |
[oracle@arista demos]$ cd /var/tmp/.oracle [oracle@arista .oracle]$ ls s#2502.1 s#2654.1 s#2769.1 s#2794.1 s#3429.1 s#4498.1 sEXTPROC1521 s#2502.2 s#2654.2 s#2769.2 s#2794.2 s#3429.2 s#4498.2 s#2567.1 s#2659.1 s#2773.1 s#2864.1 s#4301.1 s#7148.1 s#2567.2 s#2659.2 s#2773.2 s#2864.2 s#4301.2 s#7148.2 [oracle@arista .oracle]$ pgrep tns 15 [oracle@arista .oracle]$ ps -ef | grep tns root 15 2 0 12:27 ? 00:00:00 [netns] oracle 5680 2514 0 18:20 pts/0 00:00:00 grep tns |
There is no Listener process running but there is a [netns] process owned by Root. It stands for Network Namespace process of Linux and is spawned by the [ktthreadd] (Kernel Thread Daemon), used to spawn other kernel threads. There is nothing required to be done about [netns] by a DBA.
If the listener process is started(the output is from a previous session when I started the listener), you can see the socket files created by the listener process in my 12.1.0.1 version running on OEL 6.5 below,
|
1 2 3 4 5 6 7 8 9 10 11 12 |
[root@arista .oracle]# ps -ef | grep tns root 15 2 0 19:21 ? 00:00:00 [netns] oracle 3022 1 0 21:36 ? 00:00:00 /u01/app/oracle/product/12.1.0/dbhome_1/bin/tnslsnr LISTENER -inherit root 4067 3145 0 21:36 pts/3 00:00:00 grep tns [root@arista .oracle]# pwd /var/tmp/.oracle [root@arista .oracle]# ls s#2502.1 s#2654.1 s#2769.1 s#2794.1 s#3429.1 s#4301.1 sEXTPROC1521 s#2502.2 s#2654.2 s#2769.2 s#2794.2 s#3429.2 s#4301.2 s#2567.1 s#2659.1 s#2773.1 s#2864.1 s#3022.1 s#7148.1 s#2567.2 s#2659.2 s#2773.2 s#2864.2 s#3022.2 s#7148.2 |
As I said, there are several file descriptors that are created and all of them are present under the /proc file system. Below are the file descriptors that I had got in my system,
|
1 2 3 4 5 6 |
[root@arista ~]# cd /proc [root@arista proc]# ls 1 1572 2010 2248 2614 3048 3280 dma net 10 16 2016 2250 2643 3050 3282 dri pagetypeinfo 1071 1636 2039 2251 2671 3052 3284 driver partitions <<output trimmed>> |
Since we know that our listener’s PID was 3022, let’s check that out by going into the folder with the same PID.
|
1 2 3 4 5 6 7 8 9 10 |
[root@arista proc]# cd 3022 [root@arista 3022]# ls attr cpuset latency mountstats pagemap stack auxv cwd limits net personality stat cgroup environ loginuid ns root statm clear_refs exe maps numa_maps sched status cmdline fd mem oom_adj schedstat syscall comm fdinfo mountinfo oom_score sessionid task coredump_filter io mounts oom_score_adj smaps wchan |
You can see two folders, FD and FDINFO. Let’s see what we have inside the folder FD.
|
1 2 3 |
[root@arista 3022]# cd fd [root@arista fd]# ls 0 1 10 11 12 13 14 15 16 17 18 2 27 28 3 4 5 6 7 8 9 |
And we can see that there are several file descriptors that are there.
Thus it is established that there are file descriptors and sockets that get created for the process and in our case, we have checked the same for our listener process.
So how does these two get fit into the dynamic registration done by LREG process (and in the previous release, PMON)? Well, it’s simple. The processes use system calls to see that whether the listener is up or not(listener doesn’t need to be there for the database to be started). If there is no listener process found, the wait continues and after a certain time(3000 milliseconds) , a retry is done. This keeps on going till the listener is started and when that happens, the file descriptor opened by the listener is “binded” by the process to estabilish the connection between each other.
To find what the LREG process is doing, I used STRACE os utility to trace it’s working. The output for the PMON process in 11203 wasn’t entirely the same but I guess, that’s expected as it wasn’t a process dedicated for just getting the instance registered with the listener. On the other hand, LREG’s sole purpose is the dynamic registratior so that might explain the little difference between the system calls used between the two processes.
To see what happens when the listener is not present, I just started the database and used strace over it. Below is the excerpt of the output of STRACE,
|
1 2 3 4 5 6 7 8 9 10 |
[oracle@arista .oracle]$ strace -p 2581 Process 2581 attached - interrupt to quit epoll_wait(12, {}, 1024, 3000) = 0 getrusage(0x1 /* RUSAGE_??? */, {ru_utime={0, 891000}, ru_stime={0, 551000}, ...}) = 0 getrusage(0x1 /* RUSAGE_??? */, {ru_utime={0, 891000}, ru_stime={0, 551000}, ...}) = 0 times(NULL) = 431716614 epoll_wait(12, {}, 1024, 3000) = 0 getrusage(0x1 /* RUSAGE_??? */, {ru_utime={0, 891000}, ru_stime={0, 552000}, ...}) = 0 getrusage(0x1 /* RUSAGE_??? */, {ru_utime={0, 891000}, ru_stime={0, 552000}, ...}) = 0 times(NULL) = 431716914 |
We can see few system calls here , starting with epoll_wait that is used to represent the time waiting for events over file descriptors.The last argument is showing the time in milliseconds-3000 milliseconds(which means 3 seconds) . Skipping the next two lines for the resource usage consumption, the times function returning the time in seconds and if you notice, it’s having a difference of 3 between the above posted two snippets of the output.
Next snippet is going to be a little bigger and that’s where the entire action is happening.
Picking up the relevant thing from the output, the first interesting piece is the starting itself,
|
1 |
socket(PF_NETLINK, SOCK_RAW, 0) = 17 |
What is happening is that a socket function is used to work with file descriptor 17. I guess this is a fixed sort of file descriptor that’s used in establishing the connection with the listener process. The socket used here is NETLINK which is used to create a connection between the kernel and the network layers.
Moving on, the next line is
|
1 |
bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 |
Here a bind attempt with the file descriptor 17, returned from the previous statement is initiated. In the next few lines, this file descriptor would be used to initiate the connection with the PID 2581, which is the the PIF of LREG process.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=2581, groups=00000000}, [12]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\321\261xS\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"0\0\0\0\24\0\2\0\321\261xS\25\n\0\0\2\10\200\376\1\0\0\0\10\0\1\0\177\0\0\1"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 108 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\321\261xS\25\n\0\0\n\200\200\376\1\0\0\0\24\0\1\0\0\0\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\321\261xS\25\n\0\0\0\0\0\0\1\0\0\0\24\0\1\0\0\0\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=2581, groups=00000000}, [12]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\321\261xS\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"0\0\0\0\24\0\2\0\321\261xS\25\n\0\0\2\10\200\376\1\0\0\0\10\0\1\0\177\0\0\1"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 108 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\321\261xS\25\n\0\0\n\200\200\376\1\0\0\0\24\0\1\0\0\0\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\321\261xS\25\n\0\0\0\0\0\0\1\0\0\0\24\0\1\0\0\0\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 |
Now, the details of the host IP address is gathered using the file /etc/hosts.
|
1 2 3 4 5 6 7 8 9 |
open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 17 fstat(17, {st_mode=S_IFREG|0644, st_size=85, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa6722d7000 read(17, "127.0.0.1 localhost.localdomai"..., 4096) = 85 read(17, "", 4096) = 0 close(17) = 0 munmap(0x7fa6722d7000, 4096) = 0 getsockopt(0, SOL_SOCKET, SO_SNDBUF, 0x7fff025a418c, 0x7fff025a4188) = -1 ENOTSOCK (Socket operation on non-socket) getsockopt(0, SOL_SOCKET, SO_RCVBUF, 0x7fff025a418c, 0x7fff025a4188) = -1 ENOTSOCK (Socket operation on non-socket) |
An attempt to estabilish the connect with IP address is in progress.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 17 fcntl(17, F_SETFL, O_RDONLY|O_NONBLOCK) = 0 connect(17, {sa_family=AF_INET, sin_port=htons(1521), sin_addr=inet_addr("192.168.80.80")}, 16) = -1 EINPROGRESS (Operation now in progress) getsockopt(17, SOL_SOCKET, SO_SNDBUF, [169523252526333952], [4]) = 0 getsockopt(17, SOL_SOCKET, SO_RCVBUF, [169523252526404948], [4]) = 0 times(NULL) = 431716915 times(NULL) = 431716915 rt_sigaction(SIGPIPE, {SIG_IGN, ~[ILL ABRT BUS FPE SEGV USR2 TERM XCPU XFSZ SYS RTMIN RT_1], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x3b31a0f710}, {SIG_IGN, ~[ILL ABRT BUS FPE KILL SEGV USR2 TERM STOP XCPU XFSZ SYS RTMIN RT_1], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x3b31a0f710}, 8) = 0 epoll_ctl(12, EPOLL_CTL_ADD, 17, {EPOLLOUT, {u32=1872189872, u64=140352813488560}}) = 0 times(NULL) = 431716915 epoll_wait(12, {{EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=1872189872, u64=140352813488560}}}, 1024, 3000) = 1 getsockname(17, {sa_family=AF_INET, sin_port=htons(13818), sin_addr=inet_addr("192.168.80.80")}, [16]) = 0 write(17, "\0t\0\0\1\0\0\0\1;\1,\0\0 \0\377\377\177\10\0\0\1\0\0.\0F\0\0\0\0"..., 116) = -1 ECONNREFUSED (Connection refused) epoll_ctl(12, EPOLL_CTL_DEL, 17, {0, {u32=1872189872, u64=140352813488560}}) = 0 close(17) = 0 |
Since there is no listener process started yet and also, there is no file descriptor 17 associated with the process LREG yet, the connection is refused. There is no established connection on the as well and we can confirm it using the LSOF command.
|
1 2 |
[root@arista ~]# lsof -iTCP:1521 [root@arista ~]# |
Also, there is no such file-descriptor in the /proc for PID 2581 as well.
|
1 2 3 |
[root@arista 2581]# cd fd [root@arista fd]# ls 0 1 10 11 12 13 14 15 16 18 2 27 28 3 4 5 6 7 8 9 |
Now, let’s start the listener. I had kept the tracing going on when the listener was started. The very first thing to notice is that in the /proc file-system, the file-descriptor 17 is now available.
|
1 2 3 |
[root@arista ~]# cd /proc/2581/fd [root@arista fd]# ls 0 1 10 11 12 13 14 15 16 17 18 2 27 28 3 4 5 6 7 8 9 |
In the strace, there isn’t any change in the output and the same system calls that we have seen in the previous output are used except that now,there won’t be any connection refused message.
We can now see that the connection is estabilished using the same LSOF command,
|
1 2 3 4 5 |
[root@arista fd]# lsof -iTCP:1521 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME ora_lreg_ 2581 oracle 17u IPv4 52570 0t0 TCP arista.localdomain:14287->arista.localdomain:ncube-lm (ESTABLISHED) tnslsnr 7902 oracle 9u IPv6 52381 0t0 TCP *:ncube-lm (LISTEN) tnslsnr 7902 oracle 23u IPv6 52571 0t0 TCP arista.localdomain:ncube-lm->arista.localdomain:14287 (ESTABLISHED) |
We can see that file descriptor 17u(u stands for the mode, read and write) is used by the LREG process to estabilish a TCP connection. Also the socket files are created under the /var/tmp/.oracle folder for the listener process 7902.
|
1 2 3 4 5 6 |
[root@arista fd]# cd /var/tmp/.oracle/ [root@arista .oracle]# ls s#2502.1 s#2567.2 s#2659.1 s#2769.2 s#2794.1 s#2864.2 s#4301.1 s#4498.2 s#7902.1 s#2502.2 s#2654.1 s#2659.2 s#2773.1 s#2794.2 s#3429.1 s#4301.2 s#7148.1 s#7902.2 s#2567.1 s#2654.2 s#2769.1 s#2773.2 s#2864.1 s#3429.2 s#4498.1 s#7148.2 sEXTPROC1521 [root@arista .oracle]# |
In case you are wondering what is this ncube-lm thingy shown in the output, that’s
NCube-License Manager. If you check out the website, http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?&page=26, it seems that this port is assigned to Maxine Yuen. So seems like the port 1521 is not really owned but just used or shared by Oracle Corp for the listener process. Interesting isn’t it?
Before ending the post, if you are not willing to use Strace, you can opt for listener tracing as well. I haven’t posted the output of that here though.
Hope it helps, in some way!