Links
Home
Oracle DBA Forum
Frequent Oracle Errors
TNS:could not resolve the connect identifier specified
Backtrace message unwound by exceptions
invalid identifier
PL/SQL compilation error
internal error
missing expression
table or view does not exist
end-of-file on communication channel
TNS:listener unknown in connect descriptor
insufficient privileges
PL/SQL: numeric or value error string
TNS:protocol adapter error
ORACLE not available
target host or object does not exist
invalid number
unable to allocate string bytes of shared memory
resource busy and acquire with NOWAIT specified
error occurred at recursive SQL level string
ORACLE initialization or shutdown in progress
archiver error. Connect internal only, until freed
snapshot too old
unable to extend temp segment by string in tablespace
Credential retrieval failed
missing or invalid option
invalid username/password; logon denied
unable to create INITIAL extent for segment
out of process memory when trying to allocate string bytes
shared memory realm does not exist
cannot insert NULL
TNS:unable to connect to destination
remote database not found'>ora-02019
exception encountered: core dump
inconsistent datatypes
no data found
TNS:operation timed out
PL/SQL: could not find program
existing state of packages has been discarded
maximum number of processes exceeded
error signaled in parallel query server
ORACLE instance terminated. Disconnection forced
TNS:packet writer failure
see ORA-12699
missing right parenthesis
name is already used by an existing object
cannot identify/lock data file
invalid file operation
quoted string not properly terminated
hanging shutdowns (addressing the requirement for a UNIX reboot)

hanging shutdowns (addressing the requirement for a UNIX reboot)

2006-02-28       - By LiShan Cheng

Reply:     1     2     3     4     5  

I have been in a customer a couple of years ago wose backup script did
something like this

retry = 0
loop
  if retry == 3 shutdown abort and exit loop
  shutown immediate
  wait 60 seconds
  if success then exit loop else retry = retry + 1
end loop

this was used since 1998 until today, shutdown abort has been used time to
time with no errors




On 2/28/06, oracle-l-bounce@(protected) <oracle-l-bounce@(protected)>
wrote:
>
> All,
>
> I am with Jeremiah on this: A shutdown abort DOES NOT harm a database
> (at least in the five years I had used it on a set of active databases a
> few years ago). The ONLY time a Db had a problem after shutdown abort
> was in a 8i upgraded to 9i database (there was a bug a while ago which
> was related to the change of format in the redo log to support LSB which
> manifested itself when a shutdown abort was issued in between the
> upgrade before it was completed - I don't remember the specifics, but it
> manifested only during the upgrade).
>
> As to the requirement to reboot the Solaris server, was this because the
> Database did not restart and complained of 'Unable to create Shared Mem
> segment' (Or similar message)? I believe this could have been because
> you killed the background processes after a 'shutdown immediate' "hang".
> This is because once you initiate a 'shutdown immediate' and
> 'control-c'ed out of it, then you will never be able to login since any
> new attaches will complain that a shutdown is in progress, and the only
> way out is to kill the backend processes. In this case, the shared
> memory segment is never released and you get the error at database
> restart because the SHM start address is calculated to the same existing
> but currently open value, everything being equal). You can very easily
> get out of this using the example in the following real life event:
>
> In this case, I had three databases (the surviving Ist, 2nd Dbs and then
> the third whose backend had to be killed). In this case, use 'ipcs -am'
> to determine the memory segments, calculate the SGA size of the
> surviving databases and map the segment IDs using the LPIDs as shown
> below. Then use 'ipcrm -m <Key>' to kill the *right* segment (ipcrm -m
> 23175 in tis case) which will then allow you to restart the database.
> (Take it from me, I have done it many times before). In addition, the
> NATTCH column which shows 0 attaches is another giveaway!
>
> $ ipcs -am | head -2; ipcs -am | grep oracle
> IPC status from <running system> as of Thu Dec  8 13:47:57 BST 2005
> T         ID      KEY        MODE        OWNER    GROUP  CREATOR
> CGROUP NATTCH      SEGSZ  CPID  LPID   ATIME    DTIME    CTIME
> m     147840   0          --rw-r-- --   oracle      dba   oracle
> dba      0  655441920  8931 23175 13:47:22 13:47:22 11:42:07
> m          2   0xdd27ed28 --rw-r-- --   oracle      dba   oracle
> dba     16  371458048  6548 22193 13:45:01 13:45:01 14:35:12
> m     276867   0xfa9fd35c --rw-r-- --   oracle      dba   oracle
> dba      0  502874112  8931 23175 13:47:22 13:47:22 11:42:11
> m     787590   0          --rw-r-- --   oracle      dba   oracle
> dba    139  655441920 11593 23223 13:47:46 13:47:47  6:06:10
> m     716359   0xe315db0c --rw-r-- --   oracle      dba   oracle
> dba    139  502874112 11593 23223 13:47:46 13:47:47  6:06:15
>
> Ist surviving DB SQL> show sga
>
> Total System Global Area 1157681312 bytes <== LPID 23223, 139 attaches)
> Fixed Size                    73888 bytes
> Variable Size             501182464 bytes
> Database Buffers          655360000 bytes
> Redo Buffers                1064960 bytes
>
> 1158316032 = 655441920 + 502874112 (LPID 23223 - 2 segments)
>
> 2nd surviving DB SQL> show sga
>
> Total System Global Area  370548720 bytes  <== LPID 22193)
> Fixed Size                    69616 bytes
> Variable Size             328454144 bytes
> Database Buffers           40960000 bytes
> Redo Buffers                1064960 bytes
>
> John Kanagaraj <><
> DB Soft Inc
> Phone: 408-970-7002 (W)
>
> Co-Author: Oracle Database 10g Insider Solutions
> http://www.amazon.com/exec/obidos/tg/detail/-/0672327910/
>
> ** The opinions and facts contained in this message are entirely mine
> and do not reflect those of my employer or customers **
>
>
>
>
> -- --Original Message-- --
> From: oracle-l-bounce@(protected)
> [mailto:oracle-l-bounce@(protected)] On Behalf Of Roger Xu
> Sent: Monday, February 27, 2006 3:24 PM
> To: Oracle-L@(protected)
> Subject: RE: hanging shutdowns
>
> What should I do if "shutdown immediate" hangs?
> Last time, I had to reboot the Solaris Server.
>
> -- --Original Message-- --
> From: oracle-l-bounce@(protected)
> [mailto:oracle-l-bounce@(protected)]On Behalf Of Edgar Chupit
> Sent: Monday, February 27, 2006 2:12 PM
> To: Oracle-L@(protected)
> Subject: Re: hanging shutdowns
>
>
> Dear Jeremiah,
>
> First of all, I would like to mention that I don't like to shutdown
> database without any practical reason (like hardware/OS
> maintenance/upgrades/etc).
>
> And still I would like to argue that under normal circumstances startup
> force restrict + shutdown immediate (or shutdown abort, startup force,
> shutdown immediate) will run almost as fast and is as dangerous as a
> single shutdown immediate.
>
> After shutting down abort in order to perform cold backup you still need
> to startup database and close it in consistent mode. Database startup is
> not very fast process in it self, because Oracle not only needs to
> recover database into consistent state (rollback uncommitted
> transactions), but also allocate memory structures and prepare itself
> for a normal work. And to shutdown database in consistent state you
> still need to issue shutdown immediate.
>
> One of the popular reasons why shutdown immediate can take a longer time
> to proceed is because Oracle waits for SNP process to wakeup
> (Note: 1018421.102), but this can also happened when the shutdown
> immediate is called second time (after startup force), so even
> checkpointing and using startup force restrict can cause database to
> hang in shutdown immediate mode.
>
> Also, there is a Note: 46001.1 that suggest to minimize usage of
> shutdown abort on Windows systems, because it can cause "allocation
> problems when Oracle is next started.". Note: 161234.1 that describes
> situation when shutdown abort can hang. Note: 222553.1 that states that
> startup force can be safer than shutdown abort. And plenty of other
> notes that describes different problems that can occur during database
> shutdown.
>
> And surely there are many bugs that can occur after shutdown abort (but
> under normal circumstances shutdown abort is very safe).
>
> Saying all this, I would like to return to thread subject and suggest to
> the original poster to try to convince the management to switch to hot
> backups, and forget about shutting down the databases because of backup
> at all.
>
> On 2/27/06, Jeremiah Wilton <jeremiah@(protected)> wrote:
> > If you 'alter system checkpoint' before the 'shutdown abort' then it
> > should be a lot faster for the user with a hanging or prolonged
> > 'shutdown immediate'.
>
> > Jeremiah Wilton
> > ORA-600 (See ORA-600.ora-code.com) Consulting
> > Recoveries - Seminars - Hiring
> > http://www.ora-600 (See ora-600.ora-code.com).net
>
>
> --
> Best regards,
>   Edgar Chupit
>   callto://edgar.chupit
> --
> http://www.freelists.org/webpage/oracle-l
>
>
>
> For technical support please email tech_support@(protected) or you can
> call (972)721-8257.
> This email has been scanned for all viruses by the MessageLabs Email
> Security System.
>
> This e-mail is intended solely for the person or entity to which it is
> addressed and may contain confidential and/or privileged information.
> Any review, dissemination, copying, printing or other use of this e-mail
> by persons or entities other than the addressee is prohibited. If you
> have received this e-mail in error, please contact the sender
> immediately and delete the material.
> __ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ _____
> This email has been scanned for all viruses by the MessageLabs Email
> Security System. Any questions please call 972-721-8257 or email your
> request to tech_support@(protected)
> --
> http://www.freelists.org/webpage/oracle-l
>
>
> --
> http://www.freelists.org/webpage/oracle-l
>
>
>

I have been in a customer a couple of years ago wose backup script did
something like this<br><br>retry = 0<br>loop<br>&nbsp;&nbsp; if retry == 3
shutdown abort and exit loop<br>&nbsp;&nbsp; shutown immediate<br>&nbsp;&nbsp;
wait 60 seconds<br>&nbsp;&nbsp; if success then exit loop else retry = retry +
1
<br>end loop<br><br>this was used since 1998 until today, shutdown abort has
been used time to time with no errors<br><br><br><br><br><div><span class=
"gmail_quote">On 2/28/06, <b class="gmail_sendername"><a href="mailto:oracle-l
-bounce@(protected)">
oracle-l-bounce@(protected)</a></b> &lt;<a href="mailto:oracle-l-bounce
@(protected)">oracle-l-bounce@(protected)</a>&gt; wrote:</span><blockquote
class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin:
0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
All,<br><br>I am with Jeremiah on this: A shutdown abort DOES NOT harm a
database<br>(at least in the five years I had used it on a set of active
databases a<br>few years ago). The ONLY time a Db had a problem after shutdown
abort
<br>was in a 8i upgraded to 9i database (there was a bug a while ago which<br
>was related to the change of format in the redo log to support LSB which<br
>manifested itself when a shutdown abort was issued in between the<br>
upgrade before it was completed - I don't remember the specifics, but it<br
>manifested only during the upgrade).<br><br>As to the requirement to reboot the
Solaris server, was this because the<br>Database did not restart and complained
of 'Unable to create Shared Mem
<br>segment' (Or similar message)? I believe this could have been because<br
>you killed the background processes after a 'shutdown immediate' &quot;hang
&quot;.<br>This is because once you initiate a 'shutdown immediate' and
<br>'control-c'ed out of it, then you will never be able to login since any<br
>new attaches will complain that a shutdown is in progress, and the only<br>way
out is to kill the backend processes. In this case, the shared<br>
memory segment is never released and you get the error at database<br>restart
because the SHM start address is calculated to the same existing<br>but
currently open value, everything being equal). You can very easily<br>get out
of this using the example in the following real life event:
<br><br>In this case, I had three databases (the surviving Ist, 2nd Dbs and
then<br>the third whose backend had to be killed). In this case, use 'ipcs -am'
<br>to determine the memory segments, calculate the SGA size of the
<br>surviving databases and map the segment IDs using the LPIDs as shown<br
>below. Then use 'ipcrm -m &lt;Key&gt;' to kill the *right* segment (ipcrm -m<br
>23175 in tis case) which will then allow you to restart the database.
<br>(Take it from me, I have done it many times before). In addition, the<br
>NATTCH column which shows 0 attaches is another giveaway!<br><br>$ ipcs -am |
head -2; ipcs -am | grep oracle<br>IPC status from &lt;running system&gt; as of
Thu Dec&nbsp;&nbsp;8 13:47:57 BST 2005
<br>T&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ID&nbsp;&nbsp;&nbsp;&nbsp
;&nbsp;&nbsp;KEY&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MODE&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OWNER&nbsp;&nbsp;&nbsp;&nbsp;GROUP&nbsp;
&nbsp;CREATOR<br>CGROUP NATTCH&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SEGSZ&nbsp;
&nbsp;CPID&nbsp;&nbsp;LPID&nbsp;&nbsp; ATIME&nbsp;&nbsp;&nbsp;&nbsp;DTIME&nbsp;
&nbsp;&nbsp;&nbsp;CTIME<br>m&nbsp;&nbsp;&nbsp;&nbsp; 147840&nbsp;&nbsp; 0&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--rw-r-- --&nbsp;&nbsp;
oracle&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dba&nbsp;&nbsp; oracle<br>dba&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;655441920&nbsp;&nbsp;8931 23175 13:47
:22 13:47:22 11:42:07
<br>m&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2&nbsp;&nbsp;
0xdd27ed28 --rw-r-- --&nbsp;&nbsp; oracle&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
;dba&nbsp;&nbsp; oracle<br>dba&nbsp;&nbsp;&nbsp;&nbsp; 16&nbsp;&nbsp;371458048
&nbsp;&nbsp;6548 22193 13:45:01 13:45:01 14:35:12<br>m&nbsp;&nbsp;&nbsp;&nbsp;
276867&nbsp;&nbsp; 0xfa9fd35c --rw-r-- --&nbsp;&nbsp; oracle&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;dba&nbsp;&nbsp; oracle<br>dba&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;0&nbsp;&nbsp;502874112&nbsp;&nbsp;8931 23175 13:47:22 13:47:22 11:42:11
<br>m&nbsp;&nbsp;&nbsp;&nbsp; 787590&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--rw-r-- --&nbsp;&nbsp; oracle&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;dba&nbsp;&nbsp; oracle<br>dba&nbsp;&nbsp;&nbsp;&nbsp;139&nbsp;
&nbsp;655441920 11593 23223 13:47:46 13:47:47&nbsp;&nbsp;6:06:10<br>m&nbsp;&nbsp
;&nbsp;&nbsp; 716359&nbsp;&nbsp; 0xe315db0c --rw-r-- --&nbsp;&nbsp; oracle&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dba&nbsp;&nbsp; oracle<br>dba&nbsp;&nbsp;&nbsp;
&nbsp;139&nbsp;&nbsp;502874112 11593 23223 13:47:46 13:47:47&nbsp;&nbsp;6:06:15
<br><br>Ist surviving DB SQL&gt; show sga<br><br>Total System Global Area
1157681312 bytes &lt;== LPID 23223, 139 attaches)<br>Fixed Size&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;73888 bytes<br>Variable Size&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 501182464 bytes<br
>Database Buffers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
;655360000 bytes
<br>Redo Buffers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1064960 bytes<br><br>1158316032 = 655441920
+ 502874112 (LPID 23223 - 2 segments)<br><br>2nd surviving DB SQL&gt; show sga
<br><br>Total System Global Area&nbsp;&nbsp;370548720 bytes&nbsp;&nbsp;&lt;==
LPID 22193)
<br>Fixed Size&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;69616 bytes<br>Variable
Size&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
328454144 bytes<br>Database Buffers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp; 40960000 bytes<br>Redo Buffers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1064960 bytes
<br><br>John Kanagaraj &lt;&gt;&lt;<br>DB Soft Inc
<br>Phone: 408-970-7002 (W)<br><br>Co-Author: Oracle Database 10g Insider
Solutions<br><a href="http://www.amazon.com/exec/obidos/tg/detail/-/0672327910/
">http://www.amazon.com/exec/obidos/tg/detail/-/0672327910/</a><br><br>
** The opinions and facts contained in this message are entirely mine<br>and do
not reflect those of my employer or customers **<br><br><br><br><br>----
-Original Message-- --<br>From: <a href="mailto:oracle-l-bounce@(protected)">
oracle-l-bounce@(protected)</a><br>[mailto:<a href="mailto:oracle-l-bounce
@(protected)">oracle-l-bounce@(protected)</a>] On Behalf Of Roger Xu<br>Sent
: Monday, February 27, 2006 3:24 PM<br>To: Oracle-L@(protected)<br>
Subject: RE: hanging shutdowns<br><br>What should I do if &quot;shutdown
immediate&quot; hangs?<br>Last time, I had to reboot the Solaris Server.<br><br
>-- --Original Message-- --<br>From: <a href="mailto:oracle-l-bounce@(protected)
.org">
oracle-l-bounce@(protected)</a><br>[mailto:<a href="mailto:oracle-l-bounce
@(protected)">oracle-l-bounce@(protected)</a>]On Behalf Of Edgar Chupit<br
>Sent: Monday, February 27, 2006 2:12 PM<br>To: Oracle-L@(protected)<br>
Subject: Re: hanging shutdowns<br><br><br>Dear Jeremiah,<br><br>First of all, I
would like to mention that I don't like to shutdown<br>database without any
practical reason (like hardware/OS<br>maintenance/upgrades/etc).<br>
<br>And still I would like to argue that under normal circumstances startup<br
>force restrict + shutdown immediate (or shutdown abort, startup force,<br
>shutdown immediate) will run almost as fast and is as dangerous as a
<br>single shutdown immediate.<br><br>After shutting down abort in order to
perform cold backup you still need<br>to startup database and close it in
consistent mode. Database startup is<br>not very fast process in it self,
because Oracle not only needs to
<br>recover database into consistent state (rollback uncommitted<br
>transactions), but also allocate memory structures and prepare itself<br>for a
normal work. And to shutdown database in consistent state you<br>still need to
issue shutdown immediate.
<br><br>One of the popular reasons why shutdown immediate can take a longer
time<br>to proceed is because Oracle waits for SNP process to wakeup<br>(Note:
1018421.102), but this can also happened when the shutdown<br>immediate is
called second time (after startup force), so even
<br>checkpointing and using startup force restrict can cause database to<br
>hang in shutdown immediate mode.<br><br>Also, there is a Note: 46001.1 that
suggest to minimize usage of<br>shutdown abort on Windows systems, because it
can cause &quot;allocation
<br>problems when Oracle is next started.&quot;. Note: 161234.1 that describes
<br>situation when shutdown abort can hang. Note: 222553.1 that states that<br
>startup force can be safer than shutdown abort. And plenty of other
<br>notes that describes different problems that can occur during database<br
>shutdown.<br><br>And surely there are many bugs that can occur after shutdown
abort (but<br>under normal circumstances shutdown abort is very safe).
<br><br>Saying all this, I would like to return to thread subject and suggest
to<br>the original poster to try to convince the management to switch to hot<br
>backups, and forget about shutting down the databases because of backup
<br>at all.<br><br>On 2/27/06, Jeremiah Wilton &lt;<a href="mailto:jeremiah@(protected)
-600.net">jeremiah@(protected)</a>&gt; wrote:<br>&gt; If you 'alter system
checkpoint' before the 'shutdown abort' then it<br>&gt; should be a lot faster
for the user with a hanging or prolonged
<br>&gt; 'shutdown immediate'.<br><br>&gt; Jeremiah Wilton<br>&gt; ORA-600 (See ORA-600.ora-code.com)
Consulting<br>&gt; Recoveries - Seminars - Hiring<br>&gt; <a href="http://www
.ora-600 (See ora-600.ora-code.com).net">http://www.ora-600 (See ora-600.ora-code.com).net</a><br><br><br>--<br>Best regards,
<br>&nbsp;&nbsp;Edgar Chupit<br>&nbsp;&nbsp;callto://edgar.chupit<br>--<br><a
href="http://www.freelists.org/webpage/oracle-l">http://www.freelists.org
/webpage/oracle-l</a><br><br><br><br>For technical support please email <a href=
"mailto:tech_support@(protected)">
tech_support@(protected)</a> or you can<br>call (972)721-8257.<br>This email
has been scanned for all viruses by the MessageLabs Email<br>Security System.
<br><br>This e-mail is intended solely for the person or entity to which it is
<br>addressed and may contain confidential and/or privileged information.<br
>Any review, dissemination, copying, printing or other use of this e-mail<br>by
persons or entities other than the addressee is prohibited. If you
<br>have received this e-mail in error, please contact the sender<br
>immediately and delete the material.<br>__ ____ ____ ____ ____ ____ ____ ______
__ ____ ____ ____ ____ ______<br>This email has been scanned for all viruses by
the MessageLabs Email
<br>Security System. Any questions please call 972-721-8257 or email your<br
>request to <a href="mailto:tech_support@(protected)">tech_support@(protected)<
/a>.<br>--<br><a href="http://www.freelists.org/webpage/oracle-l">
http://www.freelists.org/webpage/oracle-l</a><br><br><br>--<br><a href="http:/
/www.freelists.org/webpage/oracle-l">http://www.freelists.org/webpage/oracle-l<
/a><br><br><br></blockquote></div><br>