破案记：Oracle数据库开机自动启动失败

我的一台虚机上的数据库，以前都是开机自动启动。最近忽然不自动启动了。

开机后，监听是起来的，但数据库并没有：

$ lsnrctl status

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 04-FEB-2024 20:50:11

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date                04-FEB-2024 20:28:15
Uptime                    0 days 0 hr. 21 min. 56 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/oracle/product/19c/dbhome_1/network/admin/listener.ora
Listener Log File         /opt/oracle/diag/tnslsnr/oracle-19c-vagrant/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))
The listener supports no services
The command completed successfully

$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Sun Feb 4 20:50:29 2024
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> exit
Disconnected

自动启动设置没有被改动，仍是Y：

$ cat /etc/oratab
ORCLCDB:/opt/oracle/product/19c/dbhome_1:Y

查看自动启动服务oracle-rdbms，调的就是dbstart，没有问题：

# cat /etc/systemd/system/oracle-rdbms.service
[Unit]
Description=Oracle Database(s) and Listener
Requires=network.target

[Service]
Type=forking
Restart=no
ExecStart=/opt/oracle/product/19c/dbhome_1/bin/dbstart /opt/oracle/product/19c/dbhome_1
ExecStop=/opt/oracle/product/19c/dbhome_1/bin/dbshut /opt/oracle/product/19c/dbhome_1
User=oracle

[Install]
WantedBy=multi-user.target

而且服务oracle-rdbms是启动的，这里提供了数据库启动日志的位置：

# systemctl status oracle-rdbms
● oracle-rdbms.service - Oracle Database(s) and Listener
   Loaded: loaded (/etc/systemd/system/oracle-rdbms.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2024-02-04 12:24:23 +08; 22min ago
  Process: 646 ExecStart=/opt/oracle/product/19c/dbhome_1/bin/dbstart /opt/oracle/product/19c/dbhome_1 (code=exited, status=0/SUCCESS)
 Main PID: 706 (tnslsnr)
   CGroup: /system.slice/oracle-rdbms.service
           └─706 /opt/oracle/product/19c/dbhome_1/bin/tnslsnr LISTENER -inherit

Feb 04 12:24:21 oracle-19c-vagrant systemd[1]: Starting Oracle Database(s) and Listener...
Feb 04 12:24:21 oracle-19c-vagrant dbstart[646]: Processing Database instance "ORCLCDB": log file /opt/oracle/product/19c/dbho...up.log
Feb 04 12:24:23 oracle-19c-vagrant systemd[1]: Started Oracle Database(s) and Listener.
Hint: Some lines were ellipsized, use -l to show in full.

查看数据库启动日志，这里显示了具体的错误：

# vi /opt/oracle/product/19c/dbhome_1/rdbms/log/startup.log
/opt/oracle/product/19c/dbhome_1/bin/dbstart: Starting up database "ORCLCDB"
Sun Feb  4 12:24:21 +08 2024


SQL*Plus: Release 19.0.0.0.0 - Production on Sun Feb 4 12:24:21 2024
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

SQL> Connected to an idle instance.
SQL> ORA-46693: The WALLET_ROOT location is missing or invalid.
ORA-07217: sltln: environment variable cannot be evaluated.
ORA-01078: failure in processing system parameters
SQL> Disconnected

/opt/oracle/product/19c/dbhome_1/bin/dbstart: Database instance "ORCLCDB" warm started.

想起来了，之前因为做加密实验设置了WALLET_ROOT。但这个设置我并没有改过，而且WALLET_ROOT指向的位置也是存在的，权限也没有问题：

SQL> show parameter wallet

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
ssl_wallet                           string
wallet_root                          string      /opt/oracle/wallet

查看错误信息的解释，这几个错误号需要一起看，才能找到真正原因：

$ oerr ora 46693
46693, 00000, "The WALLET_ROOT location is missing or invalid."
// *Cause:  An attempt was made to perform a keystore operation when the
//          instance initialization parameter WALLET_ROOT was missing
//          or invalid.
// *Action: Start the database after setting the WALLET_ROOT instance
//          initialization parameter to the directory where all of the
//          wallets are stored.

$ oerr ora 07217
07217, 00000, "sltln: environment variable cannot be evaluated."
// *Cause:  getenv call returned a null pointer.
// *Action: Set the environment variable and try again.

$ oerr ora 01078
01078, 00000, "failure in processing system parameters"
// *Cause:  Failure during processing of INIT.ORA parameters during
//          system startup.
// *Action:  Further diagnostic information should be in the error stack.

ORA-07217揭示了错误的根因，看下面参数的设置：

SQL> show spparameter wallet

SID      NAME                          TYPE        VALUE
-------- ----------------------------- ----------- ----------------------------
*        ssl_wallet                    string
*        wallet_root                   string      $ORACLE_BASE/wallet

原来，我设置时使用了环境变量$ORACLE_BASE，虽然这是允许的。但在启动时，此环境变量还未设置，我只是将其设置在了用户oracle的.bash_profile文件中。

解决方法是不用环境变量，而是使用绝对路径。设置后重启数据库：

alter system set wallet_root='/opt/oracle/wallet' scope=spfile;
shutdown immediate
startup

重启数据库服务器，这一回开机自动启动成功了！