Thursday, May 31, 2012

NetBackup setup over InfiniBand on Exadata:

NetBackup setup over InfiniBand on Exadata:

In order to backup all the data within the Exadata compute nodes, we need to use a dedicated media server which is directly connected to the frame through an infiniband connection.

The storage admins must specify the infiniband IP as the target. However, the master server is not on the infiniband network. We must "trick" the master server to use the proper IP. In the local /etc/hosts file, associate the primary IP on a routeable network to the infiniband hostname (hostname-priv). The master server will try to backup the hostname-priv through the routable network. When the media servers recieves the request, it will lookup the IP associated with the hostname-priv, which will (on the media server) be the infiniband IP.

On the target exadata DB host, you must specify the media server name associated to the infiniband IP in the /etc/hosts file.


Example setup:

-- NetBackup Master Server /etc/hosts
# Aliases needed to trick exadata to backup over infiniband.
10.62.121.100    atl01db01-priv.corp.dwire.com       atl01db01-priv

-- Media Server /etc/hosts
### Compute Node Private Interface details farm 1
192.168.10.1    atl01db01-priv.corp.dwire.com       atl01db01-priv

-- DB server /etc/hosts
# EXADATA NBU Media Server Infiniband
192.168.16.89    atli1.corp.dwire.com       atli1

-- The IPs of the example target server (atl01db01) are:
192.168.10.1      atl01db01-priv.corp.dwire.com    atl01db01-priv
10.62.125.100    atl01db01.corp.dwire.com           atl01db01
10.62.121.100    atl0101.corp.dwire.com               atl0101


-- bp.conf file on DB server
atlcom01db01:
$ cat /opt/openv/netbackup/bp.conf
SERVER = atli1.corp.dwire.com
CLIENT_NAME = atlcom01db01-priv

2 comments:

Anonymous said...

Hi Vijay very nice write up,

reading a paper from symantec regarding the NBU implementation on exadata they suggested to create a service for RMAN backups, but it's not clear to me if this
service is going to be created on the client network or to the IB private network.

Could you please share your config implementation?

Thanks & Regards
TK

Vijay R. Dumpa said...

We don’t use any connect string for day to day backup/restores, so no service.
Incase of ACTIVE DUPLICATE we need to create IB listener use IB ip’s in the TNS entry on source and target.


###ADD ENTRY IN LISTENER.ORA ON EXADATA SERVER
(SID_DESC =
(GLOBAL_DBNAME=PODS.WORLD)
(SID_NAME=ODS)
(ORACLE_HOME=/u01/app/oracle/product/11.2.0.2/dbhome_1)
)


###ADD ENTRY IN TNSNAMES.ORA ON SOURCE AND TARGER SERVERS
ODS_IB.WORLD =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.1)(PORT = 1522)
)
(CONNECT_DATA =
(SERVER = DEDICATED)
(UR=A)
(SERVICE_NAME = ODS.WORLD)
)
)


### Check the listener status before starting the RMAN DUPLICATE
lsnrctl status listener_ib