配置基于 CentOS5 的 iSCSI SAN

配置基于 CentOS5 的 iSCSI SAN

作者: 梁如军 (Osmond Liang)
_{Current Version: 1.0 (2008/08/28)}

内容提要

学会配置基于 CentOS5 的 iSCSI SAN 服务端
学会配置基于 CentOS5 的 iSCSI SAN 客户端

安装配置服务器端

Step #1：安装 CentOS 5

通常用作 SAN 的 Linux 主机是专用的，所以 Linux 系统本身可以直接以分区方式安装，安装时 / 分区分 10G~20G 即可
另外创建单独的分区用于网络存储，并使用 LVM

下面是一个服务器的分区方案：

/dev/sda1 – /boot
/dev/sda2 – SWAP
/dev/sda3 – /
/dev/sda5 – SAN DATA 使用的 LVM 分区

# parted -l

Model: ATA HDT722525DLA380 (scsi)
Disk /dev/sda: 250GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type      File system  标志
 1      32.3kB  107MB   107MB   主分区    ext3         启动
 2      107MB   2254MB  2147MB  主分区    linux-swap
 3      2254MB  40.0GB  37.7GB  主分区    ext3
 4      40.0GB  250GB   210GB   扩展分区
 5      40.0GB  250GB   210GB   逻辑分区               lvm

Step #2：准备存储设备

给 iSCSI-target 使用的存储设备可以是逻辑卷、磁盘或分区、甚至可以是回环设备（磁盘映像文件），下面配置最常用的逻辑卷：

[root@mySAN01 ~]# pvcreate /dev/sda5
  Physical volume "/dev/sda5" successfully created
[root@mySAN01 ~]# vgcreate sanVG /dev/sda5
  Volume group "sanVG" successfully created
[root@mySAN01 ~]# lvcreate -L 10G -n san4xen01 sanVG
  Logical volume "san4xen01" created
[root@mySAN01 ~]# lvcreate -L 10G -n san4xen02 sanVG
  Logical volume "san4xen02" created
[root@mySAN01 ~]# lvcreate -L 10G -n san4xen03 sanVG
  Logical volume "san4xen03" created
[root@mySAN01 ~]# lvcreate -L 30G -n sanDisc01 sanVG
  Logical volume "sanDisc01" created
[root@mySAN01 ~]# ll /dev/sanVG/
总计 0
lrwxrwxrwx 1 root root 27 07-21 22:16 san4xen01 -> /dev/mapper/sanVG-san4xen01
lrwxrwxrwx 1 root root 27 07-21 22:16 san4xen02 -> /dev/mapper/sanVG-san4xen02
lrwxrwxrwx 1 root root 27 07-21 22:16 san4xen03 -> /dev/mapper/sanVG-san4xen03
lrwxrwxrwx 1 root root 27 07-21 22:16 sanDisc01 -> /dev/mapper/sanVG-sanDisc01

通过上面的配置得到了4个裸磁盘设备。

Step #3：安装 SCSI-TARGET

1）设置使用 ATrpms 第三方仓库

[root@mySAN01 ~]# rpm --import http://ATrpms.net/RPM-GPG-KEY.atrpms
[root@mySAN01 ~]# cd /etc/yum.repos.d/
[root@mySAN01 ~]# vi ATrpms.repo

[atrpms]
name=CentOS-$releasever - $basearch - ATrpms
baseurl=http://dl.atrpms.net/el$releasever-$basearch/atrpms/stable
gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms
gpgcheck=1
enabled=0

/etc/yum.repos.d/ATrpms.repo

2）安装 iSCSI Enterprise Target (IET)

[root@mySAN01 ~]# yum --enablerepo=atrpms install iscsitarget iscsitarget-kmdl-`uname -r`

[root@mySAN01 ~]# rpm -qa|grep iscsi
iscsitarget-0.4.15-4.el5
iscsitarget-kmdl-2.6.18-92.1.6.el5-0.4.15-4.el5

[root@mySAN01 ~]# rpm -ql iscsitarget-kmdl-2.6.18-92.1.6.el5-0.4.15-4.el5
/lib/modules/2.6.18-92.1.6.el5/updates/iscsi/iscsi_trgt.ko

[root@mySAN01 ~]# cp -r /lib/modules/2.6.18-92.el5/updates/iscsi/ /lib/modules/`uname -r`/kernel/drivers

[root@mySAN01 ~]# less /usr/share/doc/iscsitarget-0.4.15/README

[root@mySAN01 ~]# depmod iscsi_trgt
# 若此命令不能正确执行，请重新启动系统

Step #4：配置 iSCSI Enterprise Target

1）修改 IET 的配置文件

[root@mySAN01 ~]# mv /etc/ietd.conf /etc/ietd.conf.orig
[root@mySAN01 ~]# vi /etc/ietd.conf

# Targets definitions start with "Target" and the target name.
# The target name must be a globally unique name, the iSCSI
# standard defines the "iSCSI Qualified Name" as follows:
#
# iqn.yyyy-mm.<reversed domain name>[:identifier]
#
# "yyyy-mm" is the date at which the domain is valid and the identifier
# is freely selectable. For further details please check the iSCSI spec.

Target iqn.2008-07.net.mytest:storage.iscsi.for.xen01
        Lun 0 Path=/dev/sanVG/san4xen01,Type=fileio
        Alias san4xen01

Target iqn.2008-07.net.mytest:storage.iscsi.for.xen02
        Lun 1 Path=/dev/sanVG/san4xen02,Type=fileio
        Alias san4xen02

Target iqn.2008-07.net.mytest:storage.iscsi.for.xen03
        Lun 2 Path=/dev/sanVG/san4xen03,Type=fileio
        Alias san4xen03

Target iqn.2008-07.net.mytest:storage.iscsi.san.Disc01
        Lun 3 Path=/dev/sanVG/sanDisc01,Type=fileio
        Alias sanDisc01

/etc/ietd.conf

2）启动 IET 服务

[root@mySAN01 ~]# chkconfig iscsi-target on
[root@mySAN01 ~]# service iscsi-target restart
Stopping iSCSI target service:                             [确定]
Starting iSCSI target service:                             [确定]

3）检查启动后的 IET

[root@mySAN01 ~]# cat /proc/net/iet/volume
tid:4 name:iqn.2008-07.net.mytest:storage.iscsi.san.Disc01
        lun:3 state:0 iotype:fileio iomode:wt path:/dev/sanVG/sanDisc01
tid:3 name:iqn.2008-07.net.mytest:storage.iscsi.for.xen03
        lun:2 state:0 iotype:fileio iomode:wt path:/dev/sanVG/san4xen03
tid:2 name:iqn.2008-07.net.mytest:storage.iscsi.for.xen02
        lun:1 state:0 iotype:fileio iomode:wt path:/dev/sanVG/san4xen02
tid:1 name:iqn.2008-07.net.mytest:storage.iscsi.for.xen01
        lun:0 state:0 iotype:fileio iomode:wt path:/dev/sanVG/san4xen01

安装配置客户端

Step #5：安装 iscsi-initiator

[root@linux01 ~]# yum install iscsi-initiator-utils
[root@linux01 ~]# chkconfig --level 345 iscsid on
[root@linux01 ~]# service iscsid start
[root@linux01 ~]# less /usr/share/doc/iscsi-initiator-utils-6.2.0.868/README

Step #6：配置 iscsi-initiator

Open-iSCSI 的持久配置使用 DBM 数据库实现。数据库包含两个表：

发现表：Discovery table (/var/lib/iscsi/send_targets)
节点表：Node table (/var/lib/iscsi/nodes)

使用 iscsiadm 命令工具可以管理（更新、插入、删除、查询）持久配置数据库 iscsid 守护进程可以在重启计算机时自动读取持久配置数据库，自动获取 iSCSI Target 存储设备

1）Discovery iSCSI targets

[root@linux01 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.0.198
192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen01
192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen02
192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen03
192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.san.Disc01

[root@linux01 ~]# iscsiadm -m node -l
Logging in to [iface: default, target: iqn.2008-07.net.mytest:storage.iscsi.for.xen01, portal: 192.168.0.198,3260]
Logging in to [iface: default, target: iqn.2008-07.net.mytest:storage.iscsi.for.xen02, portal: 192.168.0.198,3260]
Logging in to [iface: default, target: iqn.2008-07.net.mytest:storage.iscsi.san.Disc01, portal: 192.168.0.198,3260]
Logging in to [iface: default, target: iqn.2008-07.net.mytest:storage.iscsi.for.xen03, portal: 192.168.0.198,3260]
Login to [iface: default, target: iqn.2008-07.net.mytest:storage.iscsi.for.xen01, portal: 192.168.0.198,3260]: successful
Login to [iface: default, target: iqn.2008-07.net.mytest:storage.iscsi.for.xen02, portal: 192.168.0.198,3260]: successful
Login to [iface: default, target: iqn.2008-07.net.mytest:storage.iscsi.san.Disc01, portal: 192.168.0.198,3260]: successful
Login to [iface: default, target: iqn.2008-07.net.mytest:storage.iscsi.for.xen03, portal: 192.168.0.198,3260]: successful

[root@linux01 ~]# service iscsid stop
[root@linux01 ~]# service iscsid start

2）使用 iscsiadm 查看相关信息

[root@linux01 ~]# iscsiadm -m discovery
192.168.0.198:3260 via sendtargets

[root@linux01 ~]# iscsiadm -m discovery -P 1
SENDTARGETS:
DiscoveryAddress: 192.168.0.198,3260
    Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen01
        Portal: 192.168.0.198:3260,1
           Iface Name: default
    Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen02
        Portal: 192.168.0.198:3260,1
           Iface Name: default
    Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen03
        Portal: 192.168.0.198:3260,1
           Iface Name: default
    Target: iqn.2008-07.net.mytest:storage.iscsi.san.Disc01
        Portal: 192.168.0.198:3260,1
           Iface Name: default
iSNS:
No targets found.
STATIC:
No targets found.

[root@linux01 ~]# iscsiadm -m node
192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen01
192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen02
192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.san.Disc01
192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen03

[root@linux01 ~]# iscsiadm -m node -P 1
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen01
        Portal: 192.168.0.198:3260,1
                Iface Name: default
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen02
        Portal: 192.168.0.198:3260,1
                Iface Name: default
Target: iqn.2008-07.net.mytest:storage.iscsi.san.Disc01
        Portal: 192.168.0.198:3260,1
                Iface Name: default
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen03
        Portal: 192.168.0.198:3260,1
                Iface Name: default

[root@linux01 ~]# iscsiadm -m session
tcp: [1] 192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen01
tcp: [2] 192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen02
tcp: [3] 192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.san.Disc01
tcp: [4] 192.168.0.198:3260,1 iqn.2008-07.net.mytest:storage.iscsi.for.xen03

[root@linux01 ~]# iscsiadm -m session -P 1
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen01
        Current Portal: 192.168.0.198:3260,1
        Persistent Portal: 192.168.0.198:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.redhat:e78ec48be749
                Iface IPaddress: 192.168.0.55
                Iface HWaddress: default
                Iface Netdev: default
                SID: 1
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: Unknown
                Internal iscsid Session State: NO CHANGE
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen02
        Current Portal: 192.168.0.198:3260,1
        Persistent Portal: 192.168.0.198:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.redhat:e78ec48be749
                Iface IPaddress: 192.168.0.55
                Iface HWaddress: default
                Iface Netdev: default
                SID: 2
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: Unknown
                Internal iscsid Session State: NO CHANGE
Target: iqn.2008-07.net.mytest:storage.iscsi.san.Disc01
        Current Portal: 192.168.0.198:3260,1
        Persistent Portal: 192.168.0.198:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.redhat:e78ec48be749
                Iface IPaddress: 192.168.0.55
                Iface HWaddress: default
                Iface Netdev: default
                SID: 3
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: Unknown
                Internal iscsid Session State: NO CHANGE
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen03
        Current Portal: 192.168.0.198:3260,1
        Persistent Portal: 192.168.0.198:3260,1
                **********
                Interface:
                **********
                Iface Name: default
                Iface Transport: tcp
                Iface Initiatorname: iqn.1994-05.com.redhat:e78ec48be749
                Iface IPaddress: 192.168.0.55
                Iface HWaddress: default
                Iface Netdev: default
                SID: 4
                iSCSI Connection State: LOGGED IN
                iSCSI Session State: Unknown
                Internal iscsid Session State: NO CHANGE

3）查看相关信息

# 可以使用 dmesg 命令查看相关信息
[root@linux01 ~]# dmesg
# 也可以查看 messages 日志文件中的信息
[root@linux01 ~]# cat /var/log/messages
# 查看内核中关于 scsi 的信息
[root@linux01 ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: IET      Model: VIRTUAL-DISK     Rev: 0
  Type:   Direct-Access                    ANSI SCSI revision: 04
Host: scsi1 Channel: 00 Id: 00 Lun: 01
  Vendor: IET      Model: VIRTUAL-DISK     Rev: 0
  Type:   Direct-Access                    ANSI SCSI revision: 04
Host: scsi2 Channel: 00 Id: 00 Lun: 03
  Vendor: IET      Model: VIRTUAL-DISK     Rev: 0
  Type:   Direct-Access                    ANSI SCSI revision: 04
Host: scsi3 Channel: 00 Id: 00 Lun: 02
  Vendor: IET      Model: VIRTUAL-DISK     Rev: 0
  Type:   Direct-Access                    ANSI SCSI revision: 04

# 查看磁盘设备
[root@linux01 ~]# fdisk -l

Disk /dev/hda: 81.9 GB, 81964302336 bytes
255 heads, 63 sectors/track, 9964 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/hda1   *           1        2167    17406396    7  HPFS/NTFS
/dev/hda2            2168        9964    62629402+   f  W95 Ext'd (LBA)
/dev/hda5            2168        3824    13309821    7  HPFS/NTFS
/dev/hda6            3825        3837      104391   83  Linux
/dev/hda7            3838        9964    49215096   8e  Linux LVM

Disk /dev/sda: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sda doesn't contain a valid partition table

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 32.2 GB, 32212254720 bytes
64 heads, 32 sectors/track, 30720 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdd doesn't contain a valid partition table

# 查看 Targets 与磁盘设备名的映射关系
[root@linux01 ~]# iscsiadm -m session -P 3|egrep '^Target|Attached scsi disk'
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen01
                        Attached scsi disk sda          State: running
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen02
                        Attached scsi disk sdb          State: running
Target: iqn.2008-07.net.mytest:storage.iscsi.san.Disc01
                        Attached scsi disk sdc          State: running
Target: iqn.2008-07.net.mytest:storage.iscsi.for.xen03
                        Attached scsi disk sdd          State: running

使用 iSCSI 设备

Step #7：创建分区和文件系统

iscsi target 提供的是裸设备，为了在客户端使用文件系统级别的设备还需要做分区和创建文件系统的操作。对于要使用裸设备的情况可以省略此步骤。

1）磁盘分区

[root@linux01 ~]# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 30720.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-30720, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-30720, default 30720):
Using default value 30720

Command (m for help): p

Disk /dev/sdc: 32.2 GB, 32212254720 bytes
64 heads, 32 sectors/track, 30720 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       30720    31457264   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@linux01 ~]# partprobe

2）创建文件系统

[root@linux01 ~]# mkfs -j /dev/sdc1
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
3932160 inodes, 7864316 blocks
393215 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
240 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information:

done

This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

3）手动挂装文件系统

# 创建挂装点目录（也可以使用其他挂装点）
[root@linux01 ~]# mkdir -p /var/lib/xen/images/
# 挂装
[root@linux01 ~]# mount -t ext3 /dev/sdc1 /var/lib/xen/images/

Step #8：启动时自动挂装 iSCSI 设备

1）确保 iscsid 在启动时运行

[root@linux01 ~]# chkconfig --level 345 iscsid on
[root@linux01 ~]# chkconfig --level 345 netfs on

2）修改系统挂装表

[root@linux01 ~]# vi /etc/fstab

# 添加如下内容
/dev/sdc1     /var/lib/xen/images     ext3    _netdev  0 0

/etc/fstab

centos, SAN, iSCSI

Bookmark at

guide/iscsi_san_on_centos5.txt · 最后更改: 2008/10/18 09:21 由 osmond