iSCSI & network-attached storage: block storage over the network, explained simply

Before any protocol or command, you need a mental model of what "network storage" actually means and how it differs from the file sharing you already use.

Imagine a small office where every desk has its own filing cabinet. Each person's documents live in their own drawer, and if someone else needs a file, they walk over and borrow a photocopy. That is local storage: each machine owns its disks, and "sharing" means copying files across a network into another machine's local space. It works, but it creates duplication, version confusion, and a single point of failure for every person's data.

Now imagine the office has one large, well-organised archive room with a clerk. Nobody keeps their own cabinet anymore; instead, everyone sends requests to the clerk ("give me file 4711, blocks 0 through 256") and the clerk retrieves exactly what was asked for from the shelves. The clerk does not care who you are or what your office layout looks like; it serves raw data in response to precise instructions. That is network storage: a dedicated machine (or appliance) holds the physical disks, and client machines talk to it over the network using a protocol that says "read these blocks" or "write these blocks." The clients never see the shelf layout; they see only the data they requested.

The analogy breaks down in important ways. A real storage system does not have a human clerk making decisions about what to serve; it has hardware controllers, firmware, and protocol stacks that execute commands at microsecond latency. The "shelves" are spinning disks or flash chips organised by a controller into logical units. And critically, the network between client and storage is not a hallway you can walk; it is a shared medium with bandwidth limits, packet loss, congestion, and failure modes that have no equivalent in the office metaphor. Keeping the analogy for intuition while remembering these differences is the right approach when reading the rest of this guide.

In practical terms, network storage matters when you need: capacity beyond what a single machine can hold; a centralised location for data that multiple machines must access consistently; separation of compute from storage so you can upgrade one without touching the other; or resilience, where the storage system survives individual disk failures through redundancy. A home user with one laptop rarely needs it. A small business running a file server, a virtualisation host, or a database cluster almost always benefits.

The single most important distinction in this entire guide: the difference between handing someone a raw block of data and handing them a named file from a directory tree.

A block device is a sequence of fixed-size chunks (typically 512 bytes or 4 KiB) addressed by number. The operating system's storage stack talks to block devices in terms of "read the 64 sectors starting at LBA 1048576" or "write this 4 KiB buffer to LBA 2097152." There are no file names, no directories, no permissions on the individual blocks. The filesystem layer (ext4, NTFS, XFS, APFS) sits on top of the block device and translates "open /home/user/report.txt" into a sequence of block reads and writes. When you see /dev/sdb in Linux or Disk 2 in Windows Disk Management, you are looking at a block device: a raw sequence of sectors with no filesystem structure visible until the OS layers one on top.

A file-level protocol, by contrast, speaks in terms of files and directories. When your laptop mounts an SMB share (Windows file sharing) or an NFS export (Linux/Unix file sharing), the remote machine's filesystem is visible to you as a directory tree. You type a path like \\server\share\documents or /mnt/nfs/projects, and the protocol handles the translation between your local file operations (open, read, write, close, rename) and the server's filesystem internals. The client never sees raw block addresses; it sees names, permissions, and directory entries.

iSCSI is a block transport: it carries SCSI commands (the same command set your locally attached SATA or NVMe disk understands) over TCP/IP. The client's OS sees the remote target as if it were a locally plugged-in disk. It can partition it, format it with any filesystem the OS supports, and read/write blocks at full device speed. NFS and SMB are file transports: they carry file-operation requests (open, close, read-by-offset, write-by-offset, readdir) over TCP/IP. The client sees a directory tree, not a disk.

A common source of confusion: people say "iSCSI is faster than NFS" or "NFS is more secure than iSCSI." Both statements are incomplete. iSCSI has lower per-operation overhead because it does not carry path names, permission checks, or directory lookups in every request; it just says "write 4 KiB to LBA X." But that advantage only matters for workloads with many small random I/Os (databases, virtual machine disks). For large sequential transfers (video editing, backups), the difference narrows because both are ultimately limited by network bandwidth. And "faster" is meaningless without specifying the workload, block size, queue depth, and network conditions.

One more layer of distinction that trips people up: a storage transport and a filesystem are different layers. iSCSI is the transport (how SCSI commands travel over IP). ext4, NTFS, or XFS is the filesystem (how data is organised on top of the block device). You can put any supported filesystem on an iSCSI disk. You can also run NFS over a network without iSCSI at all. The two are independent choices that happen to both use TCP/IP as their underlying network.

The name stands for Internet Small Computer Systems Interface. Each word in that expansion maps to a real architectural decision, and understanding those decisions tells you what the protocol can and cannot do.

SCSI (Small Computer Systems Interface) is a set of commands and data structures originally designed for connecting storage devices (disks, tapes, optical drives) to a host computer over a parallel or serial bus. When your OS issues a "READ(10)" command to a locally attached disk, it is speaking SCSI: a 10-byte command descriptor block specifying the logical block address and transfer length, followed by the data payload. SCSI was designed for low-latency, high-throughput device I/O, not for general-purpose file access.

Internet in "iSCSI" does not mean "available on the public internet." It means the protocol is encapsulated in standard IP packets (specifically TCP) so it can traverse any IP network: a local Ethernet switch, a routed corporate LAN, or (with appropriate security) a WAN link. The iSCSI specification (RFC 7143, updated from RFC 3720) defines how SCSI command blocks are wrapped into iSCSI Protocol Data Units (PDUs), which are then segmented into TCP segments and IP packets. The result is that a SCSI "READ(10)" issued by a Linux application on one machine arrives at a storage target across the network, is unwrapped, and executed against the physical disk with the same semantics as if the disk were plugged in locally.

The key roles in an iSCSI deployment are:

• Initiator — the client machine that issues SCSI commands. It runs an iSCSI initiator stack (software or hardware) that translates local block I/O into iSCSI PDUs and sends them to a target. In Linux this is typically the open-iscsi package; in Windows it is the built-in "Microsoft iSCSI Initiator" service.
• Target — the storage server that receives SCSI commands and executes them against its local disks (or a backing store such as a RAID array, LVM volume, or file). It runs a target daemon (LIO/targetcli on Linux, MPIO/iSCSI Target on Windows).
• Portal — the IP address and TCP port where the target listens for initiator connections. The standard port is 3260. A single target can expose multiple portals (e.g., two NICs for redundancy).
• Logical Unit (LUN) — a named slice of storage that the target presents to an initiator. From the initiator's perspective, a LUN looks exactly like a locally attached disk: it has a size, it can be partitioned, and it accepts read/write commands at arbitrary LBAs.
• Session — a TCP connection (or set of connections) between one initiator and one target portal, carrying the SCSI command stream. A session is established during login and torn down during logout.

What iSCSI does not automatically provide is worth stating explicitly, because the absence is a common source of production incidents:

• No redundancy by default. A single iSCSI session over a single NIC and a single switch is a single point of failure. Multipathing (Chapter 13) adds independent paths, but you must configure it deliberately.
• No backups. The target stores blocks; it does not snapshot them, replicate them to another site, or retain old versions unless you build that layer separately.
• No encryption in the base protocol. iSCSI over TCP/IP is unencrypted by default. CHAP (Chapter 12) authenticates the parties but does not encrypt the data in transit. IPsec or a dedicated encrypted network is needed for payload confidentiality.
• No safe multi-initiator filesystem access. If two initiators mount the same LUN read-write with a standard filesystem (ext4, NTFS), they will corrupt each other's metadata. Cluster filesystems (GFS2, OCFS2, Veritas) or application-level locking are required for shared access.

The mental model to carry forward: an iSCSI LUN is a remote disk that your OS treats as local. Everything the OS does with a local disk (partitioning, formatting, mounting, journaling) works identically on an iSCSI LUN. The differences are all in the network layer (latency, reliability, security) and the management layer (how you create, authorise, monitor, and recover the LUN).

Tracing a single read operation from an application all the way to the physical disk and back reveals every layer that can introduce latency or failure.

When a Linux application calls read(fd, buf, 4096) on a file that lives on an iSCSI LUN, the following sequence occurs (simplified):

1. VFS layer. The kernel's Virtual File System translates the file offset into a logical block address (LBA) using the filesystem's block map. For ext4, this means consulting the extent tree to find which physical blocks on the underlying device hold the data for that file offset.
2. Block layer (blk-mq). The request is queued as a bio (block I/O) structure and submitted to the block multi-queue scheduler. If multipath is active, the device-mapper layer selects an active path before the bio reaches the iSCSI driver.
3. iSCSI initiator driver (iscsi_tcp). The driver wraps the SCSI command (CDB) and data buffer into iSCSI PDUs: a Command PDU containing the CDB, followed by Data-Out PDUs if this is a write. Each PDU is segmented into TCP segments of at most the negotiated Maximum Receive Unit (MRU), typically 262144 bytes (256 KiB) for data and smaller for headers.
4. TCP/IP stack. The TCP segments are encapsulated in IP packets, routed through the network (possibly across switches, VLANs, or routers), and delivered to the target's NIC. TCP provides reliable, ordered delivery: if a packet is lost, it is retransmitted; if it arrives out of order, it is reassembled.
5. iSCSI target daemon (LIO). The target's iSCSI layer reassembles the PDUs, extracts the SCSI CDB, and submits it to the local block layer as if it were a locally issued command. The backing store (a physical disk, an LVM volume, or a file) receives the I/O.
6. Physical storage. The disk controller reads the requested sectors from the platter (or NAND cells for SSD) and returns the data to the target's block layer.
7. Response path. The data travels back: target block layer → LIO → iSCSI PDU (Data-In) → TCP segments → IP packets → network → initiator NIC → iscsi_tcp driver → blk-mq → VFS → application buffer. A Response PDU (with status GOOD) follows the data, completing the SCSI command.

The total latency of this round trip on a well-configured 10 GbE LAN is typically 0.2–2 ms for a single 4 KiB read, dominated by the physical disk seek (for HDDs) or NAND program/erase cycles (for SSDs) plus ~50–200 µs of network transit and protocol processing. On a congested or misconfigured network, retransmissions can add 100 ms or more per lost packet, which is why storage networks are often isolated from general user traffic (Chapter 6).

The diagram above shows the happy path. In practice, several things can interrupt it: a switch drops a frame (TCP retransmits after ~200 ms RTO), the target's disk queue is saturated (the SCSI command waits in the block layer before execution), or the initiator's session times out because no response arrived within the negotiated node.session.timeo.replacement_timeout. Each of these failure modes has a distinct signature in the logs and monitoring tools, which Chapters 17 and 18 cover in detail.

"NAS" is not a protocol; it is an architecture. Confusing the two leads to bad procurement decisions and misconfigured networks.

NAS (Network-Attached Storage) refers to a storage system that exposes file-level access over a network. A NAS appliance might run NFS, SMB, or both simultaneously. It has its own filesystem on its own disks, and clients mount shares rather than seeing raw disks. The term "NAS" describes the service model (files over the network) and the appliance form factor (a dedicated box with disks in it), not a specific wire protocol. When someone says "we need a NAS," they mean "we need a box that shares files to our workstations," and the protocol choice (NFS vs SMB) is a separate decision.

SMB (Server Message Block, formerly "Common Internet File System") is Microsoft's file-sharing protocol. It carries file operations (open, read, write, lock, directory enumeration) over TCP port 445. Windows clients and servers use it natively; Linux can mount SMB shares via smbclient or the cifs kernel module. SMB is well-suited for office file sharing, print spooling, and Windows-specific features like named pipes and alternate data streams. It is not a block protocol: you cannot use SMB to present a raw disk to a virtualisation host.

NFS (Network File System) is the Unix/Linux equivalent of SMB. Versions 3 and 4 carry file operations over TCP port 2049 (NFSv3/v4) or 20048/2049 (NFSv4.1+). NFS is the default choice for Linux clusters, HPC workloads, and any environment where POSIX semantics (file permissions, hard links, atomic renames) matter. Like SMB, it is a file protocol: the client sees a directory tree, not a disk.

Fibre Channel (FC) is a separate physical and link-layer technology (not just a protocol) that carries SCSI commands over dedicated optical or copper cabling at 4/8/16/32/64 Gb/s. FC SANs (Storage Area Networks) use dedicated switches and host bus adapters (HBAs). Compared to iSCSI, FC has lower latency (~10–50 µs vs ~200–500 µs on a 10 GbE LAN), no TCP overhead, and hardware offload via HBA. However, it is significantly more expensive: dedicated FC switches, HBAs, and cabling cost several times what equivalent Ethernet gear costs. For most small-to-medium deployments, iSCSI over 10/25/100 GbE delivers sufficient performance at a fraction of the cost.

The decision is workload-driven, not ideology-driven. A SQL Server database needs block-level access with low latency and the ability to manage its own transactions: iSCSI or FC. A team of designers sharing PSD files benefits from a file-level protocol with permissions and versioning: SMB or NFS. A virtualisation cluster running 50 VMs needs many block devices with multipathing: iSCSI (or FC in a data centre budget). There is no universal "best" option, and any vendor telling you otherwise is selling something.

The network between initiator and target is the most common source of iSCSI problems that are not actually iSCSI problems. Getting the physical and logical network right before touching any storage software saves hours of debugging.

Separation. The single highest-impact architectural decision is whether storage traffic shares a network with general user traffic (web browsing, video calls, file copies) or runs on a dedicated path. On a small LAN with 1 GbE links and light usage, sharing is fine: a 4 KiB iSCSI read takes ~0.5 ms even with moderate background traffic. On a busier network, or when you have latency-sensitive workloads (databases, virtual machines), dedicating a VLAN or a separate physical switch for storage eliminates the class of problems where a video conference saturates the uplink and your database I/O stalls. You do not need separate cabling for this; a dedicated VLAN on the same switch is sufficient in most cases. What matters is that storage frames are not queued behind best-effort traffic.

Addressing and subnets. iSCSI targets and initiators communicate over TCP, so they need routable IP addresses. The target's portal address (e.g., 10.0.50.10:3260) must be reachable from the initiator. In a flat LAN this is trivial. Across routed subnets, ensure that intermediate routers do not drop or deprioritise TCP port 3260 traffic, and that any firewall between the two subnets has an explicit rule allowing it. A common mistake: the storage VLAN is on a different subnet than the server VLAN, and the admin configures the target but forgets the inter-VLAN route, so discovery times out with no error message beyond "no targets found."

Link speed and MTU. iSCSI performance scales with link speed up to the point where the storage itself becomes the bottleneck. A 1 GbE link delivers ~110 MB/s sustained; a 10 GbE link ~1.1 GB/s. If your disks (or SSDs) can only do 250 MB/s, a 1 GbE link is sufficient and upgrading to 10 GbE buys you nothing for that workload. However, if you are running many LUNs concurrently or doing large sequential transfers (backups, VM replication), the network becomes the limit before the disks do.

MTU and jumbo frames. The standard Ethernet MTU is 1500 bytes. "Jumbo frames" raise this to 9000 bytes, reducing per-packet overhead (fewer headers, fewer interrupts). For iSCSI, the benefit is real but modest: perhaps 5–15% throughput improvement on large sequential transfers, and a slight latency reduction because each SCSI data transfer requires fewer TCP segments. The critical rule: every device in the path must agree on the MTU. If the initiator NIC is set to 9000 but an intermediate switch port is still at 1500, frames are silently dropped and TCP retransmits kick in, making performance worse than if you had used standard MTU everywhere. Verify with ip link show on Linux or Get-NetAdapter on Windows at every hop. If you cannot guarantee end-to-end MTU consistency, use 1500.

Redundant paths. For production storage, a single NIC, single cable, and single switch is a single point of failure. The minimum redundancy for a small deployment: two NICs on the initiator, two NICs on the target, connected to two separate switches (or two uplinks on a redundant pair). This gives you independent paths for MPIO (Chapter 13). The physical cabling should take different routes where possible (different cable trays, different switch slots) so that a single cut cable or failed switch port does not eliminate all paths simultaneously.

This chapter sets up the storage server side on a supported Linux distribution. The examples use Debian 12 (Bookworm) and RHEL 9 / AlmaLinux 9 as reference environments; the concepts are identical across distributions, but package names and service management differ.

The modern Linux iSCSI target stack is LIO (Linux-IO), managed through the targetcli command-line tool. LIO is the kernel's in-kernel target framework; it exposes SCSI targets to initiators over iSCSI (and, with additional configuration, Fibre Channel or iSER). The user-space daemon targetclid provides a RESTful interface and the targetcli shell. On Debian/Ubuntu the package is linux-iscsi-target (or targetcli-fb); on RHEL/Fedora it is targetcli.

After installation, verify the daemon is running and the iSCSI port is listening:

Backing store selection. Before creating any target, decide what physical storage the LUN will use. Options include:

• A dedicated physical disk (e.g., /dev/sdb) — simplest, best performance, but no flexibility. You must verify the disk is empty or intentionally disposable before using it (Chapter 8 covers verification).
• An LVM logical volume — flexible sizing, can be grown later, sits on top of one or more physical disks. Recommended for most deployments.
• A file (e.g., /var/lib/iscsi/disk.img) — easiest for testing, but performance depends on the underlying filesystem and disk. Not recommended for production.

If using LVM, create a volume group and logical volume before configuring the target:

Firewall. If ufw (Debian) or firewalld (RHEL) is active, open TCP port 3260 for the initiator's IP range:

Distribution differences to be aware of:

• Debian/Ubuntu: the service is target, the config persists in /etc/rtslib-fb/, and the targetcli shell auto-loads saved config on boot.
• RHEL/Fedora: the service is also target, but the default backing-store path and some LIO parameters differ slightly. RHEL 9 includes a hardened default that disables iSCSI until explicitly enabled in /etc/tgt/targets.conf (older versions) or via targetcli.
• Arch Linux: the package is targetcli, and you must manually enable the target.service unit. The LIO kernel module may need to be loaded explicitly if not in /etc/mkinitcpio.conf.

At this point your storage server has the target software installed, a backing store prepared (or planned), and the network path open. The next chapter creates the actual iSCSI target that initiators will connect to.

This is where the abstract "target" becomes a concrete, addressable object that an initiator can discover and log in to. Each step below is explained before the command is shown.

Step 1: Verify the backing store is safe to use. Before any configuration, confirm the disk or volume you intend to expose contains no data you need:

Step 2: Open the targetcli shell and create the backend storage object.

Step 3: Create the iSCSI target (the "IQN" identity).

Step 4: Create a portal (the IP address and port where initiators connect).

Step 5: Expose the LUN (map the backend to the target).

Step 6: Authorise an initiator (access control).

Step 7: Save the configuration and verify.

Cleanup (if you need to remove the target):

The initiator side is where your server or workstation "plugs in" to the remote target and sees a new disk appear. On Linux, this is handled by the open-iscsi package and its companion tools.

Step 1: Discover the target. Discovery tells the initiator "which targets exist on this network?" There are two methods; sendTargets (static) is the most common for small deployments:

Step 2: Log in to the target (establish the session).

Step 3: Identify the new disk.

Step 4: Verify persistence.

Step 5: Check session health.

Multipath (if applicable): If the target has two portals and the initiator has two NICs, install and configure device-mapper-multipath. After logging in to both portals, multipath -ll should show a single multipath device (e.g., mpathb) with two paths. The OS then uses /dev/mapper/mpathb instead of the individual /dev/sdX devices, and I/O is distributed or failover-managed across both paths.

Windows has included a software iSCSI initiator since Windows 7 / Server 2008 R2. It is sufficient for most workloads and requires no additional purchase. The interface is graphical (iSCSI Initiator console) but every operation can be scripted with iscsicli.

Step 1: Open the iSCSI Initiator.

• Press Win+R, type iscsicpl, press Enter. (Or: Server Manager → Tools → iSCSI Initiator on Server editions.)
• If the "Microsoft iSCSI service" is not running, start it from Services (services.msc) and set it to Automatic.

Step 2: Discover the target.

• In the iSCSI Initiator window, go to the Discovery tab.
• Click Discover Portal.
• Enter the target's IP address (e.g., 10.0.50.10) and port (3260). Click OK.
• The target's IQN should appear in the list below. If it does not, verify: the target is listening on that IP, the Windows machine can ping the target, and no firewall is blocking TCP 3260 in either direction.

Step 3: Connect to the target.

• Go to the Targets tab. The discovered target should be listed with its IQN and portal address.
• Select the target and click Log On.
• In the dialog, check Persist logon information so the connection survives a reboot.
• If the target has multiple portals, check Multipath (this requires the MPIO feature to be enabled — see below).
• Click OK. You should see "The logon attempt was successful."

Step 4: Identify and initialise the disk.

• Open Disk Management (diskmgmt.msc). The new disk appears at the bottom of the list, marked as "Offline."
• Right-click the disk → Bring Online.
• Right-click again → Initialize Disk. Choose MBR (for disks ≤ 2 TB) or GPT (for larger). This writes a partition table to the first sectors of the disk.
• Create a volume: right-click the unallocated space → New Simple Volume → follow the wizard to assign a drive letter and format (NTFS for general use, ReFS for storage spaces or deduplication).

Step 5: Verify persistence.

MPIO on Windows: If you need multipath (two NICs, two target portals), enable the "Multipath-IO" feature via Server Manager → Add Roles and Features (on Server) or Programs → Turn Windows features on or off (on client). Then in the iSCSI Initiator's Log On dialog, check the Multipath box. Windows MPIO will present a single disk with two paths; Get-PhysicalDisk and Get-Disk in PowerShell show the path status.

Version differences: Windows 10/11 Pro and Server editions include the iSCSI initiator by default. Windows Home does not. The initiator on Server 2016+ supports iSCSI over TCP (iSCSI) but not iSER (iSCSI Extensions for RDMA) without additional configuration. For most deployments, standard iSCSI over TCP is all you need.

An iSCSI LUN is a blank disk until you put a filesystem on it. This chapter covers the partitioning, formatting, and mounting steps on Linux, with explicit warnings about the single most dangerous mistake in this area.

Partitioning (Linux example, using a verified empty test device):

Formatting:

Mounting with persistence:

Drive letters (Windows): On Windows, the iSCSI disk appears in Disk Management as an uninitialised disk. You bring it online, initialise it (GPT for > 2 TB), create a partition, and assign a drive letter through the New Simple Volume wizard. The drive letter is persistent unless you change it manually. For scriptable environments, use diskpart or PowerShell (Get-Disk | Set-Disk -IsOffline $false, etc.) to avoid the GUI.

Why device names are not stable: On Linux, /dev/sdb is assigned based on the order in which the kernel detects SCSI devices. If you reboot with a different set of local disks, or if the iSCSI session takes slightly longer to establish than a local disk, the iSCSI LUN might appear as /dev/sdc instead. This is why /etc/fstab entries should use UUIDs or /dev/disk/by-id/ symlinks, and why monitoring scripts should identify disks by their WWN or IQN rather than by device name.

An unauthenticated iSCSI target on an open network is equivalent to leaving a hard drive in a public parking lot. This chapter covers the layers of protection you should apply, in order of implementation difficulty.

CHAP (Challenge-Handshake Authentication Protocol). CHAP is iSCSI's built-in authentication mechanism. During the login negotiation, the target sends a random challenge string; the initiator responds with a one-way hash (MD5) of the challenge combined with a shared secret. The target verifies the hash matches its copy of the secret. If it does, the session is authenticated. CHAP comes in two forms:

• One-way CHAP (initiator authenticates to target): the target knows the initiator's secret. Configured on the target side as a per-initiator ACL secret.
• Mutual CHAP (both sides authenticate): the initiator also verifies the target's identity using a second shared secret. This protects against rogue targets that an attacker might place on the network to capture I/O traffic.

To configure one-way CHAP on a LIO target:

What CHAP does and does not do: CHAP authenticates the parties at login time. It does not encrypt the data that flows after login. Once the session is established, SCSI commands and data travel in cleartext over TCP. An attacker on the same network segment who captures packets can read every block written to the LUN. CHAP prevents an unauthorised machine from connecting; it does not prevent a connected (or sniffing) attacker from reading the stream.

Network segmentation. The most effective "encryption" for iSCSI is often simply: do not put the storage traffic on a network where untrusted devices exist. A dedicated VLAN, physically separated switch, or at minimum an ACL-restricted subnet means that even if the data is unencrypted, only authorised machines can see it. This is the approach most data centres take: storage VLANs are isolated from user VLANs, and access to the management interface (where you configure targets) is further restricted to an admin subnet.

IPsec. For environments where network segmentation is not sufficient (e.g., iSCSI over a WAN link, or a multi-tenant cloud), IPsec in transport mode can encrypt the TCP payload. Both the initiator and target must have matching IPsec policies (pre-shared key or certificate-based). This adds CPU overhead for encryption/decryption and introduces another configuration surface to manage. It is not enabled by default in any iSCSI implementation; you configure it at the OS/network level, not in the iSCSI stack.

Management-plane protection. The targetcli interface (or Windows iSCSI Target Manager) is an administrative control plane. If an attacker gains access to it, they can create new targets, modify ACLs, or delete LUNs. Protect the management interface: require SSH key authentication (disable password login), restrict source IPs with a firewall, and audit /var/log/syslog (or Windows Event Log) for targetcli commands.

Secrets management. CHAP passwords, IPsec pre-shared keys, and any API tokens for storage management should not be hardcoded in shell scripts or committed to version control. Use a secrets manager (HashiCorp Vault, AWS Secrets Manager, or at minimum a restricted-permissions file) and rotate them on a schedule. If a CHAP password is compromised, the attacker has full block-level access to your LUNs until you rotate it.

A single iSCSI path — one NIC, one cable, one switch, one target portal — is a single point of failure. Multipathing creates two or more independent paths so that the loss of any single component does not interrupt I/O.

What "independent" means. Two paths are genuinely independent only if no single component failure can take down both simultaneously. This means: different NICs (not two ports on the same NIC sharing a PHY), different cables, ideally different switches (or at minimum different switch ports with separate uplinks to a redundant pair), and different target portals (ideally on different NICs of the target). Two iSCSI sessions that both traverse the same physical switch port are not independent; if that switch port fails, both paths die. This is the most common misunderstanding in small deployments: people add a second NIC to the initiator but plug both cables into the same switch, and then wonder why "multipathing" didn't protect them when the switch went down.

Linux multipath (device-mapper-multipath). When two or more iSCSI sessions present the same LUN (same target IQN, same LUN ID) to the initiator, the device-mapper-multipath daemon merges them into a single logical device (/dev/mapper/mpathX). The path selection policy determines how I/O is distributed:

• round-robin (default): I/O is spread across all active paths, utilising both links simultaneously. Best for throughput.
• queue-length: sends more I/O to the path with the shorter queue. Useful when one path is faster (e.g., 10 GbE + 1 GbE).
• service-time: similar to queue-length but accounts for expected service time per I/O.

Failover behaviour. When a path fails (cable pulled, NIC error, switch port down), the multipath daemon detects the failure within a few seconds (configurable via polling_interval in /etc/multipath.conf) and marks the path as faulty. I/O is redirected to the remaining active path with minimal interruption — typically a brief stall of 1–5 seconds while in-flight I/O completes or times out. The application may see a few I/O errors during the transition; for databases, this is where the filesystem's journal or the database's own recovery mechanism takes over.

Validating redundancy. Do not assume your multipath setup works until you test it. The safe test: with the system under light load, unplug one of the two cables (or disable one NIC: sudo ip link set eth1 down). Verify that multipath -ll shows one path as faulty and the other still active, and that I/O continues (run fio or dd in the background). Reconnect the cable and confirm the path recovers. If I/O stops completely when you remove one path, your multipath is not configured correctly — both paths are likely sharing a single point of failure.

Windows MPIO. Windows handles iSCSI multipathing through the Multipath-IO feature (enabled via Server Manager or PowerShell: Install-WindowsFeature -Name Multipath-IO). When enabled and the initiator logs in with both portals, Disk Management shows a single disk with "Optimal" path status. Get-PhysicalDisk and Get-Disk in PowerShell expose path details. The failover behaviour is similar to Linux: a brief I/O stall, then automatic redirection. Test by disabling one NIC in Device Manager while I/O is running.

"How fast is my iSCSI?" has no single answer. The number you get depends on block size, queue depth, read/write ratio, whether the data is in cache or on platter, and how congested the network is. This chapter gives you the tools to measure what your workload actually experiences.

Key metrics:

• Throughput (MB/s or GB/s): total data transferred per second. Limited by network bandwidth, disk speed, or the application's I/O pattern, whichever is lowest.
• IOPS (I/O Operations Per Second): number of completed read/write operations per second. A database doing 4 KiB random reads cares about IOPS; a backup doing 1 MiB sequential writes cares about throughput.
• Latency (ms or µs): time from when an I/O is issued to when it completes. For interactive workloads, average and 99th-percentile latency matter more than peak throughput.
• Queue depth: number of outstanding (in-flight) I/Os. Higher queue depth can increase throughput on rotational disks (more concurrent seeks) but increases latency per operation. SSDs handle high queue depths well; HDDs have a practical limit.

fio — the standard benchmarking tool. fio is a flexible I/O generator that can simulate any workload pattern. The following examples are non-destructive: they write to a file on an already-mounted filesystem, not to a raw block device. Benchmarking a raw device (/dev/sdb1) with random writes will destroy the filesystem; only do that on a dedicated test LUN you intend to reformat afterwards.

iostat — monitoring live I/O.

Network statistics.

Interpreting results without false expectations. A 10 GbE link has a theoretical maximum of ~1.25 GB/s (12.5 Gb/s after encoding overhead). In practice, TCP/IP overhead, iSCSI protocol processing, and disk speed limit you well below that. Expect: ~800 MB/s–1.1 GB/s for large sequential transfers on SSD-backed storage over 10 GbE; ~50–200 MB/s for random 4K I/O (limited by seek time on HDDs or NAND controller on SSDs); and significantly less if the network is shared, MTU is mismatched, or the target's disk queue is saturated. If your numbers are far below these ranges, the bottleneck is not iSCSI — it is the disks, the network, or a configuration error.

A successful iSCSI login tells you the session is up. It does not tell you whether the disks are filling up, whether a drive is failing, or whether the network is developing errors. Monitoring closes that gap.

Capacity planning. The most common "outage" in small iSCSI deployments is not a hardware failure; it is someone running out of space at 2 a.m. because nobody was watching the free-space percentage. Set up alerts at 80% and 90% utilisation on each LUN's filesystem. On Linux, this is as simple as a cron job or a monitoring agent checking df --output=pcent /mnt/iscsi-data. On Windows, use Performance Monitor counters or a PowerShell script checking (Get-Volume).DriveType.

SMART and hardware health. On the target side, the physical disks that back your LUNs can fail silently. Run smartctl -a /dev/sdX (from smartmontools) on each backing disk and watch for: reallocated sector count increasing, pending sector count > 0, or the overall SMART assessment reporting "FAILED." On SSDs, watch the percentage-used (wear) and available spare capacity. A successful iSCSI session with a dying backing disk will continue to serve I/O until the disk actually fails at the moment you need it — monitoring is the only way to catch it in time.

Thin provisioning and overcommitment. If your target uses thin provisioning (LVM, ZFS, or a virtualisation layer that allocates blocks on first write rather than at creation), the LUN can appear to have 100 GiB free while the underlying pool is at 95% capacity. A sudden write burst can then exhaust the pool and cause I/O errors across all LUNs on that pool. Monitor the pool utilisation, not just the individual LUN's reported free space. sudo lvs (for LVM) or zpool list (for ZFS) show actual used vs allocated.

Network error monitoring. Even on a "dedicated" storage network, errors happen: a switch port develops a CRC error rate, a cable degrades over time, or a NIC firmware bug causes occasional packet drops. These do not kill the session (TCP retransmits), but they add latency and reduce throughput in a way that is hard to diagnose without looking at the counters. ethtool -S <interface> on both initiator and target, checked periodically, will show you if errors are accumulating.

iSCSI gives you a remote disk. It does not give you a backup. The distinction is critical: a snapshot on the same target that holds your production LUN is one step away from the data it is supposed to protect.

Backups vs snapshots vs replicas vs RAID vs HA — what each actually protects against:

• RAID protects against disk failure (one or more disks die, the array survives). It does not protect against: accidental deletion, ransomware, filesystem corruption, wrong LUN targeted, or a failed rebuild that takes down the array.
• Snapshots (LVM, ZFS, Windows VSS) capture a point-in-time copy of the data on the same storage system. They protect against: accidental file deletion, application bugs that corrupt data after the snapshot was taken. They do not protect against: target hardware failure, site disaster, or an attack that deletes data before you notice and take a snapshot.
• Replicas / replication (synchronous or asynchronous copies to another system) protect against: single-site failure, target outage. Synchronous replication adds latency to every write; asynchronous has a recovery point objective (RPO) of seconds to minutes.
• Backups (full/incremental copies to independent storage, ideally off-site) protect against: everything above, plus ransomware, logical corruption, and human error. A backup is the only layer that can restore data from a point in time when the primary and all replicas are corrupted or destroyed.
• High availability (HA) (multipathing, redundant targets, failover clusters) protects against: single-component failure causing downtime. It does not protect against data loss; it protects against unavailability.

Consistency considerations. A naive backup of a running database (copy the LUN while transactions are in flight) produces an inconsistent snapshot: some transactions are half-written, journal entries reference pages that have since changed. Application-aware backups solve this: for PostgreSQL, use pg_basebackup; for SQL Server, use VSS snapshots with the SQL Server VSS writer; for a virtual machine, use the hypervisor's quiesced snapshot (which sends a flush/fsync to the guest before capturing). For a raw iSCSI LUN that you are backing up at the block level, the safest approach is to freeze the I/O (stop the application or unmount the filesystem) before taking the snapshot, then resume.

Recovery point and time objectives. RPO (Recovery Point Objective): how much data can you afford to lose? "Last night's backup" means an RPO of up to 24 hours. "Every 15 minutes" means an RPO of 15 minutes. RTO (Recovery Time Objective): how quickly must the system be back after a failure? "Within 4 hours" vs "within 15 minutes" drives whether you need a hot standby, a warm restore, or can tolerate a cold rebuild from backup. These are business decisions, not technical ones — but they determine your backup frequency, retention, and whether you need replication.

A safe conceptual recovery workflow:

1. Identify the failure: is it a single file deleted, a corrupted LUN, a dead target, or a site-wide disaster?
2. Select the appropriate restore source: latest snapshot (for recent deletion), last full backup (for corruption), replica (for target failure), off-site backup (for site loss).
3. Restore to a different LUN or system first. Verify the data is intact (checksums, application integrity checks) before promoting it to production.
4. Redirect clients to the restored system (update iSCSI target IQN, DNS, or mount points).
5. Verify the application works: run a test transaction, check file integrity, confirm user access.
6. Document what happened, why the backup/replica was sufficient, and what would have made the recovery faster.

The golden rule: a snapshot on the same storage system that holds your production data is not a backup. If the target's disks fail, its RAID controller dies, or the building catches fire, your snapshots die with it. At minimum, maintain one backup copy on physically independent storage (a different machine, a different rack, ideally a different site). Test your restore at least quarterly — an untested backup is a hope, not a plan.

"The initiator can't see the target" is the most common iSCSI problem, and it has about eight distinct causes. Work through this table in order; each row is a separate failure mode with its own fix.

Systematic approach: before changing any configuration, collect evidence. On the initiator: sudo iscsiadm -m session -P 3, dmesg | tail -30, and journalctl -u open-iscsi --since "5 min ago". On the target: sudo targetcli /iscsi/<IQN>/ show, sudo journalctl -u target --since "5 min ago", and sudo ss -tlnp | grep 3260. The logs will almost always tell you which layer failed (network, authentication, ACL, LUN mapping) before you need to guess.

The session is logged in, the target is healthy, but something between the network layer and the application is wrong. This chapter separates the investigation by layer so you are not chasing ghosts across the entire stack.

Layer 1: Network. If I/O is slow or intermittent, check the network first because it is the most common culprit and the easiest to verify:

Layer 2: iSCSI session. If the network is clean but I/O still misbehaves:

Layer 3: Block device / multipath. If the session is fine but the disk is missing or slow:

Layer 4: Filesystem. If the block device exists but mounting fails or I/O is slow:

Layer 5: Application. If all lower layers are healthy but the application reports errors: check its own logs for I/O timeout settings (databases often have a lock_timeout or i/o timeout parameter that is too aggressive for network storage), and verify it is using O_DIRECT or the correct I/O scheduler for the workload.

Slow transfers — specific causes:

• MTU mismatch: jumbo frames on one end, standard MTU on the other. Fix: align MTU at every hop or use 1500 everywhere.
• Network saturation: another workload (backup, replication) is consuming the same link. Fix: separate storage VLAN or schedule non-critical I/O off-peak.
• Target disk queue saturation: too many concurrent LUNs sharing one physical disk. Fix: check iostat -x on the target; if %util is 100% on the backing device, the disk is the bottleneck.
• TCP window / buffer limits: on very high-bandwidth links (25/100 GbE), default TCP buffers may be too small. Fix: increase net.core.rmem_max and net.core.wmem_max in sysctl.
• Packet loss: a bad cable or switch port dropping frames causes TCP retransmits (200 ms+ per loss). Fix: check NIC error counters, replace cable, move to a different switch port.

The cardinal rule of troubleshooting: collect evidence before changing configuration. A "fix" that happens to change two variables simultaneously (reboot + reconfigure) tells you nothing about which one actually mattered. Change one thing, verify the effect, then move to the next. If you are not sure what is broken, the monitoring data from Chapters 14 and 15 will point you at the correct layer.

This is a complete, reproducible lab that connects one Linux target to one Linux initiator over a simple network. Follow it step by step; every command is shown, every expected result is stated, and every destructive step is flagged.

Lab topology:

• Target: Debian 12, IP 192.168.50.10/24, one spare disk (/dev/sdb, 50 GiB, empty)
• Initiator: Debian 12, IP 192.168.50.20/24
• Network: single switch (or direct crossover cable), no firewall between the two machines, MTU 1500 on both NICs

Phase 1: Prepare the target (on 192.168.50.10)

1. Install the target software: sudo apt update && sudo apt install -y targetcli-fb
2. Verify the spare disk is empty: lsblk -f /dev/sdb — expect no filesystem, no partitions. [DESTRUCTIVE from here on for /dev/sdb]
3. Create a file-based backing store (simplest for a lab):
   sudo targetcli backstores/fileio create lab-disk /var/lib/iscsi/lab.img size=10G
   This creates a 10 GiB sparse file. No physical disk is touched.
4. Create the target:
   sudo targetcli iscsi create iqn.2024-01.com.lab:target1
5. Create a portal on the lab IP:
   sudo targetcli /iscsi/iqn.2024-01.com.lab:target1/portals create 192.168.50.10 3260
6. Map the LUN:
   sudo targetcli /iscsi/iqn.2024-01.com.lab:target1/luns create /backstores/fileio/lab-disk
7. Authorise the initiator (use a placeholder IQN for now; you will set the real one on the initiator in Phase 2):
   sudo targetcli /iscsi/iqn.2024-01.com.lab:target1/acls create iqn.2024-01.com.lab:initiator1
8. Save and verify:
   sudo targetcli saveconfig && sudo targetcli /iscsi/iqn.2024-01.com.lab:target1 show
   Expected: portal listed, LUN 0 (10 GiB) listed, one ACL entry.

Phase 2: Configure the initiator (on 192.168.50.20)

1. Install the initiator software: sudo apt update && sudo apt install -y open-iscsi
2. Set the initiator's IQN to match the target's ACL:
   echo "IQN=iqn.2024-01.com.lab:initiator1" | sudo tee /etc/iscsi/initiatorname.iscsi
3. Restart the service to pick up the new IQN: sudo systemctl restart open-iscsi
4. Discover the target:
   sudo iscsiadm -m discovery -t sendtargets -p 192.168.50.10:3260
   Expected: 192.168.50.10:3260,1 iqn.2024-01.com.lab:target1
5. Log in:
   sudo iscsiadm -m node -T iqn.2024-01.com.lab:target1 -p 192.168.50.10:3260 --login
   Expected: "Login to [192.168.50.10:3260] successful."
6. Identify the new disk:
   lsblk -o NAME,SIZE,TRAN | grep iscsi
   Expected: a device (e.g., sda or sdb) of 10 GiB with TRAN=iscsi.

Phase 3: Use the disk (on the initiator)

1. [DESTRUCTIVE] Format the LUN:
   sudo mkfs.ext4 -L lab-data /dev/sdX (replace sdX with the actual device name from step 6). This writes a filesystem to the iSCSI LUN. The data lives on the target's file.
2. Mount it:
   sudo mkdir -p /mnt/lab && sudo mount /dev/sdX /mnt/lab
3. Verify:
   df -h /mnt/lab && echo "hello from iSCSI" | sudo tee /mnt/lab/test.txt
   Expected: 10 GiB filesystem mounted, file written successfully.
4. Check on the target that the file is growing:
   ls -lh /var/lib/iscsi/lab.img (on 192.168.50.10). The sparse file should now show a size reflecting the written data.

Phase 4: Verify persistence (reboot the initiator)

1. Make the login persistent: sudo iscsiadm -m node -T iqn.2024-01.com.lab:target1 -p 192.168.50.10:3260 --login (re-running login saves the node entry).
2. Reboot the initiator: sudo reboot
3. After reboot, check: lsblk | grep iscsi — the disk should reappear automatically. If it does not, run sudo systemctl status open-iscsi and check the journal for login errors.

Cleanup (when the lab is done)

1. On the initiator: unmount (sudo umount /mnt/lab), log out (sudo iscsiadm -m node -T iqn.2024-01.com.lab:target1 -p 192.168.50.10:3260 --logout).
2. On the target: delete the target (sudo targetcli /iscsi delete iqn.2024-01.com.lab:target1), delete the backing file (rm /var/lib/iscsi/lab.img), save config.
3. Both machines: no further action needed; the iSCSI packages can remain installed for future use.

Moving from a working lab to a production deployment is not about adding more features; it is about making sure every failure mode has a defined response, every configuration change is documented, and the system can be recovered without heroics.

Before you go live, answer these questions in writing:

• Availability: If one NIC, one cable, one switch port, or one target disk fails, does I/O continue? (Requires multipath + redundant paths.) What is your acceptable downtime window?
• Capacity: How much growth do you expect in 12 months? Can the LUN be grown online (LVM extend, Windows "Extend Volume") without a full backup/restore cycle? What happens at 100% capacity — does I/O fail gracefully or corrupt data?
• Security: Is CHAP (or mutual CHAP) configured? Is the storage VLAN isolated from user traffic? Are management interfaces restricted? Are secrets stored in a secrets manager, not in shell history?
• Monitoring: Do you have alerts for: session loss, path failure, capacity > 80%, SMART failures, NIC errors, and target CPU/memory saturation? Who receives the alerts, and what is the on-call response procedure?
• Backup and recovery: Where is the backup copy (independent storage)? How often is it taken? When was the last restore test, and did it succeed? What is your RPO and RTO, and are they acceptable to the business?
• Documentation: Is there a current diagram of the topology (IPs, VLANs, cabling, LUN mappings)? Is the target configuration exported and version-controlled (targetcli saveconfig output in a repo)? Can a second person rebuild the system from documentation alone?
• Change management: What is the procedure for adding a new initiator, growing a LUN, or replacing a disk? Is there a maintenance window for target updates? Do you have a rollback plan for each change type?

Final verification checklist:

Glossary:

Next steps:

• If you are running Windows Server, explore the built-in iSCSI Target feature (Server Manager → File and Storage Services → iSCSI → Disks) for a GUI-based alternative to LIO.
• For virtualisation workloads, investigate how your hypervisor (Hyper-V, Proxmox, ESXi) presents iSCSI LUNs to VMs and whether hardware-assisted MPIO is available.
• If you outgrow software targets, look at dedicated storage appliances (Synology, QNAP, TrueNAS) that expose iSCSI alongside SMB/NFS from a single management interface.
• For high-performance environments, research NVMe-oF (NVMe over Fabrics), which replaces the SCSI layer with NVMe commands and can reduce latency by 30–50% compared to iSCSI on equivalent hardware.

This guide is a living document. If you find an error, an unclear explanation, or a step that does not match your distribution's behaviour, the goal is to fix it here so the next reader hits fewer walls than you did.