Computer Networking from Scratch: Bits, Physical Media, Hub Collision Domains, and Switch MAC Addressing First Principles
If you know nothing about computer networking, how can you build an intuitive, solid first-principles foundation? Deconstruct network communication from absolute zero: how computers represent 0 and 1 via voltages and light pulses; why twisted pair copper cables are spiraled to cancel noise; single-mode vs multi-mode optical fiber physics; the legacy Hub era, shared media, collision domains, and CSMA/CD exponential backoff; how Layer 2 switches eliminate collisions using MAC address tables (CAM) and Ethernet frames; and dissect ARP broadcast requests, unicast replies, gratuitous ARP, and spoofing mitigation.
Computer Networking Masterclass: From Ethernet Principles to Hyperscale InfiniBand Architecture
A ground-zero masterclass to 10,000-GPU AI networking: from physical voltages, twisted pairs, and optical fibers to hubs, collision domains, switches, and MAC/ARP; step-by-step binary derivations of IP addressing, subnet masks, default gateways, VLANs, DHCP, and NAT; in-depth DNS, Socket 5-tuples, TCP 11-state machines, and BBR congestion control; Linux kernel NAPI, sk_buff, and eBPF XDP; datacenter traditional 3-tier vs 2-tier Spine-Leaf fabrics and EVPN-VXLAN; leading up to AI supercomputing: lossless RoCEv2, native InfiniBand NDR/XDR link speeds, credit flow control, Rail-Optimized Fat-Trees, Adaptive Routing, SHARP, and bare-metal OFED/NCCL performance tuning.
Browse all 12 chapters in this series ▾
- 01 Computer Networking from Scratch: Bits, Physical Media, Hub Collision Domains, and Switch MAC Addressing First Principles Reading
- 02 IP Addressing and Subnetting First Principles: Binary Arithmetic, Subnet Masks, CIDR, and Default Gateway Routing
- 03 Enterprise LAN Infrastructure: VLAN Segmentation (802.1Q), Dynamic DHCP, and NAT Port Forwarding
- 04 Application & Transport Layer Bridges: DNS Resolution, Sockets, Ports, and UDP vs TCP Foundations
- 05 Network & Transport Layers in Depth: IP Routing, CIDR, TCP 11-State Machine, and Sliding Window First Principles
- 06 TCP Congestion Control Evolution & High-Performance Transport: From Reno and Cubic to BBR Mathematical Models, and HTTP/2 to HTTP/3 (QUIC)
- 07 Linux Kernel Networking Subsystem in Depth: From NIC Drivers, NAPI, and Ring Buffers to eBPF XDP Wire-Speed Forwarding
- 08 Modern Data Center Network Architecture First Principles: Clos Topologies, Leaf-Spine Fabrics, BGP Underlay, and EVPN-VXLAN Large Layer-2 Virtualization
- 09 RDMA High-Performance Networking Foundations: Kernel Bypass, Zero-Copy, Queue Pairs, and Lossless RoCEv2 (PFC/ECN) Architecture
- 10 InfiniBand Architecture First Principles: Physical Link Rates, Credit-Based Link Flow Control, and Subnet Manager Fabric Orchestration
- 11 InfiniBand AI Cluster Networking in Practice: Fat-Tree Topologies, Rail-Optimized Architecture, Adaptive Routing (AR), and In-Network Reduction (SHARP)
- 12 Production InfiniBand Deployment, Cluster Operations, and NCCL Tuning: OFED Drivers, OpenSM HA, ibdiagnet Fabric Auditing, and GPUDirect RDMA
Introduction: The Essence of Networking — How Do Two Computers "Talk"?
Picture a simple scenario: two computers sit on a desk—Computer A and Computer B. You want to transfer a file from Computer A to Computer B.
Inside a computer, every piece of text, image, and video exists solely as a string of binary digits (0s and 1s, or bits).
- How do you deliver these bits from Computer A's memory directly into Computer B's memory?
- Can you simply connect the two motherboards with a copper wire?
- If an office has 100 computers, do you run physical wires between every single pair?
These core questions drove the evolution of computer networking over the past half-century.
This article is Chapter 1 of our masterclass: Computer Networking: From Ethernet Principles to Hyperscale InfiniBand Architecture. We set aside academic jargon to start from ground zero, tracing how tiny electrical currents and laser pulses give rise to the physical layer, hub collision domains, Layer 2 switching, and Ethernet MAC/ARP addressing.
1. Physical Layer (Layer 1): Bits, Voltages, Cables, and Fiber Optics
1.1 The Physical Manifestation of 0 and 1: Voltage Levels and Light Pulses
Computers are electronic devices. They cannot directly perceive abstract numbers; they respond only to voltage shifts or the presence/absence of light pulses:
NRZ Voltage Encoding of Binary Bits:
High Voltage (+5V) -------\ /-------\
\ / \
Low Voltage ( 0V) \-----/ \-------
Binary Bit Stream: 1 0 1 0 - Electrical Signaling (Copper Cables): Data is represented by voltage transitions across conductors—e.g., high voltage indicates
1, low voltage indicates0; - Optical Signaling (Fiber Optics): Semiconductor lasers or LEDs pulse light down glass strands—light pulses indicate
1, dark intervals indicate0.
1.2 Why Are Twisted-Pair Cables Spiraled?
A standard RJ45 Ethernet cable (such as Cat5e or Cat6) contains eight thin copper wires twisted into four distinct pairs. Why twist the wires around each other instead of running them straight and parallel?
Differential Signaling and Noise Cancellation:
Wire 1 (TX+) : + + + + + + + + + + (Positive Signal)
Wire 2 (TX-) : - - - - - - - - - - (Inverted Negative Signal)
\ / \ / \
\ / \ / \ <- Spiraled Twisting
\ / \ / \
External Noise: ~~~~~ Penetrates both wires with near-identical voltage spike ~~~~~
Receiver Differential Amp: V_out = (TX+ + Noise) - (TX- + Noise) = TX+ - TX- (Noise Cancelled!) - First Principle: Eliminating Electromagnetic Interference (EMI): Physical environments are flooded with ambient electromagnetic radiation from power lines, fluorescent ballasts, and radio signals. Ethernet cables employ Differential Signaling, transmitting complementary voltages across twisted pairs. Because the two wires are twisted closely around one another, ambient noise couples equally into both conductors. When the receiving transceiver computes the difference between the two wires, common-mode noise cancels out, preserving signal integrity.
1.3 Optical Fiber: Single-Mode vs Multi-Mode
When link distances exceed 100 meters (the physical limit of copper Ethernet) or require 100 Gbps to 400 Gbps speeds, high-frequency signal attenuation makes copper impractical. Networks turn to Fiber Optics:
| Characteristic | Multi-Mode Fiber (MMF) | Single-Mode Fiber (SMF) |
|---|---|---|
| Core Diameter | Wide ( or ) | Narrow (approx. ) |
| Light Source | Low-cost VCSEL / LED | Precision Long-Wavelength Laser Diode |
| Propagation | Multiple light modes bounce via total internal reflection | Single axial mode propagates without internal reflection |
| Limiting Factor | Modal Dispersion (pulse spreading over distance) | Near-zero modal dispersion |
| Wavelength | 850 nm | 1310 nm / 1550 nm |
| Reach | Tens to hundreds of meters (intra-datacenter racks) | Kilometers to tens of kilometers (campus backbones, metro/WAN) |
| Jacket Color | Aqua / Orange (OM3/OM4) | Yellow (OS1/OS2) |
2. From Hubs to Switches: The Evolution of Collision and Broadcast Domains
With transmission media established, consider four computers (A, B, C, D) in a room. How do we interconnect them?
2.1 The Legacy Era: Hubs and the "Collision Domain"
Early networks relied on Hubs. A hub's internal circuitry acts as a simple electrical splitter connecting all ports in parallel:
flowchart TD
subgraph HubNetwork["Hub Network: Shared Physical Layer (Electrical Broadcast)"]
Hub["Hub (Physical Layer Multi-Port Repeater)"]
NodeA["Host A (Transmitting)"] -->|Voltage enters| Hub
Hub -->|Replicates raw electrical signal| NodeB["Host B"]
Hub -->|Replicates raw electrical signal| NodeC["Host C"]
Hub -->|Replicates raw electrical signal| NodeD["Host D"]
end
- Mechanism: When Host A transmits electrical pulses on Port 1, the hub amplifies and blindly repeats the voltage onto Ports 2, 3, and 4;
- The Core Hazard: Collisions: If Host A and Host B transmit simultaneously, their electrical signals collide on the shared conductor. The overlapping voltages corrupt both transmissions;
- Collision Domain: A network segment where simultaneous transmissions from two or more devices cause electrical collisions. All hosts plugged into a hub share a single collision domain.
The Classical Solution: CSMA/CD
To manage contention on shared media, Ethernet network adapters implemented CSMA/CD (Carrier Sense Multiple Access with Collision Detection):
- Carrier Sense: Listen before transmitting. If voltage on the wire indicates active traffic, wait;
- Collision Detection: Listen while transmitting. If voltage spikes unexpectedly, a collision occurred;
- Jamming Signal: Upon detecting a collision, abort transmission immediately and broadcast a 32-bit jam signal so all nodes recognize the collision;
- Exponential Backoff: Each node waits a randomized backoff interval before attempting retransmission, avoiding immediate re-collisions.
2.2 The Layer 2 Revolution: Network Switches and CAM Tables
Shared media scales poorly: ten computers sharing a 100 Mbps hub effectively yield an average throughput of only 10 Mbps per host. To solve this, engineers developed the Network Switch.
Operating at the Data Link Layer (Layer 2), a switch parses frame headers using dedicated memory and ASICs:
flowchart TD
subgraph SwitchNetwork["Modern Switch: Dedicated Bandwidth and Collision Isolation"]
Switch["Layer 2 Switch
Maintains Content-Addressable Memory (CAM Table)"]
HostA["Host A (MAC_A)"] <-->|Dedicated Full-Duplex Link| Switch
HostB["Host B (MAC_B)"] <-->|Dedicated Full-Duplex Link| Switch
HostC["Host C (MAC_C)"] <-->|Dedicated Full-Duplex Link| Switch
end
How Switches Eliminate Collisions
Each switch port constitutes an isolated collision domain. When Host A transmits to Host B, the switch crossbar matrix forwards traffic exclusively between Ports 1 and 2. Hosts C and D see zero electrical noise and retain full wire bandwidth on their ports.
3. Data Link Layer (Layer 2): MAC Addresses and Ethernet Frames
How does a switch know which port should receive Host A's data? It inspects the hardware identifier burned into each network interface card: the MAC Address.
3.1 What Is a MAC Address?
A MAC (Media Access Control) address is a globally unique 48-bit hardware identifier burned into a network interface's ROM:
- Represented as 12 hexadecimal characters, e.g.,
08:00:27:B2:3C:9A; - First 24 bits (OUI, Organizationally Unique Identifier): Assigned by the IEEE to hardware vendors (e.g., Intel, Apple, Cisco, NVIDIA/Mellanox);
- Last 24 bits: Vendor-assigned serial number for that specific interface.
3.2 Anatomy of the Ethernet II Frame
At Layer 2, data moves encapsulated within Frames:
Standard Ethernet II Frame (IEEE 802.3):
+----------------+-----+-------------+-------------+------+---------------+---------+
| Preamble | SFD | Dest MAC | Source MAC | Type | Payload | FCS |
| (7 Bytes) | 1B | (6 Bytes) | (6 Bytes) | 2B | (46 - 1500B) | (CRC,4B)|
+----------------+-----+-------------+-------------+------+---------------+---------+ - Preamble & Start Frame Delimiter (SFD, 8 bytes total): Alternating patterns of
10101010...ending in10101011. This bit sequence allows receiver clock circuits to synchronize with incoming bit edges; - Destination MAC & Source MAC (6 bytes each): Identifies the sending and receiving hardware interfaces on the local link;
- EtherType (2 bytes): Identifies the encapsulated Layer 3 protocol:
0x0800: IPv4;0x86DD: IPv6;0x0806: ARP;
- Payload (46 to 1500 bytes): The encapsulated upper-layer packet. Minimum payload size is 46 bytes (padded if necessary to meet the 64-byte minimum frame size required by CSMA/CD slot time constraints). Maximum transmission unit (MTU) defaults to 1500 bytes;
- Frame Check Sequence (FCS / CRC32, 4 bytes): A cyclic redundancy check computed by the sender across the entire frame. The receiver recalculates the checksum; if noise flipped any bit in transit, the frame is discarded at the hardware level.
4. Switch CAM Table Learning and the ARP Protocol
4.1 How Switches Learn: The CAM Table Lifecycle
When a switch boots up, its MAC address table (CAM Table) is empty:
sequenceDiagram
autonumber
actor A as Host A (Port 1, MAC_A)
actor SW as Switch (Empty CAM Table)
actor B as Host B (Port 2, MAC_B)
actor C as Host C (Port 3, MAC_C)
Note over A: Host A sends frame: Src=MAC_A, Dst=MAC_B
A->>SW: Frame arrives on Port 1
Note over SW: 1. Learn Source MAC:
"Port 1 connects to MAC_A!" Injects into CAM Table
Note over SW: 2. Lookup Destination MAC:
MAC_B not in CAM Table!
SW-->>B: 3. Unknown Unicast Flooding: Floods copy to Port 2
SW-->>C: Floods copy to Port 3 (Host C drops frame: not intended recipient)
Note over B: Host B replies: Src=MAC_B, Dst=MAC_A
B->>SW: Frame arrives on Port 2
Note over SW: 4. Learn Source MAC:
"Port 2 connects to MAC_B!" Injects into CAM Table
Note over SW: 5. Unicast Forwarding:
Table indicates MAC_A is on Port 1. Forwards directly to Port 1!
SW->>A: Host A receives reply (Port 3 receives nothing!)
- Aging Timer: Learned MAC entries typically time out after 300 seconds of inactivity. This allows the switch to adapt dynamically if a machine is unplugged or moved to a different switch port.
4.2 ARP: Resolving IP Addresses to MAC Addresses
Applications address destinations by IP (e.g., 192.168.1.20), but Layer 2 switches forward based on physical MAC addresses. How does a host discover the MAC address associated with a target IP?
It uses the Address Resolution Protocol (ARP).
sequenceDiagram
autonumber
actor A as Host A (192.168.1.10)
actor Switch as Layer 2 Switch
actor B as Host B (192.168.1.20)
Note over A: Needs to send packet to 192.168.1.20 (No MAC cached)
A->>Switch: ARP Request (Broadcast)
DstMAC = ff:ff:ff:ff:ff:ff
"Who has 192.168.1.20? Tell 192.168.1.10!"
Switch-->>B: Switch floods broadcast frame out all ports
Note over B: Host B matches target IP
B->>Switch: ARP Reply (Unicast)
DstMAC = MAC_A, SrcMAC = MAC_B
"192.168.1.20 is at MAC_B!"
Switch->>A: Switch forwards unicast reply to Host A
Note over A: Injects entry into local ARP cache (arp -a)
- ARP Request (Broadcast): The destination MAC is set to
ff:ff:ff:ff:ff:ff(all ones broadcast). The switch replicates the frame to all active ports on the local network; - ARP Reply (Unicast): The owner of the requested IP returns its MAC address via a directed unicast frame;
- Gratuitous ARP: When an interface comes online, it broadcasts an ARP request querying its own assigned IP. If another host responds, the operating system detects an IP Address Conflict and alerts the user.
5. Practical Inspection Commands on Linux
Inspect Layer 1 and Layer 2 state on a Linux system using standard tools:
# 1. Inspect physical link state, negotiated duplex, and speed (Layer 1)
ethtool eth0
# Key output fields:
# Speed: 1000Mb/s <- Negotiated link rate
# Duplex: Full <- Full-duplex (Zero collision domain)
# Link detected: yes <- Physical carrier detected
# 2. Query hardware MAC addresses of local interfaces
ip link show eth0
# Output contains: link/ether 52:54:00:12:34:56 brd ff:ff:ff:ff:ff:ff
# 3. Display the OS ARP cache table (Layer 2 to Layer 3 mappings)
ip neigh show
# Sample output:
# 192.168.1.1 dev eth0 lladdr 00:50:56:c0:00:08 REACHABLE 6. Summary and Next Steps
The Physical and Data Link layers establish the foundation for local network communication:
- The Physical Layer encodes bits as voltage transitions or light pulses, using twisted pairs and low-dispersion optical fibers to resist noise and attenuation;
- Legacy Hubs shared physical media, bounding hosts within a single collision domain and relying on CSMA/CD;
- Layer 2 Switches eliminate collisions using MAC addresses and CAM table learning, delivering full-duplex wire-speed forwarding;
- ARP bridges logical IP addresses to physical MAC addresses.
However, Layer 2 networks cannot scale globally on their own:
- Broadcast Storms: If billions of devices shared a flat Layer 2 fabric, a single ARP broadcast would flood the entire Internet;
- Flat Address Space: MAC addresses are randomly burned into hardware without geographic or topological hierarchy; switches cannot store billions of entries in memory;
- Why must we layer IP Addresses on top of MAC addresses?
- What is a Subnet Mask, and how does binary bitwise arithmetic separate network prefixes from host identifiers?
- How does the Default Gateway rewrite frame headers when forwarding packets across subnets?
In Chapter 2 of our masterclass, we explore network-layer math: IP Addressing and Subnetting First Principles: Binary Arithmetic, Subnet Masks, CIDR, and Default Gateway Routing!
Frequently Asked Questions (FAQ)
Q1: Why do we need IP addresses if every network interface already has a unique MAC address?
MAC addresses are flat identifiers, whereas IP addresses are hierarchical routing locators.
- MAC addresses resemble personal identity numbers: Your ID number remains the same whether you are in Tokyo, London, or San Francisco. If the global Internet routed on MAC addresses alone, every router on Earth would need to track billions of flat MAC entries in memory, causing hardware forwarding tables to collapse;
- IP addresses resemble postal addresses: Structured hierarchically as Country State City Street Building. Core routers inspect only high-level network prefixes (e.g., regional aggregate routes) to forward packets across continents. The specific MAC address is resolved via ARP only on the final local hop. This hierarchical structure enables global route aggregation and internet-scale routing.
Q2: What is the fundamental difference between a Collision Domain and a Broadcast Domain?
- Collision Domain (Layer 1): A physical segment where simultaneous transmissions from multiple devices collide at the electrical/signal level. Hubs cannot isolate collision domains; every port on a Layer 2 switch represents an independent collision domain;
- Broadcast Domain (Layer 2/3): The set of all devices that receive a broadcast frame (destination MAC
ff:ff:ff:ff:ff:ff). Standard switches forward broadcasts out all active ports, so all ports share one broadcast domain; only Layer 3 routers or switches segmented with VLANs divide broadcast domains.
Q3: How does ARP Spoofing compromise local networks, and how do switches mitigate it in hardware?
Attack Mechanism: ARP is stateless and lacks authentication. An attacker repeatedly sends forged ARP replies to a victim host stating: "I am the gateway 192.168.1.1, and my MAC is AA:AA:AA:AA:AA:AA". The victim overwrites its local ARP cache, redirecting all outbound internet traffic through the attacker's machine and enabling Man-in-the-Middle (MITM) inspection. Hardware Mitigation: Enterprise switches deploy Dynamic ARP Inspection (DAI). The switch snoops DHCP handshakes to build a trusted <IP-MAC-Port> binding database, dropping any ARP response packet on untrusted ports whose IP-MAC pairing does not match the hardware-validated binding table.