ZeroTier Intranet Penetration for Web Crawling: No Public IP Required (Part 2)

Start from our web crawling and data collection basics guide if you’re new

ZeroTier Intranet Penetration: How It Works and When to Use a Moon
In Part 1, we introduced frp, a lightweight intranet penetration tool that meets most personal needs. frp is free, open-source, easy to configure, supports multiple protocols, and performs well. Its main tradeoffs are: adding a new port usually means editing config and restarting, and you still need a public server as a relay (non-P2P traffic consumes server bandwidth).

This part introduces ZeroTier intranet penetration, a remote networking solution that can work even when you don’t have a public IP. ZeroTier creates a virtual LAN (overlay network) so devices can reach each other via virtual IPs. For small personal use, ZeroTier Central is free up to 25 devices (across all networks), with no credit card required. 

Note: your original text mentioned “1 network and 20 hosts free.” ZeroTier’s current docs/pricing indicate the free tier is up to 25 devices; wording and limits have changed over time. 

What to expect in real networks (important for crawling)

ZeroTier prefers direct connections when NAT traversal succeeds; otherwise, traffic can be relayed via ZeroTier infrastructure (“roots”) which may increase latency. ZeroTier documents root server presence in multiple regions/cities (e.g., Los Angeles, Miami, Tokyo, Zurich, San Francisco), and IPs may change over time. 

If you need better control over relay paths (common in some regions), you can deploy your own roots / “Moon” style infrastructure. ZeroTier’s current docs describe private root servers as the recommended approach; the “Moon” concept is still documented but noted as deprecated in the glossary. 

ZeroTier Key Features (Beginner-Friendly Summary)

• No public IP required on endpoints: devices join a virtual network and receive a virtual IP.
• Cross-platform: Windows/macOS/Linux/iOS/Android, plus many router/NAS environments.
• Central management: authorize devices, assign managed IPs, and configure routes in ZeroTier Central. 
• Optional self-host optimization: deploy private roots / Moon-style setup to improve stability/latency where relaying is suboptimal. 

Installation

ZeroTier Central:

https://my.zerotier.com/

Client download:

https://www.zerotier.com/download/

Step 1: Create a Network (ZeroTier Central)

1. Register and log in to ZeroTier Central.
2. Click Create A Network.
3. Copy your Network ID (16-digit).

Access Control

• Set to Private (recommended): devices must be manually authorized before joining.
• Public is not recommended for real environments, especially when crawling systems or internal services are involved.

(Your screenshots remain valid here; no content change required.)

Step 2: Join Devices to the Network

On each device, install ZeroTier and join using the Network ID:

zerotier-cli join <YOUR_NETWORK_ID>

Then go back to ZeroTier Central → Members list → Authorize the new device (required if the network is Private).

Step 3: Connectivity Test (Ping)

After two devices are authorized, test reachability using their ZeroTier virtual IPs:

ping 10.243.0.2

If latency is high or packet loss occurs, it often indicates traffic is being relayed rather than direct.

Why Build a Moon / Private Root Optimization Node?

If your devices frequently fall back to relaying and you want more predictable performance, you can deploy self-hosted infrastructure. ZeroTier’s documentation frames this as deploying private root servers; the older “Moon” terminology is still commonly used in community guides, but ZeroTier notes moons are deprecated in the glossary. 

This tutorial keeps your original operational steps (Moon-style) because it matches how many users currently optimize paths in practice.

Setting Up Your Own Moon Relay Node (Server Side)

1) Preparation

• A cloud server (prefer higher bandwidth)
• Linux (Ubuntu/CentOS)
• Security group: open UDP 9993 (ZeroTier default port)

2) Install ZeroTier

curl -s https://install.zerotier.com/ | sudo bash

3) Start service and enable auto-start

sudo systemctl start zerotier-one
sudo systemctl enable zerotier-one

4) Join your network

sudo zerotier-cli join <YOUR_NETWORK_ID>

5) Generate Moon config

cd /var/lib/zerotier-one/
sudo zerotier-idtool initmoon identity.public > moon.json

Edit moon.json and fill your server’s public IP in the relevant section (as in your screenshot).

6) Generate the signed moon file

sudo zerotier-idtool genmoon moon.json
ls

You will get a file similar to:

000000xxxxxxxxxxxxxx.moon

7) Put it into moons.d and restart

sudo mkdir -p moons.d
sudo mv 000000xxxxxxxxxxxxxx.moon moons.d/
sudo systemctl restart zerotier-one

8) Download the .moon file to your local machine

For example, using your preferred SSH/SFTP tool (FinalShell, scp, etc.). (Keep your original sz approach if that’s your workflow.)

Client Side (Windows Example)

1. Open ZeroTier data directory (default):

C:\ProgramData\ZeroTier\One

(Enable hidden folders if needed.)

2. Create folder:

moons.d

3. Copy the .moon file into:

C:\ProgramData\ZeroTier\One\moons.d

4. Restart the ZeroTier service:

• Win + R → services.msc
• Find “ZeroTier One” → Restart

5. Verify peers/routes

Open Administrator CMD, then:

cd "C:\ProgramData\ZeroTier\One"
zerotier-cli listpeers

If you can see the moon node listed, it indicates the client has loaded the configuration.

Route Tuning Notes (Keep Your Existing Checklist)

Your existing “route activation delay / ping tips / delete default route” checklist is useful operational guidance. Keep it, but ensure the text is unambiguous:

• First ping gateway/relay-related IP, then ping other nodes.
• Route changes may take time to propagate; repeat ping tests.
• If you intend to force a custom route, remove/avoid conflicting default routes in the network config.

(Keep your screenshots and sample ping outputs as-is.)

Security Notes (ZeroTier Best Practices)

When building a virtual LAN for crawling or internal services, prioritize these:

1. Access control

• Use Private networks and manually authorize members. 
• Regularly remove unused/unknown devices.

2. Protect identity keys

• identity.secret is the device identity credential; treat it as sensitive and back it up before OS reinstall.

3. Account security

• Enable 2FA in account settings.
• Avoid weak passwords; rotate if needed.

4. Sensitive data

• Overlay encryption exists, but for highly sensitive services, still prefer application-layer security (HTTPS/SSH), especially across heterogeneous environments.

Summary

ZeroTier intranet penetration is a practical way to connect devices for web crawling and internal service access without requiring public IPs on endpoints. If your traffic is frequently relayed and performance is unstable, deploying a self-hosted optimization node (private roots / Moon-style setup) can improve predictability—while ZeroTier’s docs indicate private root servers as the recommended direction and note moons as deprecated in terminology.