Network Extension Setup for macOS

System Extensions are a modern replacement for certain classes of legacy kernel extensions (KEXTs) on macOS. Starting with macOS 10.15 (Catalina), Apple introduced System Extensions as a more secure and stable alternative that runs in user space rather than kernel space. Network Extension functionality can be implemented using either the traditional App Extension approach or the System Extension approach.

Overview

System Extensions differ from traditional Network Extensions in several key ways:

  • Architecture: System Extensions run as standalone processes with their own main.swift entry point, while Network Extensions run as app extensions

  • Distribution: System Extensions require Developer ID signing and notarization for distribution outside the Mac App Store

  • Activation: System Extensions must be explicitly activated using OSSystemExtensionManager before first use

  • Configuration: System Extensions use different Info.plist structure and entitlements

When to Use System Extensions

Use System Extensions when:

  • Distributing your app outside the Mac App Store

  • Building enterprise or developer tools

  • Requiring maximum performance and stability

  • Targeting macOS 10.15 and later

Use traditional Network Extensions when:

  • Distributing through the Mac App Store

  • Supporting older macOS versions

  • Building sandboxed applications

Key Differences: Network Extensions vs System Extensions

Aspect
Network Extension
System Extension

Entry Point

No main.swift (NSExtension handles lifecycle)

Requires main.swift with NEProvider.startSystemExtensionMode()

Info.plist Key

NSExtension dictionary

NetworkExtension dictionary

Entitlement

packet-tunnel-provider

packet-tunnel-provider-systemextension

Sandbox

App Sandbox enabled

No App Sandbox

Activation

Automatic on first VPN start

Manual via OSSystemExtensionManager

Distribution

Mac App Store or Developer ID

Developer ID only (requires notarization)

Prerequisites

Before implementing System Extensions, ensure you have:

Implementation Steps

1. Create System Extension Target

Create a new target in your Xcode project for the System Extension provider:

  1. In Xcode, go to File > New > Target

  2. Select System Extension (under macOS)

  3. Name it appropriately (e.g., "HydraProviderSE" or "WireGuardProviderSE")

2. Configure main.swift

System Extensions require a main.swift file as the entry point. This file must call NEProvider.startSystemExtensionMode():

This code initializes the Network Extension provider in System Extension mode and starts the main dispatch loop.

3. Configure Info.plist

System Extensions use a different Info.plist structure compared to Network Extensions.

System Extension Info.plist:

Traditional Network Extension Info.plist (for comparison):

Key Differences:

  • System Extension uses NetworkExtension dictionary with NEMachServiceName and NEProviderClasses

  • Network Extension uses NSExtension dictionary with NSExtensionPointIdentifier and NSExtensionPrincipalClass

4. Configure System Extension Entitlements

System Extension provider targets require specific entitlements:

Important: System Extensions do NOT include com.apple.security.app-sandbox (unlike Network Extensions).

Traditional Network Extension entitlements (for comparison):

5. Configure Main App Entitlements

Your main macOS application needs additional entitlements to install and manage System Extensions:

Critical: The com.apple.developer.system-extension.install entitlement is required for the main app to install System Extensions.

6. Implement System Extension Activation

System Extensions must be explicitly activated before first use. Create a helper class to manage activation:

7. Configure SDK with usesSystemExtension Flag

When configuring the VPN SDK, set the usesSystemExtension flag to true in the VPNGroupData:

8. Activate System Extension Before VPN Start

Call the system extension activation before initializing the VPN SDK:

9. Implement ProviderDelegate

The ProviderDelegate implementation is similar for both Network Extensions and System Extensions:

Code Signing and Provisioning

Development

For development, you'll need:

  1. Development Certificate: Mac Development certificate from your Apple Developer account

  2. Provisioning Profiles: Create separate provisioning profiles for:

    • Main application (macOS App Development)

    • Each System Extension provider (macOS App Development)

Naming Convention:

  • VPNSDK Demo macOS SE Development - Main app

  • VPNSDK Demo Hydra Provider macOS SE Development - Hydra provider

  • VPNSDK Demo WireGuard Provider macOS SE Development - WireGuard provider

Distribution

For distribution outside Xcode, you'll need:

  1. Developer ID Application Certificate: For signing the main app

  2. Developer ID Application Certificate: For signing the System Extension providers

  3. Notarization: All System Extensions must be notarized by Apple before distribution

  4. DMG Creation: Package your app in a signed and notarized DMG installer

Distribution Process:

  1. Sign the System Extension providers with Developer ID certificate

  2. Sign the main application with Developer ID certificate

  3. Create a DMG installer

  4. Sign the DMG

  5. Notarize the DMG with Apple

  6. Staple the notarization ticket to the DMG

Known Issues and Troubleshooting

Code Signature Invalid Error in Development

When launching a System Extension demo app in development mode from Xcode, you may encounter a "code signature invalid" error. This is a known issue with the current implementation.

Workaround:

  • Build and run the app using the distribution configuration

  • Install the notarized DMG for testing

  • The issue does not occur in production builds that are properly signed and notarized

Alternative: Disable System Integrity Protection (at your own risk)

System Integrity Protection (SIP) in macOS protects the entire system by preventing the execution of unauthorized code. The system automatically authorizes apps that the user downloads from the App Store. The system also authorizes apps that a developer notarizes and distributes directly to users. The system prevents the launching of all other apps by default.

To disable SIP, run the computer in Recovery Mode and then run the following command in Terminal application from the Utilities menu:

After restart, SIP will be disabled and the System Extension should work properly based on the activation request.

Warning: Disabling SIP reduces the security of your Mac. Only disable SIP for development purposes on non-production machines, and re-enable it when no longer needed by running csrutil enable in Recovery Mode.

For more information, see Disabling and Enabling System Integrity Protection.

User Approval Required

On first activation, macOS will prompt the user to approve the System Extension in System Preferences > Security & Privacy. Your app should handle this gracefully:

  1. Listen for requestNeedsUserApproval delegate callback

  2. Show user instructions to approve the extension in System Preferences

  3. Retry activation after approval

Extension Activation Timeout

System Extension activation may take several seconds. Ensure your UI provides appropriate feedback:

Testing

Testing System Extensions

  1. Development Testing:

    • Build and run from Xcode

    • Note: Code signature errors may occur in development mode

    • Use Console app to view logs from the System Extension

  2. Distribution Testing:

    • Create a notarized DMG

    • Install the DMG on a clean test machine

    • Verify System Extension activation prompts appear

    • Test VPN connection for both Hydra and WireGuard protocols

  3. Verification:

    • Check System Preferences > Security & Privacy for extension approval

    • Use systemextensionsctl list in Terminal to view installed extensions

    • Monitor system logs for extension lifecycle events

Best Practices

  1. Always set usesSystemExtension correctly: Mismatch between this flag and actual target type will cause connection failures

  2. Handle user approval gracefully: Provide clear instructions for approving the extension

  3. Test on clean machines: System Extension installation behaves differently on development machines

  4. Use build configurations: Separate SYSTEM_EXTENSION build configuration helps manage conditional compilation

  5. Monitor activation status: Implement proper delegate callbacks to track activation progress

  6. Provide clear error messages: System Extension errors can be cryptic - translate them for users

  7. Test upgrade scenarios: Ensure your app handles replacing existing System Extensions correctly

Demo Application Reference

The VPNSDK Demo for Apple includes a complete System Extension implementation with:

  • Separate targets for Hydra and WireGuard System Extensions

  • Proper entitlements configuration for all targets

  • System Extension activation helper class

  • Build configurations for development and production

  • Code signing configuration with provisioning profiles

  • Example DMG creation and notarization workflow

The demo supports multiple configurations:

  • iOS: Traditional Network Extensions

  • macOS (App Extension): Traditional Network Extensions

  • macOS (System Extension): Modern System Extensions with Developer ID distribution

Additional Resources

Last updated

Was this helpful?