Troubleshooting Azure IoT Edge
This guide provides common troubleshooting steps and solutions for issues you might encounter with Azure IoT Edge.
Common Issues and Solutions
1. IoT Edge Agent or IoT Edge Hub Not Running
The IoT Edge Agent (iotedged) and IoT Edge Hub (edgeHub) are critical components. If they are not running, your modules won't deploy or communicate.
Diagnosis:
- Check the status of the IoT Edge service:
systemctl status iotedge - View logs for the IoT Edge Agent:
journalctl -u iotedge -f - View logs for the edgeHub module (if it's supposed to be running):
iotedge logs edgeHub
Solutions:
- Restart the service:
sudo systemctl restart iotedge - Check configuration: Ensure your
config.yamlfile is correctly formatted and contains valid connection strings or certificates. - Resource limitations: The device might be running out of memory or CPU. Check system resource usage.
2. Modules Not Deploying or Crashing
Problems with module deployment or execution can stem from various sources, including configuration errors, image issues, or resource constraints.
Diagnosis:
- Check the status of your deployed modules using the Azure portal or Azure CLI.
- Examine the logs for individual modules:
iotedge logs - Verify module identity registration in IoT Hub.
- Ensure the correct container image is specified and accessible (e.g., from Azure Container Registry).
Solutions:
- Correct deployment manifest: Double-check the
deployment.jsonfile for syntax errors, incorrect module names, image URIs, or environment variables. - Image pull issues: If using a private registry, ensure authentication credentials are provided correctly in the deployment manifest.
- Resource allocation: Modules might be crashing due to insufficient memory or CPU. Adjust resource limits if necessary.
- Dependency errors: Ensure any required parent modules or services are running.
Network Connectivity Issues: Devices often struggle with intermittent or blocked network access. Ensure your firewall rules allow communication to and from IoT Hub and any necessary container registries.
3. Device Not Connecting to IoT Hub
Connection problems can prevent your IoT Edge device from communicating with Azure IoT Hub.
Diagnosis:
- Verify the device's network connectivity.
- Check the IoT Hub connection string or certificate validity in the device's configuration.
- Inspect IoT Edge Agent logs for connection errors.
- Ensure the correct ports are open (e.g., 443, 8883 for MQTT/TLS).
Solutions:
- Re-provision the device: If connection strings or certificates are compromised or incorrect, re-provisioning the device with new credentials can help.
- Network configuration: Ensure DNS resolution is working and that there are no proxy or firewall issues blocking traffic to IoT Hub endpoints.
- Time synchronization: Ensure the device's clock is synchronized with a reliable time server, as certificate validation relies on accurate timestamps.
4. Large Log Files or Disk Usage
Over time, logs can consume significant disk space. Regular management is crucial.
Diagnosis:
- Check disk space usage on the device:
df -h - Identify large log files, often found in
/var/log/syslogor within the IoT Edge module log directories.
Solutions:
- Configure log rotation: Set up
logrotateto manage IoT Edge agent and module logs. - Limit log verbosity: Adjust the log level in the IoT Edge Agent configuration if excessive logging is not required.
- Clean up old logs: Manually remove old, unnecessary log files, but be cautious not to delete critical current logs.
Using Azure portal diagnostics: The Azure portal offers powerful tools for viewing device twin properties, module twin properties, and direct method invocation, which can be invaluable for remote troubleshooting.
Troubleshooting Tools and Commands
Here are some essential commands for diagnosing issues on your IoT Edge device:
- Check service status:
systemctl status iotedge - Start/Stop/Restart service:
sudo systemctl start/stop/restart iotedge - View service logs:
journalctl -u iotedge -f - View module logs:
iotedge logs - List modules:
iotedge list - Get device info:
iotedge check(runs a diagnostic check) - Show device twin:
iotedge device-twin show - Show module twin:
iotedge module-twin show