Article ID: TECH-HTTP-001
Last Updated: January 2025
Category: Technical Support / Integration
Tags: HTTP Export, Troubleshooting, Gateway Timeout, Integration Errors

📋 Article Summary

This guide helps customers diagnose and resolve issues with receiving automated energy measurement exports via HTTP. It covers common errors, their solutions, and provides a systematic approach to troubleshooting.

🎯 Applies To

• Customers using HTTP/HTTPS export functionality
• System administrators managing HTTP endpoints
• Technical support teams

⚡ Quick Solution Finder

📝 Prerequisites

Before troubleshooting, verify your HTTP server meets these requirements:

✅ Accepts minimum 1MB payload size
✅ Returns HTTP 200 OK status on successful receipt
✅ Supports POST, PUT, or POST MULTIPART methods
✅ Can handle multiple transactions for large exports (>2,000 measurements)

🔧 Common Error Types and Solutions

1. Gateway Timeout Errors

Error Details:

• Message: Gateway Timeout
• Type: CLIENT_ERROR
• HTTP Status: 504

Root Causes:

• Server processing exceeds 60-second timeout
• Proxy/load balancer timeout too restrictive
• Backend performance bottlenecks

Step-by-Step Solution:

Increase Timeout Settings

For Nginx:

# Add to your server or location block
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
proxy_send_timeout 300s;

For Apache:

# Add to httpd.conf or .htaccess
Timeout 300
ProxyTimeout 300

For IIS:

<system.webServer>
  <httpProtocol>
    <customHeaders>
      <add name="Keep-Alive" value="timeout=300" />
    </customHeaders>
  </httpProtocol>
</system.webServer>

Optimize Processing Performance

• Monitor CPU/memory during exports
• Implement asynchronous processing
• Consider returning 200 immediately and processing in background

2. Read Timeout Errors

Error Details:

• Message: Read timed out
• Type: INTERNAL_ERROR
• Frequency: Very common (261-262 status codes)

Root Causes:

• Network connectivity interruption
• Firewall blocking sustained connections
• SSL/TLS handshake issues

Step-by-Step Solution:

Network Diagnostics

# Test connectivity
telnet your-server.com 443

# Check SSL certificate
openssl s_client -connect your-server.com:443

Firewall Configuration

• Whitelist our export service IPs (contact support for list)
• Ensure persistent connections are allowed
• Check rate limiting rules

Server Health Check

# Monitor connections during export window
netstat -an | grep :443 | grep ESTABLISHED

3. DNS Resolution Errors

Error Details:

• Message: Name or service not known
• Type: CLIENT_ERROR
• Example: ops.autodeskbuildingops.com: Name or service not known

Root Causes:

• Incorrect hostname/URL
• DNS configuration issues
• Domain expiration or changes

Step-by-Step Solution:

Verify Domain Configuration

# Check DNS resolution
nslookup your-domain.com
dig your-domain.com

# Verify from our perspective
curl -I https://your-domain.com/your-endpoint

Common Fixes

• Ensure domain is publicly resolvable
• Check for typos in configured URL
• Verify SSL certificate matches domain

4. Internal Server Errors

Error Details:

• Message: Internal Server Error
• Type: CLIENT_ERROR
• HTTP Status: 500

Root Causes:

• Application crashes when processing payload
• Insufficient server resources
• Code bugs or misconfigurations

Step-by-Step Solution:

Enable Application Error Logging

Check Resource Limits

# Memory
free -m

# Disk space
df -h

# Process limits
ulimit -a

Test with Sample Payload (see Debugging Steps)

🔍 Comprehensive Debugging Steps

Step 1: Enable Verbose Logging (CRITICAL)

⚠️ Important: Verbose logging is essential for troubleshooting. Without detailed logs, we cannot effectively help diagnose issues.

Nginx Configuration:

# /etc/nginx/nginx.conf
error_log /var/log/nginx/error.log debug;
access_log /var/log/nginx/access.log combined;

# Log request body for debugging
log_format postdata '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $bytes_sent '
                    '"$http_referer" "$http_user_agent" "$request_body"';

Apache Configuration:

# Enable debug logging
LogLevel debug
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined

# Log POST data (use cautiously)
DumpIOInput On
DumpIOOutput On

Application-Level Logging (Node.js Example):

const fs = require('fs');
const logStream = fs.createWriteStream('./export-debug.log', { flags: 'a' });

app.use((req, res, next) => {
  const timestamp = new Date().toISOString();
  const logEntry = {
    timestamp,
    method: req.method,
    url: req.url,
    headers: req.headers,
    contentLength: req.headers['content-length'],
    ip: req.ip
  };
  
  logStream.write(JSON.stringify(logEntry) + '\n');
  
  // Log response
  const originalSend = res.send;
  res.send = function(data) {
    logStream.write(`Response: ${res.statusCode} at ${new Date().toISOString()}\n`);
    originalSend.call(res, data);
  };
  
  next();
});

Step 2: Test Endpoint Manually

Basic Test:

curl -X POST https://your-server.com/endpoint \
  -H "Content-Type: application/json" \
  -d '{
    "measurements": [{
      "device_id": 47653,
      "device_name": "Test Device",
      "measurement_time(UTC)": "2021-08-10T08:48:00Z",
      "resolution(minutes)": 1,
      "site_id": 3026,
      "site_name": "Test Site",
      "current(A)": 3.7,
      "voltage(V)": 230,
      "power(W)": 807.4,
      "power_factor": 0.95,
      "energy(Wh)": 13.5
    }]
  }' \
  -w "\n\nHTTP Code: %{http_code}\nTotal Time: %{time_total}s\nConnect Time: %{time_connect}s\nStart Transfer: %{time_starttransfer}s\n" \
  -v

Large Payload Test:

# Create large test file
echo '{"measurements":[' > large-test.json
for i in {1..2000}; do
  echo '{"device_id":'$i',"device_name":"Device'$i'","measurement_time(UTC)":"2021-08-10T08:48:00Z","resolution(minutes)":1,"site_id":3026,"site_name":"Test Site","current(A)":3.7,"voltage(V)":230,"power(W)":807.4,"power_factor":0.95,"energy(Wh)":13.5},' >> large-test.json
done
echo ']}' >> large-test.json

# Test with large payload
curl -X POST https://your-server.com/endpoint \
  -H "Content-Type: application/json" \
  -d @large-test.json \
  -w "\nPayload Size: %{size_upload} bytes\nTime: %{time_total}s\n"

Step 3: Monitor During Export Window

Identify Export Schedule

• Check your export configuration for scheduled times
• Note timezone differences (exports use UTC)

Real-time Monitoring Script:

#!/bin/bash
# monitor-exports.sh

LOG_FILE="/var/log/nginx/access.log"
ERROR_FILE="/var/log/nginx/error.log"

echo "Monitoring exports... Press Ctrl+C to stop"
tail -f $LOG_FILE $ERROR_FILE | grep -E "POST|PUT|500|502|504"

Resource Monitoring:

# Run during export window
top -b -n 1 > resource-snapshot.txt
iostat -x 1 10 >> resource-snapshot.txt

✅ Diagnostic Checklist

Use this checklist to ensure all common issues are addressed:

• [ ] Endpoint Configuration
  • [ ] Accepts POST/PUT/POST MULTIPART methods
  • [ ] Returns exactly HTTP 200 OK (no redirects)
  • [ ] No authentication or properly configured
• [ ] Capacity & Performance
  • [ ] Can handle 1MB+ payloads
  • [ ] Timeout settings >60 seconds at all layers
  • [ ] Sufficient CPU/memory during peak times
• [ ] Network & Security
  • [ ] Firewall allows incoming connections
  • [ ] SSL/TLS certificates valid and not expired
  • [ ] DNS resolves correctly from external networks
• [ ] Logging & Monitoring
  • [ ] Verbose logging enabled
  • [ ] Logs retained for at least 7 days
  • [ ] Monitoring alerts configured

📊 Data to Collect for Support

When contacting support, please provide:

1. Error Information

Export Configuration ID: [Your ID]
Error Message: [Exact error from logs]
Failure Timestamps: [UTC times]
Frequency: [How often it occurs]

2. Server Logs (REQUIRED)

Include at least 3 failed attempts showing:

• Complete request headers
• Response codes
• Processing duration
• Any error stack traces

3. Configuration Details

Web Server: [Type and Version]
OS: [Operating System]
Timeout Settings: [All relevant timeouts]
Proxy/LB: [If applicable]
SSL Certificate: [Expiry date]

4. Test Results

• Manual curl test output
• Large payload test results
• Resource usage during failures

5. Network Information

Public IP: [Your server IP]
Firewall Rules: [Relevant rules]
Rate Limits: [If configured]

💡 Best Practices

Implement Robust Logging

# Python example with rotation
import logging
from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    'export-receiver.log',
    maxBytes=10485760,  # 10MB
    backupCount=10
)
logging.basicConfig(handlers=[handler], level=logging.DEBUG)

Quick Response Pattern

// Return 200 immediately, process async
app.post('/export', (req, res) => {
  res.status(200).send('OK');
  
  // Process in background
  setImmediate(() => {
    processExportData(req.body);
  });
});

Health Check Endpoint

@app.route('/health')
def health_check():
    return {'status': 'healthy', 'timestamp': datetime.utcnow()}, 200

Monitor Endpoint Availability

• Use uptime monitoring services
• Set up alerts for failures
• Track response time trends

🆘 Still Need Help?

If issues persist after following this guide:

1. Collect all diagnostic information listed above
2. Enable verbose logging and capture multiple failed attempts
3. Run the manual tests and save outputs
4. Contact Support with:
   • This article reference (TECH-HTTP-001)
   • All collected diagnostic data
   • Your export configuration ID

Remember: Failed exports are automatically retried for up to 7 days, ensuring no data loss during troubleshooting.