Protobuf and External Integration ​

Purpose and Scope ​

This document describes ecapture's protobuf-based event serialization format and external integration capabilities. Starting from version 0.8.0, ecapture provides structured event streaming through Protocol Buffers, enabling real-time integration with external tools, GUIs, and analysis platforms. For information about text-based console output, see 4.1 Text Output Mode. For PCAP file generation, see 4.2 PCAP Integration.

The protobuf format enables:

• Structured event serialization for programmatic consumption
• WebSocket streaming for real-time event delivery
• Integration with the eCaptureQ graphical interface
• HTTP APIs for remote configuration and event forwarding
• Compatibility with tools like Burp Suite for security analysis

Protobuf Event Structure ​

LogEntry and Event Schema ​

ecapture uses a hierarchical protobuf schema where all output is wrapped in a LogEntry message. Each LogEntry can contain either log messages or captured events:

Protobuf Event Serialization Flow

Sources: pkg/event_processor/iworker.go:214-227, user/event/ievent.go:51

Event Type Enumeration ​

The type field in the Event message corresponds to the ParserType enumeration, indicating how the payload should be interpreted:

The parser type is determined dynamically based on payload content inspection and assigned when the event worker completes processing.

Sources: pkg/event_processor/iparser.go:40-47, pkg/event_processor/iworker.go:218

Event Serialization in the Pipeline ​

EventProcessor to Protobuf Conversion ​

When events flow through the event processing pipeline, they are converted to protobuf format at the output stage. The conversion occurs in the eventWorker.Display() method:

Event-to-Protobuf Conversion Sequence

The key distinction is whether the configured logger implements the CollectorWriter interface. If not, events are serialized as protobuf; otherwise, they are formatted as human-readable text.

Sources: pkg/event_processor/iworker.go:174-228, user/event/ievent.go:54-70

Protobuf vs. Text Mode Selection ​

The output mode is determined by the logger type passed to NewEventProcessor():

Logger Type Determines Serialization Format

This design allows the same event processing pipeline to serve both human-readable console output and structured protobuf streaming without code duplication.

Sources: pkg/event_processor/iworker.go:198-228, pkg/event_processor/processor.go:206-215

WebSocket Streaming Server ​

Server Architecture ​

Starting from version 1.4.0, ecapture includes an embedded HTTPS server that provides WebSocket endpoints for real-time event streaming. The server runs concurrently with the capture modules:

Embedded HTTPS Server Architecture

The server listens on localhost:28256 by default (configurable via --listen flag) and provides WebSocket connections for clients to subscribe to the event stream.

Sources: CHANGELOG.md:82-87, README.md:82-86

WebSocket Protocol ​

Clients connect to the WebSocket endpoint and receive a continuous stream of protobuf-serialized LogEntry messages. Each WebSocket message contains exactly one marshaled LogEntry:

Clients must:

1. Establish WebSocket connection to wss://localhost:28256/ws
2. Continuously read binary WebSocket messages
3. Unmarshal each message as pb.LogEntry
4. Process based on log_type discriminator

Sources: CHANGELOG.md:16, README.md:304-308

Connection Lifecycle ​

WebSocket Connection Lifecycle

Sources: CHANGELOG.md:82-89

eCaptureQ GUI Integration ​

Remote Mode Architecture ​

eCaptureQ is the official cross-platform GUI client for ecapture. It connects to ecapture's WebSocket endpoint to receive events and provides real-time visualization. The remote mode architecture enables Windows/macOS clients to connect to Linux servers:

eCaptureQ Remote Mode Integration

Sources: README.md:287-302, README_CN.md:267-281

Integrated Mode vs. Remote Mode ​

eCaptureQ supports two deployment models:

In integrated mode, eCaptureQ spawns the ecapture process and connects to its WebSocket endpoint via localhost. In remote mode, users configure the remote server address, and eCaptureQ establishes a WebSocket connection over the network.

Sources: README.md:293-296, README_CN.md:273-276

HTTP Configuration API ​

Remote Configuration Updates ​

ecapture exposes HTTP endpoints for dynamic configuration updates without restarting the capture process. The API enables remote control of capture parameters:

Configuration API Endpoints

HTTP Configuration API Structure

The API accepts JSON payloads to modify capture behavior at runtime. For example, changing the target PID or updating BPF filters without stopping the capture.

Sources: README.md:321-323, README_CN.md:302-303

Configuration Update Flow ​

Configuration Update Sequence

The module may need to reload eBPF programs if certain parameters (like PID filters) are changed. The API provides atomic configuration updates to prevent inconsistent state.

Sources: README.md:321-323, CHANGELOG.md:27

Event Forwarding Integration ​

Burp Suite Integration ​

ecapture supports forwarding captured HTTP/HTTPS events to external tools like Burp Suite for security analysis. This enables SSL/TLS traffic inspection without certificate installation:

Event Forwarding to External Tools

Events are reconstructed as HTTP requests and forwarded to configured endpoints. Burp Suite can intercept and analyze the traffic as if it were a proxy, but without requiring certificate trust.

Sources: README.md:325-327, README_CN.md:305-306

Forwarding Configuration ​

Event forwarding is configured via command-line flags or the HTTP configuration API:

The forwarder converts each parsed event into an HTTP request payload and sends it to the configured endpoint. Response status is logged but does not affect capture.

Sources: README.md:325-327

Custom Client Implementation ​

WebSocket Client Example ​

For custom integrations, clients can connect directly to ecapture's WebSocket endpoint. A minimal Python client example:

# Pseudocode structure - not actual code
import websocket
import protobuf_generated.log_entry_pb2

def on_message(ws, message):
    log_entry = log_entry_pb2.LogEntry()
    log_entry.ParseFromString(message)
    
    if log_entry.log_type == LOG_TYPE_EVENT:
        event = log_entry.event_payload
        print(f"Event: {event.uuid}")
        print(f"Payload: {event.payload}")

ws = websocket.WebSocketApp(
    "wss://localhost:28256/ws",
    on_message=on_message
)
ws.run_forever()

The protobuf schema definitions are available in the repository at protobuf/PROTOCOLS.md for generating language-specific bindings.

Sources: CHANGELOG.md:16, README.md:304-308

Protobuf Schema Generation ​

Clients must generate protobuf bindings from the .proto definition files in the protobuf/ directory:

Protobuf Schema Compilation for Multiple Languages

The repository includes pre-generated Go bindings at protobuf/gen/v1/. For other languages, developers run protoc with the appropriate language plugin against the source .proto files.

Sources: pkg/event_processor/iworker.go:29, user/event/ievent.go:21

Integration Security Considerations ​

TLS Certificate Verification ​

The embedded HTTPS server uses self-signed certificates by default. Clients connecting to remote ecapture instances should:

1. Verify the server certificate fingerprint out-of-band
2. Use certificate pinning for production deployments
3. Restrict listener addresses (use --listen 127.0.0.1:28256 for local-only access)

For remote access, consider SSH tunneling or VPN instead of exposing the port directly:

# SSH tunnel example (not actual code)
ssh -L 28256:localhost:28256 user@remote-server
# Then connect local client to localhost:28256

Sources: README.md:82-86

Authentication and Authorization ​

As of version 1.5.1, ecapture does not implement authentication on the HTTP/WebSocket endpoints. The security model assumes:

• The listener is bound to localhost by default (localhost:28256)
• Remote access requires SSH tunneling or similar secure transport
• The system-level requirement for root privileges provides process isolation

Future versions may add API key authentication for remote deployments.

Sources: README.md:82-86, CHANGELOG.md:1-35